The vast majority of computers sold today have a CPU / GPU integrated together in a single chip. Most ordinary home users don't care about GPU or local AI performance that much.

In this video Jeff is interested in GPU accelerated tasks like AI and Jellyfin. His last video was using a stack of 4 Mac Studios connected by Thunderbolt for AI stuff.

https://www.youtube.com/watch?v=x4_RsUxRjKU

The Apple chips have both power CPU and GPU cores but also have a huge amount of memory (512GB) directly connected unlike most Nvidia consumer level GPUs that have far less memory.

▲

onion2k 13 hours ago | parent [-]

Most ordinary home users don't care about GPU or local AI performance that much.

Right now, sure. There's a reason why chip manufacturers are adding AI pipelines, tensor processors, and 'neural cores' though. They believe that running small local models are going to be a popular feature in the future. They might be right.

▲

swiftcoder 12 hours ago | parent [-]

It's mostly marketing gimmicks though - they aren't adding anywhere near enough compute for that future. The tensor cores in an "AI ready" laptop from a year ago are already pretty much irrelevant as far as inferencing current-generation models go.

	▲	zozbot234 12 hours ago \| parent [-]
		NPU/Tensor cores are actually very useful for prompt pre-processing, or really any ML inference task that isn't strictly bandwidth limited (because you end up wasting a lot of bandwidth on padding/dequantizing data to a format that the NPU can natively work with, whereas a GPU can just do that in registers/local memory). Main issue is the limited support in current ML/AI inference frameworks.