| ▲ | AMD will bring its "Ryzen AI" processors to standard desktop PCs for first time(arstechnica.com) |
| 39 points by Bender 3 days ago | 33 comments |
| |
|
| ▲ | tuukkah an hour ago | parent | next [-] |
| Meanwhile, the corresponding "non-standard" desktop PC is the Framework Desktop, which with the Ryzen AI Max+ 395 can use 120GB of its 128GB RAM for the GPU: How to Run a One Trillion-Parameter LLM Locally: An AMD Ryzen™ AI Max+ Cluster Guide https://www.amd.com/en/developer/resources/technical-article... |
| |
| ▲ | hedgehog 18 minutes ago | parent [-] | | Minisforum MS-S1 is the same chip but has a PCIe slot suitable for a network card. |
|
|
| ▲ | snovv_crash an hour ago | parent | prev | next [-] |
| Hoe much dedicated cache do these NPUs have? Because it's easy enough to saturate the memory bandwidth using the CPU for compute, never mind the GPU. Adding dark silicon for some special operations isn't going to make out memory bandwidth faster. |
|
| ▲ | mcraiha an hour ago | parent | prev | next [-] |
| These are mobile chips shoehorned into AM5. They aren't very good e.g. for gaming purposes. https://videocardz.com/newz/amd-ryzen-ai-400-does-not-suppor... |
| |
| ▲ | bcraven 35 minutes ago | parent [-] | | Presumably that's why the subheading is: >First wave of Ryzen AI desktop CPUs targets business PCs rather than DIYers. |
|
|
| ▲ | iso-logi 2 hours ago | parent | prev | next [-] |
| 8 Core/16 Thread, boosting up to 5.1GHz with iGPU would be pretty neat for a Plex Server or Proxmox Server with a few VMs. |
|
| ▲ | lelanthran 33 minutes ago | parent | prev | next [-] |
| It doesn't sound as impressive as I wanted :-( I wanted a better strix halo (which has 128GB unified RAM and 40cu on the 8080s (or something) iGPU). This looks like normal Ryzen mobile chips + but with fewer cus. |
| |
| ▲ | noelwelsh 6 minutes ago | parent | next [-] | | Yeah the next generation of Strix Halo is what would get me excited. I think right now TSMC has no capacity, so maybe we have to wait another year. Kinda ironic that all CPU/RAM capacity is being sold to LLM companies, and as a result we can't get the hardware needed for good local LLMs. | |
| ▲ | wtallis 13 minutes ago | parent | prev [-] | | Putting Strix Halo into the AM5 socket would make no sense. Half the memory controllers would be orphaned and the GPU would be severely bandwidth-starved (assuming that the memory controller on Strix Halo actually supports DDR5 and not just LPDDR5). |
|
|
| ▲ | Buttons840 2 hours ago | parent | prev | next [-] |
| Do we expect special AI processors to diverge from GPUs? Like, processors that can do parallel neural network computations but cannot draw graphics? |
| |
| ▲ | c0balt an hour ago | parent | next [-] | | That is already the case with datacenter "GPUs". A A100, MI300 or Intel PVC/Gaudi does not have useful graphics performance nor capabilities. Coprocessors ala NPU/VPU are also on the rise again for CPUs. | | |
| ▲ | elcritch an hour ago | parent [-] | | Great now I’m envisioning a rich guy using an A100 as his desktop GPU just to show off. Which begs the question if that’s even possible. | | |
| |
| ▲ | dagmx 2 hours ago | parent | prev | next [-] | | That’s already the norm no? Pretty much every hardware vendor has an NPU | |
| ▲ | jiggawatts an hour ago | parent | prev [-] | | Yes. Even the latest NVIDIA Blackwell GPUs are general purpose, albeit with negligible "graphics" capabilites. They can run fairly arbitrary C/C++ code with only some limitations, and the area of the chip dedicated to matrix products (the "tensor units") is relatively small: less than 20% of the area! Conversely, the Google TPUs dedicate a large area of each chip to pure tensor ops, hence the name. This is partly why Google's Gemini is 4x cheaper than OpenAI's GPT5 models to serve. Jensen Huang has said in recent interviews that he stands by the decision to keep the NVIDIA GPUs more general purpose, because this makes them flexible and able to be adapted to future AI designs, not just the current architectures. That may or may not pan out. I strongly suspect that the winning chip architecture will have about 80% of its area dedicated to tensor units, very little onboard cache, and model weights streamed in from High Bandwidth Flash (HBF). This would be dramatically lower power and cost compared to the current hardware that's typically used. Something to consider is that as the size of matrices scales up in a model, the compute needed to perform matrix multiplications goes up as the cube of their size, but the other miscellaneous operations such as softmax, relu, etc.. scale up linearly with the size of the vectors being multiplied. Hence, as models scale into the trillions of parameters, the matrix multiplications ("tensor" ops) dominate everything else. |
|
|
| ▲ | cebert 3 days ago | parent | prev | next [-] |
| AMD marketing is hoping the “AI” branding is a positive. Antidotally, I know many consumers who are not sold on AI. This branding could actually hurt sales. |
| |
| ▲ | aljgz 2 hours ago | parent | next [-] | | We are dealing with a hype, but the reality is that AI would change everything we do. Local models will start being helpful in [more] unobtrusive ways. Machines with decent local NPUs would be usable for longer before they feel too slow. | | |
| ▲ | vbezhenar an hour ago | parent | next [-] | | For some people maybe. I don't want to use local AI and NPU will be dead weight for me. Can't imagine a single task in my workflow that would benefit from AI. It's similar to performance/effiency cores. I don't need power efficiency and I'd actually buy CPU that doesn't make that distinction. | | |
| ▲ | wtallis an hour ago | parent | next [-] | | > Can't imagine a single task in my workflow that would benefit from AI. You don't do anything involving realtime image, video, or sound processing? You don't want ML-powered denoising and other enhancements for your webcam, live captions/transcription for video, OCR allowing you to select and copy text out of any image, object and face recognition for your photo library enabling semantic search? I can agree that local LLMs aren't for everybody—especially the kind of models you can fit on a consumer machine that isn't very high-end—but NPUs aren't really meant for LLMs, anyways, and there are still other kinds of ML tasks. > It's similar to performance/effiency cores. I don't need power efficiency and I'd actually buy CPU that doesn't make that distinction. Do you insist that your CPU cores must be completely homogeneous? AMD, Intel, Qualcomm and Apple are all making at least some processors where the smaller CPU cores aren't optimized for power efficiency so much as maximizing total multi-core throughput with the available die area. It's a pretty straightforward consequence of Amdahl's Law that only a few of your CPU cores need the absolute highest single-thread performance, and if you have the option of replacing the rest with a significantly larger number of smaller cores that individually have most of the performance of the larger cores, you'll come out ahead. | | |
| ▲ | throwa356262 41 minutes ago | parent [-] | | Is everyone a content creator these days? Besides, most of what you mentioned doesn't run on NPU anyway. They are usually standard GPU workload. | | |
| ▲ | wtallis 36 minutes ago | parent [-] | | None of what I listed was in any way specific to "content creators". They're not the only ones who participate in video calls or take photos. And on the platforms that have a NPU with a usable programming model and good vendor support, the NPU absolutely does get used for those tasks. More fragmented platforms like Windows PCs are least likely to make good use of their NPUs, but it's still common to see laptop OEMs shipping the right software components to get some of those tasks running on the NPU. (And Microsoft does still seem to want to promote that; their AI PC branding efforts aren't pure marketing BS.) | | |
| ▲ | anematode 21 minutes ago | parent [-] | | The issue is that the consumer strongly associates "AI" with LLMs specifically. The fact that machine learning is used to blur your background in a video call, for example, is irrelevant to the consumer and isn't thought of as AI. |
|
|
| |
| ▲ | fodkodrasz an hour ago | parent | prev | next [-] | | Never wanted to do high quality voice recognition? No need for face/object detection in near instant speed for your photos, embedding based indexing and RAG for your local documents with free text search where synonyms also work? All locally, real-time, with minimal energy use. That is fine. Most ordinary users can benefit from these very basic use cases which can be accelerated. Guess people also said this for video encoding acceleration, and now they use it on a daily basis for video conferencing, for example. | |
| ▲ | orbital-decay an hour ago | parent | prev [-] | | Also similar to GPU + CPU on the same die, yet here we are. In a sense, AI is already in every x86 CPU for many years, and you already benefit from using it locally (branch prediction in modern processors is ML-based). | | |
| ▲ | wtallis 41 minutes ago | parent [-] | | > Also similar to GPU + CPU on the same die, yet here we are. I think the overall trend is now moving somewhat away from having the CPU and GPU on one die. Intel's been splitting things up into several chiplets for most of their recent generations of processors, AMD's desktop processors have been putting the iGPU on a different die than the CPU cores for both of the generations that have an iGPU, their high-end mobile part does the same, even NVIDIA has done it that way. Where we still see monolithic SoCs as a single die is mostly smaller, low-power parts used in devices that wouldn't have the power budget for a discrete GPU. But as this article shows, sometimes those mobile parts get packaged for a desktop socket to fill a hole in the product line without designing an entirely new piece of silicon. |
|
| |
| ▲ | wood_spirit an hour ago | parent | prev [-] | | So I’ve got a lot warmer to believing that AI can be a better programmer than most programmers these days. That is a low bar :). The current approach to AI can definitely change how effective a programmer is: but then it is up to the market to decide if we need so many programmers. The talk about how each company is going to keep all the existing programmers and just expect productivity multipliers is just what execs are currently telling programmers; that might change when the same is execs are talking to shareholders etc. But does this extrapolate to the current way of doing AI being in normal life in a good way that ends up being popular? The way Microsoft etc is trying to put AI in everything is kinda saying no it isn’t actually what users want. I’d like voice control in my PC or phone. That’s a use for these NPUs. But I imagine it is like AR- what we all want until it arrives and it’s meh. | | |
| ▲ | snovv_crash an hour ago | parent [-] | | When I interview people for a job I'm not looking to hire an average programmer, though. |
|
| |
| ▲ | himata4113 2 hours ago | parent | prev | next [-] | | I'd actually love to have an NPU that isn't useless on my 285k. | |
| ▲ | skirmish an hour ago | parent | prev | next [-] | | Indeed, I was buying a laptop for my wife, and she was viscerally against "Ryzen AI": I don't want a CPU with builtin AI to spy on my screen all the time! | |
| ▲ | kijin an hour ago | parent | prev [-] | | They can just buy a regular Ryzen 9000 series CPU, then. Maybe add a real graphics card if they're into gaming. |
|
|
| ▲ | poly2it an hour ago | parent | prev | next [-] |
| The Ryzen AI line is actually great if deployed to an entire org as the bottom tier, as it garuantees every device has a 50 TOPs NPU. We deploy local software at $STARTUP and this makes deployment to a Windows corp more predictable. |
|
| ▲ | FpUser an hour ago | parent | prev [-] |
| Well, for me personally it is a meh until RAM prices go down. Suddenly, decent PC has turned from a tool accessible to average Joe to a luxury item |
| |