| ▲ | Apple ML Research: Muse Spark Open with MLX on Mac Studio(machinelearning.apple.com) |
| 96 points by antipaul 3 hours ago | 48 comments |
| |
|
| ▲ | ks2048 an hour ago | parent | next [-] |
| So, Meta will spend millions/billions on building open-weight models and then people will send Apple $20,000 to run them. Seems like a good deal for Apple. |
| |
| ▲ | Juvination an hour ago | parent | next [-] | | Exactly why does Apple need to do any model training/development? They are out here selling the shovels. | | |
| ▲ | rswail 16 minutes ago | parent | next [-] | | Exactly my thought as I read it. They make and sell the hardware that will actually make AI something everyone can used without needing to rely on OpenAI/Anthropic et al. Apple has never had a moat, anyone else could have built this sort of hardware but they haven't. So they don't care which model wins, because it will always move out to the edge of the network, and they own a large chunk of that edge. | | |
| ▲ | drob518 9 minutes ago | parent [-] | | Well, the moat is that cpu/npu development is very hard, requires talented engineers, and requires access to cutting edge fab process, all of which in turn costs a lot of $$$. So, sure, “anyone” who has all that can definitely breach Apple’s moat. |
| |
| ▲ | bredren 33 minutes ago | parent | prev [-] | | Apple’s Photos ecosystem is privacy oriented and as a result some (now) baseline features require the use of local models. The Foundation models keep a surprisingly decent set of data available for fast filtering by a local LLM. I’ve found a slower, second pass, is needed by Qwen to clean it up. (I am running on the new Mac Mini m5 pro 64gb.) I’ve run my own tests using a date range, a short description of a red cedar planter box build I did last year, and a set of known correct photos. There is some cool stuff you can do without sending your family’s pics up to any of the frontier model providers. | | |
| ▲ | cj00 24 minutes ago | parent [-] | | I've found some strange guardrails with the Foundation models. Like blocking describing of any photo with a single woman in it. It does this relatively consistently but with a different subset every time. Like out of 10 photos in the set, it's skips 6 then the next run, it skips 3. Have you seen guardrail issues like this? I switched to Gemma 4 instead. |
|
| |
| ▲ | loeg 11 minutes ago | parent | prev | next [-] | | How big is the local LLM market? I have trouble imagining Apple will move enough units to make $20k/ea worth it for them as a business segment. | |
| ▲ | GeekyBear 41 minutes ago | parent | prev | next [-] | | How many 5080 16 Gig cards would you have to buy to run the same model, and what would that cost? | | |
| ▲ | fluidcruft 25 minutes ago | parent [-] | | idk but naive me sees the article say the model needed 158GB of RAM so I guess 10x 16GB GPUs? Apparently retail price is $1k per 5080 16GB but they're actually selling at $1.5k so I guess $15k for just the GPUs. | | |
| ▲ | petu 18 minutes ago | parent [-] | | 158GB number is for Mistral Medium 3.5 128B model, Muse Spark is way larger. Muse Spark Open would require more than 1.6TB of RAM in its native precision (BF16) for the model parameters alone. In a 4-bit quantized (NVFP4) format, the model memory footprint is lowered to about 493GB.
| | |
| ▲ | bigyabai 11 minutes ago | parent [-] | | So about 24gb/expert at NVFP4. If you compressed the KV cache, you might be able to get away with 2x 24GB cards and stream the experts. | | |
| ▲ | petu 2 minutes ago | parent [-] | | If you're happy with generation speed in 5 t/s range, sure -- PCIe 5.0 x16 is only 64GB/s. |
|
|
|
| |
| ▲ | reilly3000 an hour ago | parent | prev | next [-] | | They have the best strategy of bigtechco: let others figure out the squishier layers while they focus on selling the best hardware. Privacy has incredible value to a lot of individuals and industries. $20-80k in hardware that *appreciates* the moment you buy it is a reasonable investment for a lot of SMBs, and still less than a single Nvidia rig. Privacy pays. | |
| ▲ | informal007 28 minutes ago | parent | prev | next [-] | | It won't be free for Apple to use Meta's model in the future. | |
| ▲ | rvz 32 minutes ago | parent | prev [-] | | Both companies are profitable and AI is not their main line of business. This affects pure AI companies. As open weight models continue to match frontier models, the race to $0 will continue to benefit supporters of running open weight models locally. |
|
|
| ▲ | peri-cl an hour ago | parent | prev | next [-] |
| I think they're sandbagging Qwen pretty heavily by benchmarking Meta's model at 4-bit, against Qwen only at native BF16 (and nothing else shown). That's not a quant most users would choose. I get that this is cross-brand marketing for Apple's partner, Meta; but really Muse Spark isn't even in the top *ten* models I'd consider if I had a 512 GB M5 in my paws. Or a cluster of them. |
|
| ▲ | magnusokik 6 minutes ago | parent | prev | next [-] |
| It's a good article which showcases direction of travel, i am sure mac studio will get more and more powerful in coming generations but for now it's way to expensive for local setup. |
| |
| ▲ | kingcauchy 3 minutes ago | parent [-] | | yeah especially because it's not very easy to configure everything sensibly for a multi-use case setup, they really need better mac containers on the mac itself |
|
|
| ▲ | tony_cannistra 2 hours ago | parent | prev | next [-] |
| Not exactly your average "desk", but still neat. In this post, we show how to run the latest frontier-class model — Muse Spark Open, with nearly a trillion parameters — right from your desk, using the latest version of MLX, released today. On a cluster of four Mac Studios, Muse Spark Open generates over 36 tokens per second (see Figure 1), well above average human reading speed. Beginning on a single Mac Studio (up to 512GB of unified memory), then scaling out across multiple Macs with the RDMA over Thunderbolt driver, below we share an overview of the range of models you can now run on Mac Studio.
|
| |
| ▲ | bensyverson an hour ago | parent [-] | | Yes, I'm very glad they're doing this, but as of 2026, it's a bit academic. Pricing for the 512GB Mac Studio hasn't even been announced yet, but the 256GB version starts at $9499. So when you start talking about four of those puppies, it's substantial. Someone else will do the math on how many years it would take to spend $40k on Muse Spark Open via OpenRouter. | | |
| ▲ | mbreese 32 minutes ago | parent | next [-] | | But there are definitely people out there that will need to run these models locally (contractual or legal requirements). By validating a Mac Studio as an option for running local models, it might be the more appealing option. For most of us though, you're right -- the OpenRouter option is far more cost effective. | |
| ▲ | bitcurious 35 minutes ago | parent | prev | next [-] | | Puts into perspective how cheap the frontier models are via subs or even API. | | |
| ▲ | mlsu 29 minutes ago | parent [-] | | You can't really compare API prices though. The cards that they are running frontier models on in the datacenter can service hundreds of simultaneous requests and are at nearly 100% utilization. This thing would sit on your desk and be used to service a single user for a couple hours per day. |
| |
| ▲ | hirako2000 an hour ago | parent | prev | next [-] | | It seems Apple figured they could sell more Mac Studios on the basis of frontier local AI models, profit to be made while users haven't figured out the math yet. | | |
| ▲ | bensyverson an hour ago | parent [-] | | Don't get me wrong; in the long term I see the price of local inference falling as their chips continue to get 15-20% faster every year, and meanwhile the "intelligence per billion parameter" chart is rising. There will be a magic point where they cross, and "good enough" reaches your phone. That's when Apple starts to drink Nvidia's milkshake. | | |
| ▲ | cyanydeez 41 minutes ago | parent [-] | | Apple and NVIDIA prices are not going to fall. Their competitors will. Anyone locked into these two ecosystems are willingly going along with it. | | |
| ▲ | bensyverson 8 minutes ago | parent [-] | | It’s not about the price of Apple products falling—I was commenting that the price of inference will continue to fall (chips getting faster). If “good enough” inference is in your pocket, this benefits Apple and harms Nvidia. |
|
|
| |
| ▲ | LoganDark an hour ago | parent | prev [-] | | There was the M3 Ultra 512GB Mac Studio, though. | | |
| ▲ | mft_ an hour ago | parent [-] | | …which is in the region of €20k on eBay, so the calculus doesn’t really shift… | | |
| ▲ | LoganDark an hour ago | parent [-] | | It was only like $14k new. Why are you people acting like nobody has this stuff? People were already building clusters when that thing first came out. The research isn't bunk just because the refresh hasn't landed yet, my god. | | |
| ▲ | unshavedyak 31 minutes ago | parent [-] | | Maybe you (or i?) missed the point - it wasn't that nobody has those machines, it was that at $40k for 4x the 256GB versions is a substantial price, and so the parent parent was suggesting that it would be compared against OpenRouter. Ie how long would it take someone to spend $40k worth of credits on OpenRouter? I've had the same thought on self hosting for the past 2 years. Eg i could invest $10k into hardware or spend those on credits. I chose credits, because by the time i spend 10k i suspect the market will have dramatically shifted in requirements, compute, etc. |
|
|
|
|
|
|
| ▲ | janderson215 2 hours ago | parent | prev | next [-] |
| Hilarious how they managed to put out an entire press release about Meta’s technology without mention Meta other than what is in the code references. |
| |
| ▲ | Someone an hour ago | parent | next [-] | | It’s not a press release and not about Meta’s technology. The title (“Local and Distributed Large LLM Inference with MLX and RDMA on Mac Studio”) doesn’t even mention Muse. It’s more like an advertorial for their tech. | |
| ▲ | browningstreet an hour ago | parent | prev [-] | | Still weird, and against brand, for them to lead with this particular model from that particular company. The groans are still earned. | | |
| ▲ | enos_feedler an hour ago | parent | next [-] | | I find it bizarre of Apple to do anything publicly in partnership with Meta. I always thought the reasons Apple snubbed Facebook were out of values and principles not business. That could be wrong. Now that Meta has a line of business that Apple can finally charge a platform tax on there is new reason to start sleeping with them. | | |
| ▲ | origini an hour ago | parent [-] | | Its still weird though. The datacenters are not full of Mac studios but Nvidia hardware. They are known for macbooks. And they are not willing to build proper apple servers for all the build systems people have. | | |
| ▲ | robotresearcher an hour ago | parent [-] | | They did, and few people bought them, so they stopped. | | |
| ▲ | myrandomcomment 30 minutes ago | parent [-] | | I worked at a company where one of the pieces of hardware we were building at the time used a PowerPC CPU (this is back in ~2007). We had a rack of Apple xServe for build and test. The xServe was nice pieces of hardware compared to the x86 kit we had, more like Sun gear in their attention to detail and ease of use. At my current company we compile for x86 and devs can do a full build and test environment on their laptops (which are 95% MacBook Pros) but it is unclear what will happen when Rosetta support is pulled. Now the interesting thing is we plan to move our stuff over to target the AWS Graviton (and other cloud ARM cores) as those instances are cheaper for generally the same performance (for what was are doing anyway) so the point will be moot. There will be some crossover point where having build and test servers in a rack will be cheaper than doing it in the cloud. At that point having Apple ARM based servers would be a win assuming we can build a binary that works for Graviton, et.al. |
|
|
| |
| ▲ | Someone an hour ago | parent | prev [-] | | They don’t lead with it. The title of the actual article is “Local and Distributed Large LLM Inference with MLX and RDMA on Mac Studio”. The submitter changed it to “Apple ML Research: Muse Spark Open with MLX on Mac Studio” | | |
| ▲ | browningstreet an hour ago | parent [-] | | Second paragraph: "In this post, we show how to run the latest frontier-class model — Muse Spark Open, with nearly a trillion parameters..." |
|
|
|
|
| ▲ | comandillos 35 minutes ago | parent | prev | next [-] |
| A 800B model that performs way worse than other 300B ones. Well... |
| |
| ▲ | peri-cl 18 minutes ago | parent | next [-] | | Why buy one 256 GB Studio, when you could buy two 512 GB's for lower-quality output? Local LLM's really sharpen your senses to model size. You could fit a car in the price difference! | |
| ▲ | otterley 28 minutes ago | parent | prev [-] | | What are you using to determine performance? |
|
|
| ▲ | _pdp_ 38 minutes ago | parent | prev | next [-] |
| I am super excited about this. While right now Mac Studio is not a budget device and it certainly costs more than the subsidised subscription, the promise that one day we can run these models on commodity hardware is exciting. |
|
| ▲ | isodev an hour ago | parent | prev | next [-] |
| Preparing to upsell those very expensive Macs coming later this month, are we. |
|
| ▲ | butterlesstoast an hour ago | parent | prev [-] |
| I -- love -- an article -- full of -- em -- dashes. |
| |
| ▲ | ManuelKiessling 20 minutes ago | parent | next [-] | | If used correctly, I find the em dash one of the most elegant and reader-friendly ways to structure certain parts of a text — that it got such a bad reputation lately really is a shame! | |
| ▲ | jonathaneunice 3 minutes ago | parent | prev [-] | | Em dashes—love ’em!—but they do. not. need. personal. space. |
|
