Apple’s Photos ecosystem is privacy oriented and as a result some (now) baseline features require the use of local models.

The Foundation models keep a surprisingly decent set of data available for fast filtering by a local LLM.

I’ve found a slower, second pass, is needed by Qwen to clean it up. (I am running on the new Mac Mini m5 pro 64gb.)

I’ve run my own tests using a date range, a short description of a red cedar planter box build I did last year, and a set of known correct photos.

There is some cool stuff you can do without sending your family’s pics up to any of the frontier model providers.

Nvidia is also selling the shovle but they train their own LLM and by doing so they can fine tune their CUDA/Chip to have a better shovel than others.

Nvidia's own model does not have to beat OpenAI or any SOTA model. Nvidia just need to build up the in house expert and knowledge to make a better chip since hyperscalers and none hyperscaler like Meta is building their own AI chip now.

Apple can chose not to train their own model to learn how to sell a better shovel.

Exactly my thought as I read it.

They make and sell the hardware that will actually make AI something everyone can used without needing to rely on OpenAI/Anthropic et al.

Apple has never had a moat, anyone else could have built this sort of hardware but they haven't.

So they don't care which model wins, because it will always move out to the edge of the network, and they own a large chunk of that edge.

apple is pretty well set up for this because they do all their hardware in house. if they wanted to, they could pivot in this direction. Ternus is also a hardware guy.

They have been sold out of a lot of units, due to people developing AI, so apparently, it is significant.

I think the bet is that it will get bigger, both at the consumer level, and with orgs eventually discovering ready-made solutions for having their own on-prem inference.

100,000 units a year would be $2 billion annually.

And usually, most of a company's revenue comes from the heaviest spenders.

So figure there are some out there who will by tens of these because they have some "idea to change the world". (Whether true or not.) Some startups, etc.

I'd wager, across planet earth right now, it should be trivial to push 100,000 units per year. Probably closer to 500,000 units per year. So that's an easy $10 billion a year.

$2-10 billion is meaningful for any company. Even one as large is Apple.

Especially if it's a first step towards a stronger foothold in the AI inference space.

They can then expand down market to grow the pie further.

For example, with truly dedicated inference boxes.

And given the demand for $7,000 Sparks and sold out $5,000 GPUs, the market for these $20,000 machines is likely much higher than my 100,000 unit lower bound. (Earth is a big place. It doesn't take much excitement to generate massive sales in a space with currently insatiable demand.)

If we're being frank here, Apple's AI strategy lags behind Nvidia, AMD and even Google. Nvidia has CUDA, AMD ships CDNA, Google has the TPU family, and Apple... well, Apple discontinued the rackmount Mac Pro, forfeit the HPC ARM datacenter to Nvidia's Grace CPUs, refuses to sign CUDA drivers, and spent half a decade designing raster GPUs instead of GPGPUs.

All of their competition has real inference and training stacks. Apple ships an iGPU and a smartphone NPU, they're reliant on Nvidia and Google hardware to deliver Apple Intelligence.

idk but naive me sees the article say the model needed 158GB of RAM so I guess 10x 16GB GPUs? Apparently retail price is $1k per 5080 16GB but they're actually selling at $1.5k so I guess $15k for just the GPUs.