> something competitive with Nvidia for AI training

Apple is counting on something else: model shrink. Every one is now looking at "how do we make these smaller".

At some point a beefy Mac Studio and the "right sized" model is going to be what people want. Apple dumped a 4 pack of them in the hands of a lot of tech influencers a few months back and they were fairly interesting (expensive tho).

▲

JumpCrisscross 6 hours ago | parent | next [-]

> Apple is counting on something else: model shrink

The most powerful AI interactions I've had involved giving a model a task and then fucking off. At that point, I don't actually care if it takes 5 minutes or an hour. I've cued up a list of background tasks it can work on, and that I can circle back to when I have time. In that context, smaller isn't even the virtue at hand–user patience is. Having a machine that works on my bullshit questions and modelling projects at one tenth the speed of a datacentre could still work out to being a good deal even before considering the privacy and lock-in problems.

▲

raincole 3 hours ago | parent | next [-]

Cool? And it has nothing to do with what kind of consumer hardware Apple should sell. If your use cases are literally "bigger model better" then the you should always use cloud. No matter how much computing power Apple squeezes into their device it won't be a mighty data center.

	▲	gizajob 2 hours ago \| parent \| next [-]
		For running the model once it’s been trained, all a datacenter does is give you lower latency. Once the devices have a large enough memory to host the model locally, then the need to pay datacenter bills is going to be questioned. I’d rather run OpenClaw on my device plugged into a local LLM rather than rely on OpenAI or Claude.
	▲	3 hours ago \| parent \| prev [-]
		[deleted]

▲

jiggawatts 2 hours ago | parent | prev [-]

What "tooling" do you use to let AIs work unattended for long periods?

▲

root_axis 6 hours ago | parent | prev | next [-]

> At some point a beefy Mac Studio and the "right sized" model is going to be what people want.

It's pretty clear that this isn't going to happen any time soon, if ever. You can't shrink the models without destroying their coherence, and this is a consistently robust observation across the board.

▲

sipjca 6 hours ago | parent [-]

I don’t think it’s about literally shrinking the models via quantization, but rather training smaller/more efficient models from scratch

Smaller models have gotten much more powerful the last 2 years. Qwen 3.5 is one example of this. The cost/compute requirements of running the same level intelligence is going down

▲

HerbManic 4 hours ago | parent | next [-]

I have said for a while that we need a sort of big-little-big model situation.

The inputs are parsed with a large LLM. This gets passed on to a smaller hyper specific model. That outputs to a large LLM to make it readable.

Essentially you can blend two model type. Probabilistic Input > Deterministic function > Probabilistic Output. Have multiple little determainistic models that are choose for specific tasks. Now all of this is VERY easy to say, and VERY difficult to do.

But if it could be done, it would basically shrink all the models needed. Don't need a huge input/output model if it is more of an interpreter.

▲

kyboren 4 hours ago | parent | prev [-]

Yes, but bigger models are still more capable. Models shrinking (iso-performance) just means that people will train and use more capable models with a longer context.

	▲	sipjca 2 hours ago \| parent [-]
		Of course they are! Both are important and will be around and used for different reasons

▲

Forgeties79 7 hours ago | parent | prev [-]

Cheaper than what you’d expect though. You could get a nice setup for $20-40k 6mo ago. As far as enterprise investments go, that’s a rounding error.

	▲	a1o 6 hours ago \| parent \| next [-]
		Not all enterprises are the same, I imagine many companies have different departments working with local optimums, so someone who could benefit from it to get more productivity might not have access to it because the department that is doing hardware acquisition is being measured in isolation.
	▲	zer00eyz 6 hours ago \| parent \| prev [-]
		Drop that down to 5k, and make it useful. Give every iPhone family a in house Siri that will deal with canceling services and pursuing refunds. Your customer screw up results in your site getting an agent drive DDOS on its CS department till you give in. Siri: "Hey User, here's your daily update, I see you haven't been to the gym, would you like me to harass their customer service department till they let you out of their onerous contract?"