I would love to see any good research projects about it but i have the feeling that Frontier with MoE is making too fast of a progress so that a customized model would always be worse and that the MoE part is actually going somehow in this direction.

On the other hand, at the GTC was a talk about coding in different lanugage (like spanish) and explaining that the quality between spanish and english is relevant different.

But i have not found a good article about the impact of learning data with practical experiments or even if the order of the learning data matters.

At least I think i remember that Meta mentioned having better and less data can be better than more data with lower quality.

As long as these models can explain to you facts about any other topics, its still overfitted for the task though.

I kick myself a couple times a week for not getting the 512GB Mac Studio in February. I was holding out for an M4 or M5 chip...

Pulled the trigger?

Bought an M1 64 GB for 2000 euro’s second hand a year ago. That was sweet

If you're dropping thousands on API tokens, you're going to be slowed down at least 10x trying to do everything on a single MBP.

The models people are spending thousands on require more on the range of 600-800gb memory.

128gb hardly runs deepseek v4 flash which is almost free via api pricing.

Don't forget about energy usage, you'll probably never break even vs same model on openrouter.

I'm waiting for the speed/quality per dollar metric to go down a little bit further and then I will def run it at home.

Its not just that you send a sentence to an API endpoint, you always send EVERYTHING to that agent as a context.

You want to analyse your spending history? You now send everything to someone.

Either no one cares but understands this implication on how easy it is to really capture you or no one really things about it.

But i'm a lot more diligent on what I send. I disabled the gemini activity feature for example because google started telling me that my stuff could be reviwed by humans.

"With dirt cheap models like deepseek-v4-flash that will run "forever" on $10, the answer for me is clearly: no."

When it's free, you are the product.

It's the size of a big vm. There's nothing wrong with reserving that much working space for one item.

If you don't mind exfiltrating all your IP to the API provider

Problem is that might go away or get nerfed.

Haha wow. I’m trying to even imagine the AI landscape in 15 years and I can’t.

I’m quite optimistic about the long-term future of local LLMs for privacy and cost control reasons. An LLM running on my own hardware, even if it’s not a laptop but a home server, is one where I don’t need to worry about token limits, token fees, privacy, and “rug-pulling” from the vendor.

In the short term, the big challenge is being able to afford hardware that can run a ~30B model. Last month I got to experiment with LLMs on a NVIDIA RTX 6000 Ada Generation as a visiting researcher during my summer break. I see the power of local LLMs for agentic coding; they’re no Claude, but they are quite useful. I wish I had gotten into local LLMs before hardware has gotten prohibitively expensive and in some cases unavailable; Apple discontinued certain Mac Minis and Mac Studios with high amounts of RAM due to the RAM shortage.

Hopefully high RAM prices don’t become a new normal, though the next year or two doesn’t look good.

> I don't understand the desire to run own AI models for programming locally.

Privacy. Security. Not bulk uploading your trade secrets and intellectual property to Sam and Dario’s servers.

> A model you can run on a loptop is simply not going to work as well as it's needed for programming

The models you can run on a high-spec laptop today are approximately where frontier models were 12-18mo ago (albeit at a lower tok/s rate). If you scan back through hn comments from that era, you’ll find plenty of people saying “this is powerful enough to massively increase my productivity”.

I've never done it but would be interested because it cuts out the burden of worrying about costs. Maybe I'm mistaken on energy cost here. There's a constant raincloud that follows me around regarding limits, and it would be nice to shake that.

I've been able to accomplish incredible feats (for myself) since GPT-4, so model intelligence is secondary.

For some companies there might be a need to run them locally. For instance, Apple decided to run LLMs on the phone locally. I guess it depends on how important latency and privacy are. Perhaps Meta is looking at how much interest for those local models is there.

Yet.