I still have this naive notion that we don't need LLMs for code generation and editing. Does a system need the knowledge of the full works of Shakespeare to be able to output Javascript?

Maybe people smarter than me know better but couldn't there be a middle ground where an IDE/Editor has an embedded engine (doesn't need to be a full-on LLM) that doesn't require external tool calls and token spend?

If an organization is paying $2400/year per developer for tokens and a highly intelligent editor/IDE comes around that charges $1000/yr and gets more output at a fixed cost, its a no-brainer of a decision.

You know what, considering there was a recent "small" open weights LLM released recently that meets 90% of my coding needs I'm inclined to agree.

Qwen3.8-Flash-Next - relatively small, it runs on 6 6 year old GPUs on my home PC happily running 5 simultaneous 262k sessions with additional 10 cached in RAM (bought back when you didn't have to remortgage your house for Ram) and it has been the first local model that is not a toy.

But there is a class of problems where I still reach for Anthropic's fable...

However, I have a hunch bordering with certainty Anthropic is achieving such great results by doing a lot of harness tricks.

For example opus 4.8, is not much better on coding than before mentioned Qwen model, but gets amazing results on factual knowledge stuff (the knowing all works of Shakespeare thing). How hard would it be to add a general knowledge RAG to requests that contain relevant questions and beat all benchmarks like that? Not very hard.

So I think there is big innovation to be had in harnesses, routers, inference and so on.

As to money spent on AI per developer my current client (a fortune 200 software company) spends $500 per month. That is $6k a year. A lot more than your examples. And many people run out of their quota pretty quickly.

Forgive my naive understanding of LLMs - but how do you get semantic understanding of a codebase, such that it knows what changes to make/why/where, without a wider understanding of language more broadly?

I'm using the word understanding loosely there, but I couldn't think of another word.

I think the amount of knowledge to correctly work on code is more than you'd think, because at the end of the day writing code without an understanding of the environment it exists in/for is likely to not fit the problem correctly. Maybe it doesn't need knowledge of _Shakespeare_ per se, but if you were working on a virtual tabletop having knowledge of tabletop games can help with identifying the right implementation to use, knowing what kind of constraints to consider, etc.

I suppose how much literary support you need in your model depends directly on how erudite the comments are.

Would there be problem domains in which the more educated LLM would perform better? Are your names directly related to concepts from said domain,

LLM comments: "I think it may be a potential bug that the sum VATAddedTax gets added to the TaxFreeItems".

I mean it seems a bit unnecessary but also maybe it can help in unexpected ways.

I think this already happens through Mixture of Experts which is now build in to ost models.

But finding out what an LLM needs to understand from a business side to write your code good, is an otpimzation which no one cares currently.

I'm pretty sure we either stay on big full frontier models for a long time, just use them for everything or we will start to see more and more people doing finetuning/project specific training like java + german + english + business contxt xy;

It will be an indicator for the whole industry.

> Maybe people smarter than me know better but couldn't there be a middle ground where an IDE/Editor has an embedded engine (doesn't need to be a full-on LLM) that doesn't require external tool calls and token spend?

IDEA already have small LLM for one line code completion IIRC.

But the gain people want from LLM is generally "here, add this entire feature" or "here, go thru every dependency's changelog and update code to work with latest version". Those are not small LLM tasks

I have hope we will get there eventually, once all the hype/wealth extraction/boys club giving all their buddies money cycles end, and the specialized tools with real value start to emerge.