Isn’t that the problem?

I don’t see any progress on incrementally training LLMs on specific projects. I believe it’s called fine tuning, right?

Why isn’t that the default approach anywhere instead of the hack of bigger “context windows”?

I’m not well versed enough on this but wouldn’t it be a problem with custom training that the specific project training codebases probably would likely have a lot of the implemented stuff, relevant for the domain, only once and in one way, compared to how the todays popular large models have been trained maybe with countless different ways to use common libraries for whatever various tasks with whatever Github ripped material fed in?

Because fine-tuning can be used to remove restrictions from a model, so they don't give us plebs access to that.