Can anyone explain to me what this massive volume of code could actually be doing? They obviously have variations of model architectures, training infrastructure, serving infrastructure, and the source for their apps.

Why would the volume that any of this be increasing exponentially? And what are all the engineers doing when (besides the model quality) their products don't seem to be changing very much?

What they do not have is a version of chatgpt.com that is not a slow, awkward, glitchy mess.

As a user of Astra, it writes code that appears to work. It does not necessarily write sensible code that is architected in a way that makes sense based on what it does. I just watched it do a major rework of a couple thousand lines of code by keeping the original organization and kludging up an adapter to make old code have new semantics.

I would expect that this approach, coupled with lots of fun asking for changes and none of the grunt work of actually thinking about the result, could result in exponentially increasing amounts of mess.

I haven’t used a recent Anthropic model - maybe it’s better at this.

For one, they have the internal models that us normies don't have. Also, machine learning as a field is probably the most automate-able field for AI. It's 100% on computers without "real world" collisions at all, AI can do the whole stack today even with publicly available models. And since it's so empirical, you can make a lot of progress by just running more experiments with more compute.

Engineers are working on multiple sessions, and LLM is doing most of the commits/pushes. This is becoming norm at big techs.

If you are wondering what these changes are, you can take a peek at https://github.com/openclaw/openclaw/commits/main/

This isn't to say that such pace of iteration is good or bad. That's a separate discussion.

RL environments. RL environments as far as the eye can see

> Why would the volume that any of this be increasing exponentially?

Because the limiting factor for software has always been the supply, not the demand. When the average human engineer would ship 100 lines of production code / week, you could only afford to do the most critical things. But if code gets cheaper to produce, the projects that were marginal become worth doing, and the projects that were obviously insane become marginal. And that's exactly what is happening as models get cheaper and better, leading to the frontier of worthwhile projects expanding rapidly.

> And what are all the engineers doing when (besides the model quality) their products don't seem to be changing very much?

That's got some nice But other than that, Mrs. Lincoln, how was the play? energy to it!

At some point I imagine they will be writing their own software for the entire training and inference stack. Kernels, libraries, why not? With compute so constrained it must be beneficial to optimize everything as much as possible

[facetious]

many yaks to shave. many bikesheds to paint.

if you're paid by the line, why not go to town...

becuase devs stop coding by hand and start prompting AI even for smallest of tasks, for example "hey codex, check grafana alerts and tell me x, y, z" instead of navigating it by hand and figuring out in grafana UI

Everytime you use sites it should record the code, for example. machine speed code needs machine speed vcs.

> Why would the volume that any of this be increasing exponentially?

Surely it's obvious?

Slop cannons firing at a rate so high no human can hope to restrain them.

Such a colossal volume of slop that even the most well engineered, efficient technology on the planet, written by one of the greatest coders of all time, is no longer sufficient to contain it.

It's the billions of scripts we used not to commit. And billions more.

RL environments

It's AI slop. AI coding agents write much more voluminous code, in my experience. The base code is more expansive, and there are more tests, and there are many, many more commits as you keep fixing and fixing and fixing over and over and over.

Anyone who has fallen into the ADR pit with agents could tell you this. Its just slop, slop to fix slop, docs about slop, slop to fix the docs, etc. 600 commits and 200k LOC/Docs for software that used to be 15 commits and 10k LOC +README and no real value to be found.

Or it's yet another iteration to get another test with overbuilt providence catalogling, boated code, overbuilt nonsense. Suddenly you're 2.5 Billion tokens deep and you just wonder "What the actual Feck did they do, how did they burn that many tokens? It's just a web app with a plugin system, my god."

Maybe it could be faking code and ai usage?

I've seen it on other workplaces, but from what I see from open ai and anthropic, the code shouldn't be as extended.

But maybe I'm crazy lol