Does this train on LLM output, or is this more like iterative self prompt improvement?

Their statement is that they regard lines of code shipped as indicative of self-improvement. So, while a well written coding agent might be a few thousand LOC, Athropic's is bloated like a decomposing whale and over 500K LOC ! What more proof do you need?

Have you tried reading the article? It answers your question.

Don't ask people to explain the article to you if you're too lazy to open it yourself.