I’ve asked Astra to build me an app for a prototype I created quickly using Sonnet.

It’s been 2 days and it made no real progress on the actual app. It created docs, scripts, workflows, and it’s doing a bunch of reviewing on every PR.

I told it that I just need an MVP.

I’m pretty sure an average senior engineer would have finished that task much quicker, and guaranteed with more readable, higher-quality code. Meanwhile, I think I’ve easily crossed 100k tokens so far on nothing.

Funny world we’re living in that this is “SOTA” and “AGI”.

I’m genuinely curious what these OAI and A/ engineers are working on that they praise these models so much. I did not see any improvement since Opus 4.5.

Also, I’m really unimpressed by any “one shot” demo that’s out there in the wild. It means nothing for serious software engineering.

I don't know how to make apps or evaluate code, but with astra I having been making an iOS app on my own for the first time and it's going great. my app is not terribly complex but requires using bluetooth and other intricacies which I thought would be tough. but it's going really well. I'm not asking it to one-shot it though, I'm going feature by feature, testing and building up.

yes, at first it would run simulator tests on all font sizes but it stopped after I asked it not to do that until UI review

maybe sol would have done the same thing, idk. but I find the whole process to be really nice with astra. I use it on high unless it says something is impossible then i go max and ask it to find alternatives (happened once)

>Also, I’m really unimpressed by any “one shot” demo that’s out there in the wild. It means nothing for serious software engineering.

If a person, or team of people, can build a demo quickly then it's good odds that they can build the real version (though, famously, not a guarantee). However, it turns out that a machine that can spit out 100 demos of whatever can't actually build the real thing.

Similarly, a chess engine rated to 1000 Elo doesn't play like a 1000 rated human being. The mistakes that each make to reach the equivalent level are different in size, frequency and kind. The thing that makes a human reach a good demo is very close to the skillset to reach the finished article. This isn't so for LLMs but we have yet to update our priors.

For me it also produces totally overengineered tests that are tightly coupled to the implementation. For example testing existence of css classes (in a template based go prooject ...) instead of behaviour.