| ▲ | Someone1234 an hour ago | ||||||||||||||||||||||
My big hope for this release is - Have they got their massive over-engineering under control? Say what you will about Opus 5.5, but it has a good "gut instinct" for how large/complex to design the solution. The previous Astra just absolutely loved pointless abstractions, excessive numbers of files, and just over-complicating very simple things within an inch of their life. And, yes, you can prompt your way out of anything. But needing a default prompt to tell OAIs models to work better isn't preferable, particularly when that sometimes gets lost in sub-agents and the over-engineering returns into parts of the work. | |||||||||||||||||||||||
| ▲ | pettijohn 20 minutes ago | parent | next [-] | ||||||||||||||||||||||
Oh my god yes. 6.x has been terrible at overcomplicating things. I have to prompt it to find the simplest solution that meets the requirements and then ask it to simplify and tell it to STICK TO THE PLAN. In addition it feels like they overfit for persistence. I created a new work tree and forgot to copy .env files. I caught codex hammering away and docker and psql commands and stopped it and asked it to explain what it was doing. So now my agents.md says "if documented tooling fails STOP and seek human assistance." | |||||||||||||||||||||||
| ▲ | scandinavian an hour ago | parent | prev | next [-] | ||||||||||||||||||||||
I've been using both sol and astra to write some python and rust, and they generate the sloppiest code I've seen in a long time. I would be embarrassed to commit anything they write. The rust code especially was terrible. Had it implement a reader/writer for a binary format and it was just littered with magic numbers, terribly named variables, very compact code visually etc. It produces working code, so I guess the GPT models are mostly for people who let AI take the wheel 100%. | |||||||||||||||||||||||
| |||||||||||||||||||||||
| ▲ | seizethecheese 13 minutes ago | parent | prev | next [-] | ||||||||||||||||||||||
I was skeptical Sol 6.1 actually over-engineers compared to Opus 5.5, as I just switched from one to the other and haven't noticed a difference, but it looks like you're probably right! We're building a battle system for OpenCode (https://agentduel.com/leaderboard) and have data on how large the diff was compared to opponents for both: Opus 5.5: 0.42x (based on 299 battles) Sol 6.1: 1.42x (based on 265 battles) Data on this is still pretty small. P=0.14. It's also affected by large differences, this is average not median. | |||||||||||||||||||||||
| ▲ | afro88 22 minutes ago | parent | prev | next [-] | ||||||||||||||||||||||
I have been bitten by this 3 times and each time Opus 5.5 brought it back to something sane. The last time I left Opus in a /goal to get behaviour matching the docs, and it decided on it's own that a rewrite was the best path (and it was, and it worked) | |||||||||||||||||||||||
| ▲ | pants2 an hour ago | parent | prev | next [-] | ||||||||||||||||||||||
Lol I made the mistake of having Astra design and build a complex system without any input (experimentally). Seemed functional at first and shipped it internally and it had dozens of complex bugs, so I started digging into the architecture and it was a monstrosity! Definitely 10X more complex than it needed to be. | |||||||||||||||||||||||
| |||||||||||||||||||||||
| ▲ | holzhaker1 an hour ago | parent | prev | next [-] | ||||||||||||||||||||||
yeah i feel like with astra they went all in on 3D modelling which encourages "just one tweak, just one more tweak" which makes it detailed enough, but in coding that same behaviour just sucks. | |||||||||||||||||||||||
| |||||||||||||||||||||||
| ▲ | uw_rob an hour ago | parent | prev | next [-] | ||||||||||||||||||||||
Honestly I don't even know if you can prompt your way out of these types of behaviors - stylistic tics and judgement seem to be extremely difficult for the models to overcome. It seems we are tied to what the labs do unless we want to dive into and get dirty with models yourself. In retrospect with time it becomes obvious that the previous generation of models couldn't be prompt engineered to do what the current generation of models do. There's just no way you will prompt engineer Opus 4.8 to write as well as Opus 5.5. I'm guessing with time we'll see that there was no way you could prompt engineer Sol 5.1 to be as token efficient as Sol 5.2 | |||||||||||||||||||||||
| |||||||||||||||||||||||
| ▲ | incrudible 37 minutes ago | parent | prev [-] | ||||||||||||||||||||||
I have limited experience with OpenAI models, but I have to say that Opus 5.5 has not improved on the terrible writing style, and I am struggling to prompt my way out of it. It always needs an extra pass dedicated just to rewrite all the prose. | |||||||||||||||||||||||
| |||||||||||||||||||||||