I think they neutered Fable. When it first came out it was indeed revolutionary. But what we have today, is not what we had before the ban.

Medium vs High? Why? From all the charts I've seen the performance jump is pretty large from med -> high (not as noticeable from high -> xhigh).

> - what "maintainable" is is probably some high dimensional space described by these signals; it'd probably require some human labeling to figure out where this space is

this is a nicely succinct way to put this - a multi-dimensional space where no single metric is really useful

state space of the system is interesting too. I would guess that for any production software with dependencies like databases/third parties that might be too hard to measure, but if you can silo off parts of your system into bounded state machines, it may be a value metric on some module behind a clean interface.

I think the kubernetes control loop model is a great instance of this, a handful of scoped components that own a control loop across a well-defined state machine, that can operate / recover in the face of most network partitions or downtime - the promise of CRDTs but rather more a pragmatic approach to it

yeah this was just a start - the fastest cheapest thing we could try for a brand new model.

I'm hoping to do some more work with sol/fable in the mix as well as exploring more languages and curating the problem set to include more of the benchmark

I also kinda felt like opus4.5 was dumber than 4.1 personally, maybe a little biased since 4.5 was 2.5x faster and 2.5x cheaper seems to indicate its a smaller model

no i'm spinning those up at some point this week. here's the first few prompts I used (claude opus 5 as the research orchestrator), (these were interspersed with lots of tools and assistant messages but it should get you kicked off.

> fetch this article for slopcodebench and help me run an eval on a subset of problems with opus 5 https://arxiv.org/html/2603.24755v1 > Get all the context, fetch any mentioned repos, and then propose a plan to me.

> i have an anthropic API key in .... > Let's do the three challenges with Opus 4.8 and Opus 5 and Fable please. I like your minimal set. Let's try it. What do you need from me?

> Actually I changed my mind. I want to do two of the easy ones you picked and then I want you to pick the one with more checkpoints, maybe one of the harder ones, not the very hardest one but one with a higher number of checkpoints.

> Actually let's do one easy, one medium, and one hard problem please. If we have a hard problem I'd like to see that.

> lets rock - i think lets just do opus 4.8 and sonnet 5 and opus 5 since we our ZDR will block fable

At least for Claude Code, putting "run /simplify at the end" in an "implement the plan" skill helps a little. It still often leaves new code in bizarre places, and/or with bad/alien-sounding names and comments.

I agree this is an option, and the next thing on my radar is to try with a more realistic "factory-shaped" harness where you have feedback from linters and other models after each coding episode that refines the architecture.

For readability specifically, I've found it hard to get the models to do this with prompting. If you've talked to opus/fable for a long time on prose writing you probably felt this too

Clearly the first step of slopbench2 should be to have the agent first write its own harness!

Yeah the main reason I skipped fable was because we have a ZDR with anthropic and I didn’t feel like spinning up another account to circumvent that. Next run will have fable and sol

i laughed at the pelican bit its good

yes the labs will always prioritize the vibeslop dopamine casino as far as I can tell - making the models useful and addictive for unsophisticated users, sometimes at the expense or at the very least at the ignorance of the needs of power users

i hope that is because you hate slop and not because you write it

yes sol is still my daily driver for most coding tasks

I did find opus 5 quite handy for general knowledge work and visual design, without the cost of fable (e.g. the graphics in this post are made by opus 5)

but its not noticeably better than opus 4.8 in those regards, and I would not miss it if forced to go back to 4.8

We've been running automated code reviews on claude with a bunch of skills/subagents with different specialties. Any review feedback is then fed back into claude to fix. Since switching to Opus 5 I've noticed the reviews are overly pedantic, and that leads to feedback loops where each fix generates more feedback, which requires more fixes, i.e. slop. I had, for example, a simple SQL migration script with a single CREATE TABLE. After a few rounds of review, it ballooned into a complicated 200 line script.

I'm not ready to blame Opus 5 for being stupid. Perhaps we have a prompt buried somewhere that's essentially asking it to be pedantic, and it's just obeying the prompt.