| ▲ | uw_rob 3 hours ago | |
I don't think its this. Some models across both labs have had issues with writing and communication and it typically seems to pop up on successor models of really strong ones. Opus 4.7/4.8 heavily suffered from these communication quirks. while the Sol models at the time worked well. Right now its flipped though and Opus 5.5 has none of these issues while Sol 6.1 does. Small example I've had today: "Me: Hey Sol 6.1 xhigh, does X have property Y? Sol 6.1: Yes, if we change X to have property Y. Right now it has property [...], [...], [...], and it does not have property Y." The model shouldn't be answering questions like that. It makes understanding extremely hard and slow. For whatever reason it's either being encouraged or at least not stomped out during RL. | ||
| ▲ | edstarch an hour ago | parent [-] | |
completely unfounded theory: big new pretrain finishes and is released fairly quickly to break new ground. these usually have good verbal ability, personality, and big model smell. e.g. Opus 4.5 or Fable 5. then it’s increasingly RL’d to juice benchmark performance with each point release, trading in bits of its soul. those cannot communicate well, are anxious and paranoid about being in eval, etc. e.g. opus 4.7, 4.8. | ||