| ▲ | kasey_junk 17 hours ago | |
> We trust them that what is behind the string "opus-5-5" is consistent and what we signed up for. Do you run evals as a matter of course? Everyone I know that does shows high levels of variance across them under the best case. That’s not to say that the labs don’t downgrade models at times, just that without any measurement it’s a non-interesting assertion. | ||