Remix.run Logo
RDTvlokip 5 hours ago

I have a question, as it happens: Do you think the benchmarks and models were trained on benchmark datasets to skew the results, even though in real-world applications we realize they're not that great?

sinuhe69 3 hours ago | parent [-]

Recent incident with the Rio 3.5 model clearly shows that many coding models are specifically trained/fine tuned for the benchmarks.