Remix.run Logo
▲ grey-area an hour ago

And yet we have to contrast that with reports like this of model capability falling recently:

https://news.ycombinator.com/item?id=50422069

It's really very hard to pin down progress in this domain, companies are definitely targeting benchmarks.

One bit of evidence I would look at is the maths results this last year which have been impressive, that was a huge expensive brute force effort, but it is still good progress.