Remix.run Logo
theLiminator 6 hours ago

I wish someone ran some sort of representative benchmark suite every X days to see if this occurs.

micycle1 6 hours ago | parent | next [-]

https://marginlab.ai/trackers/codex/

CamperBob2 6 hours ago | parent [-]

Unfortunately that seems to be monitoring Sol, not Astra, unless I'm missing something.

embedding-shape 6 hours ago | parent | prev [-]

I'm not sure how you could run such a benchmark without leaving it possible for the labs to easily detect and fudge the results.