They posted a long document explaining it all https://x.com/Lon/status/2101034933284417614

They're not measuring a fixed set of questions. This was post-hoc analysis on whatever prompts they were running each day.

Anyone can understand why it would go up or down depending on the work they're doing that day. This analysis is silly.

https://xxcancel.com/Lon/status/2101034933284417614

Ah I didn't catch that. Yeah without controlling the inputs this doesn't prove much.