| ▲ | Catloafdev 2 hours ago | |
It's not strictly compute, because this has noticeably improved in open-weight models too, such as Gemma and Qwen. I suspect they noticed this issue and adjusted their training to be better about it over time. | ||
| ▲ | ahmedfromtunis an hour ago | parent [-] | |
I built a toy news-summarizing agent with Gemma 4, and it was so frustrating, actually, because of the cut-off date. The model wasted over half the token budget, each time, on internal debates over the current date. When generating a World Cup summary, for example, it refused to believe qualification rounds were over and refused to even call the web searching tool to collect the data. I injected the current datetime at the very beginning of the system prompt, but Gemma refused to believe it! The m-effer insisted the timestamp was fake and hypothesized it was being evaluated in a synthetic lab test with simulated future dates! No amount of system prompting could convince it to trust the clock. That was the most frustrating and bizarre "bug" I ever faced! | ||