Remix.run Logo
super256 3 hours ago

It's a "problem" of compute, I think. If you query without an account on ChatGPT you will see the model look up less stuff and research less, than when you have a paid account and choose "medium" or "high" in the effort slider.

Which makes sense, because of you have looked into search and crawlers you notice that search is actual quite expensive (which is why e.g. Kagi charges a few bucks for search every month).

Catloafdev 2 hours ago | parent [-]

It's not strictly compute, because this has noticeably improved in open-weight models too, such as Gemma and Qwen. I suspect they noticed this issue and adjusted their training to be better about it over time.

ahmedfromtunis an hour ago | parent [-]

I built a toy news-summarizing agent with Gemma 4, and it was so frustrating, actually, because of the cut-off date.

The model wasted over half the token budget, each time, on internal debates over the current date.

When generating a World Cup summary, for example, it refused to believe qualification rounds were over and refused to even call the web searching tool to collect the data.

I injected the current datetime at the very beginning of the system prompt, but Gemma refused to believe it!

The m-effer insisted the timestamp was fake and hypothesized it was being evaluated in a synthetic lab test with simulated future dates!

No amount of system prompting could convince it to trust the clock.

That was the most frustrating and bizarre "bug" I ever faced!