Remix.run Logo
SparkyMcUnicorn a day ago

100% sure. Tested in the Anthropic workbench[0] to double check and got the same result.

The web interface has a prompt that defines a cutoff date and who's president[1].

[0] https://console.anthropic.com/workbench

[1] https://docs.anthropic.com/en/release-notes/system-prompts#c...

mikeshi42 20 hours ago | parent [-]

Can confirm the workbench does with `claude-sonnet-4-20250514` returns Biden (with a claimed April 2024 cutoff date) while Chat returns Trump (as encoded in the system prompt, with no cutoff date mention). Interesting

jaapz 13 hours ago | parent | next [-]

They encoded that trump is president in the system prompt? That's afwully specific information to put in the system prompt

rafram 8 hours ago | parent [-]

Most of their training data says that Biden is president, because it was created/scraped pre-2025. AI models have no concept of temporal context when training on a source.

People use "who's the president?" as a cutoff check (sort of like paramedics do when triaging a potential head injury patient!), so they put it into the prompt. If people switched to asking who the CEO of Costco is, maybe they'd put that in the prompt too.

jaakl 19 hours ago | parent | prev [-]

Some models do have US 2025 president election results explicitly given in system prompt. To fool all who use it for cutoff check.