I don't think age of the puzzle even matters, all models have search capacities these days

zakisaad 35 minutes ago | parent | next [-]

Model weights (what is being tested here) don't inherently "access the web" when inference is running. If the model has access to a web search tool, that's a different story.

▲

criemen 3 hours ago | parent | prev | next [-]

> all models have search capacities these days

one would hope that they disable websearch and internet access (maybe all tools?) when doing generalization testing?

▲

dexwiz 3 hours ago | parent | prev [-]

Is search part of the model or the harness?

	▲	Cycl0ps 2 hours ago \| parent [-]
		Maybe that's a rhetorical question but just in case - the search would always be part of the harness. A model is only handling next-token prediction for a given input. That token may be something like [[web search]] to invoke a tool call but the actual call would be handled by the harness.