On balance I'm not that surprised that it can guess games by looking at the weights of a tiny neural net. A human can do the same by looking at the weights of an MLP trained on mnist, for example. With that simple structure, it's easy to look at the input layer and see that features correspond to the spatial location of objects of interest.

I'm much more surprised that it can do this on larger models, although the details seem to suggest that it's given a lot more freedom to run forward passes and do limited experiments, not just eyeball the weights.

▲

jgbuddy an hour ago | parent | next [-]

> although the details seem to suggest that it's given a lot more freedom to run forward passes and do limited experiments

No tools or forward passes are allowed

	▲	Imnimo 33 minutes ago \| parent [-]
		Under experiment 8 (the one where it fixes performance on GSM8K) it says: >Then, we allow it to select prompts from the training sets of GSM8K and MATH to perform a budget of forward passes (average batch of 64 sequences, max 512) on, varying the budget across 10, 25 and 50. Again, no training or gradient information is allowed.

▲

kachnuv_ocasek an hour ago | parent | prev [-]

> A human can do the same by looking at the weights of an MLP trained on mnist

Excuse me?

	▲	astahlx an hour ago \| parent \| next [-]
		Yes. Just look at the reasoning logs that are part of the work. The reasoning is clearly about the input layer. At least for the game examples.
	▲	Imnimo an hour ago \| parent \| prev [-]
		See for example: https://scikit-learn.org/stable/auto_examples/neural_network...