We don't know the specifics of what type of ML related training Astra underwent, and it wouldn't be crazy to find out that small model experiments are a big part of model training.

What's surprising though is that I'd expect this type of training to be exclusive to internal model snapshots or at least heavily guardrailed against in public models.

▲

david-gpu an hour ago | parent [-]

I was wondering exactly that. It is not surprising that commercial frontier labs would have been training models with tasks specifically targeted to lead to recursive self-improvement.

Now, from that broad goal to being able to surgically alter model weights directly... That seems like a big gap to me, but it is possible that the model is just so darn beyond human skills that it has bridged that gap in a way a sane human would have never.

We will undoubtedly see other academic labs explore this idea further. Right now my mind is blown, but I don't want to take their results at face value, either.

	▲	PhunkyPhil 21 minutes ago \| parent \| next [-]
		I'm in no way familiar with LLM/ML architecture, but an N trillion parameter model is an enormous amount of training data... itself? When you predict a token you have the neurons/weights that were activated. Couldn't you run that backwards and put it in it's own training data?
	▲	dumberquestions 44 minutes ago \| parent \| prev [-]
		If model weights have a logical and coherent structure, I don't think it would be surprising for models to have some understanding of them, the same they can understand language and code and images, given enough data. What would be surprising is if this happened without any specific training on model experiments and interpretability.