>But to be clear, nobody understands why the evolved weights of a NN make the decisions that they do.

We might not understand particular "emergent" capabilities, but the low level mechanism is not just understood, but a deterministic algorithm with a handful of basic componets, that are well understood themselves.

▲

conscion 2 hours ago | parent [-]

> We might not understand particular "emergent" capabilities

The emergent capabilities are the only capabilities we care about

	▲	coldtea 2 hours ago \| parent [-]
		For allignment maybe. For the core functionality and the optimizations we don't really need to know how the emergent capabilities decide on particular answers. Which is why we could build LLMs before those features ...emerged for us to see, and why we can just code LLMs with the numerical NN algorithms we use, and do now have to go in and change individual weights.