Eternal question about pre-training, i.e. on what medium this pre-training or pre-pre-training has been performed ... if we assume that the platform is CMOS-based piece of hardware, then the most primitive trainable element would be a two-state learning automaton, that can be assembled out of a couple of dozen transistors. This is effectively a kind of a single cell bacteria. You then need to organize these automata into collectives which can start to exchange tokens and reinforcements between themselves and the training environment, and these collectives do so at a higher level and so on, and that's what can be seen as NCAs. (*) A good source of idea on this approach can be found in the book of M.L. Tsetlin: Automata Theory and Modelling of Biological Systems, 1973 - https://shop.elsevier.com/books/automation-theory-and-modeli...