| ▲ | HarHarVeryFunny 2 days ago | ||||||||||||||||
> I don't even see any evidence that 'neuromorphic' architecture is brain inspired in a non-trivial sense I'm just giving the definition of the word neuromorphic. Individual systems claiming to be neuromorphic can be judged on their own merits, but that does not change the definition of the word. >> brains operate in asynchronous dataflow type fashion. >That's a questionable statement. To the best of my knowledge, there is no consensus as of 2025 of how to model even a single neuron You're confusing two things - do we know absolutely everything there is to know about every type of neuron and synapse, in order to build a 100% accurate model of them? No. Do we know that neurons operate asynchronously to each other, only activating when their inputs change? Yes. > When I asked Steve Furber what 'neuromorphic meant, he said: "There are many things today described as neuromphric. Mead would not call SpiNNaker as neuromorphic!" Of course not - it was just an ARM-based message passing system, built to be able to efficiently simulate spiking neural networks (modelled as message passing), but with no particular specialization for that task. > I asked him if SpiNNaker can accelerate back-propagation and he said: "no, because the brain does not do back-propagation". That's a silly question, and you got the silly answer you deserved. SpiNNaker is basically just an ARM cluster with fast message passing. It can be used to accelerate anything relative to running on slower hardware. If you wanted to train an ANN on it, using backprop, you certainly could, but why would you WANT to mix spiking neural nets and backprop ?! > If that is the goal, they are also not very successful. On CMOS silicon changes from 0 to 1 or 0 to 1 is what consumes most of the power, this would make the constant spiking expensive, no? Sure, but biological neurons don't spike constantly. They spike only when an input changes that causes them to activate - maybe on average only 10's of times per second. This is the whole point of an async (no clock)/dataflow chip design - for chip elements to only consume power when their inputs change - not to route a GHz clock to them that continuously draws power (by flipping between 0 and 1 billions of times a second) even if the element's inputs are only changing 10's of times a second (or whatever). | |||||||||||||||||
| ▲ | GregarianChild 2 days ago | parent [-] | ||||||||||||||||
> definition of the word neuromorphic The definition of the term is so vague as to be useless for scientific progress. What exactly is excluded by "brain inspired"? If I paint my computer grey because the brain is grey, it's brain inspired? You know that regular expressions were invented to model neurons [1]? If nothing is excluded the term is useless. But in reality, the term is used explicitly to deceive as already done in the 19th century see [2]. Heraclitus says somewhere something to the effect that "good thinking is dry". Established terminology like • Asynchronous CPU • Delay Insensitive Circuit • Quasi-Delay-Insensitive Circuit • Clockless CPU • Asynchronous circuit • Self-timed circuit is dry and useful. 'Neuromorphic' suggests an understanding of how the brain works that is just not there in 2025. > Do we know that neurons operate asynchronously to each other, only activating when their inputs change? Yes. I strongly disagree. We do not know how exactly the brain codes information. If you have timed coding, it's not asynchronous. And that doesn't even address the chemical signals that abound in the brain. The Hodgkin-Huxley equations give a good approximation to how the neuron produces a "spike" - the signal that travels down the axon. But it's only an approximation. Every type of ion channel in the cell adds another 10 or so unkown parameters to the equations, which can only be approximately found experimentally. So we have a fairly good picture of how a single neuron behaves in most circumstances, but not complete. Ion channels are are crucial to how the brain works. They are the nonlinearity that makes computation possible, like the transistor is to the computer. There are several hundred types of ion channel that are known about so far and probably thousands more that are not yet discovered. > That's a silly question Not in the context I asked: I asked him after he had suggested that 'neuromorphic' would be great to reduce the energy consumption of LLMs. I wanted to know why/how, given that LLMs are trained today with BP. I give him the benefit of the doubt here, since we were in a rush and could not go deeper into the subject. > mix spiking neural nets and backprop ?! It's an active area of research to do just this. Google SpikeProp, e.g. [3]. Lot's of papers in this space at the moment. Why? I don't know as I don't follow this space. Potential reasons: (i) BP is natural, (ii) BP works super well for DeepLearning, (iii) the 'neuromorphic' community has failed so spectacular and want to try something that works, (iv) to get funding, (v) we really do not know how the brain works, and in that case, why not try out crazy things? [1] S. Kleene, Representation of events in nerve nets and finite automata. https://www.dlsi.ua.es/~mlf/nnafmc/papers/kleene56representa.... [2] K. S. Kendler, A history of metaphorical brain talk in psychiatry. https://www.nature.com/articles/s41380-025-03053-6 | |||||||||||||||||
| |||||||||||||||||