I have always wondered if the neural engine could be used for training - pretty excited for part 3 of this to see if the juice is actually worth the squeeze

In principle most if not all inference hardware should be usable for training.

Efficiency is the question.