There used to be a lot of research into using deep NNs to train decision trees, which are themselves much less of a black box and can actually be reasoned about. I wonder where that all went?

History is littered with great ideas that lost people's interest and focus. A sad realization is that the focus may never return to them either.