Ok, so assuming that this all holds: you could then create a model that is substantially larger than the ones that we have now and at some point the model would become sparse: there would be ample space to accommodate all of the information in the input without weights ever carrying more than one piece of information. The degree to which you get overlap is then a function of the ratio between the size of the model and the size of the input and this is pretty much in line with the conventional way of looking at models (unless I'm grossly mistaken). So by reducing the size of the model relative to the size of the input you force the model to co-locate multiple pieces of information to the point that they start to resemble each other more and more.

But from my earliest experiments with this stuff that's not really what happens. Yes, you get 'feature like' regions in the models. But those feature like regions, even though they encode something particular do not encode concepts the way we would normally use them and I fail to see how this pathway would lead to an explainable model in the sense that would could use something simpler than the model itself to predict its output with a high degree of fidelity.