I was, for a time, a neuroscience major and have had this same thought. I'm concerned that we're treating these systems as engineered systems when they are closer to evolved or biological systems. They at at least different in that we can study them much more precisely than biological systems because their entire state is visible and we can run them on arbitrary inputs and measure responses.

I agree, what we do is much closer to growing them than to engineering them. We basically engineer the conditions for growth, and then check the results and try again.

My best argument that insights from neuroscience will transfer to neural networks, and vice versa:

For sufficiently complex phenomena (e.g., language), there should only be one reasonably efficient solution to the problem, and small variations on that solution. So there should be some reversible mapping between any two tractable solutions to the problem that is pretty close to lossless, provided both solutions actually solve the problem.

And, yeah, the main advantage of neural networks is that they're white-box. You can also control your experiments in a way you can't in the real world.