As I understand it, some of the silver was siphoned off in trade with China (via the trans-Pacific route to Manila). China needed continuing silver imports because silver was not used there in the form of standardized coinage, but rather in ingots that were weighed and subdivided, with inevitable continuing loss.

It was very surprising to see zero mention of the Manila galleon[0] trade route in the linked article, even if technically the question was about gold rather than silver. The simple answer to the question of what happened to the money was that Spain spent it! The impacts on SE Asia were profound and are still being felt today.

[0] https://en.wikipedia.org/wiki/Manila_galleon