It is regulatory costs these days. Most mines currently operating in the US were grandfathered into current regulatory regimes, they'd likely never be developed today.

This creates a perverse incentive where it is often cheaper to reprocess low-grade ore from an existing mine than to jump through the regulatory hoops and decades of lawsuits to develop a new mine with high-grade ore. Refining a low-grade ore in the US often is not cost competitive on the global market, so there isn't much incentive to do so even though you've already mined the material.

The US needs to make it fast and efficient to develop new high-grade ore deposits. America has extraordinary mineral wealth as a matter of geology but we barely even explore in the US anymore because even if you find it you can't develop it. This has been the case since circa the 1980s or 1990s.

Price controls on gold up until the late 1970s didn't help either, since it discouraged gold exploration. Many high-value mineral deposits in the US have been discovered as a side-effect of gold exploration. The price controls disappeared but were almost immediately replaced with regulatory regimes that made it unprofitable to develop new mines.

Many rare earth deposits in the US were discovered as a side-effect of uranium prospecting. The US government stopped subsidizing uranium mining ~1970, which was the main reason it was being done at all, and so people stopped discovering associated minerals around the same time.

Should the price of these domestically-produced minerals include the environmental impacts on the watersheds, superfund'd land, and Americans living next to the mining operation?

Or is that just the inefficiency introduced by them pesky regulations you're trying to make more "fast and efficient"?

If you don't price all that in, some might say you're asking some locals and counties to give a pretty major subsidy to some private mine owner.