The FDA regulates physical goods. They require literal factories and shipping to get these products anywhere. They can put stops on these products pretty easily. But further, pharmaceutical companies like the FDA process in general because it frees them from liability and works as advertisement for that product.

AI models are a finished product when the training is done. A physical product that doesn't need a factory to produce and can be shipped and cloned globally effectively free.

The better comparison is media. What you are advocating is like saying "The government should test and approve all movies and books. We shouldn't have those industries police themselves". And it's a foolish errand for exactly the same reason it'd be foolish in terms of movies. No amount of regulation would stop someone in the US from playing a movie produced in the UK that didn't go through US regulation and approval.