How will the firewall for LLM look like? Because the problem is real, there will be a solution. Manually approve domains it can do HTTP requests to, like old school Windows firewalls?

Yes, curated whitelist of domains sounds good to me.

Of course, everything by Google they will still allow.

My favourite firewall bypass to this day is Google translate, which will access arbitrary URL for you (more or less).

I expect lots of fun with these.

Correct. Any ci/cd should work this way to avoid contacting things it shouldn't.