What do they test? In general, I don't understand the whole "AI safety" stuff. What are the scenarios we need to be protected from and how is this going to be accomplished?

I ask myself this a lot. It took decades of sustained malfeasance, real-world case studies, and lobbying for the EPA and OSHA to be established, largely against the wishes of the businesses they regulate. Today, it is primarily the AI businesses themselves asking for regulation on the basis of as-of-yet largely hypothetical risks, rather than ordinary citizens seeking protection from tangible negligence---and the government is eager to accommodate. I think few, if any, authentic anti-AI activists have a taxpayer-funded "safety testing" regime as their end goal.

Testing whether they will hack critical infrastructure, cellphone towers to sewer systems.