It's beyond insane that Astra achieves above 50%. Are models allowed to "play against" the specific bot they're up against? What explains the fact that it drops to single-digits against strong bots?

Yes during the 1 hour window they are allowed to play practice matches against opponents they are graded against - though they do not have access to their opponents source code.

They drop to single digits against the A tier and S tier bots because those bots are just generally harder opponents to beat.