accepting AGPL means you're vowing to publish derivative work. if you aren't legally allowed to publish that work, you violated its terms. the Court may not order you to publish to cure the violation, if you lack the rights to do so, but they can still order you to pay damages.

if I sign a contract saying I'll produce a million Iron Man action figures, but I don't have the IP for Iron Man, I can't just shrug my shoulders and say "well, you can't make me." the Court would make me pay damages.