Have you tried using Browserless/similar to scrape around tricky hosts?

No, I haven't tried Browserless. So far, it has all been from a single residential IP which is probably the bigger issue with Liverpool than the WAF challenge itself. Once I have a valid session cookie I can solve the JS challenge fine, the rate limit is per-IP. Rotating residential proxies (or Browserless behind one) might be the answer... I'm just reluctant at this stage to bite the bullet on the cost for a single (albeit huge) council. Have you used it for similar stuff?