The most obvious downside for me was remote terminal windows (e.g. using ttyd) being unusable because canvas rendering was "broken".

That one at least is easy to fix, since firefox shows a little icon in url bar if it is blocking canvas data, and the remote site asked for it. You just click on it to whitelist that site.

I've had more issues personally with resist fingerprinting making major sites completely unusable (drupal.com, walmart.com..)