> if the server operator was malicious, they could just push different client-side JavaScript

Same as with OS updates, browser updates, dependencies used by the OS, dependencies used by the browser. Also you can run malicious software such as keyloggers and you're compromised.

That argument doesn't mean E2E (even web based) is snake oil. Browsers just give you more points of failure.

The difference is: in web based cryptography, you get the cipher text and the code to decrypt it from the same source. Hijacking OS updates is arguably much harder than hijacking one particular web server, and there is pretty much no effective defense against malicious OS updates.