Just serving a whole SVG file directly never even crossed my mind as an option, very cool.

A while back I used a giant interactive SVG as the UI for a site I was prototyping, albeit wrapped in a normal HTML page. It was easy to set up and worked reasonably well, but I found that in Firefox performance started to degrade beyond a few thousand elements and so converted everything (except some accessibility features) to use a canvas instead. (The core of the old version is still deployed here if you want to see how far you can push it: https://freeclimbs.org/wall/demo/edit-set )

See svija.com/en — all SVG websites ;-)