But even if you SSR them you still need to send at least 100KB (and probably far more) of JS down to the browser to hydrate state and handle any interactivity.

You need JS to handle interactivity no matter what. And 100KB is nothing, a single image is larger.