When a format has so much flexibility, the world will glom onto the first thing that it solves and put it into the wild to solve that problem. The rest of the capabilities fall by the way side, yet not removed from the format. They're just ignored. MP4 can do so much more than the typical deliverable of a video stream and an audio stream. The spec allows for multiple video streams, multiple audio streams, subtitles, Flash like interactivity to allow self contained DVD style programming of menus to allow for chapter navigation, audio/sub selection, multiangle, etc.

MPEG4 supports interactive 3D scenes (read: games!) via bit-encoded VRML which can include JavaScript. I don't know any software which support this features (even without interactivity), but it is here.

I'd wrote reader for this bit encoding many years ago when company I'd been employed at made custom VRML engine for Samsung featurephones (project was dead on arrival as at this moment smartphones / Android become available and all this lineage of featurephones was killed).

> MP4 can do so much more

None of that has fallen by the wayside though? I have seen examples of all of those in the wild except for interactive menus and multiangle. Multiple video streams is incredibly rare to encounter but I have run into it a few times.

The argument in favor of flexibility (and jxl) is that if you optimize things for the "average" web user (as the essay seems to be suggesting) then fairly mundane usecases require you to start juggling formats, support becomes spotty, and things start breaking. It's nice to have generous limits within which you can be confident that things will "just work" for the end user. Even just on my own system I'd much rather use a single format rather than dealing with app x not supporting format y.

tl;dr jxl is the mp4 of image formats and that's exactly why I like it.

The only things I agree with the essay about are progressive decoding and decoding speed. Particularly the latter badly needs to be improved.