Or could it be that the fact that LaTeX is a layer on top of TeX is a cause of additional complexity and that you still have to deal with TeX, so the abstraction is neither complete nor opaque? I assume in Typst they don't (yet) have this issue. Another reason could be missing static typing and thus less formal means to find bugs? You "missing abstractions" finding would point to that.

Yeah, absolutely, the sheer number of moving parts is related to this. There's TeX-the-language, there's LaTeX-the-format (a big layer of macros and conventions), the different LaTeX standards (LaTeX2ε, LaTeX3, whatever, I don't know the ins and outs). Then there's the implementation: all the different engines (pdftex, XeTeX, LuaTex), then there are the distributions, then there are build systems and wrapper scripts to handle all the mysterious compilation-cycle incantations that depend on all the other stuff (latexmk, etc.).

All these layers and alternatives have proliferated partly because of age and the necessity of new solutions to new problems. I also think they're partly coping with fundamental limitations, design decisions that seemed like (or very much were!) a good idea in 1978, that sort of thing.