Well, LaTeX3 was sort of what you describe, but has been dropped:

https://www.latex-project.org/latex3/

If nothing else, the support for Unicode via UTF-8 has been a big win, and the new programming model and Lua allow some pretty cool stuff, if I do say so myself --- I'm still impressed by Alan Xiang's solution here:

https://tex.stackexchange.com/questions/722886/how-to-write-...

I agree that the preamble stuff is kind of a trainwreck --- I've always held that someone needs to put together a templated version where quite a large number of options are available, but commented out, so that one could work up a version of a document by just uncommenting the appropriate lines --- it really feels like something which LyX should have done.

The problem of course is the old greybeards are accustomed to tomes such as:

https://ctan.math.washington.edu/tex-archive/macros/latex/co...