This is very close to the philosophy behind something I've been building:

https://github.com/simonschnake/Mark2Tex

I built Mark2TeX while writing my thesis in Obsidian. I wanted to write the prose in Markdown, but keep a normal LuaLaTeX document for math, citations, cross-references, etc.

So Mark2TeX lets me use Markdown-like files and drop into real LaTeX whenever I need it.

I deliberately didn't try to build a complete Markdown/Pandoc replacement. It's basically an attempt to under-engineer the Markdown/LaTeX boundary.