Indeed you can get quite far with only the built-in modes. One pain point though is the lack of many major modes, for example there’s no built-in markdown-mode, json-mode, julia-mode, or typst-mode, to mention a few.

I recently did try to run Emacs without any packages for a while. To my surprise, AUCTeX was not one of the packages I missed, the built-in latex-mode works well. I did miss CDLaTeX though.