Abandon hope.

I just treat ~ as a system-owned configuration area, and put my actual files (documents, photos, etc.) in a completely different hierarchy under /.

"/home/${USER}" for whatever junk programs are going to stick there, "/home/${USER}/home" for my "real" home directory.

I have been doing this for decades. My files are in a sub-directory of $HOME. It also makes it very obvious when a piece of software does not treat your $HOME with respect.