What internal tools are you having the LLM access? I have a server, but I just host whatever MCP servers and what not on device, not on the server.

I also pretty much exclusively work at my desktop - if you use multiple systems I can see where this matters.

just certain dashboards that display my personal financial information, housing stuff, and some goals/trackers I have around projects I run.

I do work mainly on my desktop, but let's say I'm traveling right, I'd have to turn it off and was missing access to some pretty important information, so I ended up getting a VM and hosting some of those things on there instead. I still use both, but things I want available, I'll host it on my "homelab" and secure both machines via tailscale. it's pretty easy and it ensures that only my phone, my main mac, and my homelab have access to each other and nothing else.