how is this any different to naively running a webserver or database server or, in fact, anything as yourself?

People are forgetting the basic rules of operating systems. Do not run sensitive programs as a user account - especially not as a root account!

a little bit - if I install apache2 from a debian package I have at least some sense of warm and fuzzy feeling that it would be a very rare and exceptional event for like, a rogue apache2 binary to go rampaging through my system or uploading everything it finds to god knows where.

Installing "all of this was built in the last 3 months and is cutting edge" LLM stuff is very different than like, installing mariadb and then doing 'sudo mysql_secure_installation', since you mentioned database servers.

It's like comparing FedEx flying cargo airplanes in/out of Memphis in the most boring route way hundreds of times a day vs. some guy in a red bull sponsorship painted pylon racing aerobatic craft performing at an airshow in difference of risk.