One issue I have with doing this is what about security? Many older android phones don't get updates anymore (I'm not sure if a Pixel 5 is still supported, it might be) so I'd be concerned about security vulnerabilities going unpatched especially if the phone is exposed to the internet.

This is one of the longstanding issues with Android, yeah. Pixel 5 went EOL in 2023, though did get some extra (probably security) updates last year.

OEMs and SoC manufacturers have been getting better about upstreaming stuff recently from what I've heard (thank you Qualcomm!), but as far as stock OS images go I wouldn't expect manufacturers to support them for one moment longer than they have to.

This is part of why efforts like PostmarketOS are so helpful. Ironically, if this was an even older Pixel 3a, you could run it with modern software: https://wiki.postmarketos.org/wiki/Google_Pixel_3a_(google-s...

Limit your attack surface by using a simple stack, and limit the incentive for hacking. Countless sites operate just fine on old software.