Instead of Rasp 4s, just get a dell optiplex micro, hp elitedesk, lenovo m920q etc - much more stable, quite low power, and can also use it as a true node, and comes with quick sync for video related tasks.

Install proxmox onto each, create talos VM(s) on each for kubernetes nodes, and you are off to races.

I've got worker nodes on m920qs and proxmox VMs too for workers, but like the clean segregation of a HA control node group.

I doubt it matter much either way tbh. K8S is inherently resilient.

I did have the rasps on hand though...not sure I'd go and specifically buy them just to achieve said segregation.