The experience you are describing has overwhelmingly not been my own, nor anyone in my space I know.

I can only speak most recently for EKS, but the cost is spent almost entirely on compute. I’m a one man shop managing 10,000 containers. I basically only spend on the compute itself, which is not all that much, and certainly far, far less than hiring a sys admin. Self hosted anything would be a huge PITA for me and likely end up costing more.

Yes, you can avoid kubernetes and being a “slave” to cloud providers, but I personally believe you’re making infrastructure tradeoffs in a bad way, and likely spending as much in the long run anyway.

maybe my disconnect here is that I mostly deal with full production scale applications, not hobby projects I am hosting on my own network (nothing wrong with that, and I would agree k8s is overkill for something like that).

Eventually though, at scale, I strongly believe you will need or want a control plane of some type for your container fleets, and that typically ends up looking or acting like k8s.