Remix.run Logo
63stack 2 days ago

Word for word my experience with operating k8s.

stego-tech 2 days ago | parent [-]

To be clear: I like K8s! It’s fun to be able to write some YAML, apply it, and be done!

But the sheer work of getting to that point, safely and securely? It ruins the experience for me, personally.

spwa4 2 days ago | parent [-]

What would you like setup to look like?

Would there be real interest in a kubernetes distro that takes IPs and a (set of) domain names, and boots up on N nodes, installing letsencrypt, so that you can do a deployment and have ingress actually working?

stego-tech 2 days ago | parent [-]

For homelabs and SMBs, that is definitely an opportunity. Other areas I’d like to see more growth in:

* “No-Code Kubernetes”, that lets staff design basic (or even not-so-basic) deployments using a web-based GUI. We’re seeing more of this from a “understand how things work while they run” perspective, but I haven’t personally come across any “here’s your building blocks and explainers of the environment variables, go build” solutions.

* An “ESXi-ified” K8s. Talos comes so, so close to this, but I’d love something that was as easy to deploy into production as ESXi was on SD cards. Deploy as an appliance on bare metal or as a VM, and voila, Control Plane with an IP schema, network layer, AD CA/ACME support (including Let’s Encrypt), and a basic load balancer/ingress. Changing the setup is as simple as adding a basic text file with the control plane IP and join string (worker nodes), with a simple flag to add it as another control plane node for HA.

* Renewed focus on etcd management ease. A lot of the cert track focuses extensively on etcd management through kubectl, which is an unnecessary abstraction layer for things like backups, failovers, and redundancies in smaller IT departments.

* Automated migrations. The K8s evangelists hype it as being able to manage VMs, which would be great if kube-virt was standard (it’s an add-on). I’d like to see K8s either formally integrate it into the baseline or more distros make it a checkbox option at cluster creation. Part of that should also be support for automatic deployment creations for existing VMs in a hypervisor, by analyzing current settings and suggesting the YAML or JSON to replicate that VM in K8s with appropriate IP address, current storage, and ACLs

From a tech standpoint, the foundations for K8s success have long been polished into a mirror shine. At this point it’s usability and accessibility that remain broadly unaddressed, especially if we want more people and companies using iterative, composable infrastructure.

just_mc 2 days ago | parent [-]

Check out Harvester: https://harvesterhci.io/