Kubernetes and its features

I recently read an article in the Dutch Eos Psyche & Brein magazine titled “Waarom je de makkelijke oplossing over het hoofd ziet”. It goes about explaining why is it that when faced with a challenge people tend to pick an additive solution in contrast to a subtractive one. The author cites the article “People systematically overlook subtractive changes” from Nature where the authors describe the results of their research about why people tend to chose solutions that add stuff to solve a problem instead of the, in a lot of cases, simpler way of removing stuff instead.

This got me thinking about the current state of the container orchestration platforms and the way that they are designed and built. Bear with me for a moment.

From the inception of Borg (“Large-scale cluster management at Google with Borg”) the architecture of container management systems has been, just like the applications they were made to run, highly distributed, and that makes sense. Why would we make one single application that has to manage critical workloads across thousands of machines? That would be the biggest weakest link in the chain if there ever was one. In no way do I want to suggest that distributed, decoupled admin planes are a bad idea, on the contrary. However, one of the doors that such a design approach opens is the possibility of adding new functionality.

Let’s look at the Kubernetes API for example. Over the years new APIs and resources have been added that can manage practically anything in your application’s lifecycle: Ingresses, Jobs, Persistent Volumes. Kube has become a one-stop-shop for everything needed to run distributed applications at scale. But then you start to wonder: does it need that? Do we want to have everything being managed and stored by one singular system on a comparatively small amount of storage (etcd)? At wat point is Kube just another layer of AWS-like functionality on top of AWS?

If we zoom in to one specific resource, the ConfigMap, we can already see one of those extra functionalities that you wonder if we really need. Applications use a lot of environment variables for configuration on runtime. Kube provides us with a handy way of storing and giving the applications those environment variables, or even mounting whole files to them, whenever needed. All of that data get stored in the central control plane in etcd. If you use a managed Kubernetes solution you could say that that data is safe, however if you are managing it yourself, you have a lot of weight on your shoulders. Why not use something already existing with a much higher uptime and reliability guarantees? Something like the AWS SSM Parameter Store for example. Applications can get everything they need from the Parameter Store in a safe way without ever having to rely on plain-text environment variables in their runtimes.

I can give more examples: why rely on volumes or EFS to write files to if you can just write them to S3?

In any case, this is what I am trying to say: do we really need to add more complexity to what a container orchestrator or can we simplify it and make it easier for people not only to manage it, but to contribute to it?