Working to deploy our services to Kubernetes, we came across the issue of managing configuration of our services for multiple namespaces, a usual task whenever one manage deployments.

As we’re using Helm to manage our components/apps/services resources description and their deployments, the Kubernetes resources files are generated on Tiller (the server component of Helm that runs inside Kubernetes cluster). I believe Helm is architected like that to allow rollback management by letting Tiller track releases. Helm client is just pushing charts and values to Tiller.

We encountered some issue when the state maintained by Tiller diverge from the state that Kubernetes observe. For example if resources created by Helm are modified externally (by kubectl for example), Helm won’t apply any change if its state is identical to the state sent by Helm client, Tiller does not compare its records with the ones in Kubernetes.

That situation creates discrepancies that are repeatedly source of issues so we decided to explore alternative way to manage our configurations.

We looked into ksonnet (https://ksonnet.io/), which generates templates “client side” (no server component) and has great conventions for managing multiple environments (dev, etc). Honestly we had a hard time figuring out how to use jsonnet language, it seemed over complicated and has a syntax that we found hard to grasp. After having worked with Helm and template files that look just like the resources definition in Kubernetes it did not seem like the right path for us.

Looking back to helm, we found out about the ‘helm template’ command that generates the resources definition locally (client side). Using this command we’re able to output the resources definitions using our values files, and commit them into git. We use kubectl to apply them to the target cluster. And voilà. The desired state of our deployments is managed in Git where developers and CI/CD pipeline can make changes. Then this state can be pushed directly to Kubernetes so it can update our deployments.