Deploying to Kubernetes from CI with helm
2019-03-05
Update 09/2019: I switched from CircleCI to Github Actions
I use helm and CircleCI to update the production and staging deployments of this blog (yes I actually have a staging deployment for a blog) in my kubernetes cluster running on Google Cloud (GKE). The helm chart for this project has two deployments, two services, and one ingress resource that routes requests to either the production or staging service based on the HTTP HOST header of the request.
Two deployments, two services, one ingress
$ tree ./helm ./helm ├── Chart.yaml ├── templates │ ├── NOTES.txt │ ├── _helpers.tpl │ ├── deployment-develop.yaml │ ├── deployment-master.yaml │ ├── ingress.yaml │ ├── service-develop.yaml │ └── service-master.yaml └── values.yaml
Deployment in CI
// .circleci/config.yml - run: name: Deploy to Kubernetes command: | if [[ "$CIRCLE_BRANCH" == "develop" ]]; then helm upgrade --install blog --reuse-values --debug \ --set develop_image=cflynnus/blog:${CIRCLE_BRANCH}-${CIRCLE_SHA1} \ ./helm elif [[ "$CIRCLE_BRANCH" == "master" ]]; then helm upgrade --install blog --reuse-values --debug \ --set master_image=cflynnus/blog:${CIRCLE_BRANCH}-${CIRCLE_SHA1} \ ./helm fi kubectl rollout status deployment/${PROJECT_NAME}-${CIRCLE_BRANCH}-app
I ran into a problem when I added a new variable to my helm chart values.yaml file.
// values.yaml develop_image: “cflynnus/blog:latest” master_image: “cflynnus/blog:latest” google_analytics: “UA-000000000-1” # New variable
My CI uses helm upgrade
with the --reuse-values
flag to update either my
production or my staging environment, depending on whether I’ve just pushed to
my master or develop branches. This flag instructs tiller to generate the yaml
kubernetes resources using the computed values from my last deployment and
accepting values passed to helm upgrade
via the --set
flag. This works most
of the time but becomes an issue when you want to add a new value to your
values.yaml file in your helm chart. When running helm upgrade --reuse-values
--set ...
tiller will not use new values in values.yaml. This leads to
template errors, where the template expects certain values to be present but
they are not.
The way I get around this, is to do idempotent deploy from my local machine
with the new variable, then push code to my remote git/github repo and have my
CI run the deployment job. Since the value is already present in tiller,
using --reuse-values
will lead to the correct yaml being generated.
// This won't cause any changes to kubernetes resources since no resources use 'google_analytics' yet $ helm upgrade --install YOUR_DEPLOYMENT --set google_analytics="xxxxxxxxxxxxxxx" .
After doing this, code can be pushed to the remote git repository and trigger the CI deployment job shown above.