Home / server / Questions / 791715
Accepted
Drux
Asked: 2016-07-25 18:42:32 +0800 CST

Using environment variables in Kubernetes deployment spec

  • 772

I currently use a Kubernetes spec Deployment.yaml for deploying a service. The spec includes a verbatim reference to a specific IP address (marked as <static-ip-address> below):

spec:
  type: LoadBalancer
  loadBalancerIP: <static-ip-address>

I am concerned about pushing information such as passwords or IP addresses into remote Git repositories. Can I avoid this e.g. by making use of environment variables, e.g. with a deployment spec and actual deployment roughly as follows:

spec:
   type: LoadBalancer
   loadBalancerIP: ${SERVICE_ADDRESS}

and

export SERVICE_ADDRESS=<static-ip-address>
kubectl create -f Deployment.yaml

Obviously this specific syntax does not work yet. But is something like this possible and if so how?

I'd prefer not relying on a separate provisioning tool. Secrets and ConfigMaps seem promising, but apparently they cannot be consumed in a way that suits this purpose. If I could directly reference a static IP address that was defined with gcloud compute addresses create service-address that would be best.

environment-variables kubernetes google-kubernetes-engine

4 Answers

  1. A much easier/cleaner solution: envsubst

    In deploy.yml:

    LoadbalancerIP: $LBIP

    Then just create your env var and run kubectl like this:

    export LBIP="1.2.3.4"
envsubst < deploy.yml | kubectl apply -f -

    You just put regular Bash variables into whatever file you want to use, in this case the YAML manifest, and have ensubst read that file. It will output the file with the env vars replaced by their values. You can also use it to create new files like this:

    envsubst < input.yml > output.yml

    envsubst is available in e.g. Ubuntu/Debian gettext package.

    • 32
  2. Best Answer

    There was another pleasantly simple solution: I have a Google Compute Address my-address defined, and I can apparently use it in the service spec like so: loadBalancerIP: my-address.

    With this as "external" source for IP addresses and secrets for passwords there is no more need for a provisioning tool (or templates) for my simple use case (within a GKE environment).

    OBSOLETE NOW: I have decided on using a provisioning tool of sorts, namely "built-in" sed, after all.

    My Deployment.yaml now contains a "template variable" e.g. in

    loadBalancerIP: $$EXTERNAL_IP

    and I deploy the service with, say, 1.2.3.4 as external IP address with

    cat Deployment.yaml | sed s/\$\$EXTERNAL_IP/1.2.3.4/ | kubectl create -f -
    • 2

  3. You can write a simple pre-processor to do variable substitution on your yaml files (or you can use jsonnet to accomplish the same thing on json configuration files).

    There is some discussion around adding templates directly into Kubernetes configuration but it isn't yet implemented or available.

    • 0

  4. Until templates are available, the easiest way to do it is to run a job which uses the Kubernetes API to update the service. A short shell script in an alpine-based image, coupled with a secret (containing the IP address) and a configmap (containing the template), should be simple enough. The difficult bit is correctly using the authentication and authorization features of the apiserver.

    https://stackoverflow.com/questions/30690186/how-do-i-access-the-kubernetes-api-from-within-a-pod-container gives an example of accessing the API. Obviously, you'll want to POST to /api/v1/namespaces/default/services instead of the GET in that example.

    • 0