Docker Registry Service Account Validation.

5 min readMay 31, 2020

Prerequisites

Authenticated and Authorized Docker Registry.
A Kubernetes Cluster (If you don’t have one, you can use Kind).
Go (If you want to custom build the code).

Docker Authentication

Recently, I have deployed a fully authenticated and authorized docker registry using docker auth. After we are authenticated with docker registry using docker login, it generates ~/.docker/config.json file which contains the encoded details of credentials which we have used to authenticate with docker registry. Whenever we deploy an application using an image from private registry, credentials from ~/.docker/config.json will be validated.

Whenever we deploy an application which use an image from authenticated private docker registry, it will look for the credentials in ~/.docker/config.json file on the node. The contents of the ~/.docker/config.json will looks like this.

{
        "auths": {
                "https://private.docker.registry:5000/": {
                        "auth": "cHJhZGVlcDpwYXNzd29yZA=="
                }
        }
}

If you need to pass these credentials to an application which needs to deployed on Kubernetes, we can pass them as imagePullSecrets in container spec. Two ways we can do that.

Passing docker credentials as imagePullSecrets in pod Spec
Updating ServiceAccount with imagePullSecrets to auto mount secrets in pods.

First method is simply straight forward. We create a secret of type kubernetes.io/dockerconfigjson and pass the secret name in imagePullSecrets in Container or Pod Spec to auto pull the images from private docker registry. We can create secret as following. The official documentation is here.

kubectl create secret generic regcred --from-file=.dockerconfigjson=~/.docker/config.json --type=kubernetes.io/dockerconfigjson

Second method is to update the ServiceAccount with imagePullSecrets instead of specifying in Pod spec. Whenever, a pod is deployed using the ServiceAccount, imagePullSecrets are auto added in Pod spec. We can update the ServiceAccount as following. The official documentation is here.

kubectl patch serviceaccount default -p '{"imagePullSecrets": [{"name": "regcred"}]}'

Service Account Validation

The main problem which I have faced is that, Whenever I deploy, I need to update the ServiceAccount of each namespace . I have a docker credentials which are common for all the users in my team. This common user have permissions to pull images from the common namespace public. Each member would have to create credentials secret from ~/.docker/config.json file and pass in imagePullSecrets . So, I needed to automate this process to create the credentials secret in every namespace whenever the service account is created. If you want to create secret in all the namespaces and update all the service accounts whenever they are created, you need to pass all as TARGET_SERVICE_ACCOUNT . To achieve this, I have created a Kubernetes ValidatingWebhookConfiguration which automates the process of creating secret in target namespace and updates the service account with imagePullSecrets . You can find a folder Webhook in my repo, Which contains all the required files to deploy the Webhook in your cluster.

How it works

The main use of this Webhook is to create the secret in every namespace and update the service account with imagePullSecrets. You can specify the credential secret name and namespace to Webhook by environment variables which will be created in every namespace of target service account.

env:
- name: SOURCE_SECRET_NAME
  value: regcred         # Docker credentials secret
- name: SOURCE_SECRET_NAMESPACE
  value: sample          # Namespace of secret 
- name: TARGET_SERVICE_ACCOUNT
  value: default         # service account name where credential secret needs to be updated and created.

You can pass all as value in environment variable TARGET_SERVICE_ACCOUNT to update all the service accounts and create secret in all the namespaces or you can leave to pass this environment variable. The webhook server update all the service accounts by default.

The secret regcred is read from source namespace sample and the same secret will be created in namespace whenever the target service account default is created and also service account default is updated with imagePullSecrets. Their is delay of 2 seconds in creation of secret and updating the service account to allow the Kubernetes api server to create the service account. You can change this delay in custom build.

You can pass other environment variables to webhook to configure the server. The following are the default values on which server would be running.

env:
- name: HOST
  value: "0.0.0.0"  # Change the value to run the server on different host
- name: PORT
  value: "8888"     # Change the value to run the server on different port
- name: TLSCERT     
  value: "/etc/webhook/certs/cert.pem" # Path to Certificate location 
- name: TLSKEY     
  value: "/etc/webhook/certs/key.pem" # Path to key location
- name: DEBUG
  value: "false" # Keep true to run server in debug mode
- name: JSONLOG
  value: "false" # Keep true to print logs in JSON format

How to install

As you see in the above environment variables, The webhook server run on HTTPS and we need to signed generate certs for the api server send the request to Webhook. To generate these certs you can run this script as following.

bash generate_certs.sh --service webhook --namespace webhook --certSecret cert --keySecret key

The above command will generate the signed certs and also creates two secrets in the specified namespace. These certs can used to mount inside the pod for the webhook to run on HTTPS. If you check manifest.yaml after you run this, you see that most of the values are replaced and CA Certificate is updated in ValidationWebhookConfiguration . The following part of the script updated the YAML file.

# Modifying manifest.yaml, Replacing CABUNDLE and Namespace
CA_BUNDLE=$(cat server-cert.pem | base64 | tr -d '\n')sed -i "s/caBundle: .*$/caBundle: ${CA_BUNDLE}/g" ./manifest.yaml
sed -i "s/namespace: .*$/namespace: ${namespace}/g" ./manifest.yaml
sed -i -e "s*SERVICE_NAME*${service}*g" ./manifest.yaml
sed -i -e "s*SERVER_CERT*${certSecret}*g" ./manifest.yaml
sed -i -e "s*SERVER_KEY*${keySecret}*g" ./manifest.yaml

Now you can deploy the webhook as following.

kubectl apply -f manifest.yaml

After the pod comes into running state, you can test the webhook by creating a service account. If you check in that namespace, the source secret will be created and target service account will be updated with imagePullSecrets.

If you want to delete all the deployed resources, secrets and certificates generated, you can delete them using same script as following.

bash generate_certs.sh --service webhook --namespace webhook --certSecret cert --keySecret key --delete true

There you go. it’s done. Thank you.

Source Code

https://github.com/PradeepTammali/authz-docker-registry