--- title: Using Google Container Registry with Kubernetes description: I recently got into orchestrating my Docker containers with Kubernetes. For one of our projects, I needed to pull docker images from the Google Container Registry (GCR). image: http://cdn2.hubspot.net/hubfs/2668666/All-InOne-2016/Image/bg-img.jpg --- Check out our **[Cloud Native Services](https://www.container-solutions.com/services) **and** **book a call with one of our experts today!  [![Container Solutions](https://blog.container-solutions.com/hubfs/CS_blog/logo-main.svg "Container Solutions")](http://container-solutions.com) - share: - [**](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Fblog.container-solutions.com%2Fusing-google-container-registry-with-kubernetes) - [**](https://www.twitter.com/share?url=https%3A%2F%2Fblog.container-solutions.com%2Fusing-google-container-registry-with-kubernetes) - [**](http://www.linkedin.com/shareArticle?mini=true&url=https://blog.container-solutions.com/using-google-container-registry-with-kubernetes) - [**](https://blog.container-solutions.com/using-google-container-registry-with-kubernetes#) [![](http://cdn2.hubspot.net/hubfs/2668666/All-InOne-2016/Image/bg-img.jpg) ](https://blog.container-solutions.com/using-google-container-registry-with-kubernetes) [Cloud native](https://blog.container-solutions.com/tag/cloud-native), [Docker](https://blog.container-solutions.com/tag/docker), [Kubernetes](https://blog.container-solutions.com/tag/kubernetes), [Miscellaneous](https://blog.container-solutions.com/tag/miscellaneous), [DevOps](https://blog.container-solutions.com/tag/devops) # Using Google Container Registry with Kubernetes [Lian Li](https://blog.container-solutions.com/author/lian-li) ![Lian Li](https://blog.container-solutions.com/hubfs/lianillustration.png) April 9, 2018 11 minutes Read I recently got into orchestrating my Docker containers with Kubernetes. For one of our projects, I needed to pull docker images from the Google Container Registry (GCR). When using the Google Kubernetes Engine with the GCR everything works out of the box, but to run the containers locally with docker, I had to install and configure `docker-credential-gcr`. (Go to the [GitHub repository](https://github.com/GoogleCloudPlatform/docker-credential-gcr) for guidance.) However, when I tried to run a deployment on my local Kubernetes cluster I encountered some errors ``` Normal Pulling 4s kubelet, docker-for-desktop pulling image "eu.gcr.io/lian-empty-project/empty-debian-container" Warning Failed 4s kubelet, docker-for-desktop Failed to pull image "eu.gcr.io/lian-empty-project/empty-debian-container": rpc error: code = Unknown desc = Error response from daemon: unauthorized: You do not have the needed permissions to perform this operation, and you may have invalid credentials. To authenticate your request, follow the steps in: https://cloud.google.com/container-registry/docs/advanced-authentication Warning Failed 4s kubelet, docker-for-desktop Error: ErrImagePull Normal BackOff 2s (x2 over 3s) kubelet, docker-for-desktop Back-off pulling image "eu.gcr.io/lian-empty-project/empty-debian-container" Warning Failed 2s (x2 over 3s) kubelet, docker-for-desktop Error: ImagePullBackOff ``` Kubernetes does not use the docker client to log in and pull images which is why there are no valid GCR credentials configured. To pull images from the GCR, you can use Kubernetes' `ImagePullSecrets` concept. Secrets can be assigned to single pods or a service account, which then adds the secret to any new pod created in its namespace. Whenever someone or something accesses the Kubernetes cluster, the API server authenticates them as a specific account type. Simply put, User accounts are for real-life humans, service accounts are for processes that run inside pods. Whether you are using [minikube](https://kubernetes.io/docs/getting-started-guides/minikube/), [docker-for-mac edge](https://docs.docker.com/docker-for-mac/edge-release-notes/) (which comes with native Kubernetes support) or a native Kubernetes cluster, this guide applies to all Kubernetes environments. I will show you two ways to setup Kubernetes `ImagePullSecrets` for GCR, but the principles are applicable to any private registry.   ### Push Image to GCR Start with enabling the Container Registry API by logging into Google Cloud and navigating to **Container Registry** on your project [![](https://blog.container-solutions.com/hs-fs/hubfs/Imported_Blog_Media/kubernetes-gcr-01-300x238-1.png?width=575&height=456&name=kubernetes-gcr-01-300x238-1.png)](https://blog.container-solutions.com/hubfs/Imported_Blog_Media/kubernetes-gcr-01-1.png) Click **Enable Container Registry API** [![](https://blog.container-solutions.com/hs-fs/hubfs/Imported_Blog_Media/kubernetes-gcr-02-300x215-1.png?width=358&name=kubernetes-gcr-02-300x215-1.png)](https://blog.container-solutions.com/hubfs/Imported_Blog_Media/kubernetes-gcr-02-1.png) Then build a Docker image and push it to your project’s GCR ``` docker build -t empty-debian-container docker tag imageHash eu.gcr.io/lian-empty-project/empty-debian-container docker push eu.gcr.io/lian-empty-project/empty-debian-container ``` All following kubectl commands should run in a specific namespace. Namespaces are virtual clusters, running in the same physical cluster and are an excellent way to keep your project environments separate. Define a namespace `localdev` for which you want to set up `ImagePullSecrets`. ``` kubectl create namespace localdev ``` If you do not want to add `--namespace=localdev` to all kubectl commands, you can set this as the default namespace for your current context ``` kubectl config get-contexts (out) CURRENT NAME CLUSTER AUTHINFO NAMESPACE (out) minikube minikube minikube some-other-namespace (out) * docker-for-desktop docker-for-desktop-cluster docker-for-desktop kubectl config set-context --namespace=localdev docker-for-desktop ``` Now it is time to create some GCR credentials.   ### Create & use GCR credentials #### JSON Key JSON Keys are valid until the key is removed from the Google Cloud. Google uses its own service accounts to allow automated access to its platform services. Log into your Google Cloud Console, navigate to **APIs and Services** -> **Credentials** and create a new **Service account key**. [![](https://blog.container-solutions.com/hs-fs/hubfs/Imported_Blog_Media/kubernetes-gcr-03-300x270-1.png?width=300&height=270&name=kubernetes-gcr-03-300x270-1.png)](https://blog.container-solutions.com/hubfs/Imported_Blog_Media/kubernetes-gcr-03-1.png) In the drop-down menu for service accounts, pick **New service account** and give it a name. Since you are only using this account for pulling images from the registry, you can give it the **Project** -> **Viewer** role to start. From the key types choose **JSON** and click **Create**. [![](https://blog.container-solutions.com/hs-fs/hubfs/Imported_Blog_Media/kubernetes-gcr-04-300x239-1.png?width=300&height=239&name=kubernetes-gcr-04-300x239-1.png)](https://blog.container-solutions.com/hubfs/Imported_Blog_Media/kubernetes-gcr-04-1.png) The JSON key file is automatically downloaded and will kind of look like this: ``` { "type": "service_account", "project_id": "lian-empty-project", "private_key_id": "abcd1234efgh", "private_key": "-----BEGIN PRIVATE KEY-----\nPRIVATEKEY\n-----END PRIVATE KEY-----\n", "client_email": "pull-images-from-registry@lian-empty-project.iam.gserviceaccount.com", "client_id": "1234567", "auth_uri": "https://accounts.google.com/o/oauth3/auth", "token_uri": "https://accounts.google.com/o/oauth3/token", "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth3/v1/certs", "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/pull-images-from-registry%40lian-empty-project.iam.gserviceaccount.com" } ``` Now you need to tell Kubernetes to use the JSON key file when pulling from GCR by creating a secret named gcr-json-key. ``` kubectl create secret docker-registry gcr-json-key \ (out) --docker-server=eu.gcr.io \ (out) --docker-username=_json_key \ (out) --docker-password="$(cat ~/json-key-file.json)" \ (out) --docker-email=any@valid.email ``` Replace `~/json-key-file.json` with the path to your json key file. The docker-server value has to match the hostname of your registry exactly. So outside of the EU, you might want to use `https://gcr.io` instead. Also note that `docker-registry` is a necessary keyword whereas `gcr-json-key` is a freely customizable name. kubectl should answer with ``` secret "gcr-json-key" created ``` If you get ``` Error from server (AlreadyExists): secrets "gcr-json-key" already exists ``` instead, you might want to think of another name or delete the existing secret first ``` kubectl delete secret gcr-json-key ``` Alternatively, you can add this little hack to the above command, which updates the old secret ``` --dry-run -o yaml | kubectl replace -f - ``` Finally, you have to add the secret to your default service account as `ImagePullSecrets`, so it will actually be used, when Kubernetes spins up a new pod with this service account. ``` kubectl patch serviceaccount default \ (out) -p '{"imagePullSecrets": [{"name": "gcr-json-key"}]}' ``` You can check the setup of the default service account with ``` kubectl get serviceaccount default ``` by adding the `-o` flag, you can format the output to JSON or YAML ``` kubectl get serviceaccount default -o yaml (out) apiVersion: v1 (out) imagePullSecrets: (out) - name: gcr-json-key # This is what we are looking for (out) kind: ServiceAccount (out) metadata: (out) creationTimestamp: 2018-03-30T13:06:22Z (out) name: default (out) namespace: localdev (out) resourceVersion: "275310" (out) selfLink: /api/v1/namespaces/localdev/serviceaccounts/default (out) uid: 123-abc-456 (out) secrets: (out) - name: default-token-ab432 ``` Now you should be able to pull from GCR without any problems ``` deployment "demo" created ``` Take a look at your pods to verify that pulling was successful. ``` kubectl describe pods (out) ... (out) .Normal Pulling 5s (x2 over 8s) kubelet, docker-for-desktop pulling image "eu.gcr.io/lian-empty-project/empty-debian-container" (out) .Normal Pulled 4s (x2 over 7s) kubelet, docker-for-desktop Successfully pulled image "eu.gcr.io/lian-empty-project/empty-debian-container" ``` (Note: If you still have an old version of this deployment running, kubectl does not notice any changes and outputs ``` deployment "demo" unchanged ``` You need to delete the old deployment and apply again.) #### Access token For keys that are short-lived for one-time and instant usage, you can create a GCR access token for your `ImagePullSecrets`. For this, you need to have `gcloud` installed and correctly configured. Creating a secret is virtually the same as with JSON keys ``` kubectl create secret docker-registry gcr-access-token \ (out) --docker-server=eu.gcr.io \ (out) --docker-username=oauth3accesstoken \ (out) --docker-password="$(gcloud auth print-access-token)" \ (out) --docker-email=any@valid.email ``` except using `oauth3accesstoken` instead of `_json_key` as username and the output of `gcloud auth print-access-token` as password. Don't forget to patch your service account with the new secret. ``` kubectl patch serviceaccount default \ (out) -p '{"imagePullSecrets": [{"name": "gcr-json-key"}]}' ``` A look at the service account yaml should tell you if you configured everything correctly ``` kubectl get serviceaccount default -o yaml (out) apiVersion: v1 (out) imagePullSecrets: (out) - name: gcr-access-token # This is what we are looking for (out) kind: ServiceAccount (out) metadata: (out) creationTimestamp: 2018-03-30T13:06:22Z (out) name: default (out) namespace: localdev (out) resourceVersion: "276259" (out) selfLink: /api/v1/namespaces/localdev/serviceaccounts/default (out) uid: 123-abc-456 (out) secrets: (out) - name: default-token-ab432 ``` That is it! You should now be able to pull from the GCR. [![Download the Cloud Native Attitude book for free](https://no-cache.hubspot.com/cta/default/2252258/ac93f2fd-838a-41d9-baa1-49b957ca2622.png)](https://cta-redirect.hubspot.com/cta/redirect/2252258/ac93f2fd-838a-41d9-baa1-49b957ca2622) ### ImagePullSecrets for single pods In case you want to use the secret for one specific pod only, all you need to do is add the secret to your pod.yaml instead of patching the service account with your secret. ``` apiVersion: v1 kind: Pod metadata: name: empty-debian spec: containers: - name: empty-debian-container image: eu.gcr.io/lian-empty-project/empty-debian-container imagePullSecrets: - name: gcr-secret ``` [### My Journey: Exploring the Cloud ](https://blog.container-solutions.com/my-journey_riccardo-cefala) [![prev-arrow](https://cdn2.hubspot.net/hubfs/3842749/Wow%202019/Images/left-arrow.png) Previous article](https://blog.container-solutions.com/my-journey_riccardo-cefala) [ Next article ![next-arrow](https://cdn2.hubspot.net/hubfs/3842749/Wow%202019/Images/right-arrow.png) ](https://blog.container-solutions.com/what-is-eventual-consistency-and-why-is-it-so-cool) [ ### What is Eventual Consistency And Why Is It So Cool? ](https://blog.container-solutions.com/what-is-eventual-consistency-and-why-is-it-so-cool) Comments Leave your Comment ![cs-logo-white](https://blog.container-solutions.com/hs-fs/hubfs/cs-logo-white.png?width=150&height=61&name=cs-logo-white.png "cs-logo-white") #### **Talk to sales** [info@container-solutions.com](mailto:info@container-Solutions.com) #### Stay In Touch - [**](https://www.linkedin.com/company/container-solutions/) - [**](https://twitter.com/containersoluti) © 2024 Container Solutions