In this second article on our "how to" series on External Secrets we’re looking at the Google Cloud Secret Manager. We are going to configure External Secrets to use a Secret Manager instance as a secret provider.
But first, a quick recap. initiated by Container Solutions and GoDaddy, External Secrets is an Open Source Kubernetes operator that integrates external secret management systems including AWS Secrets Manager, HashiCorp Vault, Google Secrets Manager, Azure Key Vault, amongst others, and is designed to enable the synchronization of secrets from external APIs into Kubernetes. You can find comprehensive project documentation here.
In this tutorial we will configure GCP Secret Manager in order to have a safe way to access secrets, and then configure External-Secrets to fetch information from our Secret Manager. A first post in this series covered the AWS Secrets Manager, and in future articles we’ll explore how to set up external-secrets with Hashicorp Vault and Azure KeyVault.
Getting started
To properly follow this tutorial, make sure you have installed the following tools:
- minikube - or any other kubernetes cluster
- external-secrets deployed
- gcloud cli installed
Configuration
External Secrets supports the configuration of several authentication methods for the GCP Secret Manager provider. In this guide we are using authentication through Service Account keys, as it doesn’t need any other GCP Resources. We are going to go through the following steps:
- Set up GCP Secret Manager
- Configure External-Secrets
Set up GCP Secret Manager
The following steps are all going to be used with gcloud CLI. The first thing we should do is configure properly the setup to the target project which we would like the Secret Manager to be, and enable its APIs:
project=<project-name-here>
gcloud gcloud config set project $project
gcloud services enable secretmanager.googleapis.comAfter that, we can create a secret inside Secret Manager, and then configure a service account to access it
echo -ne '{"password":"itsasecret"}' | gcloud secrets create mysecret --data-file=-
gcloud iam service-accounts create external-secrets
gcloud secrets add-iam-policy-binding mysecret --member "serviceAccount:external-secrets@$project.iam.gserviceaccount.com" --role "roles/secretmanager.secretAccessor"Here, we are following the least privilege principle, as we are only binding mysecret to the external-secrets service account. You can achieve a more wider permission through the command line gcloud projects add-iam-policy-binding $project --member "serviceAccount:external-secrets@$project.iam.gserviceaccount.com" --role "roles/secretmanager.secretAccessor", which would give external-secrets service account permission to access all secrets within the project.
After we have our service account configured, the next step is to create a key to that service account:
gcloud iam service-accounts keys create key.json --iam-account=external-secrets@$project.iam.gserviceaccount.com
kubectl create secret generic gcpsm-secret --from-file=secret-access-credentials=key.jsonConfiguring External Secrets
After setting up GCP Secret Manager, the next step is to create a SecretStore and an ExternalSecrets to fetch mysecret object that we’ve created earlier:
>cat <<EOF | kubectl apply -f -
apiVersion: external-secrets.io/v1alpha1
kind: SecretStore
metadata:
name: gcp-backend
spec:
provider:
gcpsm:
auth:
secretRef:
secretAccessKeySecretRef:
name: gcpsm-secret
key: secret-access-credentials
projectID: $project
---
apiVersion: external-secrets.io/v1alpha1
kind: ExternalSecret
metadata:
name: gcp-external-secret
spec:
secretStoreRef:
kind: SecretStore
name: gcp-backend
target:
name: secret-to-be-created
data:
- secretKey: password_file
remoteRef:
key: mysecret
EOFAnd that’s it! We can check that our Secret has been created in Kubernetes:
kubectl get secret secret-to-be-created -o jsonpath='{.data.password_file}' | base64 -d
{"password":"itsasecret"}%
We can also check ExternalSecrets status information:
kubectl get es
NAME STORE REFRESH INTERVAL STATUS
gcp-external-secret gcp-backend 1h SecretSyncedA Note on SecretStore object
When we deploy a SecretStore reference to a backend provider, External Secrets needs to have any credentials information required to access that prodiver in the same namespace as the SecretStore is deployed. This allows developers to handle Secret Manager access as they wish using their own credentials, in a self-service manner.
However, this approach is not always desirable. For example, sometimes compliance policies may require us to isolate any backend access from the developer. To allow that External Secrets comes with a second backend resource called ClusterSecretStore. This is a cluster-scoped kubernetes object which can fetch credentials from any namespace, allowing users to deploy a Shared-Resource topology, as per picture below:
To see how this works, let's start by first creating two separate namespaces
kubectl create namespace admin-namespace
kubectl create namespace developer-namespaceThen, let’s create our credentials secret within the admin-namespace, and create our ClusterSecretStore
kubectl create secret generic gcpsm-secret --namespace admin-namespace --from-file=secret-access-credentials=key.json
cat <<EOF | kubectl apply -f -
apiVersion: external-secrets.io/v1alpha1
kind: ClusterSecretStore
metadata:
name: gcp-backend
spec:
provider:
gcpsm:
auth:
secretRef:
secretAccessKeySecretRef:
name: gcpsm-secret
key: secret-access-credentials
namespace: admin-namespace
projectID: $project
EOFNote that the only difference is that we need to specify a namespace where the credentials are stored.
After, we can simulate a developer experience to create an ExternalSecret within their own namespace:
cat <<EOF | kubectl apply -f -
apiVersion: external-secrets.io/v1alpha1
kind: ExternalSecret
metadata:
name: gcp-external-secret-no-credentials
namespace: developer-namespace
spec:
secretStoreRef:
kind: ClusterSecretStore
name: gcp-backend
target:
name: secret-to-be-created
data:
- secretKey: password_file
remoteRef:
key: mysecret
EOFAnd that’s it! We can see that the only information the developer needed to know is the ClusterSecretStore name that they should reference, and the secret key they want to sync within their namespace.
Of course, we can check the secret has been fetch with the following command:
kubectl get secret -n developer-namespace secret-to-be-created -o jsonpath='{.data.password_file}' | base64 -d
{"password":"itsasecret"}%And to check ExternalSecret status:
kubectl get es -n developer-namespace
NAME STORE REFRESH INTERVAL STATUS
gcp-external-secret-no-credentials gcp-backend 1h SecretSyncedWrapping things up
In this post we’ve seen how to configure External Secrets to allow using the GCP Secret Manager as a provider. We also have explored different deployment setups: As a Service and Shared Store. You can check more about these subjects in the official external-secrets documentation.
Wrapping up what we have done:
- Set up GCP Secret Manager
- Configure External Secrets
- Deploy Shared Store topology
In the next post, we’ll see how to configure Hashicorp Vault as a SecretStore! Stay tuned!
Previous article
