VMware Secrets Manager

Quickstart

Get Your Hands Dirty

This is a quickstart guide to get you up and running with VSecM.

Prerequisites

  • Minikube: You can install VMware Secrets Manager on any Kubernetes cluster, but we’ll use Minikube in this quickstart example. Minikube is a tool that makes it easy to run Kubernetes locally.
  • make: You’ll need make to run certain build tasks. You can install make using your favorite package manager.
  • Docker: This quickstart guide assumes that Minikube uses the Docker driver. If you use a different driver, things will still likely work, but you might need to tweak some of the commands and configuration.
  • Helm: Helm is a package manager for Kubernetes. It allows developers and operators to more easily package, configure, and deploy applications and services onto Kubernetes clusters. In this quickstart guide, it is expected that you have helm installed on your system. Helm will be used to manage the deployment of the VMware Secrets Manager onto your Kubernetes cluster. If you don’t have Helm installed, you can follow the instructions on the official Helm website to install it.

I Have a Kubernetes Cluster Already

If you are already have a cluster and a kubectl that you can use on that cluster, you won’t need Minikube, so you can skip the steps related to initializing Minikube and configuring Minikube-related environment variables.

Also, if you are not using minikube, you will not need a local docker instance either.

A Video Is Worth A Lot of Words

Here’s a video that walks you through the steps in this quickstart guide:

Clone the Repository

Let’s start by cloning the VMware Secrets Manager repository first:

cd $WORKSPACE
git clone https://github.com/vmware-tanzu/secrets-manager.git
cd secrets-manager

Initialize Minikube

Next, let’s initialize Minikube:

cd $WORKSPACE/secrets-manager

# If you have a previous minikube setup, you can delete it with:
# make k8s-delete

make k8s-start

Configure Docker Environment Variables

Next, let’s configure the Docker environment variables for Minikube.

We don’t strictly need this for the quickstart, but it’s a good idea to do it anyway.

eval $(minikube -p minikube docker-env)

Install SPIRE and VMware Secrets Manager

Next we’ll install SPIRE and VMware Secrets Manager on the cluster.

helm repo add vsecm https://vmware-tanzu.github.io/secrets-manager/
helm repo update
helm install vsecm vsecm/vsecm

This will take a few minutes to complete.

Verify Installation

Let’s check that the installation was successful by listing the pods int the spire-system , vsecm-system and keystone-system namespaces:

kubectl get po -n spire-system

# Output:
NAME                          READY   STATUS    RESTARTS      
spire-agent-wdhdh             3/3     Running   0             
spire-server-b594bdfc-szssx   2/2     Running   0             

kubectl get po -n vsecm-system

# Output:
NAME                              READY   STATUS    RESTARTS
vsecm-keystone-c54d99d7b-c4jk4    1/1     Running   0          
vsecm-safe-6cc477f58f-x6wc9       1/1     Running   0          
vsecm-sentinel-74d648675b-8zdn2   1/1     Running   0

All the pods look up and running, so we can move on to the next step.

List Available Commands

make help lists all the available make targets in a cheat sheet format:

make help

# The output will vary as we add more commands to the Makefile.
# It will contain useful information about the available commands.

Deploy an Example Workload

Now, let’s deploy an example workload to the cluster to test VMware Secrets Manager in action.

cd $WORKSPACE/secrets-manager
export VSECM_VERSION=latest
make example-sdk-deploy

This will take a few moments too.

When done you would ba able to list the pods in the default namespace:

kubectl get po -n default

# Output
NAME                       READY   STATUS    RESTARTS   AGE
example-6cbb96b768-dhm7c   1/1     Running   0          6m44s

Let’s check the logs of our example workload:

kubectl logs example-6cbb96b768-dhm7c

The output will be something similar to this:

2024/03/25 17:30:32 fetch
2024/03/25 17:30:32 [TRACE] RpyyPfqx Sentry:Fetch 
https://vsecm-safe.vsecm-system.svc.cluster.local:8443/workload/v1/secrets
2024/03/25 17:30:32 [TRACE] RpyyPfqx Sentry:Fetch svid:id:  
spiffe://vsecm.com/workload/example/ns/default/sa/example/n
/example-6cbb96b768-dhm7c
Failed to read the secrets file. Will retry in 5 seconds...
Secret does not exist
2024/03/25 17:30:37 fetch
2024/03/25 17:30:37 [TRACE] kUWlDyo3 Sentry:Fetch 
https://vsecm-safe.vsecm-system.svc.cluster.local:8443/workload/v1/secrets
2024/03/25 17:30:37 [TRACE] kUWlDyo3 Sentry:Fetch svid:id:
spiffe://vsecm.com/workload/example/ns/default/sa/example/n/
example-6cbb96b768-dhm7c
Failed to read the secrets file. Will retry in 5 seconds...
Secret does not exist
2024/03/25 17:30:42 fetch
2024/03/25 17:30:42 [TRACE] dorrPbVN Sentry:Fetch 
https://vsecm-safe.vsecm-system.svc.cluster.local:8443/workload/v1/secrets
2024/03/25 17:30:42 [TRACE] dorrPbVN Sentry:Fetch svid:id:  
spiffe://vsecm.com/workload/example/ns/default/sa/example/n/
example-6cbb96b768-dhm7c
Failed to read the secrets file. Will retry in 5 seconds...
Secret does not exist
... truncated ...

Our sample workload is trying to fetch a secret, but it can’t find it.

Here’s the source code of our sample workload to provide some context:

package main

// ... truncated headers ...

func main() {

	// ... truncated irrelevant code ...

	for {
		log.Println("fetch")
		d, err := sentry.Fetch()

		if err != nil {
			println("Failed. Will retry in 5 seconds...")
			println(err.Error())
			time.Sleep(5 * time.Second)
			continue
		}

		if d.Data == "" {
			println("No secret yet... will check again later.")
			time.Sleep(5 * time.Second)
			continue
		}

		fmt.Printf(
			"secret: updated: %s, created: %s, value: %s\n",
			d.Updated, d.Created, d.Data,
		)
		time.Sleep(5 * time.Second)
	}
}

What the demo workload does is to try to fetch a secret every 5 seconds using the sentry.Fetch() function.

sentry.Fetch() is a function provided by the VMware Secrets Manager; it establishes a secure mTLS connection between the workload and VSecM Safe to fetch the secret.

Since this workload does not have any secret registered, the request fails and the workload retries every 5 seconds.

Since this is a quickstart example, we won’t dive into the details of how the workload establishes a secure mTLS connection with the VSecM Safe. We’ll cover this in the following sections.

For the sake of this quickstart, we can assume that secure communication between the workload and the VSecM Safe is already taken care of for us.

Register a Secret

Now, let’s register a secret and see what happens.

To register a secret we’ll need to find the vsecm-sentinel pod in the vsecm-system namespace and execute a command inside the pod.

Let’s get the pod first:

kubectl get po -n vsecm-system

Here’s a sample output:

NAME                              READY   STATUS    RESTARTS   
vsecm-keystone-c54d99d7b-c4jk4    1/1     Running   0          
vsecm-safe-6cc477f58f-x6wc9       1/1     Running   0          
vsecm-sentinel-74d648675b-8zdn2   1/1     Running   0

vsecm-sentinel-74d648675b-8zdn2 is what we need here.

Let’s use it and register a secret to our example workload:

kubectl exec vsecm-sentinel-74d648675b-8zdn2 -n vsecm-system -- \
safe -w "example" -n "default" -s "VSecMRocks"

Sentinel Command Line Help

VSecM Sentinel comes with a command line tool called safe. safe allows you to register secrets to VSecM Safe, delete secrets, or list existing secrets.

You can execute safe -h or safe --help to get a list of available commands and options.

You’ll get an OK as a response:

OK

For the command safe -w "example" -n "default" -s "VSecMRocks"

  • -w is the workload name
  • -n is the namespace
  • -s is the secret value

But how do you know what the workload name is?

That’s where ClusterSPIFFEID comes in:

kubectl get ClusterSPIFFEID

And here’s the output:

NAME             AGE
example          73m
vsecm-keystone   73m
vsecm-safe       73m
vsecm-sentinel   73m

ClusterSPIFFEID with an Analogy

Imagine the ClusterSPIFFEID as a badge maker for an organization.

If anyone could create or modify badges (SVIDs), they could make one for themselves that mimics the CEO’s badge, gaining access to restricted areas.

Hence, only trusted personnel (with elevated privileges) are allowed to manage the badge maker.

Make sure your guard your ClusterSPIFFEID with proper RBAC rules.

Let’s see the details of this example SPIFFE ID:

kubectl describe ClusterSPIFFEID example

And the output:

Name:         example
Namespace:    
Labels:       <none>
Annotations:  <none>
API Version:  spire.spiffe.io/v1alpha1
Kind:         ClusterSPIFFEID
Metadata:
  Creation Timestamp:  2024-03-25T17:17:58Z
  Generation:          1
  Resource Version:    1651
  UID:                 e8af0138-7b3a-438e-9d58-21ab35a97b15
Spec:
  Pod Selector:
    Match Labels:
      app.kubernetes.io/name:  example
  Spiffe ID Template:          
  spiffe://vsecm.com/workload/example/
  ns/{{ .PodMeta.Namespace }}/
  sa/{{ .PodSpec.ServiceAccountName }}/
  n/{{ .PodMeta.Name }}
  Workload Selector Templates:
    k8s:ns:default
    k8s:sa:example
Status:
  Stats:
    Entries Masked:             0
    Entries To Set:             1
    Entry Failures:             0
    Namespaces Ignored:         4
    Namespaces Selected:        6
    Pod Entry Render Failures:  0
    Pods Selected:              1
Events:                         <none>

For the sake of keeping things simple because this is a quickstart, we can assume that someone has created this example SPIFFE ID for us, and using this SPIFFE ID, our example workload can securely communicate with the VSecM Safe.

Verifying Secret Registration

Since we’ve registered a secret, let’s see if our example workload can fetch the secret now and display it in its logs.

kubectl logs example-6cbb96b768-dhm7c

And the output would be something like this:

2024/03/25 17:36:13 fetch
2024/03/25 17:36:13 [TRACE] ZjmdoNn9 Sentry:Fetch https://vsecm-safe.
vsecm-system.svc.cluster.local:8443/workload/v1/secrets
2024/03/25 17:36:13 [TRACE] ZjmdoNn9 Sentry:Fetch svid:id:  spiffe://vsecm.com/
workload/example/ns/default/sa/example/n/example-6cbb96b768-dhm7c
secret: updated: "2024-03-25T17:34:25Z", created: 
"2024-03-25T17:34:25Z", value: VSecMRocks
2024/03/25 17:36:18 fetch
2024/03/25 17:36:18 [TRACE] kcOZQXeH Sentry:Fetch https://vsecm-safe.
vsecm-system.svc.cluster.local:8443/workload/v1/secrets
2024/03/25 17:36:18 [TRACE] kcOZQXeH Sentry:Fetch svid:id:  spiffe://vsecm.com/
workload/example/ns/default/sa/example/n/example-6cbb96b768-dhm7c
secret: updated: "2024-03-25T17:34:25Z", created: "2024-03-25T17:34:25Z", 
value: VSecMRocks

As you can see, the secret is now fetched and displayed in the logs.

The beauty of this approach is when we change the secret using VSecM Sentinel, the workload will automatically fetch the new value, without having to restart itself.

Where to Go From Here

This quickstart is meant to give you a quick overview of how you can use VMware Secrets Manager to securely manage secrets in your Kubernetes clusters.

After successfully completing this quickstart, you can try the following:

Suggest edits ✏️

results matching ""

    No results matching ""

    «« previous next »»