Using a Let’s Encrypt Cluster Issuer for Certificate Manager

I’m a big fan of Let’s Encrypt as its a great service that provides free TLS certificates for your applications and websites. This post summarizes the steps to setup Let’s Encrypt as the cluster issuer for certificate manager.

A Cluster Issuer enables your applications to automatically request TLS certificates from Let’s Encrypt. It basically avoids having to do the following manually:

  • Typing certbot certonly --manual --cert-name something.domain.com --preferred-challenge=dns to create a manual TLS request.
  • Then going to your DNS service and creating the TXT record.
  • Then downloading the cert.pem and privkey.pem.
  • Then creating a secret to use the new certificate.

Pre-requisites

  1. A Kubernetes cluster, I’m using a TKG cluster
  2. A domain name managed by a well-known domain registrar (I’m using Cloud Flare, but Route 53 and others can also be used)

Step 1. Install Cert Manager into your Kubernetes cluster

# Install Tanzu Standard Repository
tanzu package repository add tanzu-standard --url projects.registry.vmware.com/tkg/packages/standard/repo:v2024.2.1 --namespace tkg-system

# Create namespace for cert-manager tanzu packages
k create ns my-packages

# Install cert-manager 
tanzu package install cert-manager --package cert-manager.tanzu.vmware.com --namespace my-packages --version 1.12.2+vmware.2-tkg.2

# Install cert-manager custom resource definitions
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.14.4/cert-manager.crds.yaml

Step 2. Create a Secret for Cloud Flare

I am using Cloud Flare as my DNS provider. Cloud Flare has an API that can be used with an API Token or an API Key. I am using an API Key to allow Let’s Encrypt to verify domain ownership with Cloud Flare.

You can get your API Key by following this screenshot.

Then create the following file secret-cloudflare.yaml.

apiVersion: v1
kind: Secret
metadata:
  name: cloudflare-api-key-secret
  namespace: cert-manager
type: Opaque
stringData:
  api-key: <your-cloud-flare-api-key>
  # - or -
  # api-token: your-api-token

Step 3. Create the Let’s Encrypt Cluster Issuer

I am using Let’s Encrypt as the certificate issuer and it will check the validity of the certificate request against the domain ownership in Cloud Flare using the Secret created in Step 2.

Create another file named cluster-issuer-production.yaml.

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-production
spec:
  acme:
    email: <your-email-address>
    # Letsencrypt Production
    server: https://acme-v02.api.letsencrypt.org/directory
    # - or -
    # Letsencrypt Staging
    # server: https://acme-staging-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: example-issuer-account-key
    solvers:
    - dns01:
        cloudflare:
          email: <your-cloudflare-email-account>
          apiKeySecretRef:
            name: cloudflare-api-key-secret
            key: api-key

Step 4. Apply both files to create the Secret containing the Cloud Flare API Key and the Cluster Issuer.

kubectl apply -f secret-cloudflare.yaml

kubectl apply -f cluster-issuer-production.yaml

Your cluster is now ready for automatically issuing TLS certificates using Certificate Manager.

Example Application

The following is an example application manifest that uses the letsencrypt-production ClusterIssuer to request a TLS certificate from Let’s Encrypt named nginx.k8slabs.com.

My test domain k8slabs.com is running in Cloud Flare.

The manifest has the following sections:

  • namespace – creates the nginx namespace for all of the resources below
  • service – ClusterIP service for nginx to expose the nginx pod created by the
  • statefulset – creates the statefulset that will deploy the nginx pods
  • certificate – issued by the ClusterIssuer using Let’s Encrypt and checks validity against the DNS records in CloudFlare
  • httpproxy (ingress) – creates an ingress and uses the certificate created by the ClusterIssuer to expose the nginx application over secure TLS

Sample application nginx-statefulset-contour-tls.yaml

---
apiVersion: v1
kind: Namespace
metadata:
  name: nginx
  labels:
    name: nginx
---
apiVersion: v1
kind: Service
metadata:
  name: nginx-service
  namespace: nginx
spec:
  selector:
    app: nginx
  ports:
    - name: http
      port: 80
      targetPort: 80
      protocol: TCP
  type: ClusterIP
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: nginx
  namespace: nginx
spec:
  selector:
    matchLabels:
      app: nginx
  serviceName: "nginx-service"
  replicas: 1
  template:
    metadata:
      labels:
        app: nginx
    spec:
      terminationGracePeriodSeconds: 10
      containers:
      - name: nginx
        image: k8s.gcr.io/nginx-slim:0.8
        ports:
        - containerPort: 80
          name: nginx
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: nginx
  namespace: nginx
spec:
  secretName: nginx
  issuerRef:
    name: letsencrypt-production
    kind: ClusterIssuer
  dnsNames:
    - 'nginx.k8slabs.com'
---
apiVersion: projectcontour.io/v1
kind: HTTPProxy
metadata:
  annotations:
  labels:
    app: nginx
  name: nginx-httpproxy
  namespace: nginx
spec:
  routes:
  - conditions:
    - prefix: /
    pathRewritePolicy:
      replacePrefix:
      - prefix: /
        replacement: /
    services:
    - name: nginx-service
      port: 80
  virtualhost:
    fqdn: nginx.k8slabs.com
    tls:
      secretName: nginx

Visual View

Running Kubernetes Dashboard with signed certificates

In a previous post I went through how to deploy the Kubernetes Dashboard into a Kubernetes cluster with default settings, running with a self-signed certificate. This post covers how to update the configuration to use a signed certificate. I’m a fan of Let’s Encrypt so will be using a signed wildcard certificate from Let’s Encrypt for this post.

In a previous post I went through how to deploy the Kubernetes Dashboard into a Kubernetes cluster with default settings, running with a self-signed certificate. This post covers how to update the configuration to use a signed certificate. I’m a fan of Let’s Encrypt so will be using a signed wildcard certificate from Let’s Encrypt for this post.

You can prepare Let’s Encrypt by referring to a previous post here.

Step 1. Create a new namespace

Create a new namespace for Kubernetes Dashboard

kubectl create ns kubernetes-dashboard

Step 2. Upload certificates

Upload your certificate and private key to $HOME/certs in pem format. Let’s Encrypt just happens to issue certificates in pem format with the following names:

cert.pem and privkey.pem

All we need to do is to rename these to:

tls.crt and tls.key

And then upload them to $HOME/certs where our kubectl tool is installed.

Step 3. Create secret

Create the secret for the custom certificate by running this command.

kubectl create secret generic kubernetes-dashboard-certs --from-file=$HOME/certs -n kubernetes-dashboard

You can check that that secret is ready by issuing the following command:

kubectl describe secret -n kubernetes-dashboard kubernetes-dashboard-certs
Name:         kubernetes-dashboard-certs
Namespace:    kubernetes-dashboard
Labels:       <none>
Annotations:  <none>

Type:  Opaque

Data
====
tls.key:  1705 bytes
tls.crt:  1835 bytes

Step 4. Edit the deployment

We need to download the deployment yaml and then edit it to ensure that it uses the Let’s Encrypt signed certificates.

Run the following command to download the Kubernetes Dashboard deployment yaml file

wget https://raw.githubusercontent.com/kubernetes/dashboard/v2.5.0/aio/deploy/recommended.yaml

Now edit it by using your favorite editor and add in the following two lines

        - --tls-cert-file=/tls.crt
        - --tls-key-file=/tls.key

under the following

Deployment – kubernetes-dashboard – spec.template.spec.containers.args

kind: Deployment
apiVersion: apps/v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
spec:
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      k8s-app: kubernetes-dashboard
  template:
    metadata:
      labels:
        k8s-app: kubernetes-dashboard
    spec:
      securityContext:
        seccompProfile:
          type: RuntimeDefault
      containers:
        - name: kubernetes-dashboard
          image: kubernetesui/dashboard:v2.5.0
          imagePullPolicy: Always
          ports:
            - containerPort: 8443
              protocol: TCP
          args:
            - --tls-cert-file=/tls.crt
            - --tls-key-file=/tls.key
            - --auto-generate-certificates

Step 5. Expose Kubernetes Dashboard using a load balancer

Let’s expose the app using a load balancer, I’m using NSX ALB (Avi) but the code below can be used with any load balancer.

Continue editing the recommended.yaml file with the following contents from line 32:

kind: Service
apiVersion: v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
spec:
  ports:
    - port: 443
      targetPort: 8443
  selector:
    k8s-app: kubernetes-dashboard
  type: LoadBalancer

Save changes to the file. Now we’re ready to deploy.

If you want to use the Avi Services API (K8s Gateway API). Then add labels to the service, like this. This will ensure that the service uses the Avi gateway.

kind: Service
apiVersion: v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
    ako.vmware.com/gateway-name: gateway-tkg-workload-vip
    ako.vmware.com/gateway-namespace: default
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
spec:
  ports:
    - port: 443
      targetPort: 8443
  selector:
    k8s-app: kubernetes-dashboard
  type: LoadBalancer

Step 6. Deploy Kubernetes Dashboard

Deploy the app with the following command:

kubectl apply -f recommended.yaml

Step 7. Get full access to the cluster for Kubernetes Dashboard

To get full cluster access to the kubernetes-dashboard account run the following

kubectl create clusterrolebinding add-on-cluster-admin --clusterrole=cluster-admin --serviceaccount=kubernetes-dashboard:kubernetes-dashboard

Step 8. Obtain the login token

To login we’ll need to obtain a token with the following code:

kubectl describe -n kubernetes-dashboard secret kubernetes-dashboard-token

Copy just the token and paste it into the browser to login. Enjoy a secure connection to Kubernetes Dashboard. Enjoy!