Playing with Tanzu – persistent volume claims, deployments & services

Learning the k8s ropes…

This is not a how to article to get vSphere with Tanzu up and running, there are plenty of guides out there, here and here. This post is more of a “lets have some fun with Kubernetes now that I have a vSphere with Tanzu cluster to play with“.

Answering the following question would be a good start to get to grips with understanding Kubernetes from a VMware perspective.

How do I do things that I did in the past in a VM but now do it with Kubernetes in a container context instead?

For example building the certbot application in a container instead of a VM.

Lets try to create an Ubuntu deployment that deploys one Ubuntu container into a vSphere Pod with persistent storage and a load balancer service from NSX-T to get to the /bin/bash shell of the deployed container.

Let’s go!

I created two yaml files for this, accessible from Github. You can read up on what these objects are here.

FilenameWhats it for?What does it do?Github link
certbot-deployment.yamlk8s deployment specificationDeploys one ubuntu pod, claims a 16Gb volume and mounts it to /dev/sdb and creates a load balancer to enable remote management with ssh.ubuntu-deployment.yaml
certbot-pvc.yamlpersistent volume claim specificationCreates a persistent volume of 16Gb size from the underlying vSphere storage class named tanzu-demo-storage.
The PVC is then consumed by the deployment.
ubuntu-pvc.yaml
Table 1. The only two files that you need.

Here’s the certbot-deployment.yaml file that shows the required fields and object spec for a Kubernetes Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: certbot
spec:
  replicas: 1
  selector:
    matchLabels:
      app: certbot
  template:
    metadata:
      labels:
        app: certbot
    spec:
      volumes:
      - name: certbot-storage
        persistentVolumeClaim:
         claimName: ubuntu-pvc
      containers:
      - name: ubuntu
        image: ubuntu:latest
        command: ["/bin/sleep", "3650d"]
        imagePullPolicy: IfNotPresent
        volumeMounts:
        - mountPath: "/mnt/sdb"
          name: certbot-storage
      restartPolicy: Always
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: certbot
  name: certbot
spec:
  ports:
  - port: 22
    protocol: TCP
    targetPort: 22
  selector:
    app: certbot
  sessionAffinity: None
  type: LoadBalancer

Here’s the certbot-pvc.yaml file that shows the required fields and object spec for a Kubernetes Persistent Volume Claim.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: certbot-pvc
  labels:
    storage-tier: tanzu-demo-storage
    availability-zone: home
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: tanzu-demo-storage
  resources:
    requests:
        storage: 16Gi

First deploy the PVC claim with this command:

kubectl apply -f certbot-pvc.yaml

Then deploy the deployment with this command:

kubectl.exe apply -f certbot-deployment.yaml

Magic happens and you can monitor the vSphere client and kubectl for status. Here are a couple of screenshots to show you whats happening.

kubectl describe deployment certbot
Name:                   certbot
Namespace:              new
CreationTimestamp:      Thu, 11 Mar 2021 23:40:25 +0200
Labels:                 <none>
Annotations:            deployment.kubernetes.io/revision: 1
Selector:               app=certbot
Replicas:               1 desired | 1 updated | 1 total | 1 available | 0 unavailable
StrategyType:           RollingUpdate
MinReadySeconds:        0
RollingUpdateStrategy:  25% max unavailable, 25% max surge
Pod Template:
  Labels:  app=certbot
  Containers:
   ubuntu:
    Image:      ubuntu:latest
    Port:       <none>
    Host Port:  <none>
    Command:
      /bin/sleep
      3650d
    Environment:  <none>
    Mounts:
      /mnt/sdb from certbot-storage (rw)
  Volumes:
   certbot-storage:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  certbot-pvc
    ReadOnly:   false
Conditions:
  Type           Status  Reason
  ----           ------  ------
  Available      True    MinimumReplicasAvailable
  Progressing    True    NewReplicaSetAvailable
OldReplicaSets:  <none>
NewReplicaSet:   certbot-68b4747476 (1/1 replicas created)
Events:
  Type    Reason             Age   From                   Message
  ----    ------             ----  ----                   -------
  Normal  ScalingReplicaSet  44m   deployment-controller  Scaled up replica set certbot-68b4747476 to 1
kubectl describe pvc
Name:          certbot-pvc
Namespace:     new
StorageClass:  tanzu-demo-storage
Status:        Bound
Volume:        pvc-418a0d4a-f4a6-4aef-a82d-1809dacc9892
Labels:        availability-zone=home
               storage-tier=tanzu-demo-storage
Annotations:   pv.kubernetes.io/bind-completed: yes
               pv.kubernetes.io/bound-by-controller: yes
               volume.beta.kubernetes.io/storage-provisioner: csi.vsphere.vmware.com
               volumehealth.storage.kubernetes.io/health: accessible
Finalizers:    [kubernetes.io/pvc-protection]
Capacity:      16Gi
Access Modes:  RWO
VolumeMode:    Filesystem
Mounted By:    certbot-68b4747476-pq5j2
Events:        <none>
kubectl get deployments
NAME        READY   UP-TO-DATE   AVAILABLE   AGE
certbot     1/1     1            1           47m


kubectl get pods
NAME                         READY   STATUS    RESTARTS   AGE
certbot-68b4747476-pq5j2     1/1     Running   0          47m


kubectl get pvc
NAME          STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS         AGE
certbot-pvc   Bound    pvc-418a0d4a-f4a6-4aef-a82d-1809dacc9892   16Gi       RWO            tanzu-demo-storage   84m

Let’s log into our pod, note the name from the kubectl get pods command above.

certbot-68b4747476-pq5j2

Its not yet possible to log into the pod using SSH since this is a fresh container that does not have SSH installed, lets log in first using kubectl and install SSH.

kubectl exec --stdin --tty certbot-68b4747476-pq5j2 -- /bin/bash

You will then be inside the container at the /bin/bash prompt.

root@certbot-68b4747476-pq5j2:/# ls
bin   dev  home  lib32  libx32  mnt  proc  run   srv  tmp  var
boot  etc  lib   lib64  media   opt  root  sbin  sys  usr
root@certbot-68b4747476-pq5j2:/#

Lets install some tools and configure ssh.

apt-get update
apt-get install iputils-ping
apt-get install ssh

passwd root

service ssh restart

exit

Before we can log into the container over an SSH connection, we need to find out what the external IP is for the SSH service that the NSX-T load balancer configured for the deployment. You can find this using the command:

kubectl get services

kubectl get services
NAME        TYPE           CLUSTER-IP   EXTERNAL-IP   PORT(S)        AGE
certbot     LoadBalancer   10.96.0.44   172.16.2.3    22:31731/TCP   51m

The IP that we use to get to the Ubuntu container over SSH is 172.16.2.3. Lets try that with a putty/terminal session…

login as: root
certbot@172.16.2.3's password:
Welcome to Ubuntu 20.04.2 LTS (GNU/Linux 4.19.126-1.ph3-esx x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

This system has been minimized by removing packages and content that are
not required on a system that users do not log into.

To restore this content, you can run the 'unminimize' command.

The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.


The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.

$ ls
bin   dev  home  lib32  libx32  mnt  proc  run   srv  tmp  var
boot  etc  lib   lib64  media   opt  root  sbin  sys  usr
$ df
Filesystem     1K-blocks   Used Available Use% Mounted on
overlay           258724 185032     73692  72% /
/mnt/sdb        16382844  45084  16321376   1% /mnt/sdb
tmpfs             249688     12    249676   1% /run/secrets/kubernetes.io/servic                         eaccount
/dev/sda          258724 185032     73692  72% /dev/termination-log
$

You can see that there is a 16Gb mount point at /mnt/sdb just as we specified in the specifications and remote SSH access is working.

Published by Hugo Phan

@hugophan

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: