Installing Contour and Envoy with Container Service Extension and VMware Cloud Director

This post summarizes how to get Contour and Envoy up and running with Kubernetes clusters running in VMware Cloud Director.

Pre-requisites

A Kubernetes cluster deployed by Container Service Extension in VCD
NSX Advanced Load Balancer setup for the Kubernetes cluster
A VMware Cloud Director deployment

Step 1. Upload an SSL certificate for Contour to VCD

Obtain the cluster ID from Kubernetes Container Clusters, copy the entire Cluster ID.

Navigate to Certificate Management, Certificates Library, and click on Import

I used a Let’s Encrypt signed certificate. I wrote about using Let’s Encrypt in a previous post.

For the friendly name, paste in the Cluster ID and append to the end “-cert”

Continue the wizard by uploading the certificate and the private key for the certificate.

Step 2. Install Helm

helm repo add bitnami https://charts.bitnami.com/bitnami
helm repo update
helm fetch bitnami/contour

tar xvf contour-<version>.tgz

Step 3. Running Envoy as a non-root user

Envoy is configured to run as a non-root user by default. This is much more secure but we won’t be able to use any ports that are lower than 1024. Therefore we must change the values.yaml file for contour.

Edit the values.yaml file located in the directory that you untar the tkz file into and search for

envoy.containerPorts.http

Change the http port to 8080 and the https port to 8443.

It should end up looking like this:

  containerPorts:
    http: 8080
    https: 8443

Step 4. Installing Contour (and Envoy)

Install Contour by running the following command

helm install ingress <path-to-contour-directory>

You should get one daemonset named ingress-contour-envoy and deployment named ingress-contour-contour. These spin up two pods.

You will also see two services starting, one called ingress-contour with a service type of ClusterIP and another called ingress-contour-envoy with a service type LoadBalancer. Wait for NSX ALB to assign an external IP for the envoy service from your Organization network IP pool.

This IP is now your Kubernetes cluster IP for ingress services. Make a note of this IP address. My example uses 10.149.1.116 as the external IP.

Step 5. Setup DNS

The next step to do is to setup DNS, I’m using Windows DNS in my lab so what I’ve done is setup a sub domain called apps.vmwire.com and also setup an A record pointing to *.apps.vmwire.com.

*.apps.vmwire.com 10.149.1.116

DNS is now setup to point *.apps.vmwire.com to the external IP assigned to Envoy. From this point forward, any DNS request that hits *.apps.vmwire.com will be redirected to Contour.

Testing ingress with some apps

Download the following files from my Github.

https://raw.githubusercontent.com/hugopow/cse/main/shapes.yaml

https://raw.githubusercontent.com/hugopow/cse/main/shapes-ingress.yaml

They are two yaml files that deploys a sample web application and then exposes the applications using Contour and Envoy.

You don’t have to edit the shapes.yaml file, but you will need to edit the shapes-ingress.yaml file and change lines 9 and 16 to your desired DNS settings.

In this example, Contour will use circles.apps.vmwire.com to expose the circles application and triangles.apps.vmwire.com to expose the triangles application. Note that we are not adding circles. or triangles. A records into the DNS server.

Lets deploy the circles and triangles apps.

kubectl apply -f shapes.yaml

And then expose the applications with Contour

kubectl apply -f shapes-ingress.yaml

Now open up a web browser and navigate to http://circles.<your-domain> or http://triangles.<your-domain> and see the apps being exposed by Contour. If you don’t get a connection, its probably because you haven’t enabled port 80 through your Edge Gateway.