Tag: cloud director

Using Let’s Encrypt certificates with Cloud Director

Let’s Encrypt (LE) is a certificate authority that issues free SSL certificates for use in your web applications. This post details how to get LE setup to support Cloud Director specifically with a wildcard certificate.

Let’s Encrypt (LE) is a certificate authority that issues free SSL certificates for use in your web applications. This post details how to get LE setup to support Cloud Director specifically with a wildcard certificate.

Certbot

LE uses an application called certbot to request, automatically download and renew certificates. You can think of certbot as the client for LE.

First you’ll need to create a client machine that can request certificates from LE. I started with a simple CentOS VM. For more details about installing certbot into your preferred OS read this page here.

Once you get yours on the network with outbound internet access, you can start by performing the following.

 # Update software
 yum update
 
 # Install wget if not already installed
 yum install wget
 
 # Download the certbot application.
 wget https://dl.eff.org/certbot-auto
 
 # Move certbot into a local application directory
 sudo mv certbot-auto /usr/local/bin/certbot-auto
 
 # Set ownership to root
 sudo chown root /usr/local/bin/certbot-auto
 
 # Change permisssions for certbot
 sudo chmod 0755 /usr/local/bin/certbot-auto

Now you’re ready to request certificates. Run the following command but of course replacing your desired domain within the ‘your.domain.here ‘.

/usr/local/bin/certbot-auto --config-dir $HOME/.certbot --work-dir $HOME/.certbot/work --logs-dir $HOME/.certbot/logs  certonly --manual --preferred-challenges=dns -d '*.vmwire.com'

This will create a request for a wildcard certificate for *.vmwire.com.

You’ll then be asked to create a new DNS TXT record on your public DNS server for the domain that you are requesting to validate that you can manage that domain. Here’s what mine looks like for the above.

This means that you can only request public certificates with LE, private certificates are not supported.

You will then see a response from LE such as the following:

IMPORTANT NOTES:
 - Congratulations! Your certificate and chain have been saved at:
   /root/.certbot/live/vmwire.com/fullchain.pem
   Your key file has been saved at:
   /root/.certbot/live/vmwire.com/privkey.pem
   Your cert will expire on 2020-12-24. To obtain a new or tweaked
   version of this certificate in the future, simply run certbot-auto
   again. To non-interactively renew *all* of your certificates, run
   "certbot-auto renew"

Updating Cloud Director certificates

Before you can use new certificate, you need to perform some operations with the JAVA Keytool to import the pem formatted certificates into the certificates.ks file that Cloud Director uses.

The issued certificate is available in the directory

/root/.certbot/live/

Navigate to there using an SSH client and you’ll see a structure like this

Download the entire folder for the next steps. Within the folder you’ll see the following files

FilenamePurpose
cert.pemyour certificate in pem format
chain.pemthe Let’s Encrypt root CA certificate in pem format
fullchain.pemyour wildcard certificate AND the LE root CA certificate in pem format
privkey.pemthe private key for your certificate (without passphrase)

We need to rename the file to something that the JAVA Keytool can work with. I renamed mine to the following:

Original filenameNew Filename
cert.pemvmwire-com.crt
chain.pemvmwire-com-ca.crt
fullchain.pemnot needed
privkey.pemvmwire-com.key

Copy the three new files to one of the Cloud Director cells, use the /tmp directory.

Now launch an SSH session to one of the Cloud Director cells and perform the following.

# Import the certificate and the private key into a new pfx format certificate
openssl pkcs12 -export -out /tmp/vmwire-com.pfx -inkey /tmp/vmwire-com.key -in /tmp/vmwire-com.crt

# Create a new certificates.ks file and import the pfx formatted certificate
/opt/vmware/vcloud-director/jre/bin/keytool -keystore /tmp/certificates.ks -storepass Vmware1! -keypass Vmware1! -storetype JCEKS -importkeystore -srckeystore /tmp/vmwire-com.pfx -srcstorepass Vmware1!

# Change the alias for the first entry to be http
/opt/vmware/vcloud-director/jre/bin/keytool -keystore /tmp/certificates.ks -storetype JCEKS -changealias -alias 1 -destalias http -storepass Vmware1!

# Import the certificate again, this time creating alias 1 again (we will use the same wildcard certifiate for the consoleproxy)
/opt/vmware/vcloud-director/jre/bin/keytool -keystore /tmp/certificates.ks -storepass Vmware1! -keypass Vmware1! -storetype JCEKS -importkeystore -srckeystore /tmp/vmwire-com.pfx -srcstorepass Vmware1!

# Change the alias for the first entry to be consoleproxy
/opt/vmware/vcloud-director/jre/bin/keytool -keystore /tmp/certificates.ks -storetype JCEKS -changealias -alias 1 -destalias consoleproxy -storepass Vmware1!

# Import the root certificate into the certificates.ks file
/opt/vmware/vcloud-director/jre/bin/keytool -importcert -alias root -file /tmp/vmwire-com-ca.crt -storetype JCEKS -keystore /tmp/certificates.ks -storepass Vmware1!

# List all the entries, you should now see three, http, consoleproxy and root
/opt/vmware/vcloud-director/jre/bin/keytool  -list -keystore /tmp/certificates.ks -storetype JCEKS -storepass Vmware1!

# Stop the Cloud Director service on all cells
service vmware-vcd stop

# Make a backup of the current certificate
mv /opt/vmware/vcloud-director/certificates.ks /opt/vmware/vcloud-director/certificates.ks.old

# Copy the new certificate to the Cloud Director directory
cp /tmp/certificates.ks /opt/vmware/vcloud-director/

# List all the entries, you should now see three, http, consoleproxy and root
/opt/vmware/vcloud-director/jre/bin/keytool  -list -keystore /opt/vmware/vcloud-director/certificates.ks -storetype JCEKS -storepass Vmware1!

# Reconfigure the Cloud Director application to use the new certificate
/opt/vmware/vcloud-director/bin/configure

# Start the Cloud Director application
service vmware-vcd start

# Monitor startup logs
tail -f /opt/vmware/vcloud-director/logs/cell.log

Copy the certificates.ks file to the other cells and perform the configure on the other cells to update the certificates for all cells. Don’t forget to update the certificate on the load balancer too. This other post shows how to do it with the NSX-T load balancer.

Check out the new certificate at https://vcloud.vmwire.com/tenant/vmwire.

Protecting Cloud Director with NSX-T Load Balancer L7 HTTP Policies

Running Cloud Director (formerly vCloud Director) over the Internet has its benefits however opens up the portal to security risks. To prevent this, we can use the native load balancing capabilities of NSX-T to serve only HTTP access to the URIs that are required and preventing access to unnecessary URIs from the rest of the Internet.

Running Cloud Director (formerly vCloud Director) over the Internet has its benefits however opens up the portal to security risks. To prevent this, we can use the native load balancing capabilities of NSX-T to serve only HTTP access to the URIs that are required and preventing access to unnecessary URIs from the rest of the Internet.

An example of this is to disallow the /provider and /cloudapi/1.0.0/sessions/provider URIs as these are provider side administrator only URIs that a service provider uses to manage the cloud and should not be accessible from the Internet.

The other article that I wrote previously describes the safe URIs and unsafe URIs that can be exposed over the Internet, you can find that article here. That article discuss doing the L7 HTTP policies using Avi. This article will go through how you can achieve the same with the built in NSX-T load balancer.

This article assumes that you already have the Load Balancer configured with the Cloud Director Virtual Servers, Server Pools and HTTPS Profiles and Monitors already set up. If you need a guide on how to do this, then please visit Tomas Fojta’s article here.

The L7 HTTP rules can be set up under Load Balancing | Virtual Servers. Edit the Virtual Server rule for the Cloud Director service and open up the Load Balancer Rules section.

Click on the Set link next to HTTP Access Phase. I’ve already set mine up so you can see that I already have two rules. You should also end up with two rules once this is complete.

Go ahead and add a new rule with the Add Rule button.

The first rule we want to set up is to prevent access from the Internet to the /provider URI but allow an IP address or group of IP addresses to access the service for provider side administration, such as a management bastion host.

Set up you rule as follows:

What we are doing here is creating a condition that when the /provider URI is requested, we drop all incoming connections unless the connection is initiated from the management jump box, this jump box has an IP address of 10.37.5.30. The Negate option is enabled to achieve this. Think of negate as the opposite of the rule, so negate does not drop connections to /provider when the source IP address is 10.37.5.30.

Here’s the brief explanation from the official NSX-T 3.0 Administration Guide.

If negate is enabled, when Connection Drop is configured, all requests not
matching the specified match condition are dropped. Requests matching the
specified match condition are allowed.

Save this rule and lets setup another one to prevent access to the admin API. Setup this second rule as follows:

This time use /cloudapi/1.0.0/sessions/provider as the URI. Again, use the Negate option for your management IP address. Save your second rule and Apply all the changes.

Now you should be able to access /tenant URIs over the Internet but not the /provider URI. However, accessing the /provider URI from 10.37.5.30 (or whatever your equivalent is) will work.

Doing this with the API

Do a PUT against /policy/api/v1/infra/lb-virtual-servers/vcloud with the following.

(Note that the Terraform provider for NSX-T doesn’t support HTTP Access yet. So to automate, use the NSX-T API directly instead.)

{
  "enabled": true,
  "ip_address": "<IP_address_of_this_load_balancer>",
  "ports": [
    "443"
  ],
  "access_log_enabled": false,
  "lb_persistence_profile_path": "/infra/lb-persistence-profiles/default-source-ip-lb-persistence-profile",
  "lb_service_path": "/infra/lb-services/vcloud",
  "pool_path": "/infra/lb-pools/vcd-appliances",
  "application_profile_path": "/infra/lb-app-profiles/vcd-https",
  "client_ssl_profile_binding": {
    "ssl_profile_path": "/infra/lb-client-ssl-profiles/default-balanced-client-ssl-profile",
    "default_certificate_path": "/infra/certificates/my-signed-certificate",
    "client_auth": "IGNORE",
    "certificate_chain_depth": 3
  },
  "server_ssl_profile_binding": {
    "ssl_profile_path": "/infra/lb-server-ssl-profiles/default-balanced-server-ssl-profile",
    "server_auth": "IGNORE",
    "certificate_chain_depth": 3,
    "client_certificate_path": "/infra/certificates/my-signed-certificate"
  },
    "rules": [
    {
      "match_conditions": [
        {
          "uri": "/cloudapi/1.0.0/sessions/provider",
          "match_type": "CONTAINS",
          "case_sensitive": false,
          "type": "LBHttpRequestUriCondition",
          "inverse": false
        },
        {
          "source_address": "10.37.5.30",
          "type": "LBIpHeaderCondition",
          "inverse": true
        }
      ],
      "match_strategy": "ALL",
      "phase": "HTTP_ACCESS",
      "actions": [
        {
          "type": "LBConnectionDropAction"
        }
      ]
    },
    {
      "match_conditions": [
        {
          "uri": "/provider",
          "match_type": "EQUALS",
          "case_sensitive": false,
          "type": "LBHttpRequestUriCondition",
          "inverse": false
        },
        {
          "source_address": "10.37.5.30",
          "type": "LBIpHeaderCondition",
          "inverse": true
        }
      ],
      "match_strategy": "ALL",
      "phase": "HTTP_ACCESS",
      "actions": [
        {
          "type": "LBConnectionDropAction"
        }
      ]
    }
  ],
  "log_significant_event_only": false,
  "resource_type": "LBVirtualServer",
  "id": "vcloud",
  "display_name": "vcloud",
  "_revision": 1
}