harbor – VMwire

Intro

The Harbor registry that comes shipped with TKG 1.5.1 uses Tanzu Packages to deploy Harbor into a TKG cluster. This version of Harbor does not support Helm Charts using ChartMuseum. VMware dropped support for ChartMuseum in TKG and are adopting OCI registries instead. This post describes how to deploy the upstream Harbor distribution that supports ChartMuseum for a helm repository. Follow this other post here to deploy Harbor with Tanzu Packages (Kapp) with support for OCI.

The example below uses the following components:

TKG 1.5.1

AKO 1.6.1

Contour 1.18.2

Helm 3.8.0

Use the previous post to deploy the per-requisites.

Step 2 – Edit the values.yaml file

You only need to change the following lines.

Line Number	Specification
5	loadBalancer or ingress (contour)
13	use TLS certificate
30 & 35	secret name (created in Step 3.)
38 & 39	FQDN of your harbor and notary DNS A records
215, 221 etc	A storage class if you don’t have a default storage class. Leave blank to use your default storage class.
355	admin password

expose:
  # Set the way how to expose the service. Set the type as "ingress",
  # "clusterIP", "nodePort" or "loadBalancer" and fill the information
  # in the corresponding section
  type: ingress
  tls:
    # Enable the tls or not.
    # Delete the "ssl-redirect" annotations in "expose.ingress.annotations" when TLS is disabled and "expose.type" is "ingress"
    # Note: if the "expose.type" is "ingress" and the tls
    # is disabled, the port must be included in the command when pull/push
    # images. Refer to https://github.com/goharbor/harbor/issues/5291
    # for the detail.
    enabled: true
    # The source of the tls certificate. Set it as "auto", "secret"
    # or "none" and fill the information in the corresponding section
    # 1) auto: generate the tls certificate automatically
    # 2) secret: read the tls certificate from the specified secret.
    # The tls certificate can be generated manually or by cert manager
    # 3) none: configure no tls certificate for the ingress. If the default
    # tls certificate is configured in the ingress controller, choose this option
    certSource: secret
    auto:
      # The common name used to generate the certificate, it's necessary
      # when the type isn't "ingress"
      commonName: ""
    secret:
      # The name of secret which contains keys named:
      # "tls.crt" - the certificate
      # "tls.key" - the private key
      secretName: "harbor-cert"
      # The name of secret which contains keys named:
      # "tls.crt" - the certificate
      # "tls.key" - the private key
      # Only needed when the "expose.type" is "ingress".
      notarySecretName: "harbor-cert"
  ingress:
    hosts:
      core: harbor.vmwire.com
      notary: notary.harbor.vmwire.com
   
---snipped---

Step 3 – Create a TLS secret for ingress

Copy the tls.crt and tls.key files in pem format to $HOME/certs/

# Create a new namespace for harbor k create ns harbor # Create a TLS secret with the contents of tls.key and tls.crt in the harbor namespace kubectl create secret tls harbor-cert --key $HOME/certs/tls.key --cert $HOME/certs/tls.crt -n harbor

This post describes how to setup Harbor to run on a standalone VM. There are times when you want to do this, such as occasions where your environment does not have internet access or you want to have a local repository running close to your environment.

I found that I was running a lot of TKG deployments against TKG staging builds in my lab and wanted to speed up cluster creation times, so building a local Harbor repository would make things a bit quicker and more reliable.

This post describes how you can setup a Harbor repository on a Photon VM.

Step 1: Setup a static IP

See the documentation https://vmware.github.io/photon/assets/files/html/3.0/photon_admin/setting-a-static-ip-address.html, and https://vmware.github.io/photon/assets/files/html/3.0/photon_admin/adding-a-dns-server.html

vi /etc/systemd/network/10-static-en.network

chmod 644 /etc/systemd/network/10-static-en.network
systemctl restart systemd-networkd

vi /etc/hostname

reboot

Step 2: Enable pings to the VM

iptables -A INPUT -p ICMP -j ACCEPT
iptables -A OUTPUT -p ICMP -j ACCEPT

Step 3: Update Photon repositories and perform updates

cd /etc/yum.repos.d/
sed  -i 's/dl.bintray.com\/vmware/packages.vmware.com\/photon\/$releasever/g' photon.repo photon-updates.repo photon-extras.repo photon-debuginfo.repo
tdnf --assumeyes update
tdnf updateinfo
tdnf -y distro-sync
tdnf install -y bindutils tar parted
reboot

Step 4: Install docker-compose

curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose
docker-compose --version
systemctl start docker
systemctl enable docker
docker version

Step 5: Add a data disk for Harbor

Add another vmdk file to the VM then run the below

fdisk -l
parted /dev/sdb mklabel gpt mkpart ext4 0% 100%
mkfs -t ext4 /dev/sdb1
mkdir /data
vim /etc/fstab

Append the following line to the end of the file

/dev/sdb1 /data ext4 defaults 0 0

mount /data
df -h

Step 6: Setup Harbor

mkdir -p /harbor /etc/docker/certs.d/harbor.vmwire.com
cd /harbor
curl -sLO https://github.com/goharbor/harbor/releases/download/v2.4.1/harbor-offline-installer-v2.4.1.tgz
tar xvf harbor-offline-installer-v2.4.1.tgz --strip-components=1

Step 7: Prepare SSL certificates

I use Let’s Encrypt and have the following three files renamed from the original Let’s Encrypt filenames:

harbor.cert

harbor_key.key and

ca.crt

harbor.cert is the wildcard certificate issued for my domain by Let’s Encrypt, it is originally named cert.pem.

harbor_key.key is orginally named privkey.pem.

ca.crt is chain.pem.

Copy all three certificate files to /etc/docker/certs.d/harbor.vmwire.com

cp harbor.cert harbor_key.key ca.crt /etc/docker/certs.d/harbor.vmwire.com/

Step 8: Edit the harbor.yml file

# Configuration file of Harbor

# The IP address or hostname to access admin UI and registry service.
# DO NOT use localhost or 127.0.0.1, because Harbor needs to be accessed by external clients.
hostname: harbor.vmwire.com

# http related config
http:
  # port for http, default is 80. If https enabled, this port will redirect to https port
  port: 80

# https related config
https:
  # https port for harbor, default is 443
  port: 443
  # The path of cert and key files for nginx
  certificate: /etc/docker/certs.d/harbor.vmwire.com/harbor.cert
  private_key: /etc/docker/certs.d/harbor.vmwire.com/harbor_key.key

[snipped]

Update line 5 with your harbor instance’s FQDN.

Update lines 17 and 18 with the certificate and private key.

You can leave all the other lines on default.

Install Harbor with the following command:

./install.sh

Check to see if services are running

docker-compose ps

Step 9: Add harbor FQDN to your DNS servers and connect to Harbor.

To upgrade, download the new offline installer and run

install.sh

Tag: harbor

Deploy Harbor Registry with Helm and expose with Ingress