Quick Start Guide

The following steps provide a quick-start guide to installing KeySafe 5 and its dependencies into an existing Kubernetes cluster.

These steps install KeySafe 5 and its dependencies. They should be followed to set up a demo environment for evaluation purposes and should not be used for production environments.

For a production deployment you must perform the following actions:

Secure the connections to MongoDB and RabbitMQ
Require authentication to access KeySafe 5
Configure HTTPS for the KeySafe 5 endpoints
Use your secure CA for certificates

Unpack the release

$ mkdir keysafe5-install
$ tar -xf nshield-keysafe5-1.0.0.tar.gz -C keysafe5-install
$ cd keysafe5-install

# Load the Docker images to your local Docker
$ docker load < docker-images/hsm-mgmt.tar
$ docker load < docker-images/sw-mgmt.tar
$ docker load < docker-images/ui.tar

# We need to use a private registry in many places
$ export DOCKER=private.registry.local

# Tag the Docker images for a private registry
$ docker tag hsm-mgmt:1.0.0 $DOCKER/keysafe5/hsm-mgmt:1.0.0
$ docker tag sw-mgmt:1.0.0 $DOCKER/keysafe5/sw-mgmt:1.0.0
$ docker tag mgmt-ui:1.0.0 $DOCKER/keysafe5/mgmt-ui:1.0.0

# Log in to ensure pushes succeed
$ docker login $DOCKER

# And push
$ docker push $DOCKER/keysafe5/hsm-mgmt:1.0.0
$ docker push $DOCKER/keysafe5/sw-mgmt:1.0.0
$ docker push $DOCKER/keysafe5/mgmt-ui:1.0.0

Set up a CA

You should use your existing CA for a production system. This is simply used an example for the purposes of having a working demo system.

The CA we will create is based on OpenSSL 1.1.1 - and is run inside a directory of your choosing, but in the examples here we will use /home/user/keysafe5-install/demoCA. In that directory, create the file demoCA.conf with the contents:

[ ca ]
default_ca      = CA_default                          # The default ca section

[ CA_default ]

dir             = /home/user/keysafe5-install/demoCA  # The directory of the CA
database        = $dir/index.txt                      # index file.
new_certs_dir   = $dir/newcerts                       # new certs dir

certificate     = $dir/cacert.pem                     # The CA cert
serial          = $dir/serial                         # serial no file
#rand_serial    = yes                                 # for random serial#'s
private_key     = $dir/private/cakey.pem              # CA private key
RANDFILE        = $dir/private/.rand                  # random number file

default_days    = 15                                  # how long to certify for
default_crl_days= 5                                   # how long before next CRL
default_md      = sha256                              # Message Digest
policy          = test_root_ca_policy
x509_extensions = certificate_extensions
unique_subject  = no
# This copy_extensions setting should not be used in a production system.
# It is simply used to simplify the demo system.
copy_extensions = copy

[ test_root_ca_policy ]
commonName = supplied
stateOrProvinceName = optional
countryName = optional
emailAddress = optional
organizationName = optional
organizationalUnitName = optional
domainComponent = optional

[ certificate_extensions ]
basicConstraints = CA:false

[ req ]
default_bits       = 4096
default_md         = sha256
prompt             = yes
distinguished_name = root_ca_distinguished_name
x509_extensions    = root_ca_extensions

[ root_ca_distinguished_name ]
commonName = hostname

[ root_ca_extensions ]
basicConstraints       = CA:true
keyUsage               = keyCertSign, cRLSign
subjectKeyIdentifier   = hash
authorityKeyIdentifier = keyid:always,issuer
basicConstraints       = critical,CA:true

Remember to update the dir value to the directory in which the demoCA.conf and the other CA files will be stored.

To generate the long-term CA key and random number source, we first make a directory called private:

$ mkdir ~/keysafe5-install/demoCA/private

Then run:

$ openssl genrsa -out ~/keysafe5-install/demoCA/private/cakey.pem 4096
$ openssl rand -out ~/keysafe5-install/demoCA/private/.rand 1024

The CA needs a self-signed certificate; as this is a short-term demo it will be valid for 6 months:

$ openssl req -x509 -new -nodes \
  -key demoCA/private/cakey.pem \
  -subj "/CN=demoCA" -days 183 \
  -out demoCA/cacert.pem \
  -config demoCA/demoCA.conf
$ cp demoCA/cacert.pem ca.crt

And finally, to finish off the configuration:

$ mkdir demoCA/newcerts
$ echo 01 > demoCA/serial
$ touch demoCA/index.txt

Install and set up the supporting software

Istio

$ istioctl install -y

RabbitMQ

We recommend that you use your standard secure RabbitMQ installation, along with your policies for authentication and virtual hosts on your production system; this is only a demo system.

First, we need to generate the TLS keys, and guest password. We need to add the network addresses through which RabbitMQ will be accessed to the certificate, and are very dependent on the configuration of the Kubernetes cluster.

$ openssl genrsa -out ~/keysafe5-install/rabbit.key 4096
$ export DNS1="*.rabbit-chart-rabbitmq-headless.rabbitns.svc.cluster.local"
$ export DNS2=rabbit-chart-rabbitmq.rabbitns.svc
$ export DNS3=rabbitmq.rabbitns.svc.cluster.local
$ export DNS4=host.docker.internal
$ export LOCALIP=127.0.0.1
$ export HOSTIP=$(hostname -I | cut -f1 -d" ")
$ openssl req -new -key ~/keysafe5-install/rabbit.key \
  -out ~/keysafe5-install/rabbitmq.csr -subj \
  "/CN=rabbitmq/C=GB/L=Cambridge" \
  -addext "keyUsage=digitalSignature" \
  -addext "extendedKeyUsage=serverAuth" \
  -addext "subjectAltName=DNS:rabbitmq,DNS:${DNS1},DNS:${DNS2},DNS:${DNS3},DNS:${DNS4},DNS:${HOSTNAME},IP:${LOCALIP},IP:${HOSTIP}"
$ openssl ca -config ~/keysafe5-install/demoCA/demoCA.conf \
  -out rabbit.crt \
  -in rabbitmq.csr -batch
$ rm rabbitmq.csr
$ kubectl create namespace rabbitns
$ kubectl create secret generic rabbitmq-certificates \
  --namespace=rabbitns \
  --from-file=ca.crt \
  --from-file=tls.crt=rabbit.crt \
  --from-file=tls.key=rabbit.key
$ kubectl -n rabbitns create secret generic rabbitmq-pw \
  --from-literal=rabbitmq-password=guest

Then install RabbitMQ.

$ helm repo add bitnami https://charts.bitnami.com/bitnami && helm repo update
$ helm install rabbit-chart \
  --set image.tag=3.9.13 \
  --set auth.username=guest \
  --set auth.existingPasswordSecret=rabbitmq-pw \
  --set auth.tls.enabled=true \
  --set auth.tls.existingSecret=rabbitmq-certificates \
  --set extraConfiguration='
    listeners.ssl.default = 5671
    ssl_options.versions.1 = tlsv1.3
    ssl_options.depth = 0
    ssl_options.verify = verify_peer
    auth_mechanisms.1 = EXTERNAL
    ssl_cert_login_from = subject_alternative_name
    ssl_cert_login_san_type = dns
    ssl_cert_login_san_index = 0' \
  --wait --timeout 5m --namespace=rabbitns bitnami/rabbitmq --version 8.32.1

You will be given a url that may be used from within the cluster. As we will use the same cluster to install KeySafe 5 we keep that stored in a variable to be used later. We have added ${HOSTNAME} to the DNS as an externally accessible address for use by the KeySafe 5 agent later.

$ export RABBIT_URL=rabbit-chart-rabbitmq.rabbitns.svc:5671

Add the virtual host that will be used for KeySafe 5 communication.

$ export RUN_RABBIT="kubectl -n rabbitns exec rabbit-chart-rabbitmq-0 -c rabbitmq -- "
$ export RABBIT_VHOST=nshieldvhost
$ ${RUN_RABBIT} rabbitmqctl add_vhost ${RABBIT_VHOST}

Then add the x509 users for KeySafe 5, enable x509 authentication, then make RabbitMQ accessible from outside the cluster. It’s also a good idea to set up the x509 keys and certificates for KeySafe 5 and its agents.

$ export AGENT_USER=keysafe5-agent
$ export KS5_USER=ks5
$ for x509user in $AGENT_USER $KS5_USER
  do
    ${RUN_RABBIT} rabbitmqctl add_user $x509user "ephemeralpw"
    ${RUN_RABBIT} rabbitmqctl set_permissions -p $RABBIT_VHOST $x509user ".*" ".*" ".*"
    ${RUN_RABBIT} rabbitmqctl clear_password $x509user
    openssl genrsa -out $x509user.key 4096
    openssl req -new -key $x509user.key -out $x509user.csr \
      -subj "/CN=${x509user}/C=GB/L=Cambridge" \
      -addext "keyUsage=digitalSignature" \
      -addext "extendedKeyUsage=clientAuth" \
      -addext "subjectAltName=DNS:${x509user}"
    openssl ca -config ~/keysafe5-install/demoCA/demoCA.conf \
      -out ${x509user}.crt -in ${x509user}.csr -batch
    rm ${x509user}.csr
  done
$ kubectl create namespace nshieldkeysafe5
$ kubectl create secret generic ks5-amqptls \
  --namespace nshieldkeysafe5 \
  --from-file=ca.crt \
  --from-file=tls.crt=ks5.crt \
  --from-file=tls.key=ks5.key
$ ${RUN_RABBIT} /opt/bitnami/rabbitmq/sbin/rabbitmq-plugins enable \
  rabbitmq_auth_mechanism_ssl
$ kubectl port-forward --address 0.0.0.0 --namespace rabbitns \
  svc/rabbit-chart-rabbitmq 5671:5671 > k8s-rabbitmq-portfoward.log 2>&1 &

Note that we need to keep a set of AMQP TLS credentials for the KeySafe 5 agent.

$ tar -zcf ~/keysafe5-install/agent-amqptls.tar.gz \
  keysafe5-agent.key keysafe5-agent.crt ca.crt

MongoDB

We recommend that you use your standard secure MongoDB Replica Set installation. This is just an example, and not production-ready.

$ kubectl create namespace mongons
$ helm install mongo-chart \
  --set image.tag=4.4.12-debian-10-r38 \
  --set architecture=replicaset \
  --set auth.enabled=true \
  --set auth.username=dummyuser \
  --set auth.password=dummypassword \
  --set auth.database=authdb \
  --set tls.enabled=true \
  --namespace=mongons \
  bitnami/mongodb --version 11.1.10

There will be a message listing the MongoDB server addresses. We save them to a variable for use later.

$ export MONGO1=mongo-chart-mongodb-0.mongo-chart-mongodb-headless.mongons.svc.cluster.local:27017
$ export MONGO2=mongo-chart-mongodb-1.mongo-chart-mongodb-headless.mongons.svc.cluster.local:27017
$ export MONGODB=${MONGO1},${MONGO2}

We then pick up the secrets in the MongoDB configuration

$ kubectl get secret --namespace mongons mongo-chart-mongodb-ca \
  -o jsonpath="{.data.client-pem}" | base64 --decode | \
  openssl pkey -out mongo-client-key.pem
$ kubectl get secret --namespace mongons mongo-chart-mongodb-ca \
  -o jsonpath="{.data.client-pem}" | base64 --decode | \
  openssl x509 -out mongo-client-cert.pem
$ kubectl get secret --namespace mongons mongo-chart-mongodb-ca \
  -o jsonpath="{.data.mongodb-ca-cert}" | base64 --decode > mongo-ca-cert.pem

We add those secrets in a format that KeySafe 5 can accept.

$ kubectl create secret generic mongodb-client-tls \
  --namespace=nshieldkeysafe5 \
  --from-file=ca.crt=mongo-ca-cert.pem \
  --from-file=tls.crt=mongo-client-cert.pem \
  --from-file=tls.key=mongo-client-key.pem
$ rm mongo-ca-cert.pem mongo-client-key.pem

Access the MongoDB shell to create a user with read/write permissions on the hsm-mgmt-db and sw-mgmt-db collections. Note that the username needs to match the subject of the client certificate, as found by the command:

$ openssl x509 -in mongo-client-cert.pem -subject | head -n 1

In this example we will use mongo-chart-mongodb.mongons.svc.cluster.local

To run the mongo client container:

$ export MONGO_RUN="kubectl -n mongons exec mongo-chart-mongodb-0 0 -- "
$ export TLS_PRIVKEY="$(${MONGO_RUN} bash -c 'cat /certs/mongodb.pem')"
$ export TLS_CERT="$(${MONGO_RUN} bash -c 'cat /certs/mongodb-ca-cert')"
$ export MONGODB_ROOT_PASSWORD=$(kubectl get secret --namespace mongons \
  mongo-chart-mongodb -o jsonpath="{.data.mongodb-root-password}" \
  | base64 --decode)
$ kubectl run --namespace mongons mongo-chart-mongodb-client \
  --rm --tty -i --restart='Never' --env="MONGODB_ROOT_PASSWORD=$MONGODB_ROOT_PASSWORD" \
  --env="TLS_PRIVKEY=$TLS_PRIVKEY" --env="TLS_CERT=$TLS_CERT" --env="MONGODB=$MONGODB" \
  --image bitnami/mongodb:4.4.12-debian-10-r38 --command -- bash

Once inside the mongo client container, we need to set up a connection to the server before we can start mongo admin and create the user. After we finish creating the user, we need to exit mongo admin, and then the mongo-client container.

$ echo "$TLS_CERT" > /tmp/tls.crt
$ echo "$TLS_PRIVKEY" > /tmp/tls.key
$ mongo admin --tls --tlsCAFile /tmp/tls.crt --tlsCertificateKeyFile /tmp/tls.key \
  --host $MONGODB --authenticationDatabase admin -u root -p $MONGODB_ROOT_PASSWORD

> use $external
> x509_user = {
   "user" : "CN=mongo-chart-mongodb.mongons.svc.cluster.local",
   "roles" : [
     {"role": "readWrite", "db": "hsm-mgmt-db" },
     {"role": "readWrite", "db": "sw-mgmt-db" },
   ]
 }
> db.createUser(x509_user)
> exit
$ exit

Install KeySafe 5

# Get Ingress IP address
$ export INGRESS_IP=$(kubectl --namespace istio-system get svc -l app=istio-ingressgateway -o jsonpath='{.items[0].status.loadBalancer.ingress[0].ip}')

# Install the KeySafe 5 backend services
$ helm install keysafe5-backend \
  --namespace=nshieldkeysafe5 \
  --set hsm_mgmt.image=$DOCKER/keysafe5/hsm-mgmt:1.0.0 \
  --set sw_mgmt.image=$DOCKER/keysafe5/sw-mgmt:1.0.0 \
  --set database.type=mongo \
  --set database.mongo.hosts="$MONGO1\,$MONGO2" \
  --set database.mongo.replicaSet=rs0 \
  --set database.mongo.auth.type=tls \
  --set database.mongo.auth.authDatabase=authdb \
  --set database.mongo.tls.enabled=true \
  --set database.mongo.tls.existingSecret=mongodb-client-tls \
  --set amqp.URL=${RABBIT_URL}/${RABBIT_VHOST} \
  --set amqp.auth.type=tls \
  --set amqp.tls.enabled=true \
  --set amqp.tls.existingSecret=ks5-amqptls \
  --wait --timeout 10m \
 helm-charts/nshield-keysafe5-backend-1.0.0.tgz

# Install the KeySafe 5 UI
$ helm install keysafe5-ui \
  --namespace=nshieldkeysafe5 \
  --set ui.image=$DOCKER/keysafe5/mgmt-ui:1.0.0 \
  --set svcEndpoint="https://${HOSTNAME}" \
  --set authMethod=none \
  --wait --timeout 10m \
helm-charts/nshield-keysafe5-ui-1.0.0.tgz

# Create the TLS secret for the Istio Ingress Gateway
$ openssl genrsa -out istio.key 4096
$ openssl req -new -key istio.key -out istio.csr \
  -subj "/CN=${HOSTNAME}" \
  -addext "keyUsage=digitalSignature" \
  -addext "extendedKeyUsage=serverAuth" \
  -addext "subjectAltName=DNS:${HOSTNAME},IP:${INGRESS_IP}"
$ openssl ca -config ~/keysafe5-install/demoCA/demoCA.conf \
  -out istio.crt -in istio.csr -batch
$ kubectl -n istio-system create secret tls \
  keysafe5-server-credential --cert=istio.crt --key=istio.key

# Configure Istio Ingress Gateway for KeySafe 5
$ helm install keysafe5-istio \
  --namespace=nshieldkeysafe5 \
  --set tls.existingSecret=keysafe5-server-credential \
  --set requireAuthn=false \
  --wait --timeout 1m \
helm-charts/nshield-keysafe5-istio-1.0.0.tgz

Access KeySafe 5

You can now access KeySafe 5 on either https://$INGRESS_IP or https://$HOSTNAME. For example, you could send curl requests:

$ curl -X GET --cacert demoCA/cacert.pem https://${HOSTNAME}/mgmt/v1/hsms | jq
$ curl -X GET --cacert demoCA/cacert.pem https://${INGRESS_IP}/mgmt/v1/pools | jq
$ curl -X GET --cacert demoCA/cacert.pem https://${HOSTNAME}/mgmt/v1/worlds | jq

You can access the Management UI in a web browser at https://$HOSTNAME

Configuring nShield client machines

To configure a host machine to be managed and monitored by this deployment, run the KeySafe 5 agent binary on the nShield client machine containing the relevant Security World or HSMs.

Configure this KeySafe 5 agent to communicate with the same AMQP service as the previous deployment.

You will need to copy the agent-amqptls.tar.gz created above.

Ensure no firewall rules are blocking the AMQP port communication between the machine exposing the AMQP port from Kubernetes and the machine running the agent.

$ sudo tar -xf keysafe5-install/keysafe5-agent/keysafe5-1.0.0-Linux-keysafe5-agent.tar.gz -C /
$ export KS5CONF=/opt/nfast/keysafe5/conf
$ cat << EOF | sudo tee $KS5CONF/config.yaml
  logging:
    level: info
    format: json
    file:
      enabled: true
      path: /opt/nfast/log/keysafe5-agent.log

  amqp:
    url: "${INGRESS_IP}:5671/${RABBIT_VHOST}"
    auth_type: "tls"
  update_interval: 15s
  health_interval: 15s
EOF

$ sudo mkdir -p $KS5CONF/amqp/tls
$ sudo tar xf agent-amqptls.tar.gz -C $KS5CONF/amqp/tls
$ sudo mv $KS5CONF/amqp/tls/keysafe5-agent.key $KS5CONF/amqp/tls/tls.key
$ sudo mv $KS5CONF/amqp/tls/keysafe5-agent.crt $KS5CONF/amqp/tls/tls.crt

$ sudo /opt/nfast/sbin/install

Uninstall

helm --namespace nshieldkeysafe5 uninstall keysafe5-istio
helm --namespace nshieldkeysafe5 uninstall keysafe5-backend
helm --namespace nshieldkeysafe5 uninstall keysafe5-ui
helm --namespace rabbitns uninstall rabbit-chart
helm --namespace mongons uninstall mongo-chart