Manual Install

The following steps provide a step-by-step guide to installing KeySafe 5 and its dependencies into an existing Kubernetes cluster.

An alternative to this guide is the Demo Deployment Script which provides a scripted means of installing KeySafe 5.

These steps install KeySafe 5 and its dependencies. They should be followed to set up a demo environment for evaluation purposes and should not be used for production environments.

Please see Hardening The Deployment for steps to harden the deployment. Entrust recommends these steps as a minimum and that additional hardening may be required dependent on your own requirements.

A production deployment will have as a minimum the following:

  • Maintained and patched versions of all the dependencies.

  • A secure CA with TLS v1.3 support for certificates. The deploy script can provide a local insecure CA.

  • A secure Kubernetes installation.

  • A secure MongoDB database. The deploy script can provide a replicated MongoDB with X.509 authentication running in Kubernetes.

  • A secure means of large object storage. The deploy script can provide local object storage within the Kubernetes cluster or be configured for using an NFS for object storage.

  • HTTPS secured by a trusted certificate for the KeySafe 5 endpoints. The deploy script will enable HTTPS connections with a self-signed insecure certificate.

  • Require authentication to access KeySafe 5. OIDC & OAUTH2 are currently supported in KeySafe 5. The deploy script will not set up authenticated access.

This set of commands are an example of how to install KeySafe 5. They may need modification to suit your environment.

Unpack the release

mkdir ~/keysafe5-1.7.0
tar -xf nshield-keysafe5-1.7.0.tar.gz -C ~/keysafe5-1.7.0
cd ~/keysafe5-1.7.0/keysafe5-k8s

Docker images

The Docker images need to be loaded onto a Docker registry that each node in your Kubernetes cluster can pull the images from.

  1. Load the Docker images to your local Docker, for example:

    docker load < docker-images/agent-mgmt.tar
docker load < docker-images/codesafe-mgmt.tar
docker load < docker-images/hsm-mgmt.tar
docker load < docker-images/sw-mgmt.tar
docker load < docker-images/ui.tar
docker load < docker-images/licence-mgmt.tar
docker load < docker-images/monitoring-mgmt.tar
docker load < docker-images/alert-manager-sidecar.tar
docker load < docker-images/prometheus.tar
docker load < docker-images/alertmanager.tar

  2. Set the DOCKER_REGISTRY variable to the registry in use, for example:

    export DOCKER_REGISTRY=localhost:5000
    If you are using a single-machine Kubernetes distribution like K3s, you may be able to create a simple unauthenticated local private Docker registry by following the instructions in Distribution Registry. However this registry is only accessible by setting the name to localhost which will not work for other configurations.

  3. Log in to the registry to ensure that you can push to it:

    docker login $DOCKER_REGISTRY

  4. Tag the Docker images for the registry, for example:

    docker tag agent-mgmt:1.7.0 $DOCKER_REGISTRY/keysafe5/agent-mgmt:1.7.0
docker tag codesafe-mgmt:1.7.0 $DOCKER_REGISTRY/keysafe5/codesafe-mgmt:1.7.0
docker tag hsm-mgmt:1.7.0 $DOCKER_REGISTRY/keysafe5/hsm-mgmt:1.7.0
docker tag mgmt-ui:1.7.0 $DOCKER_REGISTRY/keysafe5/mgmt-ui:1.7.0
docker tag sw-mgmt:1.7.0 $DOCKER_REGISTRY/keysafe5/sw-mgmt:1.7.0
docker tag licence-mgmt:1.7.0 "$DOCKER_REGISTRY"/keysafe5/licence-mgmt:1.7.0
docker tag monitoring-mgmt:1.7.0 "$DOCKER_REGISTRY"/keysafe5/monitoring-mgmt:1.7.0
docker tag alert-manager-sidecar:1.7.0 "$DOCKER_REGISTRY"/keysafe5/alert-manager-sidecar:1.7.0
docker tag prometheus:v3.5.1 "$DOCKER_REGISTRY"/keysafe5/prometheus:v3.5.1
docker tag alertmanager:v0.31.1 "$DOCKER_REGISTRY"/keysafe5/alertmanager:v0.31.1

  5. Push the KeySafe 5 images to the registry, for example:

    docker push $DOCKER_REGISTRY/keysafe5/agent-mgmt:1.7.0
docker push $DOCKER_REGISTRY/keysafe5/hsm-mgmt:1.7.0
docker push $DOCKER_REGISTRY/keysafe5/codesafe-mgmt:1.7.0
docker push $DOCKER_REGISTRY/keysafe5/mgmt-ui:1.7.0
docker push $DOCKER_REGISTRY/keysafe5/sw-mgmt:1.7.0
docker push $DOCKER_REGISTRY/keysafe5/licence-mgmt:1.7.0
docker push $DOCKER_REGISTRY/keysafe5/monitoring-mgmt:1.7.0
docker push $DOCKER_REGISTRY/keysafe5/alert-manager-sidecar:1.7.0
docker push $DOCKER_REGISTRY/keysafe5/prometheus:v3.5.1
docker push $DOCKER_REGISTRY/keysafe5/alertmanager:v0.31.1

Set up a Certificate Authority

You should use your existing CA for a production system. This is simply used as an example for the purposes of having a working demo system.

Either OpenSSL 3.0 or OpenSSL 1.1.1 may be used to create the CA, and the CA may be created in a directory of your choosing. In these examples, /home/user/keysafe5-1.7.0/keysafe5-k8s/internalCA is the example directory used. In that directory, create the file internalCA.conf with the contents:

[ ca ]
default_ca      = CA_default                          # The default ca section

[ CA_default ]

dir             = /home/user/keysafe5-1.7.0/keysafe5-k8s/internalCA  # The directory of the CA
database        = $dir/index.txt                      # index file.
new_certs_dir   = $dir/newcerts                       # new certs dir

certificate     = $dir/cacert.pem                     # The CA cert
serial          = $dir/serial                         # serial no file
#rand_serial    = yes                                 # for random serial#'s
private_key     = $dir/private/cakey.pem              # CA private key
RANDFILE        = $dir/private/.rand                  # random number file

default_days    = 15                                  # how long to certify for
default_crl_days= 5                                   # how long before next CRL
default_md      = sha256                              # Message Digest
policy          = test_root_ca_policy
x509_extensions = certificate_extensions
unique_subject  = no
# This copy_extensions setting should not be used in a production system.
# It is simply used to simplify the demo system.
copy_extensions = copy

[ test_root_ca_policy ]
commonName = supplied
stateOrProvinceName = optional
countryName = optional
emailAddress = optional
organizationName = optional
organizationalUnitName = optional
domainComponent = optional

[ certificate_extensions ]
basicConstraints = CA:false

[ req ]
default_bits       = 4096
default_md         = sha256
prompt             = yes
distinguished_name = root_ca_distinguished_name
x509_extensions    = root_ca_extensions

[ root_ca_distinguished_name ]
commonName = hostname

[ root_ca_extensions ]
basicConstraints       = CA:true
keyUsage               = keyCertSign, cRLSign
subjectKeyIdentifier   = hash
authorityKeyIdentifier = keyid:always,issuer
basicConstraints       = critical,CA:true

Remember to update the dir value to the directory in which the internalCA.conf and the other CA files will be stored. The certificates generated, unless overridden on the command line, will be valid for 15 days as specified in default_days.

To generate the long-term CA key and random number source, create a directory called private, then place them in that directory:

mkdir ~/keysafe5-1.7.0/keysafe5-k8s/internalCA/private
openssl genrsa -out ~/keysafe5-1.7.0/keysafe5-k8s/internalCA/private/cakey.pem 4096
openssl rand -out ~/keysafe5-1.7.0/keysafe5-k8s/internalCA/private/.rand 1024

The CA needs a self-signed certificate; as this is a short-term demo it will be valid for 90 days:

cd ~/keysafe5-install/
openssl req -x509 -new -nodes \
  -key internalCA/private/cakey.pem \
  -subj "/CN=internalCA" -days 90 \
  -out internalCA/cacert.pem \
  -config internalCA/internalCA.conf
cp internalCA/cacert.pem ca.crt

And finally, to finish off the configuration:

mkdir internalCA/newcerts
echo 01 > internalCA/serial
touch internalCA/index.txt

Install and set up the supporting software

Kubernetes namespace

Create a namespace in Kubernetes for KeySafe 5 installation.

kubectl create namespace nshieldkeysafe5

Istio

These instructions assume that only Istio will be used for ingress, and no other ingress controller is installed.

If Istio is not already installed, you may install a version aligned with the software version of istioctl with:

istioctl install -y

MongoDB

Entrust recommends that you use your standard secure MongoDB Replica Set installation.

From your existing set up we need to set up the tables and create a user with permissions for these tables. Ensure that a user can be created with the permissions to access your MongoDB installation.

Access Mongosh and at the command prompt enter these database commands to create the tables.

db.createRole(
  {
    role: "hsm-mgmt-db-user",
    privileges: [
        {
          "resource": {"db": "hsm-mgmt-db", "collection": ""},
          "actions": ["createIndex", "find", "insert", "remove", "update"]
        },
      ],
    roles: []
  }
)
db.createRole(
  {
    role: "sw-mgmt-db-user",
    privileges: [
        {
          "resource": {"db": "sw-mgmt-db", "collection": ""},
          "actions": ["createIndex", "dropCollection", "find", "insert", "remove", "update"]
        },
      ],
    roles: []
  }
)
db.createRole(
  {
    role: "codesafe-mgmt-db-user",
    privileges: [
        {
          "resource": {"db": "codesafe-mgmt-db", "collection": ""},
          "actions": ["createIndex", "find", "insert", "remove", "update"]
        },
      ],
    roles: []
  }
)
db.createRole(
  {
    role: "agent-mgmt-db-user",
    privileges: [
        {
          "resource": {"db": "agent-mgmt-db", "collection": ""},
          "actions": ["createIndex", "dropCollection", "find", "insert", "remove", "update"]
        },
      ],
    roles: []
  }
)
db.createRole(
  {
    role: "licence-mgmt-db-user",
    privileges: [
        {
          "resource": {"db": "licence-mgmt-db", "collection": ""},
          "actions": ["createIndex", "find", "insert", "remove", "update"]
        },
      ],
    roles: []
  }
)
db.createRole(
  {
    role: "monitoring-mgmt-db-user",
    privileges: [
        {
          "resource": {"db": "monitoring-mgmt-db", "collection": ""},
          "actions": ["createIndex", "find", "insert", "remove", "update"]
        },
      ],
    roles: []
  }
)

We now need to create the user with access to the database. Replace CN=ks5-mongo-user with the name of the user you want to use.

use $external
x509_user = {
  user : "CN=ks5-mongo-user",
  roles : [
    {"role": "agent-mgmt-db-user", "db": "admin" },
    {"role": "codesafe-mgmt-db-user", "db": "admin" },
    {"role": "hsm-mgmt-db-user", "db": "admin" },
    {"role": "licence-mgmt-db-user", "db": "admin" },
    {"role": "monitoring-mgmt-db-user", "db": "admin" },
    {"role": "sw-mgmt-db-user", "db": "admin" },
  ]
}
db.createUser(x509_user)

Type exit to exit the mongosh prompt.

When installing KeySafe 5 make sure to change the variables within the install script to reflect your Mongo deployment. Make sure to set the MONGOHOSTS and the MONGOSECRET environment variables.

Object Storage

For large object storage, create a Persistent Volume Claim, in the nshieldkeysafe5 Kubernetes namespace (the same namespace that we will deploy the application to).

Cluster-local Object Storage

If your Kubernetes cluster only has 1 worker node, you can choose to use local storage.

cat << EOF | kubectl -n nshieldkeysafe5 apply -f -
  apiVersion: v1
  kind: PersistentVolumeClaim
  metadata:
    name: data-nshield-keysafe5
  spec:
    accessModes:
      - ReadWriteOnce
    storageClassName: local-path
    resources:
      requests:
        storage: 2Gi
EOF

NFS Object Storage

If your Kubernetes cluster has more than 1 worker node, you must use a type of storage that supports distributed access, such as NFS. For details on creating a PVC for NFS object storage, please see NFS Object Storage Configuration.

Prometheus Database

Prometheus requires a persistent volume for its database and this must be created prior to installation of the Prometheus helm charts. This can only be created as local storage as NFS is not supported.

cat << EOF | kubectl -n nshieldkeysafe5 apply -f -
  apiVersion: v1
  kind: PersistentVolumeClaim
  metadata:
    name: prometheus-data-keysafe5
  spec:
    accessModes:
      - ReadWriteOnce
    storageClassName: local-path
    resources:
      requests:
        storage: 4Gi
EOF

Install KeySafe 5

Bringing all the secrets and URLs created above, install KeySafe 5.

The commands below assume that a login is not required to pull from the Docker Registry.

By default, this KeySafe 5 central platform deployment will only be able to communicate with version 1.5 or later KeySafe 5 Agents.

If you want your deployment to be able to communicate with legacy (1.4 or earlier) KeySafe 5 Agents then you must include the configuration --set messageBus.compatibilityMode=true when running the helm install command.

To send email notifications for alerts, an email server must be configured. The additional required configuration options need to be added to the instructions for installing the KeySafe 5 backend services below.

If authentication is required by the email server, then you must provide SMTP credentials as a Kubernetes Secret.

The SMTP server must support TLS and the server’s CA certificate must be in the Operating System’s trust store.

  --set monitoring_mgmt.alertmanager.email.smarthost=email.server.com:port \
  --set monitoring_mgmt.alertmanager.email.from=no-reply@yourdomain.com \
  --set monitoring_mgmt.alertmanager.email.auth.existingSecret=smtp-credentials-secret \
 
# Get Ingress IP address
export INGRESS_IP=$(kubectl --namespace istio-system get svc -l app=istio-ingressgateway -o jsonpath='{.items[0].status.loadBalancer.ingress[0].ip}')

# Install the KeySafe 5 backend services
helm install keysafe5-backend \
  --namespace=nshieldkeysafe5 \
  --set agent_mgmt.image=$DOCKER_REGISTRY/keysafe5/agent-mgmt:1.7.0 \
  --set codesafe_mgmt.image=$DOCKER_REGISTRY/keysafe5/codesafe-mgmt:1.7.0 \
  --set hsm_mgmt.image=$DOCKER_REGISTRY/keysafe5/hsm-mgmt:1.7.0 \
  --set licence_mgmt.image=$DOCKER_REGISTRY/keysafe5/licence-mgmt:1.7.0 \
  --set monitoring_mgmt.image=$DOCKER_REGISTRY/keysafe5/monitoring-mgmt:1.7.0 \
  --set sw_mgmt.image=$DOCKER_REGISTRY/keysafe5/sw-mgmt:1.7.0 \
  --set database.type=mongo \
  --set database.mongo.hosts="$MONGOHOSTS" \
  --set database.mongo.replicaSet=rs0 \
  --set database.mongo.auth.type=tls \
  --set database.mongo.auth.authDatabase=authdb \
  --set database.mongo.tls.enabled=true \
  --set database.mongo.tls.existingSecret=$MONGOSECRET \
  --set messageBus.auth.type=tls \
  --set messageBus.tls.enabled=true \
  --set messageBus.tls.serverTLS.existingSecret=agentcomms-server-certificates \
  --set messageBus.tls.existingSecret=agentcomms-client-certificates \
  --set objectStore.pvc=data-nshield-keysafe5 \
  --wait --timeout 10m \
  helm-charts/nshield-keysafe5-backend-1.7.0.tgz

# Install the KeySafe 5 Prometheus.
helm install keysafe5-prometheus \
  --namespace=nshieldkeysafe5 \
  --set HostIP=<YOUR_HOST_IP> \
  --set prometheus.image=localhost:5000/keysafe5/prometheus:v3.5.1 \
  --set prometheus.pvc=prometheus-data-keysafe5 \
  --set prometheus.sharedpvc=data-nshield-keysafe5 \
  --wait --timeout 3m \
  helm-charts/nshield-keysafe5-prometheus-1.7.0.tgz

# Install the KeySafe 5 Alertmanager.
helm install keysafe5-alertmanager\
  --namespace=nshieldkeysafe5 \
  --set HostIP=<YOUR_HOST_IP> \
  --set alertmanager.image=localhost:5000/keysafe5/alertmanager:v0.31.1 \
  --set alertmanager.sharedpvc=data-nshield-keysafe5 \
  --set sidecar.image=localhost:5000/keysafe5/alert-manager-sidecar:1.7.0 \
  --set sidecar.configPath=/etc/shared_volume/prometheus \
  --wait --timeout 3m \
  helm-charts/nshield-keysafe5-alertmanager-1.7.0.tgz

# Install the KeySafe 5 WebUI
helm install keysafe5-ui \
  --namespace=nshieldkeysafe5 \
  --set ui.image=$DOCKER_REGISTRY/keysafe5/mgmt-ui:1.7.0 \
  --set svcEndpoint="https://${INGRESS_IP}" \
  --set authMethod=none \
  --wait --timeout 10m \
  helm-charts/nshield-keysafe5-ui-1.7.0.tgz

# Create the TLS secret for the Istio Ingress Gateway
openssl genrsa -out istio.key 4096
openssl req -new -key istio.key -out istio.csr \
  -subj "/CN=${HOSTNAME}" \
  -addext "keyUsage=digitalSignature" \
  -addext "extendedKeyUsage=serverAuth" \
  -addext "subjectAltName=DNS:${HOSTNAME},IP:${INGRESS_IP}"
openssl ca -config ~/keysafe5-1.7.0/keysafe5-k8s/internalCA/internalCA.conf \
  -out istio.crt -in istio.csr -batch
kubectl -n istio-system create secret tls \
  keysafe5-server-credential --cert=istio.crt --key=istio.key

# Configure Istio Ingress Gateway for KeySafe 5
helm install keysafe5-istio \
  --namespace=nshieldkeysafe5 \
  --set tls.existingSecret=keysafe5-server-credential \
  --set requireAuthn=false \
  --wait --timeout 1m \
  helm-charts/nshield-keysafe5-istio-1.7.0.tgz

Access KeySafe 5

You can now access KeySafe 5 at https://${INGRESS_IP}.

For example, you could send curl requests as demonstrated below.

curl -X GET --cacert ca.crt https://${INGRESS_IP}/mgmt/v1/hsms | jq
curl -X GET --cacert ca.crt https://${INGRESS_IP}/mgmt/v1/hosts | jq
curl -X GET --cacert ca.crt https://${INGRESS_IP}/mgmt/v1/pools | jq
curl -X GET --cacert ca.crt https://${INGRESS_IP}/mgmt/v1/feature-certificates | jq
curl -X GET --cacert ca.crt https://${INGRESS_IP}/mgmt/v1/worlds | jq
curl -X GET --cacert ca.crt https://${INGRESS_IP}/codesafe/v1/images | jq
curl -X GET --cacert ca.crt https://${INGRESS_IP}/codesafe/v1/certificates | jq
curl -X GET --cacert ca.crt https://${INGRESS_IP}/licensing/v1/licences | jq
curl -X GET --cacert ca.crt https://${INGRESS_IP}/monitoring/v1/triggers | jq

You can access the Management UI in a web browser at https://${INGRESS_IP}.

Configure KeySafe 5 Agent machines

To configure a host machine to be managed and monitored by this deployment, run the KeySafe 5 agent binary on the KeySafe 5 Agent machine containing the relevant Security World or HSMs.

After copying over the agent tar file, extract it and start configuring: 
sudo tar -C / -xf keysafe5-1.7.0-Linux-keysafe5-agent.tar.gz
export KS5CONF=/opt/nfast/keysafe5/conf
sudo cp $KS5CONF/config.yaml.example $KS5CONF/config.yaml

Create the messagebus/tls directory and copy the ca.crt file copied from the keysafe5-1.7.0 directory on the demo machine into it.

mkdir -p $KS5CONF/messagebus/tls
cp ca.crt $KS5CONF/messagebus/tls/

Create the private key and a certificate signing request (CSR) for this specific KeySafe 5 agent.

sudo /opt/nfast/keysafe5/bin/ks5agenttls --keypath=$KS5CONF/messagebus/tls/tls.key --keygen
sudo /opt/nfast/keysafe5/bin/ks5agenttls --keypath=$KS5CONF/messagebus/tls/tls.key --csrgen

For this installation we copy the CSR to the demo machine, into the keysafe5-1.7.0 directory, then sign it using OpenSSL.

openssl ca -config ~/keysafe5-1.7.0/keysafe5-k8s/internalCA/internalCA.conf \
  -in ks5_demohost.csr \
  -out ks5_demohost.crt -batch

Transfer the resulting certificate ks5_demohost.crt to the nShield Agent machine at /opt/nfast/keysafe5/conf/messagebus/tls/tls.crt.

On the nShield Agent machine, if the hardserver is already running, use the KeySafe 5 install script to not restart it when installing the KeySafe 5 agent.

sudo /opt/nfast/keysafe5/sbin/install

Otherwise, use the nShield install script which will start both the nShield Security World software and the KeySafe 5 agent.

sudo /opt/nfast/sbin/install

Uninstall

helm --namespace nshieldkeysafe5 uninstall keysafe5-istio
helm --namespace nshieldkeysafe5 uninstall keysafe5-backend
helm --namespace nshieldkeysafe5 uninstall keysafe5-ui
helm --namespace nshieldkeysafe5 uninstall keysafe5-prometheus
helm --namespace nshieldkeysafe5 uninstall keysafe5-alertmanager
helm --namespace mongons uninstall mongo-chart