Working with CodeSafe

CodeSafe applications

To run CodeSafe applications on your system, you must have enabled the Secure Execution Engine (SEE) by purchasing and enabling an appropriate SEE activation licence as described in Enabling optional features.

You must also have loaded a valid certificate, known as a CodeSafe developer ID certificate, that can be used to verify the signature of each CodeSafe application that you want to run. These certificates will be supplied by the developer of the CodeSafe application and should be delivered to you together with the application.

CodeSafe developer ID certificates have a limited lifetime. If a certificate expires, applications that are already running will be unaffected but it will not be possible to start an application until a new certificate is loaded.

For this reason you should keep track of the expiry dates of your certificates and request a new certificate from the CodeSafe developer before your certificate expires.

You can view the expiry date of certificates by using csadmin ids list and looking at the notAfter field for each certificate.

New certificates can be loaded whilst an application is running without interrupting service.

If you want to develop your own CodeSafe applications, you must also purchase the CodeSafe developer kit.

An SEE application is typically a standalone SEE machine that is loaded automatically by the hardserver (for example, a CodeSafe C application).

Check the documentation that your CodeSafe application vendor supplies for information about how to set up and use the application, as well as for any other installation and configuration information.

CodeSafe applications are standalone applications, but each CodeSafe C application can consist of multiple parts, and its installation can include several configuration steps. For instructions on installing and configuring each application, see your application vendor’s documentation.

CodeSafe setup for the nShield 5c

To use CodeSafe applications with the nShield 5c you must exchange launcher service keys between the client and the nShield 5c by following the procedures in this section.

To use multiple nShield 5c modules simultaneously from the same client machine, see Configure client to access multiple nShield 5c HSMs.

Initial Client Configuration: Retrieve or generate the client’s public key

If the client’s launcher key pair has already been generated, view the client’s key type and public key data by issuing the following command:

hsmadmin cs5 clientinfo

The following is an example of the output expected for clientinfo:

ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDmcFY2wxtNj+di478pvOK3czNeXJoqw+itu/Hw6KRcGjDO60Zqr8eaWJcDS79DHncvdUZv0ZPbLqTCnNjN0ECQ=

You must copy the key type public key. You will need these in a later step for setting the client info in the nShield 5c CLI.

If the client does not already have a launcher key pair, generate a new one on the client machine with the following command:

hsmadmin cs5 genclientinfo

And then view and copy the key type and public key as in the previous step.

Configure nShield5c for Client Authorization:

To configure the nShield 5c to deploy CodeSafe applications, you need access to its serial console. For more information, see Configuring an nShield HSM using the Serial Console.

Put the module into Maintenance mode (this will temporarily stop nCoreapi and launcher services):
```
(cli)maintmode enable
```
The module will not be able to perform any tasks and will appear unavailable until it is put back into Operational mode.
Remove any existing client information in the nShield 5c to ensure a clean setup:
```
(cli)cs5 clearclientinfo
```

Set the client’s public key for the launcher service by running the following command :

(cli)cs5 setclientinfo <key_type> <public_key>

Where <key_type> and <public_key> are the output from the hsmadmin cs5 clientinfo command that you copied earlier. For example:

(cli)cs5 setclientinfo ecdsa-sha2-nistp256 ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789ABCDEFGHIJKLMNOPQRSTUVWXY

Enable ports for Codesafe 5, using the nShield 5c’s IP address:
```
(cli)cs5 ports enable <nshield-5c-ip-address>
```
Where <nshield-5c-ip-address> is the address on the HSM’s network interface that CodeSafe 5 will access.

This prevents KeySafe 5 from being used until cs5 ports disable is run.
Retrieve all the information the client needs to connect to the remote launcher service (ESN, IP address, and launcher service public key):
```
(cli)cs5 getserverinfo
```
Copy the ESN, IP address, and launcher service public key for use in the final client configuration. The following is an example of the output expected for getserverinfo:
```
Codesafe 5 server info in format <ESN> <IP Addresss> <Port> <Key Type> <Public Key>
B0A7-B88C-D3F4 10.194.147.134 2205 ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOEJMTeBR1546vmQ4YwT7WGlUv57PyCIShw66c9BAuD+WP5lrE8yjwJR/aBo7hrBxV95Vlgi6+4Up4fDI0uqO6s=
```
Put the module back into Operational mode:
```
(cli)maintmode disable
```

Finalize the Client Configuration:

On the client machine, enroll the nShield 5c for CodeSafe so that the client can connect to the remote launcher service with the following command:

hsmadmin cs5 enroll <ESN> <IP_Address> <Port> <Key_Type> <Public_Key>

Where ESN, IP_Address, Port, Key_Type and Public_key is the server information in the output for the cs5 getserverinfo command that you copied earlier.

You can verify the setup by running the following command:

csadmin list

The following is an example of the output expected when running csadmin list with no SEE machines installed:

B0A7-B88C-D3F4        No SEE machines currently installed

The nShield 5c is now ready for use with CodeSafe. For more information about building, signing, and running SEE machines with CodeSafe, see The csadmin utility tool.

Configure client to access multiple nShield 5c HSMs

To use multiple nShield 5c HSMs from the same client you should configure them with the same launcher client key. To do this repeat the procedures in Configure nShield5c for Client Authorization: for each HSM using the same <key_type> and <public_key> obtained from cs5 clientinfo each time.

The information used in Finalize the Client Configuration: will be unique to each nShield 5c HSM and will be that returned by the cs5 getservinfo command for the individual HSM.

Removal of client configuration

When a client machine is finished with an nShield 5c, disconnect from it using:

hsmadmin cs5 unenroll <ESN>

Where <ESN> is the ESN of the HSM from which you wish to remove configuration.

Then disable ports for Codesafe 5:

(cli)cs5 ports disable

If you disable the ports and try to run csadmin list on the client you will notice that it will be unable to connect to the launcher.

The following is an example of the output expected when the ports are disabled:

% csadmin list
B0A7-B88C-D3F4        CONNECTION ERROR: Unable to connect to 'launcher': [Errno None] Unable to connect to port 2205 on 10.194.147.134
Failed to get SEE machines list for module(s): ['B0A7-B88C-D3F4']

The csadmin utility tool

The csadmin tool is used to manage CodeSafe 5 applications.

The csadmin tool covers CodeSafe application deployment from both the perspective of a CodeSafe application developer and a CodeSafe application user. The help text displays the complete set of commands available. This document lists the commands that are relevant to the CodeSafe application user.

You must be logged in as an Administrator or a user with local administrator rights to execute csadmin commands.

You must have /opt/nfast/bin (Linux) or NFAST_HOME (Windows) in your PATH environment variable to use csadmin.

To view the help text included here while using csadmin, run a command or sub-command with the -h|--help option.

The following examples use a Linux machine for the deployment of CodeSafe applications. The same commands can be applied to a Windows machine.

$ csadmin --help
usage: csadmin [-h] [-v] {image,load,start,stop,list,destroy,sshd,ids,log,config,stats} ...

Tool to manage nShield 5s CodeSafe.

positional arguments:
  {image,load,start,stop,list,destroy,sshd,ids,log,config,stats}
    image               Perform tasks related to loadable nShield 5s CodeSafe images
    load                Load a CodeSafe 5 image onto an nShield 5s HSM
    start               Start a previously loaded image on an nShield 5s HSM
    stop                Stop an SEE machine running on an nShield 5s HSM
    list                List the SEE machines loaded on an nShield 5s HSM
    destroy             Destroy an SEE machine loaded on an nShield 5s HSM
    sshd                Manage the SSH daemon dedicated to a specific SEE machine on an nShield 5s HSM
    ids                 Manage the CodeSafe Developer Authentication Certificates on an nShield 5s HSM
    log                 Manage the SEE machine logging state
    config              Manage the SEE machine configuration
    stats               Manage the SEE machine statistics

options:
  -h, --help            show this help message and exit
  -v, --version         Print csadmin version

Manage loadable CodeSafe images

CodeSafe 5 images have the extension .cs5 and are often referred to as "CS5 files". Operations such as generating, signing, and inspecting loadable CodeSafe images are handled by csadmin image:

$ csadmin image --help
usage: csadmin image [-h] {generate,sign,inspect} ...

positional arguments:
  {generate,sign,inspect}
    generate            generate a loadable CodeSafe image.
    sign                sign a loadable CodeSafe image. Note: images are signed with an HSM.
    inspect             inspect a loadable CodeSafe image's metadata.

options:
  -h, --help            show this help message and exit

Generate loadable images

Only applicable to CodeSafe developers. See the CodeSafe 5 Developer Guide

Inspect images

The csadmin tool provides the inspect operation so developers can view the details of a .cs5 file.

The inspect operation usage:

csadmin image inspect --help
usage: csadmin image inspect [-h] [--json] CS5FILE

positional arguments:
  CS5FILE     The cs5 file being inspected

optional arguments:
  -h, --help  show this help message and exit
  --json      Show details in json format

For example:

$ csadmin image inspect myapp.cs5
Type               codesafe-container
Platform           nShield5
PackageName        MyCodeSafeApp
Version            1.0
EntryPoint         /home/see/launch_see.sh
Format             1

Sign images

Only applicable to CodeSafe developers. See the CodeSafe 5 Developer Guide

Load a CodeSafe application

If you are using a third-party developed CodeSafe application, ensure that you trust the developer of the CodeSafe application, and verify that the image you are using has a genuine certificate issued by the trusted organization. You must have loaded a valid certificate prior to loading the application.

Loading a CodeSafe image onto the module creates the SEE container and the environment needed to run the application. This is done using csadmin load:

$ csadmin load --help

usage: csadmin load [-h] [--timeout TIMEOUT] [--esn ESN] [--verbose] [--json] CS5FILE

positional arguments:
  CS5FILE            The CodeSafe 5 image file to be loaded into the HSM(s).

optional arguments:
  -h, --help         show this help message and exit
  --timeout TIMEOUT  Time to wait for service response, in seconds. Default: 30s, must be between 3s and 120s
  --esn ESN          ESN of the HSM to load CodeSafe 5 image into
  --verbose          Print verbose logs
  --json             Show result output in json format

On success, the load sub-command returns the UUID of the new SEE container, for example:

$ csadmin load test_app_1.cs5

FEDC-BA09-8765        SUCCESS
UUID: c2e75658-04aa-4634-a7f8-fad3692dd9d7

$ csadmin load --json test_app_1.cs5
{
    "FEDC-BA09-8765": {
        "succeeded": true,
        "data": {
            "uuid": "fedcba09-8765-4321-1234-567890abcdef"
        }
    }
}

Start a CodeSafe application

Although multiple SEE machines can be loaded and started at the same time, only one can talk to the ncoreapi service.

A SEE machine will only start if a valid certificate is already loaded. Certificates have an expiry date and if the certificate has expired it will not be possible to start the SEE machine until a new certificate is loaded.

Starting an SEE machine previously loaded onto the HSM launches the container and executes the predetermined entry point. This is done using csadmin start:

$ csadmin start --help
usage: csadmin start [-h] [--timeout TIMEOUT] -u UUID [--esn ESN] [--verbose] [--json]

optional arguments:
  -h, --help            show this help message and exit
  --timeout TIMEOUT     Time to wait for service response, in seconds. Default: 30s, must be between 3s and 120s
  -u UUID, --uuid UUID  The UUID of the machine being started.
  --esn ESN             ESN of the HSM hosting the SEE machine to be started.
  --verbose             Print verbose logs
  --json                Show response in json format

The start command takes as main input the UUID of the machine to be started. On success, it returns the IP(v6) address of the container, for example:

$ csadmin start --uuid fedcba09-8765-4321-1234-567890abcdef
FEDC-BA09-8765        SUCCESS
IP ADDRESS: ffff::fff:ffff:ffff:ffff

$ csadmin start --uuid fedcba09-8765-4321-1234-567890abcdef --json
{
    "FEDC-BA09-8765": {
        "succeeded": true,
        "data": {
            "ip": "ffff::fff:ffff:ffff:ffff"
        }
    }
}

To set the container to start automatically at startup and when the nShield 5 is reset, see Use set to enable autostart.

Stop a CodeSafe application

Stopping an SEE machine previously started on the HSM will gracefully terminate its programs execution and stop the container. This is done using csadmin stop. To display the sub-command usage, execute it with the --help option as shown below:

$ csadmin stop --help
usage: csadmin stop [-h] [--timeout TIMEOUT] -u UUID [--esn ESN] [--verbose] [--json]

optional arguments:
  -h, --help            show this help message and exit
  --timeout TIMEOUT     Time to wait for service response, in seconds. Default: 30s, must be between 3s and 120s
  -u UUID, --uuid UUID  The UUID of the machine being stopped.
  --esn ESN             ESN of the HSM hosting the SEE machine to be stopped.
  --verbose             Print verbose logs
  --json                Show response in json format

The stop command takes as main input the UUID of the machine to be stopped, for example:

$ csadmin stop --uuid fedcba09-8765-4321-1234-567890abcdef
FEDC-BA09-8765        SUCCESS

$ csadmin stop --json --uuid fedcba09-8765-4321-1234-567890abcdef
{
    "FEDC-BA09-8765": {
        "succeeded": true
    }
}

Destroy a CodeSafe application

Destroying an SEE machine that is currently stopped on the HSM removes the application container and any user data created during the life of the machine. This is done using csadmin destroy:

$ csadmin destroy --help
usage: csadmin destroy [-h] [--timeout TIMEOUT] -u UUID [--esn ESN] [--verbose] [--json]

options:
  -h, --help            show this help message and exit
  --timeout TIMEOUT     Time to wait for service response, in seconds. Default: 30s, must be between 3s and 300s
  -u UUID, --uuid UUID  The UUID of the machine being destroyed.
  --esn ESN             ESN of the HSM hosting the SEE machine to be destroyed.
  --verbose             Print verbose logs
  --json                Show response in json format

List CodeSafe applications

csadmin list displays a list of the SEE machines currently loaded on the HSM:

$ csadmin list --help
usage: csadmin list [-h] [--timeout TIMEOUT] [--esn ESN] [--verbose] [--json]

optional arguments:
  -h, --help         show this help message and exit
  --timeout TIMEOUT  Time to wait for service response, in seconds. Default: 30s, must be between 3s and 120s
  --esn ESN          ESN of the HSM to retrieve SEE machines list for. Can be specified multiple times for multiple HSMs.
  --verbose          Print verbose logs
  --json             Show response in json format

For example:

$ csadmin list
FEDC-BA09-8765
UUID                                   State     Name
----------------------------------------------------------------------
12345678-90ab-cdef-fedc-ba0987654321    STOPPED   nShield Docs Server
fedcba09-8765-4321-1234-567890abcdef    RUNNING   CodeSafe App 1
09876543-21fe-dcba-abcd-ef1234567890    RUNNING   CodeSafe App 2

Configure CodeSafe 5

csadmin config allows the user to set and query the state of CodeSafe’s auto-start and logging features:

$ csadmin config --help
usage: csadmin config [-h] {list,get,set} ...

positional arguments:
  {list,get,set}
    list          List CodeSafe machine configuration.
    get           Get CodeSafe machine configuration.
    set           Set CodeSafe machine configuration.

options:
  -h, --help      show this help message and exit

configuration keys:
  "autostart": "enabled" or "disabled". Automatically start the SEE machine when the module resets. Disabled by default.
  "log": "enabled" or "disabled". Capture a container's SEE logs. Disabled by default.

You must use a positional argument (list, get, set) with the config command.

List machine configuration

csadmin config list lists the current configuration keys and values of a given machine:

$ csadmin config list --help
usage: csadmin config list [-h] -u UUID [--esn ESN] [--verbose] [--json] [--timeout TIMEOUT]

options:
  -h, --help            show this help message and exit
  -u UUID, --uuid UUID  The UUID of the machine.
  --esn ESN             ESN of the HSM hosting the SEE Machine.
  --verbose             Print verbose logs
  --json                Show response in json format
  --timeout TIMEOUT     Time to wait for service response, in seconds. Default: 30s, must be between 3s and 300s

For example:

$ csadmin config list --uuid fedcba09-8765-4321-1234-567890abcdef
FEDC-BA09-8765
        autostart             : disabled
        log                   : disabled

$ csadmin config list --json --uuid fedcba09-8765-4321-1234-567890abcdef
{
    "FEDC-BA09-8765": {
        "succeeded": true,
        "data": {
            "autostart": "disabled",
            "log": "disabled"
        }
    }
}

Get machine configuration

csadmin config get queries the current value of a given machine’s specific configuration key:

$ csadmin config get --help
usage: csadmin config get [-h] -u UUID [--esn ESN] [--verbose] [--json] [--timeout TIMEOUT] key

positional arguments:
  key                   Configuration key

options:
  -h, --help            show this help message and exit
  -u UUID, --uuid UUID  The UUID of the machine.
  --esn ESN             ESN of the HSM hosting the SEE Machine.
  --verbose             Print verbose logs
  --json                Show response in json format
  --timeout TIMEOUT     Time to wait for service response, in seconds. Default: 30s, must be between 3s and 300s

For example:

$ csadmin config get log --uuid fedcba09-8765-4321-1234-567890abcdef
FEDC-BA09-8765        disabled

$ csadmin config get log --json --uuid fedcba09-8765-4321-1234-567890abcdef
{
    "FEDC-BA09-8765": {
        "succeeded": true,
        "data": {
            "log": "disabled"
        }
    }
}

Set machine configuration

csadmin config set sets a given machine’s specified configuration key to the specified value:

$ csadmin config set --help
usage: csadmin config set [-h] -u UUID [--esn ESN] [--verbose] [--json] [--timeout TIMEOUT] key value

positional arguments:
  key                   Configuration key
  value                 Configuration value

options:
  -h, --help            show this help message and exit
  -u UUID, --uuid UUID  The UUID of the machine.
  --esn ESN             ESN of the HSM hosting the SEE Machine.
  --verbose             Print verbose logs
  --json                Show response in json format
  --timeout TIMEOUT     Time to wait for service response, in seconds. Default: 30s, must be between 3s and 300s

Use set to enable logging

$ csadmin config set log enabled --uuid fedcba09-8765-4321-1234-567890abcdef
FEDC-BA09-8765
        state                 : enabled

$ csadmin config set log enabled --json --uuid fedcba09-8765-4321-1234-567890abcdef
{
    "FEDC-BA09-8765": {
        "succeeded": true,
        "data": {
            "state": "enabled"
        }
    }
}

Use set to enable autostart

If Autostart is enabled the SEE machine will automatically start after a reset of the HSM.

Before starting, the system will check that a valid certificate matching the application has been loaded. If the certificate has expired the SEE machine will not start.

For this reason it is important that you check the expiry dates on your certificates and request new certificates from the CodeSafe developer for any certificates that are near their expiry date. You can check the expiry date on certificates by using csadmin ids list and looking for the notAfter field.

$ csadmin config set autostart enabled --uuid fedcba09-8765-4321-1234-567890abcdef
FEDC-BA09-8765
        state                 : enabled

$ csadmin config set autostart enabled --json --uuid fedcba09-8765-4321-1234-567890abcdef
{
    "FEDC-BA09-8765": {
        "succeeded": true,
        "data": {
            "state": "enabled"
        }
    }
}

Manage CodeSafe 5 logging

When logging is enabled (see Use set to enable logging), the standard output and standard error of all programs running on the container is saved into a log file that can be pulled and cleared by csadmin log:

$ csadmin log --help
usage: csadmin log [-h] {get,clear} ...

positional arguments:
  {get,clear}  manage a container's SEE logging operations (get/clear)
    get        Get a container's SEE log
    clear      Clear SEE logs on modules

options:
  -h, --help   show this help message and exit

You must use a positional argument (get, clear) with the csadmin log command.

Get CodeSafe 5 logs

To obtain CodeSafe 5 logs, use csadmin log get:

$ csadmin log get --help
usage: csadmin log get [-h] --uuid UUID [--json] [--timeout TIMEOUT] [--esn ESN] [--verbose]

options:
  -h, --help            show this help message and exit
  --uuid UUID, -u UUID  The UUID of the machine to get the log from
  --json                Show response in json format
  --timeout TIMEOUT     Time to wait for service response, in seconds. Default: 30s, must be between 3s and 300s
  --esn ESN             ESN of the HSM to manage SEE logging for.
  --verbose             Print verbose logs

For example:

$ csadmin log get --uuid fedcba09-8765-4321-1234-567890abcdef
FEDC-BA09-8765        SUCCESS
Success: Started ipcdaemon

$ csadmin log get --json --uuid fedcba09-8765-4321-1234-567890abcdef
{
    "FEDC-BA09-8765": {
        "succeeded": true,
        "data": {
            "contents": "Success: Started ipcdaemon\n"
        }
    }
}

The internal log file has a maximum size of 128KB. When this size is reached, the log is cleared and a new log is started. This deletes previous data from the log. There is no rotation of the log.

Clear CodeSafe 5 logs

To clear CodeSafe 5 logs, use csadmin log clear:

$ csadmin log clear --help
usage: csadmin log clear [-h] --uuid UUID [--json] [--timeout TIMEOUT] [--esn ESN] [--verbose]

options:
  -h, --help            show this help message and exit
  --uuid UUID, -u UUID  The UUID of the machine to clear/delete the log from
  --json                Show response in json format
  --timeout TIMEOUT     Time to wait for service response, in seconds. Default: 30s, must be between 3s and 300s
  --esn ESN             ESN of the HSM to manage SEE logging for.
  --verbose             Print verbose logs

For example:

$ csadmin log clear --uuid fedcba09-8765-4321-1234-567890abcdef
FEDC-BA09-8765        SUCCESS
log: log cleared

$ csadmin log clear --json --uuid fedcba09-8765-4321-1234-567890abcdef
{
    "FEDC-BA09-8765": {
        "succeeded": true,
        "data": {
            "state": "log cleared"
        }
    }
}
[source]

Obtain machine-specific statistics

csadmin stat provides a mechanism for obtaining runtime data for CodeSafe machines on the system. It can only obtain data from machines in the RUNNING state:

$ csadmin stats --help
usage: csadmin stats [-h] [--timeout TIMEOUT] [-u UUID] [--esn ESN] [--verbose] [--json]

options:
  -h, --help            show this help message and exit
  --timeout TIMEOUT     Time to wait for service response, in seconds. Default: 30s, must be between 3s and 300s
  -u UUID, --uuid UUID  The UUID of the machine to query stats from.
  --esn ESN             ESN of the HSM hosting the SEE machine to query stats from.
  --verbose             Print verbose logs
  --json                Show response in json format

When executed without arguments, the stats command displays the runtime data of every machine currently running on the system:

$ csadmin stats
FEDC-BA09-8765
UUID                                    State     Name                       IP Address                  PID         CPU           Memory          KMem              Link        Tx bytes     Rx bytes    Total bytes
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
12345678-90ab-cdef-fedc-ba0987654321    RUNNING   hello               ffff::fff:ffff:ffff:eeee           1393      0.49 seconds    1.20 MiB       848.00 KiB        vethM19WNi    1.12 KiB      8.04 KiB      9.15 KiB
fedcba09-8765-4321-1234-567890abcdef    RUNNING   hello               ffff::fff:ffff:ffff:ffff            1610      0.42 seconds    1.26 MiB       836.00 KiB        vethsE1Pv4    1.12 KiB      4.98 KiB      6.10 KiB

$ csadmin stats --json
{
    "FEDC-BA09-8765": {
        "succeeded": true,
        "data": {
            "cs_stats": {
                "12345678-90ab-cdef-fedc-ba0987654321": {
                    "Name": "12345678-90ab-cdef-fedc-ba0987654321",
                    "State": "RUNNING",
                    "PID": "1393",
                    "CPU use": "0.51 seconds",
                    "Memory use": "1.24 MiB",
                    "KMem use": "888.00 KiB",
                    "Friendly Name": "hello",
                    "IP": "ffff::fff:ffff:ffff:eeee",
                    "Logging enabled": "False",
                    "Link": {
                        "name": "vethM19WNi",
                        "TX bytes": "1.12 KiB",
                        "RX bytes": "8.30 KiB",
                        "Total bytes": "9.42 KiB"
                    }
                },
                "fedcba09-8765-4321-1234-567890abcdef": {
                    "Name": "fedcba09-8765-4321-1234-567890abcdef",
                    "State": "RUNNING",
                    "PID": "1610",
                    "CPU use": "0.44 seconds",
                    "Memory use": "1.24 MiB",
                    "KMem use": "864.00 KiB",
                    "Friendly Name": "hello",
                    "IP": "ffff::fff:ffff:ffff:ffff",
                    "Logging enabled": "False",
                    "Link": {
                        "name": "vethsE1Pv4",
                        "TX bytes": "1.12 KiB",
                        "RX bytes": "5.24 KiB",
                        "Total bytes": "6.36 KiB"
                    }
                }
            }
        }
    }
}

When executed with the --uuid|-u argument, the command only displays the data of the specified machine:

$ csadmin stats --uuid fedcba09-8765-4321-1234-567890abcdef
FEDC-BA09-8765
UUID                                    State     Name                       IP Address                  PID         CPU           Memory          KMem              Link        Tx bytes     Rx bytes    Total bytes
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
fedcba09-8765-4321-1234-567890abcdef    RUNNING   hello               ffff::fff:ffff:ffff:ffff            1610      0.48 seconds    1.21 MiB       716.00 KiB        vethsE1Pv4    1.12 KiB      5.76 KiB      6.88 KiB

$ csadmin stats --uuid fedcba09-8765-4321-1234-567890abcdef --json
{
    "FEDC-BA09-8765": {
        "succeeded": true,
        "data": {
            "cs_stats": {
                "fedcba09-8765-4321-1234-567890abcdef": {
                    "Name": "fedcba09-8765-4321-1234-567890abcdef",
                    "State": "RUNNING",
                    "PID": "1610",
                    "CPU use": "0.49 seconds",
                    "Memory use": "1.39 MiB",
                    "KMem use": "840.00 KiB",
                    "Friendly Name": "hello",
                    "IP": "ffff::fff:ffff:ffff:ffff",
                    "Logging enabled": "False",
                    "Link": {
                        "name": "vethsE1Pv4",
                        "TX bytes": "1.12 KiB",
                        "RX bytes": "6.03 KiB",
                        "Total bytes": "7.15 KiB"
                    }
                }
            }
        }
    }
}

Manage container SSHD

The container SSHD allows connections to be forwarded from the client machine to the container. Its keys and state can be queried and managed with csadmin sshd:

$ csadmin sshd --help
usage: csadmin sshd [-h] [--timeout TIMEOUT] [--esn ESN] {keys,state} ...

positional arguments:
  {keys,state}
    keys             manage a container's SSH daemon keys
    state            manage a container's SSH daemon state

optional arguments:
  -h, --help         show this help message and exit
  --timeout TIMEOUT  Time to wait for service response, in seconds. Default: 30s, must
                     be between 3s and 300s
  --esn ESN          ESN of the HSM to manage SSHD daemon for.

Manage SSHD keys

The public keys used to connect to the container SSHD can be managed with csadmin sshd keys:

$ csadmin sshd keys --help
usage: csadmin sshd keys [-h] {getserver,getclient,setclient} ...

positional arguments:
  {getserver,getclient,setclient}
    getserver           Obtain a container's SSH daemon keys
    getclient           Obtain a container's SSH daemon client keys
    setclient           Set a container's SSH daemon client keys

optional arguments:
  -h, --help            show this help message and exit

Retrieve keys

The SSHD’s public key and the client keys set by the user can be retrieved by the getserver and getclient sub-commands respectively:

$ csadmin sshd keys getserver -h
usage: csadmin sshd keys getserver [-h] --uuid UUID [--json]

optional arguments:
  -h, --help            show this help message and exit
  --uuid UUID, -u UUID  The UUID of the machine to retrieve SSH daemon keys for
  --json                Show response in json format

getserver returns the server public keys for the container specified by the given UUID. getclient returns the client public key set by the user.

For example, getserver produces the following output, shown in plain and json format:

$ csadmin sshd keys getserver -u fedcba09-8765-4321-1234-567890abcdef
FEDC-BA09-8765        SUCCESS
KEY_TYPE: ecdsa-sha2-nistp256
KEY_BODY: AAAA.....................w4wtS4=

$ csadmin sshd keys getserver -u fedcba09-8765-4321-1234-567890abcdef --json
{
    "FEDC-BA09-8765": {
        "succeeded": true,
        "data": {
            "server_key": {
                "key_type": "ecdsa-sha2-nistp256",
                "key_body": "AAAA.....................w4wtS4="
            }
        }
    }
}

Set client keys

The client public key that is used to connect to the SSHD server on the HSM is set by the user with csadmin sshd keys setclient:

$ csadmin sshd keys setclient --help
usage: csadmin sshd keys setclient [-h] --uuid UUID
                                   (--keyfile KEYFILE | --keytype KEYTYPE KEYBODY)

optional arguments:
  -h, --help            show this help message and exit
  --uuid UUID, -u UUID  The UUID of the machine to set the SSH daemon keys for
  --keyfile KEYFILE, -f KEYFILE
                        File containing the public key body and type (a .pub file in
                        PEM format)
  --keytype KEYTYPE KEYBODY, -t KEYTYPE KEYBODY
                        Keytype of the public key (ex. ecdsa-sha2-nistp256) followed by
                        keybody

The key can be set using a keyfile (--keyfile) or by manual text entry (--keytype).

Keyfile:

When using the --keyfile option, the user passes in the path to the keyfile to be used. This is the public (.pub) key.

Accepted public key types are ecdsa-sha2-nistp521 and ecdsa-sha2-nistp256.

For example:
```
$ csadmin sshd keys setclient -u fedcba09-8765-4321-1234-567890abcdef --keyfile ~/test_key.pub
```
Where test_key.pub is a key in OpenSSH format containing the key type and key body, for example:
```
ecdsa-sha2-nistp256 AAAAE......................5s8=
```
Manual text entry:

When using the --keytype option, the user manually enters the key type followed by the key body.

For example:
```
$ csadmin sshd keys setclient -u fedcba09-8765-4321-1234-567890abcdef --keytype ecdsa-sha2-nistp256 TestKeyBody
```

Manage the SSHD state

The user can query and manage the container SSHD state using csadmin sshd state:

$ csadmin sshd state --help
usage: csadmin sshd state [-h] {enable,disable,get} ...

positional arguments:
  {enable,disable,get}
    enable              Enable a container's SSH daemon
    disable             Disable a container's SSH daemon
    get                 Get a container's SSH daemon state

optional arguments:
  -h, --help            show this help message and exit

Enable the container SSHD

The container SSHD will only start when the container has been enabled. By default, it is disabled. It can be enabled with csadmin sshd state enable:

$ csadmin sshd state enable --help
usage: csadmin sshd state enable [-h] --uuid UUID [--json]

optional arguments:
  -h, --help            show this help message and exit
  --uuid UUID, -u UUID  The UUID of the machine to enable SSH daemon for
  --json                Show response in json format

If enabled, the SSHD will start when the container is started. If the container is already running when the SSHD is enabled, the SSHD will start if it was not already running.

The enable subcommand also returns the port and address on which the SSHD is listening, for example (output in plain text and in JSON format):

$ csadmin sshd state enable -u fedcba09-8765-4321-1234-567890abcdef
FEDC-BA09-8765        SUCCESS
SSHD PORT: 3006
LISTENING ADDRESS: aaaa::aa:aaaa:aaaa:aaaa

$ csadmin sshd state enable -u fedcba09-8765-4321-1234-567890abcdef --json
{
    "FEDC-BA09-8765": {
        "succeeded": true,
        "data": {
            "sshd_port": "3006",
            "listening_address": "aaaa::aa:aaaa:aaaa:aaaa"
        }
    }
}

Disable the container SSHD

When the container SSHD state is disabled, the container SSHD will not start. This is the default state. It can also be disabled with csadmin sshd state disable:

$ csadmin sshd state disable --help
usage: csadmin sshd state disable [-h] --uuid UUID

optional arguments:
  -h, --help            show this help message and exit
  --uuid UUID, -u UUID  The UUID of the machine to disable SSH daemon for
If the container sshd is running, the disable command will also stop it.
It will not start again until is has been enabled.

Retrieve the container SSHD state

The user can also query the SSHD state with csadmin sshd state get. This displays the current SSHD state (enabled or disabled):

$ csadmin sshd state get --help
usage: csadmin sshd state get [-h] --uuid UUID [--json]

optional arguments:
  -h, --help            show this help message and exit
  --uuid UUID, -u UUID  The UUID of the machine to get the SSH daemon state for
  --json                Show response in json format

If the SSHD is disabled, the output only returns the state:

$ csadmin sshd state get -u fedcba09-8765-4321-1234-567890abcdef
FEDC-BA09-8765        SUCCESS
STATE: disabled

$ csadmin sshd state get -u fedcba09-8765-4321-1234-567890abcdef --json
{
    "FEDC-BA09-8765": {
        "succeeded": true,
        "data": {
            "state": "disabled"
        }
    }
}

If the SSHD is enabled, the output returns the port and address that the SSH server on the HSM is listening on, as well as the port on the container where connections should be forwarded to:

$ csadmin sshd state get -u fedcba09-8765-4321-1234-567890abcdef
FEDC-BA09-8765        SUCCESS
STATE: enabled
SSHD_LISTEN: 3001
CONTAINER_FORWARD: 9000
LISTENING ADDRESS: aaaa::aa:aaaa:aaaa:aaaa

$ csadmin sshd state get -u fedcba09-8765-4321-1234-567890abcdef --json
{
    "FEDC-BA09-8765": {
        "succeeded": true,
        "data": {
            "state": "enabled",
            "ports": {
                "sshd": "3001",
                "container": "9000"
            },
            "listening_address": "aaaa::aa:aaaa:aaaa:aaaa"
        }
    }
}

The container port is supplied by the user in network-conf.json.

Forward to the container SSHD

To establish port forwarding on the host machine, use the SSH local port forward command. Do this after the container SSHD is enabled, started, and its keys are set. Use of a secure tunnel over an open TCP connection to communicate with the container is recommended, so use local port forwarding instead dynamic port forwarding to enable SSH tunneling:

ssh -L local_port:destination_server_ip:remote_port ssh_server_hostname

The full details of the command:

ssh -i [PATH_TO_PRIV_KEY] -L [::1]:[LOCAL_PORT]:[CONTAINER_IP_ADDR%lxcbr0]:[CONTAINER_PORT] -f -N -p [SSHD_PORT] launcher@[LISTENING_ADDRESS]%nshield0

Where:

PRIV_KEY is the private key to the public client key that was set with setclient.
LOCAL_PORT is the port on the local client where traffic to be forwarded will be sent.
CONTAINER_IP_ADDR is the IP address of the container. This is the address returned when container is started.
CONTAINER_PORT is the port on the container that is listening for forwarded traffic. This is set in network-conf.json as "ssh_tunnel" (valid range: 1024-65535).
SSHD_PORT is the port the SSH daemon is listening on. This port is returned when SSHD is enabled and can also be found with csadmin sshd state get.
LISTENING_ADDRESS is the address the host clients use to communicate with the HSM (on the HSM side). This address is returned when SSHD is enabled and can also be found with csadmin sshd state get.

For example:

ssh -i ~/test_ssh_keys/test_key -L [::1]:8888:[ffff::fff:ffff:ffff:ffff%lxcbr0]:8888 -f -C -N -p 6789 launcher@aaaa::aa:aaaa:aaaa:aaaa%nshield0

In preparation, there must also be some endpoint in the container that listens on the specified container port, for example, a socket. An example simple python socket script is shown below:

import socket
import sys

HOST = ''
PORT = 8000 #the container port set in the network conf

soc = socket.socket(socket.AF_INET6, socket.SOCK_STREAM)

try:
    soc.bind((HOST, PORT))
except socket.error as message:
    print('Bind failed. Error code: '
        + str(message[0]) + ' Message '
        + message[1])
    sys.exit()

print('Socket binding complete')

soc.listen(9)

conn, address = soc.accept()

with conn:
    print('Connected by', address)
    while True:
        data = conn.recv(1024)
        if not data: break
        conn.send(data)

soc.close()

After the SSH instance is established, any traffic sent to the local port on the local client should be forwarded to the container on the container port. Because of the port forwarding restrictions set for the container SSHD in the authorized_keys file, attempts to forward to any port on the container other than the one specified in network_conf.json will be rejected.

Manage identities

Identities are needed in order to load and run CodeSafe applications. Developer identities (certificates) are managed with csadmin ids:

$ csadmin ids --help
usage: csadmin ids [-h] [--timeout TIMEOUT] [--esn ESN] {add,remove,get,validate,list,create} ...

positional arguments:
  {add,remove,get,validate,list,create}
    add                 Add a CodeSafe developer identity certificate to the nShield5s HSM
    remove              Remove a CodeSafe developer identity certificate from the nShield5s HSM
    get                 Get a CodeSafe developer identity certificate from the nShield5s HSM
    validate            Validate a CodeSafe developer identity certificate on the nShield5s HSM
    list                List all of the CodeSafe developer identity certificates on the nShield5s HSM
    create              Create a CSR for a named developer id key. Also create the developer id key if it does not already exist

options:
  -h, --help            show this help message and exit
  --timeout TIMEOUT     Time to wait for service response, in seconds. Default: 30s, must be between 3s and 300s
  --esn ESN             ESN of the HSM to manage developer identities certificates for.

The command allows end users to add new identities onto the HSM, remove existing identities from the HSM, retrieve (get) identities from the HSM, list identities presently installed on the HSM, and validate a specific identity present on the HSM.

Create a developer ID certificate

Only applicable to CodeSafe developers. See the CodeSafe 5 Developer Guide

List identity certificates

List identity certificates using csadmin ids list:

Listing certificates allows you to check their expiry dates by looking for the notAfter field for each certificate. You should request new certificates from the CodeSafe developer for any certificates that have expired or are near their expiry date. Entrust recommend that you remove any expired certificates using csadmin ids remove.

$ csadmin ids list --help
usage: csadmin ids list [-h] [--json] [--verbose]

options:
  -h, --help  show this help message and exit
  --json      Show response in json format
  --verbose   Print verbose logs

Add identities

Add identity certificates using csadmin ids add:

$ csadmin ids add --help
usage: csadmin ids add [-h] [--verbose] CERTFILE

positional arguments:
  CERTFILE    The developer identity certificate file to be added into the HSM(s).

options:
  -h, --help  show this help message and exit
  --verbose   Print verbose logs

Remove identities

Remove identity certificates using csadmin ids remove:

Entrust recommend that you remove any expired certificates so that only valid certificates remain in the system.

$ csadmin ids remove --help
usage: csadmin ids remove [-h] -s SERIALNO [--verbose]

options:
  -h, --help            show this help message and exit
  -s SERIALNO, --serialno SERIALNO
                        Serial number of the certificate in base ten
  --verbose             Print verbose logs

Retrieve identities

Retrieve identity certificates using csadmin ids get:

$ csadmin ids get --help
usage: csadmin ids get [-h] -s SERIALNO [--json] [--verbose]

options:
  -h, --help            show this help message and exit
  -s SERIALNO, --serialno SERIALNO
                        Serial number of the certificate in base ten
  --json                Show response in json format
  --verbose             Print verbose logs

Validate identities

Validate identity certificates using csadmin ids validate:

$ csadmin ids validate --help
usage: csadmin ids validate [-h] -s SERIALNO [--verbose]

options:
  -h, --help            show this help message and exit
  -s SERIALNO, --serialno SERIALNO
                        Serial number of the certificate in base ten
  --verbose             Print verbose logs