Build and sign example SEE machines on Linux

Build module-side C examples

Create an empty directory to build the module side examples into, for example:
```
mkdir ~/buildmodule/
```
Navigate to the empty directory:
```
cd ~/buildmodule/
```

Build the module side examples with cmake using the following commands:

cmake -DCMAKE_TOOLCHAIN_FILE=/opt/nfast/c/csd5/cmake/codesafe-toolchain-nshield5-csee.cmake /opt/nfast/c/csd5/examples/

cmake --build .

Successful builds create .cs5 images for each example. For example, the classic SEE Hello example has a .cs5 image at ~/buildmodule/n5/csee/hello/module/hello.cs5.

Building Host Side C Examples

Create an empty directory to build the host-side clients for the SEE machines, for example:
```
mkdir ~/buildhost/
```
Navigate to the directory where the host-side examples will be built:
```
cd ~/buildhost/
```
Build the host-side examples with cmake using the following commands:
```
cmake /opt/nfast/c/csd5/examples/

cmake --build .
```

Successful builds create executable host-side clients for each example. For example, the classic SEE Hello example has an executable program at ~/buildhost/n5/csee/hello/host/hello.

Build CS5 Images for Python Examples

Create an empty directory to build the Python examples into, for example:
```
mkdir ~/build_python
```
Navigate to the empty directory:
```
cd ~/build_python/
```
Build the examples with cmake using the following commands:
```
cmake /opt/nfast/python3/csd5/examples

cmake --build .
```

Successful builds create .cs5 images and executable host-side clients for each example. For example, the hello_tcp example has a .cs5 image at ~/build_python/n5/netsee/helloworld_tcp/module/helloworld_mod_tcp.cs5 and the executable program is located at ~/build_python/n5/netsee/helloworld_tcp/hostside/helloworld_host_tcp.py.

Sign CodeSafe Images

Use csadmin ids create to generate the developer ID key, if it does not already exist, as well as the CSR file in a single step. If the key already exists, it only generates the CSR.

csadmin ids create --keyname developerid --x509cname developer.entrust.com --x509country US --x509province Minnesota --x509locality Shakopee --x509org "Entrust CodeSafe" --x509orgunit "Entrust CodeSafe"

Generate key 'testdeveloperkey' ...

Loading `TestOCS':
 Module 1: 0 cards of 1 read
 Module 1 slot 0: empty
Card reading complete.

OK
Generate a CSR in 'testdeveloperkey.csr' ...
OK
Created CSR file 'testdeveloperkey.csr'. Please send it to Entrust Support

This creates the CSR file in the location where the command was run. This developer ID creation was done with TestOCS, quorum of 1/1. Exact output might vary slightly with different OCS quorums.

Send the CSR to customer support to be signed by Entrust. You must obtain the signed developer ID certificate in order to sign and load an application.

For more detailed information on Developer IDs and CSRs, see Sign and deploy CodeSafe 5 SDK apps using csadmin.
Use nfast generatekey to generate a simple ECDSA NIST521P application signing key (ASK). The following example specifies the key to be protected by the module. However, end users are encouraged to protect the key with an OCS.
```
/opt/nfast/bin/generatekey --batch --module=1 simple type=ECDSA curve=NISTP521 ident=ask plainname=ask protect=module
```

Sign the CodeSafe image, for example:

csadmin image sign --askeyname ask --devkeyname developerid --devcert ~/ca/developerid_cert.pem --out /tmp/hello-signed.cs5 ~/ca/hello.cs5

Additional examples are provided later in this chapter.

Where applicable, sign the CodeSafe image with non-ASK keys, for example:

csadmin image signextra --appname seeinteg --key seeintkeyname --out /tmp/hello-signed-extra.cs5 /tmp/hello-signed.cs5

Use csadmin ids add to install the developer ID certificate chain from Entrust.

You can use csadmin ids list to view the loaded certificate.

$ csadmin ids add entrust_developerid_cert_chain.pem
FEDC-BA09-8765        SUCCESS
$ csadmin ids list
FEDC-BA09-8765        SUCCESS
Certificates:
{'serialNumber': '1', 'subject': 'Common Name: developer.entrust.com, Organizational Unit: Entrust CodeSafe, Organization: Entrust, Locality: Shakopee, State/Province: Minnesota, Country: US', 'keyid': 'abcdef12345678900987654321fedcbaabcdef12', 'authKeyid': '0987654321fedcbaabcdef123456789009876543', 'notBefore': '2023-01-01 12:34:56+00:00', 'notAfter': '2024-01-01 12:34:56+00:00'}
{'serialNumber': '2', 'subject': 'Common Name: developer.entrust.com, Organizational Unit: Entrust CodeSafe, Organization: Entrust, Locality: Shakopee, State/Province: Minnesota, Country: US', 'keyid': '1234567890abcdeffedcba098765432112345678', 'authKeyid': 'fedcba09876543211234567890abcdeffedca098', 'notBefore': '2023-01-01 12:34:56+00:00', 'notAfter': '2024-01-01 12:34:56+00:00'}

Run NetSEE examples

NetSEE examples communicate between the client and SEE machine directly via TCP or UDP IPv6 networking to the container, unlike legacy applications, such as for Solo XC or Solo+, which required using an emulation layer on top of SEEJobs to support networking.

helloworld_tcp

To execute the helloworld TCP example that opens a socket within the container and uses the connection to transact a "helloworld" message:

Sign the .cs5 image using devcert and askeys:

csadmin image sign --askeyname ask --devkeyname developerid --devcert ~/ca/developerid_cert.pem --out ~/buildmodule/n5/netsee/helloworld_tcp/module/helloworld_mod_tcp-signed.cs5 ~/buildmodule/n5/netsee/helloworld_tcp/module/helloworld_mod_tcp.cs5

Load the signed .cs5 image using csadmin load:
```
sudo /opt/nfast/bin/csadmin load ~/buildmodule/n5/netsee/helloworld_tcp/module/helloworld_mod_tcp-signed.cs5
```
The output of csadmin load contains the UUID of the loaded container. This UUID will be required for starting the container. The UUID can always be retrieved from the output of csadmin list.

Start the container using csadmin start:

sudo /opt/nfast/bin/csadmin start --uuid fedcba09-8765-4321-1234-567890abcdef

csadmin list lists the UUIDs of all containers. The IPv6 address of the started container appears in the output of the csadmin start command. It can also be found in the output of csadmin list and csadmin stats.

Run the host-side application.

The host-side application takes three positional arguments, the IPv6 address of the container, the port number, and the message to send to the container. The port number used by this example is 8000 by default. The message can be any string of valid characters.
```
~/buildhost/n5/netsee/helloworld_tcp/hostside/helloworld_host_tcp ffff::fff:ffff:ffff:ffff%nshield0 8000 hello_module
```
Expected output:
```
nseeContainerMachineIP=ffff::fff:ffff:ffff:ffff%nshield0
nseeContainerMachinePort=8000
mesg=hello_module
Successful Connection to Socket...
Host>Sending TCP Message-->hello_module
Host>Hello World From HSM!
```
The IPv6 address is link-local and requires the zone index to be appended (typically %nshield0).

helloworld_tcp for nShield 5c

The process is the same as the 5s example, but the host-side application command will differ. Instead of IPv6, you can use the Connect’s IPv4 address:

The examples for the nShield 5c work similarly to the 5s module, but the IP addresses and ports refer to the 5c Connect network. Similarly, for the TCP example, you can use the Connect’s IPv4 address:

~/buildhost/n5/netsee/helloworld_tcp/hostside/helloworld_host_tcp 192.168.1.100 8000 hello_module

Example output:

nseeContainerMachineIP=192.168.1.100
nseeContainerMachinePort=8000
mesg=hello_module
Successful Connection to Socket...
Host>Sending TCP Message-->hello_module
Host>Hello World From HSM!

helloworld_udp

To execute the helloworld UDP example that opens a socket within the container and uses the connection to transact a "helloworld" message:

Sign the .cs5 image using devcert and askeys:

csadmin image sign --askeyname ask --devkeyname developerid --devcert ~/ca/developerid_cert.pem --out ~/buildmodule/n5/netsee/helloworld_udp/module/helloworld_mod_udp-signed.cs5 ~/buildmodule/n5/netsee/helloworld_udp/module/helloworld_mod_udp.cs5

Load the signed container using csadmin load:

sudo /opt/nfast/bin/csadmin load ~/buildmodule/n5/netsee/helloworld_udp/module/helloworld_mod_udp-signed.cs5

Example output:

FEDC-BA09-8765: Uploading ~/buildmodule/n5/netsee/helloworld_udp/module/helloworld_mod_udp-signed.cs5
FEDC-BA09-8765: creating machine
FEDC-BA09-8765        SUCCESS
UUID: fedcba09-8765-4321-1234-567890abcdef

The output of csadmin load contains the UUID of the loaded container. This UUID will be required for starting the container. The UUID can always be retrieved from the output of csadmin list.

Start the container using csadmin start:

sudo /opt/nfast/bin/csadmin start --uuid fedcba09-8765-4321-1234-567890abcdef

Example output:

FEDC-BA09-8765        SUCCESS
IP ADDRESS: ffff::fff:ffff:ffff:ffff

csadmin list will list the UUIDs of all containers. The IPv6 address of the started container appears in the output of the csadmin start command. It can also be found in the output of csadmin list and csadmin stats.

Run the host-side application.

The host-side application takes three positional arguments, the IPv6 address of the container, the port number, and the message to send to the container. The port number used by this example is 8000 by default. The message can be any string of valid characters.
```
~/buildhost/n5/netsee/helloworld_udp/hostside/helloworld_host_udp ffff::fff:ffff:ffff:ffff%nshield0 8000 hello_module
```
Example output:
```
nseeContainerMachineIP=ffff::fff:ffff:ffff:ffff%nshield0
nseeContainerMachinePort=8000
mesg=hello_module
Successful Connection to Socket...
Host>Sending UDP Message-->hello_module
Host>Hello World From HSM!
```
The IPv6 address is link-local and requires the zone index to be appended (typically %nshield0).

helloworld_udp for 5c

The process is the same as the 5s example, but the host-side application command will differ. Instead of IPv6, you can use the Connect’s IPv4 address:

The examples for the nShield 5c work similarly to the 5s module, but the IP addresses and ports refer to the 5c Connect network.

~/buildhost/n5/netsee/helloworld_udp/hostside/helloworld_host_udp 192.168.1.100 8000 hello_module

Example output:

nseeContainerMachineIP=192.168.1.100
nseeContainerMachinePort=8000
mesg=hello_module
Successful Connection to Socket...
Host>Sending UDP Message-->hello_module
Host>Hello World From HSM!

Run NetSEE examples via SSH tunnel

NetSEE examples communicate between the client and SEE machine directly through a TCP/IPv6 network connection to the container, unlike legacy applications, such as for Solo XC or Solo+, which communicate through the hardserver to the nCore API.

On the nShield 5c network, the SSHD listening address may be an IPv4 address instead of IPv6. Adjustments to the steps below may be needed to accommodate this.

helloworld_tcp via SSH Tunnel

To execute the helloworld TCP example via an SSH Tunnel that opens a socket within the container and uses the connection to transact a "helloworld" message:

Create an SSHD key for the hello example:

mkdir ~/examplekeys/
ssh-keygen -t ecdsa -f ~/examplekeys/helloworld_tcp_ecdsa_key

Modify the network-conf.json of the helloworld_tcp example to support SSH tunneling, for example:

cat ~/buildmodule/n5/netsee/helloworld_tcp/module/network-conf.json
{
        "incoming": {
                "tcp":
                {
                        "protos": ["ipv6"],
                        "ports": []
                }
        },
        "outgoing" : {
                "tcp" :
                {
                       "protos": ["ipv6"],
                       "ports": []
                }
        },
        "ssh_tunnel" : {
                 "container_port" : 8000
        }
}

+ When the container server app accepts a client connection on the specified incoming port (for example 8000), it designates and responds to the client on an ephemeral port in the range [32768-60999] as the outgoing port. This port does not have to be defined in the network-conf.json. Only one port is supported for the ssh_tunnel / container_port in this version. The ssh_tunnel port must be the only port specified for communication that is restricted to be made over SSH. To only send communication in the plain, the port should instead be specified in the incoming ports list.

Rebuild the .cs5 image with the updated network-conf.json so the loaded container will allow SSH tunneling:

sudo /opt/nfast/bin/csadmin image generate --package-name "helloworld_tcp" --entry-point /usr/bin/entrypoint --network-conf ~/buildmodule/n5/netsee/helloworld_tcp/module/network-conf.json --packages-conf ~/buildmodule/n5/netsee/helloworld_tcp/module/extra-packages-conf.json --version-str 1.0  --rootdir ~/buildmodule/n5/netsee/helloworld_tcp/module/container/ ~/buildmodule/n5/netsee/helloworld_tcp/module/helloworld_mod_tcp.cs5

Most paths used in generating the new image are paths to the file locations on the host that is building the image However, the --entry-point path is the absolute path to the entrypoint file within the container and should be /usr/bin/entrypoint, not ~/buildmodule/n5/netsee/helloworld_tcp/module/container/usr/bin/entrypoint.

Sign the new .cs5 image using devcert and askeys:

sudo /opt/nfast/bin/csadmin image sign --askeyname ask --devkeyname developerid --devcert ~/ca/developerid_cert.pem --out ~/buildmodule/n5/netsee/helloworld_tcp/module/helloworld_mod_tcp-signed.cs5 ~/buildmodule/n5/netsee/helloworld_tcp/module/helloworld_mod_tcp.cs5

Load the signed container using csadmin load:
```
sudo /opt/nfast/bin/csadmin load ~/buildmodule/n5/netsee/helloworld_tcp/module/helloworld_mod_tcp-signed.cs5
```
The output of csadmin load contains the UUID of the loaded container. This UUID will be required for starting the container and managing the SSHD keys of the container. The UUID can always be retrieved from the output of csadmin list.

Load the public key created earlier (helloworld_tcp_ecdsa_key) to the container using csadmin sshd setclient:

sudo /opt/nfast/bin/csadmin sshd keys setclient --uuid fedcba09-8765-4321-1234-567890abcdef --keyfile ~/examplekeys/helloworld_tcp_ecdsa_key.pub

Enable SSH tunneling on the container:

sudo /opt/nfast/bin/csadmin sshd state enable --uuid fedcba09-8765-4321-1234-567890abcdef

Example output:

FEDC-BA09-8765        SUCCESS
SSHD PORT: 6789
LISTENING ADDRESS: aaaa::aa:aaaa:aaaa:aaaa

The output of sshd state enable contains the SSHD Port number and the listening address of the container SSHD.

Start the container using csadmin start:
```
sudo /opt/nfast/bin/csadmin start --uuid fedcba09-8765-4321-1234-567890abcdef
```
csadmin list lists the UUIDs of all containers. The IPv6 address of the started container appears in the output of the csadmin start command. It can also be found in the output of csadmin list and csadmin stats.

Setup the SSH tunnel on the host:

Run csadmin sshd state get and collect the following information:

Container tunnel address (ffff::fff:ffff:ffff:ffff)
Container port (8000)
SSHD port (6789)
SSHD listening address (aaaa::aa:aaaa:aaaa:aaaa)

On nShield Connect the SSHD listening address may be an IPv4 or IPv6 address

Next, choose a local IP address and port number through which to access the tunnel. Typically localhost is chosen as the local IP address (127.0.0.1 or [::1])

Check the HSM’s SSHD public key by running csadmin sshd keys getserver -u UUID for comparison against the output from ssh on first use, or this could be added to your known_hosts file directly.

The SSH tunnel command is formatted as follows:

ssh -i ~/examplekeys/helloworld_tcp_ecdsa_key -L LOCAL_IP:LOCAL_PORT:[TUNNEL_ADDRESS%lxcbr0]:CONTAINER_PORT -f -N -p SSHD_PORT launcher@LISTENING_ADDRESS

Using the example data:

ssh -i ~/examplekeys/helloworld_tcp_ecdsa_key -L [::1]:8000:[ffff::fff:ffff:ffff:ffff%lxcbr0]:8000 -f -N -p 6789 launcher@aaaa::aa:aaaa:aaaa:aaaa%nshield0

When using nShield 5s the IPv6 address is link-local and requires the zone index to be appended (typically %nshield0). If you are working with a 5c network, replace the IPv6 address with the appropriate nShield5c network address (IPv4 or IPv6) for your configuration.

Run the host-side application.

The host-side application takes three positional arguments, the IPv6 address set up in the forwarding step [::1], the port number, and the message to send to the container. The port number used by this example is 8000 by default. The message can be any string of valid characters.
```
~/buildhost/n5/netsee/helloworld_tcp/hostside/helloworld_host_tcp ::1 8000 hello_module
```
Expected Output:
```
nseeContainerMachineIP=::1
nseeContainerMachinePort=8000
mesg=hello_module
Successful Connection to Socket...
Host>Sending TCP Message-->hello_module
Host>Hello World From HSM!
```

Run CSEE examples via SSH tunnel

The Classic SEE (CSEE) examples use the SEElib library’s SEEJobs functionality for communication with the host (using port 8888 in the container). These examples are identical to examples provided with previous iterations of nShield HSMs and CodeSafe. CSEE applications must be loaded using the [codesafe] section of the client-side hardserver config file, or by running the hsc_codesafe tool directly. Using this configuration support simplifies the deployment of CSEE applications and automates the csadmin operations (except for signing) and securely setting up the SSH tunnel.

hello CSEE via automatic configuration

This section describes executing the hello CSEE example. The hello example operates functionally identically to the CSEE hello example for Solo XC and Solo+.

The hello example sends a string from the host to the module. The module converts the string to uppercase and returns the string to the host.

Generate an input file containing a character string to be sent to the module.
```
echo UPPERCASElowercase > ~/inputfile
```
This input file has both uppercase and lowercase characters.

In all CSEE examples, the network-conf.json permits only secure communication via the SSH tunnel (and no plaintext communication via incoming is allowed by specifying an empty ports list), i.e. the network-conf.json in the SDK matches the below:

cat ~/buildmodule/n5/csee/hello/module/network-conf.json
{
        "incoming": {
                "tcp":
                {
                        "protos": ["ipv6"],
                        "ports": []
                }
        },
        "outgoing" : {
                "tcp" :
                {
                       "protos": ["ipv6"],
                       "ports": []
                }
        },
        "ssh_tunnel" : {
                 "container_port" : 8888
        }
}

Sign the .cs5 image using devcert and askeys:

sudo /opt/nfast/bin/csadmin image sign --askeyname ask --devkeyname developerid --devcert ~/ca/developerid_cert.pem --out /opt/nfast/custom-seemachines/hello-signed.cs5 ~/buildmodule/n5/csee/hello/module/hello.cs5

Load the signed container using the [codesafe] of the client-side config file (replacing the esn field contents with the actual ESN of the module on which to load the application) and then clearing the module e.g. with nopclearfail -c -m1 (replacing 1 with the module number in question):
```
[codesafe]
esn=5CB5-41F2-F235
image_file=/opt/nfast/custom-seemachines/hello-signed.cs5
worldid_pubname=hellosee
```
Run the host-side application.

The host-side application takes one required positional argument for the input file containing a string to convert to uppercase on the module. Module number is assumed to be 1 by default, use -m2 to specify module 2 etc. The published object name of the SEE World ID is assumed to be hellosee by default (as in the config example above); if a different name was used in the [codesafe] section of the config file, use -p parameter to specify it.
```
~/buildhost/n5/csee/hello/hostside/hello ~/inputfile
```
Example output:
```
Worldid: 0x1234abcd
UPPERCASELOWERCASE
```
The module has received the input string UPPERCASElowercase and has converted and returned it as a fully uppercase string UPPERCASELOWERCASE.

tickets CSEE via automatic configuration

This section describes executing the tickets CSEE example. The tickets example operates functionally identically to the tickets example for Solo XC and Solo+. The tickets example serves to demonstrate cryptographic functionality by encrypting and having the module decrypt a user-provided string.

Generate a simple RSA key to encrypt with:

sudo /opt/nfast/bin/generatekey --module=1 simple type=RSA pubexp=3 ident=encryptionkeytickets plainname=encryptionkeytickets protect=module nvram=no size=2048

Sign the .cs5 image using devcert and askeys:

sudo /opt/nfast/bin/csadmin image sign --askeyname ask --devkeyname developerid --devcert ~/ca/developerid_cert.pem --out /opt/nfast/custom-seemachines/seetickets-signed.cs5 ~/buildmodule/n5/csee/tickets/module/seetickets.cs5

Load the signed container using the [codesafe] of the client-side config file (replacing the esn field contents with the actual ESN of the module on which to load the application) and then clearing the module e.g. with nopclearfail -c -m1 (replacing 1 with the module number in question):
```
[codesafe]
esn=5CB5-41F2-F235
image_file=/opt/nfast/custom-seemachines/seetickets-signed.cs5
worldid_pubname=ticketsee
```
Run the host-side application.

The host-side application accepts the encryption key created earlier as an optional argument (--key). Module number is assumed to be 1 by default, use -m2 to specify module 2 etc. The published object name of the SEE World ID is assumed to be ticketsee by default (as in the config example above); if a different name was used in the [codesafe] section of the config file, use -p parameter to specify it.
```
~/buildhost/n5/csee/tickets/hostside/hosttickets --key simple,encryptionkeytickets
```

When prompted, enter a string to encrypt (for example, testencryption) and press Return:

Enter string to be encrypted (256 characters maximum): testencryption

The host encrypts the message then the module decrypts it and returns it in plain text format.

Example output:

HostSide> Loading security world key (simple,encryptionkeytickets)
HostSide> Creating World: init status was 0 (OK)
HostSide> Sending ticket for private RSA key to module
HostSide> Generating AES session key and creating blob under public RSA key
HostSide> Sending key blob to module
HostSide> Sending cipher-text to module
HostSide> decrypted cipher text received from SEE machine:
"testencryption"
HostSide> Thank you for watching. The end.

benchmark CSEE via automatic configuration

This section describes executing the benchmark CSEE example. The benchmark example operates functionally identically to the benchmark example for Solo XC and Solo+. The benchmark example will transact asynchronously with the module running multiple threads processing transactions. The benchmark example will output transactions/second data every second.

Generate a simple key for signing a ticket in the bm-machine on the module:

sudo /opt/nfast/bin/generatekey --module=1 simple type=RSA pubexp=3 ident=signingkeybenchmark plainname=signingkeybenchmark protect=module nvram=no size=2048

Sign the .cs5 image using devcert and askeys:

sudo /opt/nfast/bin/csadmin image sign --askeyname ask --devkeyname developerid --devcert ~/ca/developerid_cert.pem --out /opt/nfast/custom-seemachines/bm-machine-signed.cs5 ~/buildmodule/n5/csee/benchmark/module/bm-machine.cs5

Load the signed container using the [codesafe] of the client-side config file (replacing the esn field contents with the actual ESN of the module on which to load the application) and then clearing the module e.g. with nopclearfail -c -m1 (replacing 1 with the module number in question):
```
[codesafe]
esn=5CB5-41F2-F235
image_file=/opt/nfast/custom-seemachines/bm-machine-signed.cs5
worldid_pubname=bmsee
```
Run the host-side application.

The host-side application takes two positional arguments: the appname and the key name of the signing key created earlier. Module number is assumed to be 1 by default, use -m2 to specify module 2 etc. The published object name of the SEE World ID is assumed to be bmsee by default (as in the config example above); if a different name was used in the [codesafe] section of the config file, use -p parameter to specify it.
```
~/buildhost/n5/csee/benchmark/hostside/bm-test simple signingkeybenchmark
```
Example output:
```
Worldid: 0x1234abcd
1 759 759.00
2 1522 761.00
3 2361 787.00
4 3324 831.00
5 4238 847.60
6 5124 854.00
7 5948 849.71
8 6723 840.38
9 7579 842.11
10 8408 840.80
```