Building CodeSafe 5 applications

General SDK use

The CodeSafe 5 SDK provides the tools necessary to build and run CodeSafe applications on nShield 5 HSMs. The CodeSafe 5 applications (also known as SEE machines) are containerized. The SDK provides the structure of the container, including a root file system, libraries required for communication with the nCore API, and libraries to enable communication between the CodeSafe application and the host. The SDK provides headers and libraries for building applications and a root file system for the container containing runtime libraries, binary tools such as touch, cat, grep and optionally a Python distribution.

Prerequisites

SDK file structure overview

SDK location

The default installation location of the CodeSafe 5 SDK is:

  • Linux: /opt/nfast/c/csd5/

  • Windows: C:\Program Files\nCipher\nfast\c\csd5\

Some tools which are required for operations on CodeSafe applications (also known as SEE machines) might be found elsewhere in the main installation. For example, csadmin, which enables loading, starting, and stopping CodeSafe applications, is installed in the following default locations:

  • Linux: /opt/nfast/bin/csadmin

  • Windows: C:\Program Files\nCipher\nfast\bin\csadmin (Windows)

These cases are described in the following sections as required.

Container root file system

The container root file system is located in:

  • Linux: /opt/nfast/c/csd5/rootfs/

  • Windows: C:\Program Files\nCipher\nfast\c\csd5\rootfs\

This root file system contains two main parts: binary files and libraries.

Binaries

rootfs/bin/ (Linux) or rootfs\bin\ (Windows) contains many useful common Linux binaries that you might need within the container such as cat, grep, and touch.

rootfs/sbin/ (Linux) or rootfs\sbin\ (Windows) contains the init script for the container.

Libraries

rootfs/lib/ and rootfs/usr/lib/ (Linux) or rootfs\lib\ and rootfs\usr\lib\ (Windows) contain runtime libraries that may be needed by applications built with the SDK.

Image creation options

CS5 images are generated with csadmin image generate (see the csadmin image subcommand documentation.)

Generating an image requires the name of the CS5 file and the use of the following mandatory command-line arguments:

  • --package-name

  • --version-str

  • --entry-point

  • --network-conf

  • --packages-conf

  • --rootdir

The following items are also required:

  • A container directory (not necessarily named "container") that points to what would be the CodeSafe application’s root directory.

    This directory must include any files used by the application, including the entry point program, for example:

    container/
    ├── home
    └── usr
        └── bin
            └── entrypoint

    The container directory can be located anywhere in the host file system. Ensure you pass the full path to the generate command via the --rootdir argument, as specified in the command usage.

  • An entry point program.

    This is the program that runs when the SEE container is started (on launcher start). It must be made executable so it can be launched accordingly. In the previous example, the entry point program is in container/usr/bin/entrypoint.

  • A network configuration file. (See Example network-conf.json file.)

    The valid range for container_port is 1024 - 65535.

  • A file with extra packages information. (See Example extra-packages-conf.json file)

--entry-point points to the full path of the executable program relative to the container’s root.

Example extra-packages-conf.json file

This example shows how to specify extra packages to be included in the container image, which will be copied from the host into the container at build time.

The binaries package is required for most applications, as it includes the IPC daemon which is required to mediate between the CodeSafe application and the nCore API service.

The other packages are needed for all applications: * The python packages can be omitted if Python is not being used. * The kmlocal and cardlist packages support use of the NFKM library as they pull in Security World files to their usual /opt/nfast/kmdata locations within the container.

It is also possible to specify custom packages here, which can be copied from the host into the container at build time, and then used by the application. These may be relative paths to the nShield installation location, like the python and binaries packages in the example; absolute paths to any location on the host file system; or paths referenced via environment variables, as in the kmlocal and cardlist examples.

{
    "packages": [{
            "package": "python",
            "description": "python 3.11 binaries",
            "host_path": "python3/csd5/ppc64/usr/bin",
            "machine_path": "usr/bin",
            "exclude": ""
        },
        {
            "package": "python",
            "description": "python 3.11 libraries",
            "host_path": "python3/csd5/ppc64/usr/lib/python3.11",
            "machine_path": "usr/lib/python3.11",
            "exclude": ""
        },
        {
            "package": "binaries",
            "description": "binaries for script support 1.0.0",
            "host_path": "c/csd5/rootfs",
            "machine_path": "/",
            "exclude": ""
        },
        {
            "package": "kmlocal",
            "description": "Security World files",
            "host_path": "${NFAST_KMLOCAL}",
            "machine_path": "opt/nfast/kmdata/local",
            "exclude": ""
        },
        {
            "package": "cardlist",
            "description": "cardlist (smartcard allow-list for NFKM library)",
            "host_path": "${NFAST_CARDLIST}",
            "machine_path": "opt/nfast/kmdata/config/cardlist",
            "exclude": ""
        }
    ]
}

Example network-conf.json file

This example shows port 8888 (the default port for the nShield SEEJobs protocol) accessible securely through the SSH tunnel. TCP ports 8000 and 8001 are exposed directly on the HSM network interface for other application services, and no outgoing ports are allowed.

{
    "incoming" : {
        "tcp" : {
            "protos" : [ "ipv6" ], "ports" : [ 8000, 8001 ]
        }
    },
    "outgoing" : {
        "tcp" : {
            "protos" : [ "ipv6" ], "ports" : []
        }
    },
    "ssh_tunnel" : {
             "container_port" : 8888
    }
}

Example entry point script

#!/bin/sh
# Export any environment variables needed by the application here, for example:
# export MY_VAR=value
# Then run the application itself, usually installed in /usr/bin:
/usr/bin/myapplication
# The script may contain additional commands for diagnostics or error reporting.
# Any output from the application or from this script will be captured in the CodeSafe application logs.

CMake

The SDK installs a directory which includes CMake toolchains used for building example CodeSafe applications:

  • Linux: /opt/nfast/c/csd5/cmake

  • Windows: C:\Program Files\nCipher\nfast\c\csd5\cmake

These toolchains can serve as examples themselves for creating custom toolchains.

Include directories

The SDK provides two directories with header files that can be included along with their respective libraries to provide additional functionality in CodeSafe applications. These headers are stored in:

  • Linux:

    • /opt/nfast/c/csd5/gcc/*

    • /opt/nfast/c/csd5/include-see/*

  • Windows:

    • C:\Program Files\nCipher\nfast\c\csd5\gcc\*

    • C:\Program Files\nCipher\nfast\c\csd5\include-see\*

nShield-specific CodeSafe libraries

C/C++ libraries

The nShield-specific C libraries are located in:

  • Linux: /opt/nfast/c/csd5/lib-ppc64-linux-musl/\*

  • Windows: C:\Program Files\nCipher\nfast\c\csd5\lib-ppc64-linux-musl\*

Applications that wish to communicate with the nCore API service, or expose the SEEJobs protocol, from a C or C++ application, must link against either of two main groups of libraries:

  • Traditional SEElib (including SEEJobs):

    • seelib.a and librtusr.a libraries.

    • This exposes the SEElib API, which includes support for SEEJobs and communication with the nCore API.

    • seelib.a is not compatible with libnfstub.a / libnfkm.a.

    • This restricts to the traditional APIs that were historically referred to as CSEE APIs.

  • Standard host-side APIs:

    • These APIs are the same as those used by host applications communicating with nShield HSMs, and are not specific to CodeSafe applications.

    • The hellonshield example CodeSafe application in the SDK (/opt/nfast/c/csd5/examples/netsee/hellonshield on Linux and C:\Program Files\nCipher\nfast\c\csd5\examples\netsee\hellonshield on Windows) demonstrates use of the C Generic Stub API and NFKM API in a CodeSafe application, and can be used as a reference for this use-case. It doesn’t have a client-side component, and the example’s output is simply written to the application log.

    • C Generic Stub (the NFastApp API for nCore)

    • libnfstub.a library, and additionally add the hilibs include subdirectory, such as /opt/nfast/c/csd5/include-see/hilibs, for headers such as nfastapp.h.

    • libnfstub.a is not compatible with seelib.a but it is compatible with libnfkm.a.

    • libnfstub.a includes all of the SEElib API in a form that is compatible with the C Generic Stub, enabling SEElib (including SEEJobs) to be combined with use of the standard host-side APIs if desired.

    • NFKM (the Security World API)

    • libnfkm.a, and additionally add the sworld include subdirectory, such as /opt/nfast/c/csd5/include-see/sworld, for headers such as nfkm.h.

    • Effective use of the NFKM library will usually require the kmlocal and (if interacting with smartcards) cardlist packages to be included in the image as described in Example extra-packages-conf.json file.

Combining SEElib with C Generic Stub (NFastApp) or NFKM

When combining SEElib with NFastApp or NFKM, always link against libnfstub.a instead of seelib.a.

The traditional CodeSafe SEElib library exists as the stand-alone library seelib.a. This static library is not compatible with libnfstub.a or libnfkm.a.

As a convenience for users combining use of SEElib (SEElib_* functions) with the NFastApp_* functions in the C Generic Stub and/or the NFKM APIs, the C Generic stub static library libnfstub.a in CodeSafe 5 includes the full SEElib API.

If the goal is to migrate use of SEElib_Transact to NFastApp_Transact for consistency with client-side application code whilst still retaining support for SEEJobs, or if SEElib is being used primarily but NFKM library must also be used in the same application, then libnfstub.a must be used instead. This provides all the SEElib symbols in a form compatible with libnfstub.a and libnfkm.a. seelib.a must not be linked against in this case. The usual seelib.h header should continue to be used to obtain the declarations, but if also including C Generic Stub (such as nfastapp.h) or NFKM headers (such as nfkm.h), they should be included before seelib.h.

If SEElib_init() is called in the libnfstub.a implementation, it will initialize an NFast_AppHandle internally which is configured to marshal M_Bignum objects in the form defined by struct NFast_Bignum in the seelib.h header, which is an M_ByteBlock containing a little-endian representation of the value. This makes the default marshalling behaviour and struct representation of objects when migrating existing SEElib code from seelib.a to libnfstub.a identical.

In order to support inter-operation with other NFastApp use-cases, new APIs SEElib_init_NFastApp() and SEElib_init_NFastApp_default() have been added in seelib.h that can be called instead of SEElib_init() if required. These new functions are declared if nfastapp.h is included before seelib.h. SEElib_init_NFastApp() allows initialization of SEElib with the caller’s own NFast_AppHandle, allowing the caller to precisely control the NFastApp initialization. SEElib_init_NFastApp_default() initializes the SEElib library with the default NFastApp Bignum implementation defined in nfastapp-bignum.h. This provides maximum compatibility with typical default NFastApp initialization that would be used in most client-side application code that is being migrated to CodeSafe 5. If nfastapp-bignum.h is being used, it should be included before seelib.h.

If "manual" marshalling or unmarshalling of nCore types is being performed using functions like NF_Marshal_CertificateList or NF_Unmarshal_CertificateList, the struct NF_UserData * context set in the NF_Marshal_Context.u or NF_Unmarshal_Context.u field was set to NULL in traditional seelib.a applications. If using marshalling or unmarshalling functions when linking against libnfstub.a, the struct NF_UserData * u; field must now be set to a valid Generic Stub object. This should either be obtained in the usual manner using the NFastApp APIs, or if using the SEElib initialization extensions, it can be obtained using the new SEElib_GetUserData() function. When using seelib.a, SEElib_GetUserData() will always return NULL as it is not applicable for the implementation of the marshalling and unmarshalling functions in that version of the library.

Python modules

The nShield-specific Python modules are nfpython (for nCore interfaces) and nfkm (for NFKM interfaces, which is a superset of the nfpython module and so can be used instead of nfpython if using both APIs).

If using the Python runtime from the extra packages (see Example extra-packages-conf.json file), these modules are available as import nfpython or import nfkm without any additional configuration, and behave in the same way as they do on the host side, except that they communicate with the nCore API service via the IPC daemon within the container instead of communicating via the hardserver service.

As with the C libnfkm.a API, effective use of the Python nfkm API for Security World functionality will usually require the kmlocal and (if interacting with smartcards) cardlist packages to be included in the image as described in Example extra-packages-conf.json file.

The Python webserver example in the SDK (/opt/nfast/python3/csd5/examples/webserver on Linux and C:\Program Files\nCipher\nfast\python3\csd5\examples\webserver on Windows) demonstrates use of the nfpython and nfkm modules in a CodeSafe application, and can be used as a reference for this use-case. It can be used in conjunction with the supplied hostside client, or from a web browser.

C++ applications

CodeSafe 5 supports C++17.

You can link the seelib.a or the libnfstub.a and libnfkm.a libraries with C++ applications to use the nShield APIs. The relevant header files contain extern "C" guards so that they can be included in C++ code.

The helloplus example in the SDK (/opt/nfast/c/csd5/examples/netsee/helloplus on Linux and C:\Program Files\nCipher\nfast\c\csd5\examples\netsee\helloplus on Windows) shows how to build a simple CodeSafe 5 application with C++ 17 and demonstrates various C++ language and library features working with the cross-compiler. It doesn’t have a client-side component, and the example’s output is simply written to the application log.

If not using CMake to build your CodeSafe 5 application, the C++ compiler and linker flags defined in the SDK’s codesafe-toolchain-nshield5-csee.cmake file should be consulted to obtain the correct flags for your own build files. Note that the -static linker flag is required to statically link the C and C++ runtime libraries in this case.

The reserved C++ keyword export is used as a field name in M_Command.args.export and M_Reply.reply.export (defined in both seelib.h and nfastapp.h headers). By default, the field is renamed using the pre-processor to exportcmd when compiling with a C++ compiler. If an application has already worked around this issue by defining export before including relevant nShield headers, the pre-processor renaming will not be applied, and the user’s existing work-around will continue to behave as before. If STDMARSHAL_KEEP_EXPORT_RENAME is defined before including relevant headers, export will not be #undef-ed after the affected headers, so that code can still reference M_Command.args.export and M_Reply.reply.export for backwards-compatibility with code migrated from C to C++; this should only be done if export is not otherwise being used in the codebase. To completely disable these workarounds, define the STDMARSHAL_NO_EXPORT_RENAME macro before including relevant headers.

Compatibility

The CodeSafe 5 SDK and nShield 5 HSMs are sufficiently different from previous implementations that applications for HSM models cannot run on the nShield 5.

Applications must be rebuilt using the CodeSafe 5 SDK to run on the nShield 5.

Where possible, APIs such as SEElib and SEEJobs behave as they did with previous HSM models, but signing and deployment details for CodeSafe 5 applications differ.

Building new CodeSafe applications

A CodeSafe application is a container image with a complete filesystem which can be loaded onto a CodeSafe 5-enabled HSM as part of a container. The SEElib, Generic Stub, or nfpython library enables CodeSafe applications to interface with the nCore API via the IPC daemon.

Source code is compiled using one of the GCC (or G++) cross-compilers supplied with the CodeSafe SDK. For details of required compiler options, toolchains, makefiles and so on, see the CMake files supplied with the examples, as well as Build and sign example CodeSafe applications on Linux and Build and sign example CodeSafe applications on Windows.

The container image (the .cs5 file) must be signed using the csadmin utility.

Developer authentication

CodeSafe 5 requires a signed CodeSafe image to run CodeSafe applications on the HSM.

The CodeSafe developer needs to request a developer ID certificate by sending a Certificate Signing Request (CSR) to Entrust support. The tool used to create the CSR is integrated into the HSM software as a subcommand of csadmin.

For security purposes, a developer keypair must be created and stored within the HSM. In addition, the keypair must be OCS protected to provide authorization control on its use. The developer keypair will be created by csadmin if it does not already exist.

After the certificates are received, they are installed on the HSM and are used to sign CodeSafe application images with the csadmin tool.

The implementation of this is described in more detail in Create and sign CodeSafe 5 applications with csadmin.

Deploying CodeSafe applications

After the code has been compiled, built, and signed, the csadmin utility may be used to deploy the CodeSafe application, on a 5s or remotely on a 5c, although in most deployments this is done indirectly via the automatic-loading configuration in the host-side config file or 5c (Connect) config file, or interactively via the hsc_codesafe utility, in order to simplify deployment.

For more information on the csadmin utility, see Create and sign CodeSafe 5 applications with csadmin. For more information on automatic-loading configuration, see Automatic Configuration of CodeSafe 5 Applications via the Host Machine.

When the CodeSafe application starts, the IPC daemon is started automatically in the background, and then the entrypoint script is executed, which then runs the primary process in the container and its main() function.

CodeSafe application initialization requirements

If using SEElib, a CodeSafe application must initialize SEElib before making use of any of its functionality. This is done by calling SEElib_init(), or by using one of the Generic Stub (NFastApp) SEElib extension functions for initialization described in Combining SEElib with C Generic Stub (NFastApp) or NFKM if linking against the Generic Stub library rather than traditional seelib.a. It is recommended that this call is made immediately within the main() function of a CodeSafe application.

By default, SEElib_init() will also enable SEEJobs support for communication from the host via Cmd_SEEJob and Cmd_FastSEEJob sent via a client hardserver. This uses port 8888 within the container, and this port should be set as the ssh_tunnel port in the container configuration as shown in later examples. It is possible to enable a non-default port for SEEJobs by calling SEElib_SetPort(8000) (for example) before SEElib_init(). To disable SEEJobs listening altogether, run SEElib_SetPort(0) before SEElib_init(). If multiple processes are running within the container that need to use SEElib, only one of them can listen for SEEJobs, and so all but one of the processes must call SEElib_SetPort(0) before SEElib_init() to avoid multiple initializations of the SEEJobs listener.

If using the Generic Stub (NFastApp) library, the same initialization requirements as for host-side usage apply, and if using the NFastApp_Init() or NFastApp_InitEx() functions, rather than the SEElib extensions for NFastApp, this will not initialize the SEEJobs listener, and the usual functions such as NFastApp_Connect() and NFastApp_Transact() will be available for issuing nCore commands via the IPC daemon. If the SEElib extensions for NFastApp are used for initialization, the SEEJobs listener will be initialized by default as described in the previous paragraph, and the usual SEElib functions will be available for issuing nCore commands via the IPC daemon, including support for SEEJobs, in addition to the NFastApp functions.

SEElib Functions

After initialization, SEElib functions can be used to communicate with the nCore API via the IPC daemon that is included as part of the container image. These functions behave the same as in previous CodeSafe versions although the underlying implementation has changed.

SEElib_Transact()

To send a command to the nCore API and block waiting for a reply:

int SEElib_Transact(struct M_Command *cmd, struct M_Reply *reply)

This sends the cmd command to the nCore API and waits for the reply to be written to reply.

SEElib_Submit() / SEElib_Query()

To send a non-blocking command to the nCore API:

int SEElib_Submit(M_Command *cmd, M_Reply *reply, PEVENT ev, SEElib_ContextHandle tctx)

The cmd command is submitted to the nCore API. The transaction listener thread will call EventSet ev, if ev is non-NULL when the reply returns for this command. The reply is unmarshalled into reply and tctx is returned to the caller with SEElib_Query(M_Reply **replyp, SEElib_ContextHandle *tctx_r).

Before using the SEElib_Submit() method, SEElib_StartTransactListener() must have been called to start the transaction listener.

Unlike SEElib_SubmitCoreJob(), SEElib_Submit() does not block and wait for all other calls to SEElib_Transact() to complete.

SEElib_SubmitCoreJob / SEElib_GetCoreJobEx()

To submit a job to the nCore API:

extern int SEElib_SubmitCoreJob(const unsigned char *data, unsigned int len)

To receive a job from the nCore API:

extern int SEElib_GetCoreJobEx(unsigned char *buf, M_Word *len_io, unsigned flags)

SEElib_SubmitCoreJob() is blocking. It waits for the job to be submitted, which includes waiting for existing calls made to SEElib_Transact() to be completed. The same is true for SEElib_GetCoreJobEx().

For non-blocking calls, consider using SEElib_Submit().

Other SEElib methods

For a comprehensive list of all functionality provided via the SEElib, see: SEE API documentation.

Host/CodeSafe application communication

Host/CodeSafe application communication can be done using SEEJobs via the hardserver using an automatically configured SSH tunnel set up by using the [codesafe] configuration section of the client-side config file (or optionally using the 5c "Connect" config file if using a 5c). Alternatively (or additionally) communication between the host and the CodeSafe application can be done via TCP and UDP IPv6 networking.

Multiple processes within the container can connect to the ncoreapi service concurrently (this is multiplexed automatically by the IPC daemon), but only one process may bind to the SEEJobs port, or to any other port for implementing a custom protocol.

IPv6 vs IPv4 networking

The 5s HSM itself only exposes an IPv6 network interface, so only IPv6 can be used for direct TCP or UDP communication between the host and the CodeSafe application. However, if using a 5c (Connect form factor), then the communication is relayed automatically to the 5c’s network interface, which supports both IPv4 and IPv6, so the host can communicate with the CodeSafe application via IPv4 if desired in that case. See Working with CodeSafe for more information regarding configuration of CodeSafe on the 5c.