Preload Utility

Overview

The preload utility loads persistent cryptographic objects (keys/OCS/softcards) onto a chosen set of modules, then makes those objects available for use by applications. This removes the need for applications to load keys/cards themselves, and allows for easy sharing of keys/cards between multiple applications. Additionally, preload can manage keys, such that they are reloaded/maintained on modules to provide high availability.

Preloading is achieved via keys/cardsets being loaded, then once loaded the IDs of these objects are recorded persistently to a file (the preload file), which can be read via another application sharing the same session, and subsequently used.

Keys/cardsets must have previously been created before they can be preloaded, and all modules participating in a preload session must be in the same security world.

The preload binary can be found in /opt/nfast/bin. This binary calls the preload.py script found in /opt/nfast/python/scripts

The image below shows the relationship between preload, modules and applications:

preload modeuls and applications

Using Preload

Preload Commands

A command is needed in order to run preload. This command needs to be specified after the preload arguments.

The purpose of this command is to decide what needs to be done after preload has found and loaded all its crypto objects (OCS/softcards/keys).

$> preload [arguments] command

Preload has a choice of 3 commands:

  1. pause - continue to run the preload process forever. This is useful to load keys in one session and use them in another.

  2. exit - exit preload gracefully. This is useful to add keys to the preload session. Not available in combination with high availability mode.

  3. subprocess- execute this subprocess and exit once the subprocess has finished

The only exception to this is the --list-admin option that does not require a command.

The preload session remains open, and thus the preloaded keys remain loaded, as long as at least one instance of preload continues to run. If/when the final preload instance terminates, all loaded objects will be cleaned up.

Example showing a single key, of type simple, being loaded and then an application being launched:

$> preload -A simple -K key1 myapplication.py

Preload file location

The environment variable NFAST_NFKM_TOKENSFILE holds the path to the preload file. If it is not set, then the default location is used. A non-default location can also be set via the --preload-file option when invoking preload.

Preload Command Line Arguments

Argument Effect

--version

show program’s version number and exit

-h, --help

show help message and exit

-m MODULE_NUMBER, --module=MODULE_NUMBER

Load on specified module (may be repeated; default = all).

-c IDENT, --cardset=IDENT

Load all cardsets matching IDENT. If IDENT looks like a hash it will be interpreted as that, otherwise it will be interpreted as a name. If it is definitely a name, use --cardset-name.

--cardset-name=NAME

Load cardset(s) named NAME.

-s IDENT, --softcard=IDENT

Load all softcards matching IDENT. If IDENT looks like a hash it will be interpreted as that, otherwise it will be interpreted as a name.

--softcard-name=NAME

Load softcard(s) named NAME.

-o, --any-one

Load a single cardset.

-i, --interactive

Load cardsets interactively until told to stop.

-A APP, --appname=APP

Choose the appname for subsequent -K options.

-K IDENT, --key-ident=IDENT

Load keys with ident matching IDENT.

-n PATTERN, --name-pattern=PATTERN

Load keys with name matching PATTERN. Use * for wildcard.

--name-exact=NAME

Load keys with name NAME.

-M, --module-prot

Load all module protected keys, in addition to any others requested.

--no-cardset-keys

Do not automatically load keys protected by requested cardsets. Deprecated.

--no-token-keys

Do not automatically load keys protected by requested tokens.

--admin=KEYS

Load admin keys (separate with commas, or use all).

--list-admin

List available admin key names (for --admin).

-F, --require-fips

Require FIPS-auth to be loaded.

-N, --no-fips

Do not record FIPS auth, even if available. (overrides -F).

-H, --high-availability

High availability mode.

--polling-interval=POLLING_INTERVAL

Interval (s) between polls for changes to the module list (default=60). High availability mode only.

-f PRELOAD_FILE, --preload-file=PRELOAD_FILE

Use specified preloaded objects file, instead of the default.

-R, --reload-everything

Reload keys and tokens that are already loaded.

--show-key-info

Display key information for keys as they are loaded.

-l, --file-logging

Log to file.

-S, --no-stderr-logging

Do not log to stderr, this is independent of file logging.

--log-file=LOG_FILE

The file destination for the log, defaults to preload_%pid.log in the nfast log directory.

--log-level=LOG_LEVEL

The log level to log, options: DEBUG, INFO, WARNING, ERROR. Default is INFO, if unrecognized option it will fall back to default.

Pattern Matching

Options to preload that use pattern matching, namely --name-exact and --key-ident, can accept the following wildcards:

Wildcard Definition

*

matches everything

?

matches any single character

[seq]

matches any character in seq

[!seq]

matches any character not in seq

It is advised that all arguments that using wildcards are surrounded by quotations to ensure that they are passed to preload as intended. For example, to load all keys whose names start with keyname, the following pattern could be used:

$> preload --name-pattern 'keyname*' exit

Preload File

The IDs of preloaded crypto objects are persistently stored in a preload file.

Each entry has the following format:

Element Description

Hash

the sha1 hash of the crypto object

module

The module which this object is present

objectid

The id reference as a M_KeyID

generation

…​.

Example nfkminfo output with preloaded crypto objects:

Pre-Loaded Objects ( 10):  objecthash    module     objectid   generation
c29da3ac0d99a7c01477831ac31a4bebe283c4f8      1   0xac57be2e            1
c29da3ac0d99a7c01477831ac31a4bebe283c4f8      2   0xac57be2d            1
1080cca2be9588e6e47bcd870ebcbb133ea0561b      1   0xac57be2c            1
1080cca2be9588e6e47bcd870ebcbb133ea0561b      2   0xac57be13            1

By default the preload file location is /tmp:

/tmp/nfpriv_<username>/default

This location can be changed by using the command line option -f PRELOAD_FILE, --preload-file=PRELOAD_FILE.

Softcard Support

Softcards are now supported in preload, along with module protected keys and OCS cardsets.

In order to preload a softcard and the corresponding keys being protected by said softcard the -s or --softcard-name arguments can be used.

The -s option can be used with the softcard name or the hash of the softcard:

$> nfkminfo -s
...
Operator logical token hash               name
3768b8efb7c7324dd8a1edbe2650c2015281c877  test
$> nfkminfo -k simple aes128simplesoftcard1
...
name                  "aes128simplesoftcard1"
hash                  07c8110498dc0315455457f25564fc288c7da304
...
softcard              3768b8efb7c7324dd8a1edbe2650c2015281c877
$> preload -s test nfkminfo
...
Pre-Loaded Objects ( 4):  objecthash   module objectid  generation
07c8110498dc0315455457f25564fc288c7da304   1 0xa411c0ab 1
07c8110498dc0315455457f25564fc288c7da304   2 0xa411c09e 1
3768b8efb7c7324dd8a1edbe2650c2015281c877   1 0xa411c09d 1
3768b8efb7c7324dd8a1edbe2650c2015281c877   2 0xa411c0a0 1

This shows the softcard is loaded on modules 1 and 2. It additionally shows that the key protected by the softcard has been loaded on both modules.

No Cardset Keys

The --no-cardset-keys command line option can also be used for softcards.

This command line option will ensure that only the softcard is preloaded, and no keys protected by that cardset:

$> preload -s test --no-cardset-keys
...
Pre-Loaded Objects (  2):  objecthash   module objectid   generation
3768b8efb7c7324dd8a1edbe2650c2015281c877     2 0xa9ba32a9 1
3768b8efb7c7324dd8a1edbe2650c2015281c877     1 0xa9ba32aa 1

FIPS Auth

FIPS Auth can be made available via preload.

The command line -F will ensure FIPS auth is preloaded everywhere.

The command line -N will ensure FIPS auth is not recorded, and will negate -F.

FIPS auth is also an admin key, see Admin Key section for more information.

Admin Keys

Listing

Admin keys can be listed using the --list-admin command line option.

This should be run without a command:

$> preload --list-admin

Available admin keys are NSO, M, RA, P, NV, RTC, FIPS, MC, RE, DSEE, FTO.

Loading

Admin keys can be loaded using the --admin=KEYS command line option, supplying the value --admin=ALL to load all available admin keys. Note that admin key loading will require an ACS card being present in a slot of each module that is to be used.

Also note that the logical token of the admin key is preloaded alongside the key itself. E.G. kfips and ltfips.

High Availability

Preload provides a high availability mode. When this mode is invoked Preload will load all requested keys, and will then periodically check for modules added or removed from the security world, or for keys becoming unloaded on existing modules. Should old or new modules be found to not have the specified keys/cardsets loaded, then preload will attempt to load them. This ensures that all available/usable modules have the requested keys loaded at all times, available for use by applications. Merged keyIDs are used to ensure applications can continually use these keys without interruption or changing key IDs. Preloaded keys are not only available to one application, but to any/all applications that share the preload session.

When preload is invoked with the --high-availability or -H option, it does the following differently:

  1. Whenever preload loads a key onto the HSMs, it creates a Merged Key to represent the set of (HSM, key ID) pairs. Applications will then use these merged IDs to address the keys.

    • As discussed below, this in itself provides failback, resilience and increased availability: the Merged Key ID remains usable even if some HSMs fail or are removed from the security world.

  2. For as long as preload is running, it does the following repeatedly, once per polling interval:

    • Consult the hardserver to get a list of operational HSMs which are in the relevant security world.

    • For each Merged Key that was loaded by this instance of preload:

    • Ensure there is a valid current entry for each usable HSM.

    • To achieve this, check HSMs and load (or re-load) keys onto them as necessary, and update Merged Key contents.

    • Ensure that the individual key IDs within each Merged Key are valid: Remove any that are no longer valid and usable (e.g. those for a removed HSM).

    • Update the preload file to reflect changes, if any.

    • When finished, sleep for an interval of time, then repeat.

In summary, this mode attempts to keep preloaded crypto objects present on all usable modules in a security world (or a set of modules if requested via the -m argument) for as long as preload is running, with a keyID that remains constant, so that keys are available for use by any applications sharing the preload session.

Prerequisites for high availability mode

Users should not mix and match instances of preload with and without the -H high availability option, if those instances are sharing a session.

Managing OCS cardset-protected keys requires the following:

  • the OCS protecting the key(s) be a 1/N quorum

  • the passphrase for each card of the OCS set be identical

  • one card of the OCS set be left inserted in a slot (local or remote) for each module

  • if the card is non-persistent, it must be left in a local slot.

Differences from legacy behaviour

When running in high availability mode, certain behaviours of may differ from those outside of high availability mode. This includes the prompts for PIN entry, error messages, etc. This is due to a necessary difference in implementation between the two modes, and is expected.

Conditions for Management/Reloading

As mentioned above, preload in high availability mode will (re)load keys onto modules when a module is usable. A module will be considered usable if that module is in operational mode and in the correct world (and in the case of OCS protected keys, if a card from the OCS set is inserted into the module, locally or remotely). Preload will not attempt to perform actions that involve world administration, such as world loading or client enrolment. Users are responsible for managing worlds and client enrolment, and thus for bringing modules into a usable state.

The automatic loading/reloading of keys onto usable modules is not to be confused with forced reloading of keys provided by the -R option.

Merged Keys in the Preload File

When high availability mode is activated, all keys are represented in the preload file as Merged keys; cardsets and softcards are represented in the same way as non-high-availability mode.

Due to the fact that in high availability mode keys are represented as MergedKeys, which do not correspond to any one particular module, the module element of the preload file is no longer relevant for keys. However, for cardsets, the module field is still utilized.

For symmetric and private halves of asymmetric keys the module number is represented as a -1 and for public halves of asymmetric keys the module number is represented as a -2.

This is evident in the output from nfkminfo. (Note that nfkminfo ignores the 32-bit two’s-complement representation, thus displaying -1 and -2 as (232 - 1) and (232 - 2) respectively: 4294967295 and 4294967294):

Pre-Loaded Objects (  4):  objecthash   module objectid  generation
84749a62d0f71db7f80c5df6469c11685f7f1b78   1 0xb5c0c7fa 1
84749a62d0f71db7f80c5df6469c11685f7f1b78   2 0xb5c0c7fd 1
28dcee51dfc53387f4dc4d55538d8b5253ee85d1 4294967295 0xb5c0c7f7 1
c2afe833ae6e823a37777c633a5b3a18a9e5dfbd 4294967294 0xb5c0c7f8 1

As shown above, cardsets/softcards are still module specific.

To make nfkminfo show the preloaded objects, run it as a subprocess as part of the preload command. (see above section on using preload)

Merged Key IDs (just like single-key IDs) are shared between multiple instances of preload that are invoked by the same client (i.e. using the same ClientID). As such, applications must ensure that they perform no operations that delete or replace the merged key ID, or alter the keys that are part of that merged key ID.

Polling Interval

Preload manages its crypto objects by polling available modules, based on a polling interval.

Once per interval, if preload detects modules (new or existing) without the relevant crypto objects (keys/cards) present, it will attempt to load those missing objects.

This polling interval is configurable via the command line option --polling-interval=SECONDS

By default the polling interval is 60 seconds.

Key timeouts and use limits

It is advised to not use OCSs or keys with timeouts in high availability mode, as preload will be unable to reload objects once their timeouts have expired.

In high availability mode, there are situations where OCS/keys that have previously timed out, or reached maximum use limits, may be reloaded (and thus their limits reset) without user interaction. In general within high availability mode keys that have timed out or reached their use limits will be left in place, unusable, respecting the limits. However if the module containing those keys reboots or resets then, upon the module’s return, preload will notice that the keys are not loaded and will load them. This reloading of keys will necessarily reset timeouts and use limits. If the timeout on an OCS has reached its limit, any keys protected by that OCS will not be reloaded on newly-indoctrinated modules in the security world.

Multiple Preload instances in high availability mode

As described above, keys will be maintained by the preload instance that first introduces them, and will cease being maintained when that instance ends. (Here maintained means reloaded automatically onto relevant HSMs that lack them.)

Therefore when preload is invoked with exit (or a short-lived subprocess command) it will load the specified keys but then exit, leaving those keys unmaintained.

If a preload process is already running under high availability mode, any new preload process (with the same preload file) will gain access to the preloaded keys. As such that later instance must also be run in high availability mode (and preload will reject an attempt to run it in plain mode in this situation).

The pause command may be useful for setting up availability of keys for subsequent use by multiple applications:

First, a long-running preload instance to load keys and maintain them indefinitely:

$ preload --high-availability [...other options...] pause

Then run applications (possibly short-lived) that use those keys:

$ preload --high-availability [...other options...] app --args --for --app

Managing Keys

Given multiple preload processes run under high availability, the process that will manage the keys is the first process to find them, based on command line options.

For Example, Security World crypto objects:

crypto object name protected by

Softcard

softcard1

N/a

Key

simple_softcard1

softcard1

Key

simple_module1

module

First preload process started:

$> preload -H -s softcard1 pause

This would load the softcard softcard1 on all modules as well as the key simple_softcard1:

$> preload -H nfkminfo
...
Pre-Loaded Objects (  3):  objecthash   module objectid  generation
29235f2a0b77fc1e18641b0820fe3c93e030a02e 4294967295 0x44313d41 1
5bccb6f540802ef1da3828f6b8b0f3fc985272e6   2 0x44313d47 1
5bccb6f540802ef1da3828f6b8b0f3fc985272e6   1 0x44313d46 1
...
$> nfkminfo -k simple simplesoftcard1
...
name                  "simple_softcard1"
hash                  29235f2a0b77fc1e18641b0820fe3c93e030a02e
...
$> nfkminfo -s
...
Operator logical token hash               name
5bccb6f540802ef1da3828f6b8b0f3fc985272e6  softcard1

Second preload process started:

$> preload -H -n simple pause

This would load the key simple_module on all modules:

$> preload -H nfkminfo
...
Pre-Loaded Objects (  4):  objecthash   module objectid  generation
600bcc26336c13f2371bdbb54b1cde293ded9a15 4294967295 0x44313d29 1
29235f2a0b77fc1e18641b0820fe3c93e030a02e 4294967295 0x44313d41 1
5bccb6f540802ef1da3828f6b8b0f3fc985272e6   2 0x44313d47 1
5bccb6f540802ef1da3828f6b8b0f3fc985272e6   1 0x44313d46 1
...
$> nfkminfo -k simple simplemodule1
...
name                  "simple_module1"
hash                  600bcc26336c13f2371bdbb54b1cde293ded9a15

The evidence that the first preload process is still managing the key simple_softcard1, even though the second preload process could have loaded it, is in the objectid.

The object id for key simple_softcard1 has not changed (0x44313d41).

FIPS Auth in High Availability mode

Fips auth can be preloaded when running preload in high availability mode. In this scenario fips auth will be loaded as a high availability key (ie, reloaded/maintained on modules, as with other preloaded keys).

To enable FIPS auth use the command line option -F.

However, note that fips auth is represented differently, in comparison to other high availability mode keys, within the preload file.

The FIPS auth key is represented in the preload file multiple times: once for each module it is loaded on, and one extra time with a negative module ID as with other merged IDs. However the objectid is still a Mergedkey so will remain the same across those entries. This duplication of entries is to maintain compatibility with legacy behaviour/applications.

The following shows the pre-loaded FIPS auth objects on an estate of 3 modules - note there are 4 entries, each with the same objectid:

Pre-Loaded Objects (  4):  objecthash   module objectid  generation
aa462d0dd9dfeaa80968aadda2610ac0f6f94352   3 0xa824b9ab 1
aa462d0dd9dfeaa80968aadda2610ac0f6f94352   2 0xa824b9ab 1
aa462d0dd9dfeaa80968aadda2610ac0f6f94352   1 0xa824b9ab 1
aa462d0dd9dfeaa80968aadda2610ac0f6f94352 4294967295 0xa824b9ab 1
...
hkfips      aa462d0dd9dfeaa80968aadda2610ac0f6f94352

PKCS #11 and JCE

Both PKCS #11 and JCE applications are compatible with the high availability mode of preload, provided the PKCS #11 or JCE library that the application uses is from the 12.60 release or later. Flags or environment variables only need to be set to enable this when PKCS #11 is used for key reloading.

Use PKCS #11 for key reloading

PKCS #11 key reloading requires preload to be run in high availability mode, with the following options enabled:

  • --high-availability.

  • --no-token-keys.

  • --preload-file=PRELOAD_FILE, where PRELOAD_FILE must match the location given to PKCS #11 with the NFAST_NFKM_TOKENSFILE environment variable.

  • Either --cardset=<IDENT> or --softcard=<IDENT> (depending on whether using card set or softcard protected keys), where <IDENT> is the identifier of the card set or softcard, respectively.

    PKCS #11 key reloading is also supported for module-protected keys, but the PKCS #11 application must still be run under a preload application that is reloading tokens for another key.
    Using preload in high availability mode with Operator Card Sets has a set of restrictions, see Overview.

  • Additionally, the following option is not required, but recommended:

    --polling-interval=<POLLING_INTERVAL>, where <POLLING_INTERVAL> also determines how often PKCS #11 will attempt to reload keys. The default is 60 seconds.

For more information, see section PKCS _11 with key reloading in the Cryptographic API Integration Guide.

Unsupported options

The -H --high-availability option may not be used in conjunction with any of the following options:

  • -o --any-one

  • -i --interactive

  • exit

  • --admin

  • --reload-everything

Logging

By default preload logs to stderr.

Logs follow the format: yyyy-mm-dd hh:mm:ss: [pid]: LogLevel: message

e.g.

2019-03-27 09:45:50: [439]: INFO: loading objects

Preload can also log to a file, this behaviour is separate from stderr logging. Therefore we can disable logging or log to stderr and/or a file.

To disable stderr logging, use the command line option -S. To enable file logging use the command line option -l.

The default file location for logs is /opt/nfast/logs/preload_log_pid.log.

To change the file location, use the command line option --log-file=FILE.

As standard, preload has different log levels. These are:

  • DEBUG

  • INFO

  • WARNING

  • ERROR

  • CRITICAL

The log level is by default: INFO and can be changed via the command line option --log–level=LEVEL.

Using preloaded objects - Worked example

In order to use preloaded objects, an application needs to create a connection that reads in the preload file:

Python:

import nfkm

conn = nfkm.connection(existingobjects="") # Reads file from default location

# If no existingobjects parameter is specified,
# the connection will not attempt to read any preload file:
conn_no_preload = nfkm.connection()

If the existingobjects argument is the empty string, the connection will use the file from the default location.

Any other string should be a path to different preload file. It can then call NFKM_GetInfo to get the security world info:

Python:

world_info = nfkm.getinfo(conn)

This results in a data structure with all the preloaded objects (this list is static and created at the time of connection creation):

Python:

import nfkm

conn = nfkm.connection(existingobjects="")

world_info = nfkm.getinfo(conn)

print world_info.existingobjects

Result:

[
ExistingObjectInfo
.module= 2
.hash= 84749a62 d0f71db7 f80c5df6 469c1168 5f7f1b78
.change= 1
.id= 0xffffffff88afd208,
ExistingObjectInfo
.module= 1
.hash= 84749a62 d0f71db7 f80c5df6 469c1168 5f7f1b78
.change= 1
.id= 0xffffffff88afd20b
]

Once an application has the M_KeyID references, it can use those cryptographic objects:

objid = world_info.existingobjects[0].id

cmd = nfkm.Command(["GetLogicalTokenInfo", 0, objid])

print conn.transact(cmd)

Result:

Reply.cmd= GetLogicalTokenInfo
.status= OK
.flags= 0x0
.reply.state= Present
.hkt= 84749a62 d0f71db7 f80c5df6 469c1168 5f7f1b78
.shares= empty
.sharesneeded= 0