Audit Logging

Audit Logging on nShield HSMs provides the following features:

  • Logs generated and signed on the nShield HSM

  • Tamper detection

  • Deletion Detection

  • Administrative operations are logged

  • Key lifetime events are logged

  • Per key usage events are optionally logged

  • Optional key usage logging

  • Public key verification of audit logs

  • Compatibility with syslog and Security Information and Event Management (SIEM).

Configuring Audit Logging

Audit Logging is enabled per Security World and is configured on creation of the Security World.

Prerequisites

  • If the audit logs are to be sent to a non-default location, the configuration file must be set up before the Security World is created.

  • The Real Time Clock (RTC) on the HSM must be set and the setting confirmed after creating the Security World or indoctrinating an HSM into the Security World. The RTC on the HSM is used to timestamp audit log entries.

Configure audit log transport through syslog

The Audit Logging entries are delivered over syslog using UDP transport. This transport must be configured before Audit Logging is enabled in order to collect the initial messages.

  1. Check the syslog transport before creating an Audit Logging enabled Security World.

    Send a log message to the configured host and port using a command, for example logger, that can send messages to a syslog server over UDP.

  2. Set the Audit Log entries in the hardserver configuration file.

    [auditlog_settings]
    # Start of the auditlog_settings section
    # Hardserver settings for audit logging.
    # Each entry has the following fields:
    #
    # The port number Auditlogging server listens to .
    auditlog_port=514
    #
    # IP Address of the Auditlogging server
    auditlog_addr=WWW.XXX.YYY.ZZZ
    #
    # Copy auditlog to hardserver log. (default=no)
    #  auditlog_copy_hslog=ENUM
    auditlog_port

    This is the UDP destination port for Audit Logging syslog messages. The default is 514.

    auditlog_addr

    This is the IP address of the host to which the Audit Logging syslog messages should be delivered. The default is 0.0.0.0.

    auditlog_copy_hslog

    This indicates that the syslog messages from Audit logging should be copied to the hardserver’s log file. This provides some degree of redundancy but means that the hardserver’s log file may grow significantly. The default is no. It is not recommended on nShield network-attached HSMs.

  3. Restart the hardserver to load the updated configuration file.

  4. Push the configuration to the nShield HSMs in the Security World and restart the hardserver on each HSM. See Utilities for network-attached HSMs.

Create a Security World with Audit Logging enabled

For the overall procedure, see Creating a Security World using new-world.

Key considerations:

  • A Security World is created with Audit Logging enabled if either the --audit-logging or -G options are passed to the new-world command or the Security World is created in the common-criteria-cmts mode. This requires that the HSM is capable of audit logging and Security World creation will fail if the HSM does not support Audit Logging.

  • Additional HSMs are indoctrinated into an Audit Logging enabled Security World using the new-world command with the --program or -l options.

  • The HSM must be capable of Audit Logging. If it is not capable the indoctrination will fail. Therefore all HSMs in an Audit Logging Security World are set to Audit Logging.

When you configure an Audit Logging Security World:

  1. Audit Logging is set as enabled for this Security World and is recorded in the world file.

  2. The HSM is initialized and:

    • A flag indicating the Audit Logging status is recorded in the HSM’s EEPROM

    • A 3072bit DSA HSM specific Audit Logging Signing Key (KAL) is created and persisted in the HSM’s EEPROM

    • A Certifier Block containing the public half of KAL is generated and sent to the log receiver via the hardserver.

When you indoctrinate a new HSM into an Audit Logging Security World:

  • The world file specifies that this is an Audit Logging Security World

  • The same actions as for initializing an HSM when the Audit Logging Security World was created are performed. This means that:

    • All HSMs in an Audit Logging Security World have Audit Logging enabled

    • Each HSM has a distinct Audit Logging Signing Key.

Confirm the Audit Logging configuration

Check for AuditLogging on the state line in the output of nfkminfo.

Enabled

AuditLogging

Disabled

!AuditLogging

>nfkminfo
...
 state       0x37270009 Initialised Usable Recovery !PINRecovery !ExistingClient RTC NVRAM FTO AlwaysUseStrongPrimes !DisablePKCS1Padding !PpStrengthCheck AuditLogging SEEDebug
...

Check AuditLogging on the active modes line in the output of enquiry.

Enabled

AuditLogging

Disabled

(AuditLogging is not listed)

>enquiry
...
mode                 operational
...
active modes         AuditLogging UseFIPSApprovedInternalMechanisms AlwaysUseStrongPrimes
hardware status      OK

Disable Audit Logging

Audit Logging is set for the lifetime of the Security World.

To disable Audit Logging on an HSM:

  1. Remove that HSM from the Security World.

  2. Reinitialize the HSM using initunit.

Audit Logging architecture

Audit Logging is implemented on the nShield HSM. The Audit Logging capability is on the embedded nShield 5s card. The following image displays the nShield HSM implementation.

The audit log entries are generated on the module, the hardserver acts as a relay to a syslog infrastructure. The logging is at the level of nCore commands as processed by the HSM.

Implementation architecture

The hardserver layer implements a higher-level abstraction which is presented to application clients. The information used to manage this such as Client Identifiers etc. is not available to the HSM and therefore cannot be logged.

Audit Logging implementation

The Audit logging functionality is based on that described in RFC-5848 ( https://tools.ietf.org/html/rfc5848 ). This describes a mechanism also known as syslog-sign that adds the following capabilities to syslog:

  • Origin authentication

  • Public verification

  • Message integrity

  • Replay resistance

  • Message sequencing

  • Detection of missing messages.

It is implemented on top of a standard syslog data stream and does not use any additional protocol. The syslog-sign logging scheme adds a number of control messages to the log entries that are to be audited. These are also implemented as syslog messages. The following sections outline the audit log entries that are present in the syslog data stream for nShield Audit Logging. The signing mechanism used is DSA with a 3072 bit key and SHA-256 as the hashing mechanism.

Audit log entry

This is the log message from the entity being audited. It is in a standard format and includes operation specific data required to provide an auditing capability. As each log message is generated on the HSM, a digest operation is performed on it and the digest buffered in the HSM.

Signature Block

When sufficient digests have been accumulated (N), a Signature Block is generated as a standard log entry containing the following:

  • Digests of the previous N messages

  • Information to allow the digests to be matched with their respective log entries

  • A signature across the digests and other information.

The number of messages is dictated by the transport medium, the size of the digests, the size of the signature and the size of other data contained in the message. There is a limit to the size of messages that can be transported over syslog. The signature is performed using a log signing key. This key is generated and the private half is held securely in the HSM.

Certifier Block

Verification of the Signature Blocks requires that you, or the application performing the verification, has access to the public half of the log signing key. The Certifier Block provides a mechanism for the log verifier to get access to this key. The key is packaged in a form allowing the source of the audit logs to be verified. As the size of this information may be too large for the syslog transport medium it can be broken down into Certifier Block Fragments which are compatible with the syslog transport mechanism. When all of these fragments are received by the log verifier, it can reconstruct the public half of the log signing key and perform any consistency checks and origin verification that is needed.

Audit Log Verification process

Given the public half of the log signing key, a Signature block and its corresponding log entries, the verifier can check the signature on the Signature Block. When this is verified, the log entry digests in the Signature Block are implicitly verified. The integrity of the corresponding log entries can be verified by performing a digest on received log entries and comparing them to the corresponding verified digests in the Signature block. The image below shows the basics of this approach. For more information, see Audit Log Verification.

Log Verification

Log distribution

The nShield Audit Logging capability uses the RFC-3164 (https://tools.ietf.org/html/rfc3164) standard for distributing audit log messages. All audit log messages will have the following header prepended. This header is applied by the hardserver before sending the message and does not form part of the signed audit log messages. The signed portion of the audit log message starts at the CEF:0 CEF identifier and continues to the end of the message.

<134>MMM DD HH:MM:SS hostname CEF:0......

The PRI value of this header <134> indicates the facility local0 and a severity of info.

The syslog infrastructure used for Log distribution is out of the scope of this guide and your responsibility to implement. Log distribution for Audit Logging uses syslog as the transport medium which allows a standard protocol and message format to be used for the Audit Logging messages. This transport is compatible with SIEM systems and the wider syslog infrastructure in an organization can be used to further distribute or process the log stream. There are many possible configurations of syslog deployment, as shown below.

syslog configurations

Configuring audit log distribution

The actual implementation of the syslog infrastructure is at your discretion. Verification of the log messages requires that the verifying application has access to the audit logs from the HSMs in the Security World. The example verifier for the nShield Audit logging facility described in Audit Log Verification processes a file containing the audit log messages. It can process audit log messages from a specific HSM identified by its ESN or will use the first ESN found in the file.

It is recommended that logs from the nShield Audit Logging facility are separated from those from other applications. This can be accomplished by using the information in the audit log messages described in the section on Log Format. There are a number of entries that can be used to separate out the messages from the nShield Audit Logging facility. These include:

  • Identifying elements in the CEF header:

    • Device Vendor

    • Device Product

  • Identifying elements in the syslog header

    • Hostname or IP address of the machine hosting the hardserver which is distributing the audit log messages

  • Using a distinct port for nShield Audit Logging see configuring syslog.

The log messages can be further split into those from specific HSMs using the ESN in the audit log messages.

As an example, the following rsyslog configuration will direct all messages with the string nCipher Security to a specific log file:

:msg, contains, "nCipher Security" /var/log/hsmauditlog

A similar strategy can be used with syslog-ng:

destination d_auditlog { file("/var/log/hsmauditlog"); };
filter f_auditlog { match("nCipher Security" value("MESSAGE")); };
log { source(s_log); filter(f_auditlog); destination(d_auditlog); }

Adjust the example paths for the platform running your syslog server as appropriate. s_log is the source receiving the audit logging messages.

Refer to the documentation for your syslog implementation for information on processing and distributing log messages.

Configuring the syslog message infrastructure

It is important that the syslog infrastructure does not attempt to rewrite the log messages as this will affect the ability of the Audit Logging process to verify log messages. For example, the rsyslog default RFC-3164 parser will rewrite log messages interpreting the CEF:0 as a tag and will write CEF: 0 to the log file. This means that an Audit Logging message persisted by the default RFC-3164 parser cannot be verified as Audit Logging signs the log message starting at CEF:. You must configure your syslog infrastructure to preclude the signed part of the audit log message.

rsyslog

rsyslog can be configured to not reformat messages using the following approach:

  1. Define a formatting template as shown below in the /etc/rsyslog.conf file. This should be added in the ##### MODULES ##### section of the rsyslog configuration file.

    $template myFormat,"%rawmsg%\n"
  2. Apply this formatting template to the processing of Audit Logging messages. For this example it is assumed that messages containing nCipher Security will be persisted in the /var/log/hsmauditlog file. You can use any other selection mechanism such as storing messages for a particular HSM as identified by its ESN in separate files.

    Linux
    :msg, contains, "nCipher Security"         /var/log/hsmauditlog;myFormat
    Windows
    :msg, contains, "nCipher Security"         \var\log\hsmauditlog;myFormat
  3. If the rsyslog server is going to be used as a relay, then the format needs to be applied to any relay statements in the rsyslog configuration file and to any receivers of the syslog message.

syslog-ng

syslog-ng does not appear to rewrite messages in the same way as rsyslog. Refer to the syslog-ng documentation for information on formatting.

Audit log format

CEF format

The audit log entries are emitted from the HSM in CEF format. This provides both human readable log messages and compatibility with SIEM applications. As indicated in the previous section the audit log entries are distributed using the syslog transport mechanism.

A CEF format log message is shown below:

CEF:Version|Device Vendor|Device Product|DeviceVersion|Device Event Class ID|Name|Severity|[Extension]
Parameter Description

CEF:Version

This is mandatory and Version is currently 0.

Device Vendor

This is nCipher Security

Device Product

This identifies the family of nShield HSMs:

  • nShield Solo

  • nShield Solo XC

  • nShield Edge

  • nShield 5s

Device Version

This is the firmware version, for example 12.80.0.

Device Event Class ID

This is an identifier for the type of message:

Class ID Description

1

nCore Commands

2

Internal HSM events:

  • Periodic heartbeat

  • Secure channel establishment

3

Audit logging control messages:

  • Signature Blocks

  • Certifier Blocks

4

Reserved

5

Shutdown messages

6

Reserved

Name

This is the event being logged. For Audit Logging, it is either the nCore command that is being logged, Cmd_Destroy for example, a description of the event such as heartbeat or one of ssign or ssign-cert which identifies either a Signature Block or a Certifier block.

Severity

This is an indication of the importance of the message.

Severity Description

1

nCore Commands

2

Internal HSM events:

  • Reboot events

  • Secure channel establishment

3

nCore Commands that force a Signature Block flushing buffered
message hashes.

4

Periodic Heartbeat messages

5

Audit Logging control messages:

  • Signature Blocks

  • Certifier Blocks

6

Shutdown messages

10

HSM Error messages

CEF extensions

The rest of the log message is made up of CEF extensions. These are name/value pairs that are used to convey specific information for the log message. The name-value pairs can be processed by SIEM applications such as Arcsight and can be displayed in tabular reports of the messages received. They can be used for filtering and further processing within the SIEM application. The following table specifies the meaning and format of the extensions used by the Audit Logging facility.

Extension Name Description

esn

Electronic Serial Number (ESN) of the HSM in the following format:
XXXX-XXXX-XXXX

rtc

Time-stamp from the HSM’s Real Time Clock (RTC) as ms since the epoch (1970 Jan 01 00:00:00 UTC).

outcome

Outcome of the operation - success or failure

hkey

Identifying nCore key hash for the main key of the command being logged as a 40 character hex string

hbase

Identifying nCore key hash for the base key of a Cmd_DeriveKey command being logged as a hex string

hwrap

Identifying nCore key hash for the wrap key of a Cmd_DeriveKey command being logged or the logical token hash for key blobbing operations as a hex string

hin3-5

Identifying nCore key hashes for the remaining keys of a Cmd_DeriveKey command being logged as hex strings

hknso

Identifying nCore key hash of Security Officer’s key as a hex string

htok

Identifying nCore Logical Token hash as a hex string

shareindex

Index of share being operated on by Logical Token functions as a decimal number

sharesleft

Number of Logical Token shares left to read or write as a decimal number

tokenslot

Slot number for Logical Token operations as a decimal number

sharesneeded

Quorum required to reconstruct a Logical Token as a decimal number

sharestotal

Total number of shares for a Logical Token as a decimal number

timelimit

How many seconds after reassembly the Logical Token is usable for

shorthash

Short Logical Token hash used in Cmd_EraseShare and Cmd_ChangeSharePIN

hkm

Identifying nCore hash of module key KM

mode

Mode that a channel is opened in. One of encrypt, decrypt, sign or verify

source

Source of command. One of host, SEE or internal

flags

Flags supplied to Cmd_SetNSOPerms and Cmd_InitialiseUnitEx.

Cmd_SetNSOPerms

  • AlwaysUseStrongPrimes

  • DisablePKCS1Padding

  • FIPSLevel3Enforcedv2

  • CommonCriteriaCMTSRestrictions

Cmd_InitialiseUnitEx

  • AuditLogging

  • UseFIPSApprovedInternalMechanisms

slotcount

Count of Dynamic Slots to be configured

slotid

Dynamic Slot to create association for

prevrtc

The previous value of the HSM’s RTC as ms since the epoch. Used to indicate previous value of the RTC before a Cmd_SetRTC timestamp or an event occurring before a restart

smartcardesn

ESN of smartcard used for Dynamic Slot operations

kmltype

Type of the Module Per-Initialization Signing Key (KML) set by Cmd_InitialiseUnit(Ex)

sos

Indication of the sos code

Infrastructure extensions

The Audit Logging Implementation requires a number of infrastructure CEF extensions to provide data necessary for the RFC-5848 based signed syslog approach used. Please refer to RFC-5848 for further details on these infrastructure extensions. These CEF extensions replace the RFC-5424 Structured Data used in the original scheme but have the same meaning.

Message and reboot counters

There are two counters that are sent with all Audit Logging command log messages. The Reboot Session ID is also sent with Certifier Block and Signature Block messages.

Counter Description

seqNo

Log message sequence number as a decimal number. This a counter that has a range of 1 to 9999999999. When seqNo reaches 9999999999 it is reset to 1. It is incremented for every command log message sent. This is not part of RFC-5848 and has been added to provide a direct mechanism for detecting deleted or missing log messages.

The sequence number is unique in the context of a reboot session. When rsid is incremented seqNo is reset to 1.

rsid

Reboot Session ID as a decimal number. This a counter that has a range of 1 to 9999999999. When rsid reaches 9999999999 it is reset to 1. It is incremented every time the HSM is restarted and whenever the Global Block Counter (gbc) reaches its limit and is reset.

Client ID Session Extension

The session extension is a unique identifier that refers to an HSM’s client. This allows audit logs to be traced to the client machine that initiated a command for the HSM.

When a client is enrolled into the HSM, the following takes place:

  • A session value is calculated from both the client’s IP address and its KNETI hash.

  • The HSM automatically generates an audit log with the Cmd_SessionCreate command that contains the IP address of the client, the client’s KNETI hash and the corresponding session ID.

    For example, for an nShield 5c HSM:

    CEF:0|nCipher Security|nShield5|13.4.0|1|Cmd_SessionCreate|1|esn=F4A9-050A-EBF8 rsid=5 rtc=1678208481435 seqNo=215 source=host outcome=success description=IP:"172.23.136.106:512";KNETI:"76c5516f834376eb34e7345fa3870fde764c962d" session=75c9cd2600000001

Subsequently, whenever the client executes a loggable command on the HSM, the audit log that’s generated will contain the session field with the client’s unique identifier.

If this session is terminated (for example, if the Impath Resilience session expires after a client disconnects and doesn’t resume the session before the timeout), then the HSM will generate an audit log with the Cmd_SessionDestroy command to indicate this.

For the same HSM:

CEF:0|nCipher Security|nShield5|13.4.0|1|Cmd_SessionDestroy|1|esn=F4A9-050A-EBF8 rsid=5 rtc=1678208481437 seqNo=216 source=host outcome=success session=75c9cd2600000001
Name Description

session

A unique identifier for the client that executed a command on an HSM. This field is only available on audit logs generated inside an HSM for commands initiated by the HSM’s clients.

Certifier Block extensions

The following extensions are used in the Certifier Block where the CEF header name element is ssign-cert. The Certifier Block is used to distribute the log signing public key. It is sent by the HSM when logging is enabled and every time the HSM is restarted. This provides for redundancy as any Certifier Block from an HSM that has been configured for Audit Logging will contain the same log signing public key. The Certifier Block can extend over multiple syslog messages. The extensions identified here allow the data of a Certifier Block to be rebuilt from multiple fragments. Sufficient fragments are sent in separate ssign-cert messages to rebuild the payload block. See Certifier Block example for the details of the data included in the certifier block.

Name Description

tpbl

Total Payload Block Length. This is the length of all Certifier Block fragments.

findex

Index of this fragment (1 based) as a decimal number.

flen

Length of the fragment as a decimal number.

frag

Base 64 encoded Certifier Block fragment.

sign

DSA signature using KAL of the data in each fragment up to the sign extension. The DSA signature is DER encoded and then base64 encoded. It is present here to support consistency checking.

Signature Block extensions

The following extensions are used in the Signature Block where the CEF header name element is ssign. The Signature Block supports the verification of Audit Logging messages. The Signature Block is sized to fit within a syslog message which dictates the number of audit log messages it covers. This release supports a maximum of 10 audit log messages per Signature Block. The main data for the Signature Block is the SHA-256 hashes of the log messages covered by the block.

Name Description

gbc

Global Block Counter as a decimal number. Count of signature blocks sent in this Reboot Session prior to this Signature Block. This is a decimal number that has a range of 1 to 9999999999. When gbc reaches 9999999999 it is reset to 0. At the same time the rsid is incremented and seqNo reset to 1. It is incremented after every Signature Block is sent.

fmn

First Message Number as a decimal number. First log message seqNo in this Signature Block.

hcnt

Count of log messages included in this Signature Block as a decimal number.

hb

The log message SHA-256 hashes base64 encoded and separated by the & character. The & character does not occur in base64 encoding and avoids SIEM issues with embedded spaces.

sign

DSA signature using KAL of the data in the Signature Block up to the sign extension. The DSA signature is DER encoded and then base64 encoded.

Example Audit Logging messages

This section shows example Certifier Block, Signature Block and Audit Logging messages and shows how the CEF extensions are used together.

Certifier Block example

This is an example Certifier Block produced after a reboot of the HSM. The log messages have been reformatted for display as each one can be up to 1024 characters long. The Reboot Session ID (rsid) is 8. There are five fragments in this example. The first four are 450 characters and the final 340 long for a total length of the payload of 2140 characters. The Event Class Id is 3 and the severity is 5 identifying these as infrastructure messages.

<14>Aug 10 18:22:18 nethsm CEF:0|nCipher Security|nShield Solo XC|12.60.9|3|ssign-cert|5|esn=7109-02E0-D947 rsid=9 rtc=1628616065737 tpbl=2140 findex=1 flen=450 frag=uwAAAEQAAAArPjRMy6QnKObhNt0va3OSSdLSTSEkO2Ydxj2sRmsTvmDJ5tqXeNE0p7GkmFfgO/yU2iTl9VlSJa7Opq+BKNTw3wEAAEQAAACIWksBX/JuSf8Cyv3WRfnUWvvCAHID+OGaoYvHqqlVtW7GzRMWMg/fPeO3UV32w2i+GkORU87i6dJKybxqhwVVQwAAAKwFAAAEAAAAAAAAAAMAAAACAAAADwAAADcxMDktMDJFMC1EOTQ3AAANAAAAa0OXfBuCMHaS7g0dVUE5vnVx43guAAAABgAAAAAAAABEAAAA4P0MEe9CDHohvBeOvBwLykF99mopVY1Rdam8554PDJWwhwID6+tUn8AJGOAscYtfrOxabeJ1iC+btrmJZbrlrXQBAABEAAAAdlxXLbwppzAjT8/H+off7ikbnLNQdozuNJeNaqb4+IZwCxcNwA

<14>Aug 10 18:22:18 nethsm CEF:0|nCipher Security|nShield Solo XC|12.60.9|3|ssign-cert|5|esn=7109-02E0-D947 rsid=9 rtc=1628616065739 tpbl=2140 findex=2 flen=450 frag=s3z1d9CeG3/LQpv6zXv9mC2HoFDes/dP/x3eUAAAC7AAAADgAAAPKa01J+3cWja4OICn9Bog4a4IlRAwAAAIABAADjwRLJlKVAir+HlVAUCWojKksMqGyWGhwhMoqYP8ldIy7bb3UVQBp6M+fxVpSFFrz3bfDgJQNh/13YCAY1+r1JYvEner7cnGatDIjnMgNqQPN6alqM787pMz3/eIq0L0xI8rVy99F/foV6aFcJVCvxsjL9wIQOd4AhjIgTfPTiAEC4UTl5Eg9YkKnjZXizpTxhReSZVMjIM8Fu2sjcvzh1Q8POqYcEuU5sZhQbLVjUvRpou2HpgOTwhcXW+X4gWpMlXsVkwV7F24j4Ax6eiyaSp1HCx4savMxcyA3cxwp7dTUJnFPVr8npfqp2H3ai3khSIefMc7d8gyajJnOLlJQMzf6O5HrlVeuixd6hBdwd

<14>Aug 10 18:22:18 nethsm CEF:0|nCipher Security|nShield Solo XC|12.60.9|3|ssign-cert|5|esn=7109-02E0-D947 rsid=9 rtc=1628616065740 tpbl=2140 findex=3 flen=450 frag=B1Ku9rirlixkgEd+73tMVJ1FQz85aCWuRqJl04YB1YwFvZgvRXhHvzqLFeJZAUerKlLgIaZwDq1twoXzvHq88QcJdbr0i4+87VorPKkEjKtSSGHOVkkHhoBC8uNgYXnTBxqcqCqpZl4whuiEBmJQLcwgAAAAg8rgckmo3ArobecQooPxQ9AjYbCmAoKOUTRi7grTzPyAAQAA3Bvuz+tQ1uh5LvuKMLTtGDTTplG7ks6ZkL8b+F2UW37jfN3lap27oAZq1otU4FOP4EVvoMmNSdI4uzCPi7VgcI3AcIkdjZIwbpYf9XQwvFwMxYvdBPGhPtc/t8Lslgs97rMkES4ZciNI/NwjKp0fW4kCiSBSUQUAUcp6vgqg2vVL9naqRHhXNRJuweaRtOO60z0mBkTgCnAvscdr2ymErrWDZArHosYXJZrXghjNmXvu+rS8GvTvTc

<14>Aug 10 18:22:18 nethsm CEF:0|nCipher Security|nShield Solo XC|12.60.9|3|ssign-cert|5|esn=7109-02E0-D947 rsid=9 rtc=1628616065742 tpbl=2140 findex=4 flen=450 frag=189gfRjMpL5aBYYAkl1XqWDGHhFcltxTSzCMgWalxMOeOQxaZLbzYDtl2/udXIo0bn/PTgaOkPYTypvzFwsQM4axpDzVYCE064YVoyjUpgWBU4kMlC4JuH5ytJAM+uA67xu36Iqx3j/mjMozTR1rGJuH+b314zgHRjvfr8AA0juiXn8tdxkFFRQlzYC9Ulw4g6fOSa06ecBIOA5Q0ylvdVjqmcX2++J+snC0wTxHV+vLKWh8m7/DDjExTXKpHo5EqyQB24tHCogAEAAJRhMgdRJ9i6dxLTUzQFo7ZzcQpZFdbFF4TN5+8Z/TER2/toZg+oNQkD2Eshd/T44p3WQr1XkQFjFe3/syQu8pS4Q0inmZzx6leFN99pXUhxXPnNIIpuc8UHJWips7nQnlsa3ER6evViNQanQSBTYEmuibXRITin8NM3h0RTJekDGw1J2wjO

<14>Aug 10 18:22:18 nethsm CEF:0|nCipher Security|nShield Solo XC|12.60.9|3|ssign-cert|5|esn=7109-02E0-D947 rsid=9 rtc=1628616065743 tpbl=2140 findex=5 flen=340 frag=DjWEMgrKNb0X6hYqOPg/Ozid2ytohqdRx0Hr2zel6Ha17HgFfAkXlYaPXxdIGnrsuNwub43HU9iVByMdSe8kjOigEW/jXzTZQMFPy/Iy1+GCV3P8GW+liYE/DM5auWH1a9NWbfjGTWQK4A820Eqy9zSUzOBnbn/gCLVUUMbSAGKY41Dtx7pUNU0TQpEMydBmfaoQhyx07fj070t8Zc2Ut2e5a2s/uxwb0nSQNYJ25SBDb8UjHnhtNJVHiANHU3Qu1JcUnEKUwQW00dLeLOaMtd2Ldy+QaHHjsk7WYhboWCCSsWiHa4z3nVQsr3BrrkxBUlilWtXQlVVmEaoAAAA=

Log Messages and Signature Block

This example shows a sequence of audit log messages with a Signature Block after 10 messages. These are in the same rsid as the previous example. The log sequence number for this excerpt starts at 31 and the last log message before the Signature Block is sequence number 40. The name element identifies the command being executed by the HSM. Each of the example commands operates on an nCore Key and this is identified by the nCore key hash of the relevant key.

The Signature Block has name element ssign identifying it as a Signature Block. The gbc is 6 meaning this is the 7th Signature Block in this Reboot Session ID. The fmn is 31 and hcnt is 10 meaning that this Signature Block covers messages 31 to 40. As Audit Logs are generated this sequence will be repeated. Once this Signature Block has been received and with the log signing public key available the signature on this Signature Block can be verified and then the hashes of the individual log messages can be calculated and compared with the hashes recorded in the Signature Block for the corresponding log message to allow the detection of tampering.

<134>May 15 12:42:09 myhost CEF:0|nCipher Security|nShield Solo|12.60.2|1|Cmd_Destroy|1|esn=1111-2222-3333 rsid=8 rtc=1526388047690 seqNo=31 source=host outcome=success
hkey=c4ab637985a542e7eb3eb4838f57872d5422bbb4

<134>May 15 12:47:09 myhost CEF:0|nCipher Security|nShield Solo|12.60.2|1|Cmd_Destroy|1|esn=1111-2222-3333 rsid=8 rtc=1526388347977 seqNo=32 source=host outcome=success
hkey=c4ab637985a542e7eb3eb4838f57872d5422bbb4

<134>May 15 12:52:10 myhost CEF:0|nCipher Security|nShield Solo|12.60.2|1|Cmd_Destroy|1|esn=1111-2222-3333 rsid=8 rtc=1526388648265 seqNo=33 source=host outcome=success
hkey=c4ab637985a542e7eb3eb4838f57872d5422bbb4

<134>May 15 12:53:21 myhost CEF:0|nCipher Security|nShield Solo|12.60.20|1|Cmd_GenerateKeyPair|1|esn=1111-2222-3333 rsid=8 rtc=1526388719548 seqNo=34 source=host outcome=success
hkey=cec7b0b1ef47d4141d65fdde9f9d23e854391dea

<134>May 15 12:53:21 myhost CEF:0|nCipher Security|nShield Solo|12.60.2|1|Cmd_Export|1|esn=1111-2222-3333 rsid=8 rtc=1526388719549 seqNo=35 source=host outcome=success
hkey=cec7b0b1ef47d4141d65fdde9f9d23e854391dea

<134>May 15 12:53:21 myhost CEF:0|nCipher Security|nShield Solo|12.60.2|1|Cmd_Export|1|esn=1111-2222-3333 rsid=8 rtc=1526388719549 seqNo=36 source=host outcome=success
hkey=cec7b0b1ef47d4141d65fdde9f9d23e854391dea

<134>May 15 12:53:21 myhost CEF:0|nCipher Security|nShield Solo|12.60.2|1|Cmd_Destroy|1|esn=1111-2222-3333 rsid=8 rtc=1526388719550 seqNo=37 source=host outcome=success
hkey=cec7b0b1ef47d4141d65fdde9f9d23e854391dea

<134>May 15 12:53:21 myhost CEF:0|nCipher Security|nShield Solo|12.60.2|1|Cmd_Import|1|esn=1111-2222-3333 rsid=8 rtc=1526388719550 seqNo=38 source=host outcome=success
hkey=cec7b0b1ef47d4141d65fdde9f9d23e854391dea

<134>May 15 12:53:21 myhost CEF:0|nCipher Security|nShield Solo|12.60.2|1|Cmd_Destroy|1|esn=1111-2222-3333 rsid=8 rtc=1526388719551 seqNo=39 source=host outcome=success
hkey=cec7b0b1ef47d4141d65fdde9f9d23e854391dea

<134>May 15 12:53:21 myhost CEF:0|nCipher Security|nShield Solo|12.60.2|1|Cmd_Import|1|esn=1111-2222-3333 rsid=8 rtc=1526388719552 seqNo=40 source=host outcome=success
hkey=cec7b0b1ef47d4141d65fdde9f9d23e854391dea

<134>May 15 12:53:21 myhost CEF:0|nCipher Security|nShield Solo|12.60.2|3|ssign|5|esn=1111-2222-3333 rsid=8 rtc=1526388719552 gbc=6 fmn=31 hcnt=10hb=8ogF/vsd9SwQ+qWEnealDuczRE9XbE9Rf3k3dc51SXo=&Xq5dbTEtg0l6pHQ7n6G16X+muB6C6VzN4FKbuqZqNUQ=&tOdfZk5Uvs6W9E8mJyEU4kkNHAImwYft0v+7mHSL7VY=&ct+wqWsptfc+asw9ppvYVNbmkpqU3/WbXm5nHJPri9E=&riegp83m5c3jYt2vymNf61ov+Jf8JqCeLSyhiSDRXfA=&WGKibqrL7AEKXU3Wu2IG1VcctIIESxyXluLcAFR+qvE=&TDrCHjkuA2fS5ZBgVu4Wspta+MbdhkyrGXeHHWn9Xck=&EwflNUbAIIVVdj06PlEvjlljRoIXiprw56cqGGLbp9w=&Y1j1x6GEOofqG9XdAqk91zFhA8bczBocsttvvNND3tI=&p5YnBsabKWL4F5+2WNmIJ9DmKuSeUz5v3qogOilztRY=sign=MEUCIDa7RJlKmMRJ4KrDWxDYYok1t9ptQXqH1nPE5xLihcgoAiEAr4JWBTxsF/XrN2kJVfiVpCRicbvgTNjjjnouohS6QTM=

Commands Audited

The Audit Logging facility generates log entries on the module for a set of nCore commands and module operations. The commands and information logged for each command are described in the following sections.

Key usage logging

By default the nShield Audit Logging Facility does not log usage of keys for cryptographic operations such as sign, verify, encrypt and decrypt or their usage in channels for these purposed. Creation, Deletion and a number of other key operations are unconditionally logged by default. The Audit Logging feature provides the capability to optionally log these operations. This is determined on a per-key basis by the LogKeyUsage permission group flag on the ACL group authorizing the operation for which logging is desired. See the nCore Developer Tutorial for further information on ACLs.

The generatekey utility (see Key generation options and parameters) provides the ability to set this permission group flag when a key is generated by either:

  • Specifying logkeyusage=yes as an option on the command line

  • Answering yes to the logkeyusage question if the command is being used interactively.

When generatekey is used this flag is applied to all permission groups but is only checked by the HSM on the group authorizing the desired action.

The following example shows this set on permission group 0 of a key’s ACL.

groups[    0].flags= LogKeyUsage
                     .n_limits= 0
                     .n_actions= 2
                     .actions[    0].type= OpPermissions
                                        .details.oppermissions.perms= DuplicateHandle
ExportAsPlain GetAppData SetAppData
   ReduceACL ExpandACL Encrypt Verify UseAsBlobKey GetACL

In the following sections, the tables will indicate if this mechanism is required to generate a log message for a specific command or key.

Commands generating Audit Log messages

The following tables list the nCore commands that generate Audit Logging messages. For each command they identify command specific data that is contained in the log message and the CEF extension used to identify it.

nCore Key and Logical Token hashes are the standard nCore identifying hashes. They are used to identify a key or logical token as it is an invariant for the key or logical token. These hashes are logged as lower-case hex encoding. In some cases a short hash may be presented. This is the first 10 bytes of the hash in a lower-case hex encoding.

For each command logged the command is specified by the name element of the CEF header. The other elements of the CEF header are filled as detailed in the previous section. All commands being logged will also include the following CEF extensions:

Extension Description

esn

ESN of the HSM

rsid

Reboot Session ID

rtc

Timestamp as milliseconds since the epoch derived from the HSM’s Real Time Clock

seqNo

Sequence number of the Audit Log message

outcome

Success or failure

Identifying Log Messages: As Audit Logging will potentially be running for a long time, the identification of a log message from an HSM based on the rsid and seqNo will not hold if the HSM is not restarted before the seqNo is reset when it reaches 9999999999. In this case account can be taken of the gbc as this will increment at a slower rate than the seqNo. Therefore messages in the same rsid with the same seqNo will have significantly different values of gbc (the mapping between seqNo and gbc is determined by the Signature Block containing the message in question). When the gbc is reset the rsid is incremented so counting begins in a new Reboot Session. Account can also be taken of the value of the rtc.

Key commands

Commands with Yes in the Requires logkeyusage ACL column will only be logged if the Key’s ACL contains the LogKeyUsage Flag in the permission group authorizing the operation.

Command Command Specific Information Logged Extension Requires logkeyusage ACL

Cmd_Sign

nCore key hash

hkey

Yes

Cmd_Encrypt

nCore key hash

hkey

Yes

Cmd_Decrypt

nCore key hash

hkey

Yes

Cmd_Verify

nCore key hash

hkey

Yes

Cmd_ChannelOpen

nCore key hash

channel mode

hkey

mode

Yes

-

Mode is one of encrypt, decrypt, sign, verify

Cmd_Import

nCore key hash

hkey

No

Cmd_Export

nCore key hash

hkey

No

Cmd_Duplicate

nCore key hash

hkey

No

Cmd_GenerateKey

nCore key hash

hkey

No

Cmd_GenerateKeyPair

nCore key hash

hkey

No

nCore Key Hashes of private and public key halves are identical

Cmd_SetAppData

nCore key hash

hkey

No

Cmd_SetACL

nCore key hash

hkey

No

Cmd_Destroy

nCore key hash

hkey

No

Also used for Logical Tokens

Cmd_DeriveKey

nCore Key Hash of derived key

nCore Key Hash of base key

nCore Key Hash of wrap key

nCore Key Hash of third input key

nCore Key Hash of fourth input key

nCore Key Hash of fifth input key

hkey

hbase

hwrsp

hin3

hin4

hin5

No

Yes

Yes

Yes

Yes

Yes

The nCore Key Hashes for the input keys will only be included in the audit log if the Permission Group for the DeriveKey action has the LogKeyUsage flag.

Cmd_MakeBlob

nCore Key Hash

nCore LT Hash

hkey

hwrap

No

Cmd_LoadBlob

nCore Key Hash

nCore LT Hash

hkey

hwrap

No

Cmd_SetKM

nCore key hash

hkey

No

Cmd_RemoveKM

nCore key hash

hkey

No

Logical Token and Share Commands

These commands do not use the logkeyusage ACL mechanism and log unconditionally.

Command Command Specific Information Logged CEF Extension

Cmd_ChangeSharePIN

KM nCore Key Hash

Short LT Hash

Share Index

Slot

hkm

shorthash

shareindex

tokenslot

Cmd_Destroy

LT Hash

hkey

Cmd_Destroy is used for Logical Tokens as well as Keys

Cmd_EraseShare

Short LT Hash

Share Index

Slot

shorthash

shareindex

tokenslot

Cmd_GenerateLogicalToken

KM nCore Key Hash

nCore LT Hash

Token Shares Needed

Token Total Shares

Token time-limit

hkm

htok

sharesneeded

sharestotal

timelimit

Cmd_LoadLogicalToken

KM Hash

LT Hash

Token Shares Needed

Token Total Shares

Token time-limit

hkm

htok

sharesneeded

sharestotal

timelimit

Cmd_ReadShare

LT Hash

Share Index

SlotId

Share Left

htok

shareindex

tokenslot

sharesleft

This is the remaining number of shares required to reconstruct the Logical Token. It reduces to 0 when a quorum of the Shares have been read.

Cmd_WriteShare

LT Hash

Share Index

SlotId

htok

shareindex

tokenslot

Administrative Commands

These commands are logged unconditionally.

Command Command Specific Information Logged CEF Extension

Cmd_InitialiseUnit

Type of KML key

kmltype

DSAp3072s256

Cmd_InitialiseUnitEx

Type of KML key

InitialiseUnitEx Flags

kmltype

flags

DSAp3072s256
Combination of AuditLogging and UseFIPSApprovedInternalMechanisms

Cmd_SetNSOPerms

nCore Key Hash of Security Officers Key

SetNSOPermsFlags

hknso

flags

Combination of AlwaysUseStrongPrimes, DisablePKCS1Padding, FIPSLevel3Enforcedv2 and CommonCriteriaCMTSRestrictions

Cmd_CreateSeeWorld

Cmd_SetSEEMachine

Cmd_SetRTC

Previous RTC

prevrtc

This command is not applicable for nShield 5c.

New RTC value will be shown in the rtc extension

Dynamic Slot Commands

These commands are logged unconditionally.

Command Command Specific Information Logged CEF Extension

Cmd_DynamicSlotsConfigure

Count of Dynamic Slots to be Configured

slotcount

Cmd_DynamicSlotCreateAssociation

Slot Id for Association

slotid

EstablishSecureChannel

ESN of smartcard

smartcardesn

This internal event is logged with name element EstablishSecureChannel, a Severity of 2 and a Device Event Class Id of 2 in the CEF header and with source=internal in the CEF extensions.

Heartbeat

The heartbeat is a periodic audit log message sent every 15 minutes. This audit log message indicates that the HSM is still active. After a heartbeat event is logged a Signature Block is generated including the heartbeat log message and any outstanding audit log messages. Waiting until the heartbeat is logged before restarting the HSM will ensure outstanding log messages can be verified.

Command Command Specific Information Logged CEF Extension

heartbeat

nCore Key Hash of Security Officers Key

hknso

The heartbeat is logged in the CEF header with name element heartbeat, Severity 4, and Device Event Class Id 2. In the CEF extensions it’s logged with source=internal.

Post Reboot Logging

The nShield HSM has a number of commands and errors that cannot be logged directly when they occur. This applies primarily to errors detected during processing or self test and the reboot command Cmd_ClearUnit. The strategy adopted for these is to persist sufficient information and replay them as log entries after a successful reboot of the HSM. These reboot event messages occur after the Certifier Block has been emitted.

Each of these messages are emitted with rsid and seqNo relating to the current session and will have a prevrtc CEF element recording the RTC at the time of the event. The name element will identify the event. If the event is associated with a nCore SOS code this will be indicated by a sos CEF extension and an appropriate code. The Device Event Class Id is set to 5 and Severity will be set to 10 for errors or 6 for shutdown events. The source CEF extension will be internal. The following table lists the events replayed in a post reboot log. The available events depend on the type of HSM.

Event Id Event SOS Code

Cmd_ClearUnit

Cmd_ClearUnit

Cmd_Fail

Cmd_Fail

D

Environment_SensorFail

HV

Temperature_OutofRange

T

RNG_PeriodicTestFail

HRTP

SOS

Starting up crypto offload

HF

SOS

cache keygen failed

HR

Voltage_Tamper

V

Battery_Tamper

B

Unknown_Tamper

TAMPER

SelfTestFail

POST test timed out

HC0TTO

POST test failed: lock failure detected

HC0LC

POST test failed: TEST_STARTED

HC0TS

POST test failed: PROCESS_STARTED

HC0PS

POST test failed: CPUID_CHECK

HC0CC

POST test failed: SRAM_ALLOC

HC0SA

POST test failed: SRAM_WRITE

HC0SW

POST test failed: SRAM_READ

HC0SR

POST test failed: SRAM_FREE

HC0SF

POST test failed: CRAM_ALLOC

HC0CA

POST test failed: CRAM_GETCACHED

HC0CG

POST test failed: CRAM_WRITE

HC0CW

POST test failed: CRAM_READ

HC0CR

POST test failed: CRAM_FREE

HC0CF

POST test failed: LOCK_CHECK

HC0LC

POST test failed: RTC_CHECK

HC0RT

POST test failed: KAT_DSA

HC0KS

POST test failed: KAT_ECDSA

HC0KC

POST test failed: KAT_DES

HC0KE

POST test failed: KAT_DES3

HC0KF

POST test failed: KAT_DES3CBCMAC

HC0KO

POST test failed: KAT_AES

HC0KA

POST test failed: KAT_AESCMAC

HC0KB

POST test failed: KAT_AESCBCMAC

HC0KD

POST test failed: KAT_SHA1

HC0KH

POST test failed: KAT_SHA1HMAC

HC0KM

POST test failed: KAT_SHA224HMAC

HC0KN

POST test failed: KAT_SHA256HMAC

HC0KJ

POST test failed: KAT_SHA384HMAC

HC0KP

POST test failed: KAT_SHA512HMAC

HC0KI

POST test failed: KAT_RSA

HC0KRH

POST test failed: KAT_NISTKDF

HC0KDF

POST test failed: KAT_HASHDRBG

HC0HD

POST test failed: KAT_RSAOAEP

HC0KZ

POST test failed: KAT_25519

HC0KX

POST test failed:unknown

HC0H

As an example, the following shows a post reboot log of Cmd_ClearUnit. In this excerpt, it can be seen after the last fragment of the Certifier Block. A Signature Block is generated after the reboot log entries.

....
<134>May 16 15:08:45 myhost2 CEF:0|nCipher Security|nShield Solo XC|12.60.2|3|ssign-cert|5|esn=1111-2222-4444 rsid=2 rtc=1524140117693 tpbl=2140 findex=5 flen=340frag=3/ITRJT4T/qgd2ZEJufIzCR+nR9lngOrmogj+5JM7VMFLsWGDxUqxmFlpqs52T2zWuYIeFHGQfx9WS9PUhf2eLMyF/7onn+hFUs57/GSZlGbCnxWybfPN27oyXjHE7pfyOrWRVKlIw8UULHVezVsxeIsZuuNEsZa5gUQ++DkoTu5M2BoPr4A+6dVL2eDhOF1m2zKATfk2moW93GkA3AO7lNPV5xU76ujo2tT7Mttvg+vyddiF2UWe6n75U0FMFjlM9WnhpFAhNk9mJPrNZ5smf4i9JuNKZat+5tq5w2b/a8Sy01EVEKtJI5SSjahtp5z77RseQ8H8ytsw6oAAAA=sign=MEUCIHgrF1m7t9X5xsl/gXwlju0bPfFPjJeIeIiH8TKSN7prAiEAs3lPS62zX3TE940/Dw9/1gVradNi62wrQI+WlSI4IYU=
<134>May 16 15:08:45 myhost2 CEF:0|nCipher Security|nShield Solo XC|12.60.2|5|Cmd_ClearUnit|6|esn=1111-2222-4444 rsid=2 rtc=1524140117693 seqNo=1 source=internal prevrtc=1524140108693

<134>May 16 15:08:45 myhost2  CEF:0|nCipher Security|nShield Solo XC|12.60.2|3|ssign|5|esn=1111-2222-4444 rsid=2 rtc=1524140117693 gbc=0 fmn=1 hcnt=1 hb=nwtggjmPYA1TR07KhdOHoyytxLb7RDvg7Wpw6FfAiC4=sign=MEYCIQDxIIJZRfKsXpMMoQ3GDEkTZ/+DTuEdNLKwHQzllflMUQIhAPipdSPrBSUnarrtjMslYS4k3RPCXcNoO16xEhg/907z

Tracing Key Usage

With the information logged as detailed in the preceding sections it is possible to trace back from a Key Command to the loading of the Key, then to loading the Logical Token and reading the Shares that constitute the Logical Token.

The following example shows the notional traceback from a Cmd_Encrypt operation. This command logs the nCore Key Hash KKKKKKK. Prior to this the Key was loaded onto the HSM using Cmd_LoadBlob which correlates the nCore Key Hash with the ncore Hash of the Logical Token that authorized loading the Key. Tracing further back we can identify the shares used to reconstruct that Logical Token. In this example two shares are required identified by share indices S1 and S2. The share index identifies a specific card in an OCS cardset.

Command Key Hash Logical Token Hash Share Index

Cmd_Encrypt

KKKKKKK

Cmd_LoadBlob

KKKKKKK

LLLLLLL

Cmd_Read Share

LLLLLLL

S2

Cmd_Read Share

LLLLLLL

S1

Cmd_LoadLogicalToken

LLLLLLL

Audit Log Verification

The audit logs produced when AuditLogging feature is active can be verified using the information contained in the audit logging metadata. Every HSM enrolled into a Security World with AuditLogging enabled generates an HSM-specific log signing private key (KAL) that is maintained in the HSM’s non-volatile memory until the module is re-initialized. The public key corresponding to this private key is sent as a Certifier Block by the HSM when Audit Logging is configured either by Security World creation or by indoctrination into an existing Audit Logging Security World. Every Signature Block sent by the HSM is generated using the log signing private key. The Audit Log can be verified as follows:

  • Extract the KAL public key from the Certifier block

  • Verify the Signature Blocks

  • Verify the log message hashes in the Signature Block against hashes of the received logs to determine if any messages have been tampered

  • Identify any missing log messages.

The basics of the verification approach is shown on the Audit Log Verification diagram.

To support Audit Log verification, Entrust provide an example verification program written in Python to serve as an example for developing a more comprehensive verification solution.

Running the example verification program

The example verification program can be found in /opt/nfast/python/examples/audit-log-verifier.py (Linux) or C:\Program Files\nCipher\nfast\python\examples\audit-log-verifier.py (Windows).

This program requires the use of the nShield Python interpreter. This is necessary to provide support for the nShield specific marshalling functions used to export the log signing public key. The example verification program also requires the presence of an nShield HSM accessible to the machine on which the verification is to be performed. This is required to perform the cryptographic operations necessary to verify the log signing public key and the Signature Blocks. This HSM does not need to be the same HSM on which the logs were generated, nor does it need to be in a Security World.

The Audit Log verifier program is run with a command of the form:

python audit-log-verifier.py [-h] [-e ESN] SYSLOG

Where:

Parameter Function

-h|--help

Displays the help message

-e ESN|--esn ESN

ESN of the logevents to be verified

SYSLOG

Location of the syslog file to be verified

Make sure that you use the nShield Python.

Results

Results from the Audit Log Verifier are written to several different files and saved in a sub-directory called LogResult. See example below for more detail.

Example

Running a command of the form:

python audit-log-verifier.py AuditLogInputFile.txt

Should produce a screen output similar to the following:

FIRST LOG INSTANCE-1 for ESN:9204-02E0-D947 @ Line:1 rsid:8 ######

Verifying certifier block...
Verification of CERTIFICATE Success
Verifying a cert fragment...Line:1
Verifying a cert fragment...Line:2
Verifying a cert fragment...Line:3
Verifying a cert fragment...Line:4
Verifying a cert fragment...Line:5
Verifying SB....Lineno:7:InstanceNo:1
Verifying SB....Lineno:18:InstanceNo:1
Valid Hash list from SBs written to ValidHashes_fromSB_forInst1.txt
No entry in SB for event @ Line:19 : Seq No:12 --
No entry in SB for event @ Line:20 : Seq No:13 --

Verified cert blocks written to./LogResult/CBs.txt
Sig blocks written to./LogResult/SBs.txt
Log messages written to./LogResult/AllEvents.txt
Anything that did not match (incl.
invalid cert blockfragments)  written to :./LogResult/Inconsistent.txt

The screen output indicates the contents of the main results files which are stored in the LogResult sub-directory. The contents of the folder will vary slightly depending on the log contents and whether there were any failures, but should be similar to the following:

AllEvents.txt                      # log entries relating to events
CBs.txt                            # certificate blocks
Inconsistent.txt                   # inconsistent log entries - should be empty []
                                    (assuming no inconsistencies)
Instance.txt                       # esn number and other info relating to the log
InvalidHashes_fromSB_forInst1      # only exists if verification failed due to signature                                          block (forInst1 refers to
                                    first logging instance/world found in the
                                    log file)
SBs.txt                            # signature blocks
Tampered_logs.txt                  # contains log messages that did not verify - e.g. due
                                    to a corrupt signature block.
This file only exists
                                    if verification failed.
Unverified_logs.txt                # unverified log entries - e.g. any trailing entries
                                    from the end of the log file that lack an accompanying
                                    signature block
ValidHashes_fromSB_forInst1.txt    # valid hashes from the signature blocks
                                    (forInst1 refers to first logging
                                    instance/world found in the log file)
Verified_logs.txt                  # verified log messages

Use a text editor to examine the files as required to check the verification. Note that Inst1 in the filenames refers to the first logging world instance in the log, see Program Architecture. If the log contains messages relating to more than one logging world, files relating to subsequent instances will be tagged with Inst2, Inst3 etc.

If the verification fails, screen output should indicate the source of the failure. For example, output for a log where a log message was missing would look something like this:

FIRST LOG INSTANCE-1 for ESN:9204-02E0-D947 @ Line:1 rsid:8 ######

Verifying certifier block...
Verification of CERTIFICATE Success
Verifying a cert fragment...Line:1
Verifying a cert fragment...Line:2
Verifying a cert fragment...Line:3
Verifying a cert fragment...Line:4
Verifying a cert fragment...Line:5
Verifying SB....Lineno:7:InstanceNo:1
Verifying SB....Lineno:17:InstanceNo:1
Verifying SB....Lineno:28:InstanceNo:1
Valid Hash list from SBs written to ValidHashes_fromSB_forInst1.txt
No entry in SB for event @ Line:29 : Seq No:22 --
No entry in SB for event @ Line:30 : Seq No:23 --
@@@@@@@@ Some Log events present in the SB but missing in Log file                             # indicates missing log message
Verified cert blocks written to./LogResult/CBs.txt
Sig blocks written to./LogResult/SBs.txt
Log messages written to./LogResult/AllEvents.txt
Anything that did not match (incl.
invalid cert blockfragments)  written to :./LogResult/Inconsistent.txt

Output for a log where a log message had been tampered with or is otherwise corrupt might look like this:

FIRST LOG INSTANCE-1 for ESN:9204-02E0-D947 @ Line:1 rsid:8 ######

Verifying certifier block...
Verification of CERTIFICATE Success
Verifying a cert fragment...Line:1
Verifying a cert fragment...Line:2
Verifying a cert fragment...Line:3
Verifying a cert fragment...Line:4
Verifying a cert fragment...Line:5
Verifying SB....Lineno:7:InstanceNo:1
Verifying SB....Lineno:18:InstanceNo:1
Verifying SB....Lineno:29:InstanceNo:1
Valid Hash list from SBs written to ValidHashes_fromSB_forInst1.txt
Validating Log @ Line No:10 SeqNo:4 is Failed ----                                            # indicates tampered log entry
***** Hash Mismatch No entry in SB for event @ Line:30 : Seq No:22 --
No entry in SB for event @ Line:31 : Seq No:23 --
Verified cert blocks written to./LogResult/CBs.txt
Sig blocks written to./LogResult/SBs.txt
Log messages written to./LogResult/AllEvents.txt
Anything that did not match (incl.
invalid cert blockfragments)  written to :./LogResult/Inconsistent.txt

The tampered log line(s) will be listed in output file Tampered_logs.txt.

Output for a log where the signature block is corrupt will look something like this:

FIRST LOG INSTANCE-1 for ESN:9204-02E0-D947 @ Line:1 rsid:8 ######

Verifying certifier block...
Verification of CERTIFICATE Success
Verifying a cert fragment...Line:1
Verifying a cert fragment...Line:2
Verifying a cert fragment...Line:3
Verifying a cert fragment...Line:4
Verifying a cert fragment...Line:5
Verifying SB....Lineno:7:InstanceNo:1
Verifying SB....Lineno:18:InstanceNo:1
Signature Tampered B64 decode

//k=&s0QG1C08QBi34gaTU2+rUzp/dwtAXi9Hv0IjDvDL/yg=&Im5nW+OX0gbdlLnrFLxsZtR4meDSEXG5JXtkMmltTZU=&LGAXS1nvgHElvXhk8RVT2lCK2NMtXyD9OYTecVOaaBk=&MbJAk706yU2+QykWmtfnCV0lxn/enber8aJK3cZyxLg=&y2qxF5VGm/X/h6ZcZ5iOes7ZAFpqM/6ND8nAXzCM/bY=&kWjEaGIclJv494A1ZcUgGHJko7AeKvUUqVimhfExioU=  Length: 577
Verifying SB....Lineno:29:InstanceNo:1
Valid Hash list from SBs written to ValidHashes_fromSB_forInst1.txt
In-Valid Hash list from SBs written to InvalidHashes_fromSB_forInst1.txt
        Log entry found in Tampered SB. Line no:8 SeqNo:2
        Log entry found in Tampered SB. Line no:9 SeqNo:3
        Log entry found in Tampered SB. Line no:10 SeqNo:4
        Log entry found in Tampered SB. Line no:11 SeqNo:5
        Log entry found in Tampered SB. Line no:12 SeqNo:6
        Log entry found in Tampered SB. Line no:13 SeqNo:7
        Log entry found in Tampered SB. Line no:14 SeqNo:8
        Log entry found in Tampered SB. Line no:15 SeqNo:9
        Log entry found in Tampered SB. Line no:16 SeqNo:10
        Log entry found in Tampered SB. Line no:17 SeqNo:11
No entry in SB for event @ Line:30 : Seq No:22 --
No entry in SB for event @ Line:31 : Seq No:23 --
Verified cert blocks written to./LogResult/CBs.txt
Sig blocks written to./LogResult/SBs.txt
Log messages written to./LogResult/AllEvents.txt
Anything that did not match (incl.
invalid cert blockfragments)  written to :./LogResult/Inconsistent.txt

The failed log messages should be reported in Tampered_logs.txt in the LogResult folder.

If the certificate block is corrupt, output will be similar to that shown below. In this case, the CBs.txt file may be empty and the cert block fragments will be written to Inconsistent.txt.

FIRST LOG INSTANCE-1 for ESN:9204-02E0-D947 @ Line:1 rsid:8 ######

Verifying certifier block...
Verification of CERTIFICATE Success
Verifying a cert fragment...Line:1
Verifying a cert fragment...Line:2
Verifying a cert fragment...Line:3
Signature Tampered B64 decode
('Failed fragment:', 3, '<134>May  2 16:10:38 exampleCB1.myexample.com CEF:0|nCipher Security|nShield Solo XC|12.60.2|3|ssign-cert|5|esn=9204-02E0-D947 rsid=8rtc=4294967386734000 tpbl=2140 findex=3 flen=450 frag=B1Ku9rirlixkgEd+73tMVJ1FQz85aCWuRqJl04YB1YwFvZgvRXhHvzqLFeJZAUerKlLgIaZwDq1twoXzvHq88QcJdbr0i4+87VorPKkEjKtSSGHOVkkHhoBC8uNgYXnTBxqcqCqpZl4whuiEBmJQLcwgAAAAg8rgckmo3ArobecQooPxQ9AjYbCmAoKOUTRi7grTzPyAAQAA3Bvuz+tQ1uh5LvuKMLTtGDTTplG7ks6ZkL8b+F2UW37jfN3lap27oAZq1otU4FOP4EVvoMmNSdI4uzCPi7VgcI3AcIkdjZIwbpYf9XQwvFwMxYvdBPGhPtc/t8Lslgs97rMkES4ZciNI/NwjKp0fW4kCiSBSUQUAUcp6vgqg2vVL9naqRHhXNRJuweaRtOO60z0mBkTgCnAvscdr2ymErrWDZArHosYXJZrXghjNmXvu+rS8GvTvTc sign=MEYCIQCXhbJeFTv8oR8a51aU3Os9w2Vs9w67mzYk584Gy+MdbgIhAOD0bAwU0Vw1xOmR2oerqWKLFeawZe5rONmDMZFbJoB')
Fragment verification unsuccessful!
Adding all fragments in this CB to Inconsistent.txt
('No Valid CB for this instance:', 1)
In-Valid Hash list from SBs written to InvalidHashes_fromSB_forInst1.txt
        Log entry found in Tampered SB. Line no:6 SeqNo:1
        Log entry found in Tampered SB. Line no:8 SeqNo:2
        Log entry found in Tampered SB. Line no:9 SeqNo:3
        Log entry found in Tampered SB. Line no:10 SeqNo:4
        Log entry found in Tampered SB. Line no:11 SeqNo:5
        Log entry found in Tampered SB. Line no:12 SeqNo:6
        Log entry found in Tampered SB. Line no:13 SeqNo:7
        Log entry found in Tampered SB. Line no:14 SeqNo:8
        Log entry found in Tampered SB. Line no:15 SeqNo:9
        Log entry found in Tampered SB. Line no:16 SeqNo:10
        Log entry found in Tampered SB. Line no:17 SeqNo:11
        Log entry found in Tampered SB. Line no:19 SeqNo:12
        Log entry found in Tampered SB. Line no:20 SeqNo:13
        Log entry found in Tampered SB. Line no:21 SeqNo:14
        Log entry found in Tampered SB. Line no:22 SeqNo:15
        Log entry found in Tampered SB. Line no:23 SeqNo:16
        Log entry found in Tampered SB. Line no:24 SeqNo:17
        Log entry found in Tampered SB. Line no:25 SeqNo:18
        Log entry found in Tampered SB. Line no:26 SeqNo:19
        Log entry found in Tampered SB. Line no:27 SeqNo:20
        Log entry found in Tampered SB. Line no:28 SeqNo:21
No entry in SB for event @ Line:30 : Seq No:22 --
No entry in SB for event @ Line:31 : Seq No:23 --

Verified cert blocks written to./LogResult/CBs.txt
Sig blocks written to./LogResult/SBs.txt
Log messages written to./LogResult/AllEvents.txt
Anything that did not match (incl.
invalid cert blockfragments)  written to :./LogResult/Inconsistent.txt

Program Architecture

The program takes and reads the input syslog file containing the log messages. It optionally sets the ESN of module for which log events are to be validated, if this was passed in. If an ESN is not provided as input then the first ESN found in the syslog will be processed.

The verifier calls its parse function which segregates the messages based on ESN, and creates lists of Certifier fragments, Signature Blocks and Log events, based on matching with regular expressions.

Syslog may have gathered logs from multiple sources. As such, the verifier has a concept of a logging world, which represents a set of logs, sigblocks and certblocks that belong together, from a Security World. Based on Reboot Sequence ID, Sequence Number of the Log event, Global block counter of the Signature block and Fragment index of the Certifier block, a logging world is identified and a logging instance is created.

All records are thus given a log-instance number, such that records with the same instance number belong together.

Each event can thus be uniquely identified via a tuple. For the log messages, signature blocks and certifier blocks these are respectively (rsid and sequence number), (rsid and gbc) and (rsid and findex).

The reconstruct_CBs function is then called to validate the certifier fragments (using calls to an nShield HSM for crypto functionality). It then reconstructs the certifier blocks from the certifier fragments.

This does not require the HSM to be in the same Security World as the HSM that first generated the logs.

A list of valid and verified Certifier Blocks is created.

For any log instance one valid Certifier Block is enough to validate the events, so further certifier blocks are ignored after the first.

Next the process_sbs function is called. Signature Blocks for a supplied ESN are validated per log instance (once again via calls to the module for crypto functionality), using the KAL value taken from the Certifier block previously.

The validated Signature block hashes are maintained as a dictionary of hashes with keys as unique ids. These unique ids per instance are generated based on rsid and sequence numbers.

The process_logs function is finally called. This generates the hash of each of the log events and matches against hashes from corresponding signature blocks. Verified and Tampered log events are then written to different files in the LogResult folder.

Extended Verification

While the example verifier uses an HSM for cryptographic operation, it would be possible to use 3rd party cryptographic libraries to provide this functionality. This is outside the scope of this document.

Currently the log messages are verified against the hash in the signature blocks, and the signature of the signature blocks is verified against the key extracted from the certifier block. The certifier block its self is not verified. A potential extension to the verifier tool would be to verify the certifier block. The certifier block is signed by KLF2. This can be checked against the KLF2 value found within the module’s warrant. This would complete the chain of trust.

Additionally, the example verifier does not cope with fields that rotate back around to zero when their max size is exceeded, for example with the gbk, rsid or seqno fields. Currently logs, SBs and CBs are uniquely identified by (rsid and sequence number), (rsid and gbc) and (rsid and findex). This means that, if any of those values rotate back around to zero, we are no longer able to uniquely identify them. As a potential extension, RTC or line number values could be used to solve this.

The example verifier does not detect missing/deleted log messages in the case where a complete group of log messages are deleted, along with their corresponding SignatureBlock. Given that the SeqNo field increases for each log message, spotting missing SeqNos would reveal missing or deleted log messages. This is a potential extension.

The example verifier expects a static, unchanging log file to be supplied to it. This would be compatible with verifying a batch of log files at the end of each day, for example. A possible extension would be to extend the verifier to cope with a live stream of logs, continuously verifying them as they are generated.