stattree

stattree [<node> [<node> [...]]]

The output of stattree is statistics currently available on the host machine. Statistics are gathered both by the hardserver (relating to the server itself, and its current clients) and by each attached HSM.

A typical output fragment from stattree:

+PerModule:
   +#1:
      +ModuleObjStats:
         -ObjectCount          5
         -ObjectsCreated       5
         -ObjectsDestroyed     0
      +ModuleEnvStats:
         -MemTotal             15327232
         -MemAllocKernel       126976
         -MemAllocUser         0
      +ModuleJobStats:
         -CmdCount             169780
         -ReplyCount           169778
         -CmdBytes             3538812
         -ReplyBytes           4492764
         -HostWriteCount       169772
         -HostWriteErrors      0
         -HostReadCount        437472
         -HostReadErrors       0
         -HostReadEmpty        100128
         -HostReadDeferred     167578
         -HostReadTerminated   0
         -PFNIssued            102578
         -PFNRejected          1
         -PFNCompleted         102577
         -ANIssued             1
         -CPULoadPercent       0
      +ModuleSerialStats:
         -HostReadCount        437476
         -HostReadDeferred     167580
         -HostReadReconnect    167579
         -HostReadErrors       0
         -HostWriteCount       169774
         -HostWriteErrors      0

PerModule, ModuleObjStats, and ModuleEnvStats are <node> tags that identify classes of statistics for each hardwerver or HSM. 1 identifies an instance node.

ObjectCount, MemTotal, and the remaining items at the same level are pairs of <statistics-id>s and their values. Times are listed in seconds. Other numbers are integers, which are either real numbers, IP addresses, or counters. For example, a result -CmdCount 74897 means that there have been 74,897 commands submitted.

If you provide a <node> on the command line, stattree uses it as the starting point of the tree and displays only information at or below that <node> in the tree. Values for <node> can be numeric or textual.

$ stattree PerModule 3 ModuleObjStats
+#PerModule:
   +#3:
      +#ModuleObjStats:
         -ObjectCount          6
         -ObjectsCreated       334
         -ObjectsDestroyed     328

If you use a <statistics-id> instead of a <node> on the command line, you get an error message:

$ stattree PerModule 3 ModuleObjStats ObjectCount
+#PerModule:
   +#3:
      +#ModuleObjStats:
Unable to convert 'ObjectCount' to number or tag name.

Node tags

Connections

Statistics for connections between clients and the hardserver. There is one node for each currently active connection. Each node has an instance number that matches the log message generated by the server when that client connected. For example, when the hardserver message is Information: New client #24 connected, the client’s statistics appear under node #24 in the stattree output.

HostEnvStats

Only in network-attached HSMs
Environmental statistics for the HSM.

HostSysInfo

Only in network-attached HSMs
Further statistics for the HSM.

ModuleEnvStats

General statistics for the HSM’s operating environment.

ModuleJobStats

This tag holds statistics for the Security World Software commands (jobs) processed by this HSM.

ModuleObjStats

Statistics for the HSM’s Object Store, which contains keys and other resources. These statistics may be useful in debugging applications that leak key handles, for example.

ModulePCIStats

This tag holds statistics for the PCI connection between the HSM and the host computer. It does not apply to nShield Edge HSMs.

ModuleSerialStats

This tag is for nShield Edge HSMs only. It holds statistics for the serial connection between the HSM and the host computer.

PerModule

Statistics kept by the HSMs. There is one instance node for each HSM, numbered using the standard HSM numbering. The statistics provided by each HSM depend on the HSM type and firmware version.

ServerGlobals

Aggregate statistics for all commands processed by the hardserver since it started.
The standard statistics apply to the commands sent from the hardserver to HSMs. Commands processed internally by the server are not included here.
The Uptime statistic gives the total running time of the server so far.

Statistics IDs

ANIssued

The number of Asynchronous Notification messages issued by the HSM to the hardserver. These messages indicate such things as the clear key being pressed and the HSM being reset. In later firmware revisions inserting or removing the smartcard or changing the non-volatile memory also generate asynchronous notifications.

ChanJobsCompleted

The number of fast channel jobs completed by the HSM. The fast channel facility is unsupported on current HSMs. This number should always be 0.

ChanJobErrors

The number of low-level (principally data transport) errors encountered while processing fast channel jobs. Should always be 0 on current HSMs.

ChanJobsIssued

The number of fast channel jobs issued to the HSM. The fast channel facility is unsupported on current HSMs. This number should always be 0.

CPULoadPercent

PCIe HSMs
The current processing load on the HSM, represented as a number between 0 and 100. Because an HSM typically contains a number of different types of processing resources (for example, main CPU, and RSA acceleration), this figure is hard to interpret precisely. In general, HSMs report 100% CPU load when all RSA processing capacity is occupied; when performing non-RSA tasks the main CPU or another resource (such as the random number generator) can be saturated without this statistic reaching 100%.
Network-attached HSMs
The current utilization of the main CPU, across all cores.
If you are on a firmware version earlier than 13.1, this instead reports a load average that is scaled by 100, but could be greater than 100% if there is an average of more than one runnable thread.

ClientCount

The number of client connections currently made to the server. This appears in the hardserver statistics.

CmdBytes

The total length of all the command blocks sent for processing.

CmdCount

The total number of commands sent for processing from a client to the server, or from the server to an HSM. Contains the number of commands currently being processed.

CmdMarshalErrors

The number of times a command block was not understood when it was received. A nonzero value indicates either that the parties at each end of a connection have mismatched version numbers (for example, a more recent hardserver has sent a command to a less recent HSM that the HSM does not understand), or that the data transfer mechanism is faulty.

CryptoClientCount

Only in network-attached HSMs
The number of licensable clients connected (active and parked sessions).

CurrentTempC

The current temperature (in degrees Celsius) of the HSM main circuit board. First-generation HSMs do not have a temperature sensor and do not return temperature statistics.

DeviceFails

The number of times the hardserver has declared a device to have failed. The hardserver provides a diagnostic message when this occurs.

DeviceRestarts

The number of times the hardserver has attempted to restart an HSM after it has failed. The hardserver provides a Notice message when this occurs. The message does not indicate that the attempt was successful.

DevOutstanding

The number of commands sent by the specified client that are currently executing on one or more HSMs. When an HSM accepts a command from a client, this number decreases by 1 and QOutstanding increases by 1. Commands that are processed purely by the server are never included in this count.

HostDebugIRQs

On PCI HSMs, the number of debug interrupts received. This is used only for driver testing, and should be 0 in any production environment.

HostIRQs

On PCI HSMs, the total number of interrupts received from the host. On current HSMs, approximately equal to the total of HostReadCount and HostWriteCount.

HostReadCount

The number of times a read operation to the HSM was attempted. The HSM can defer a read if it has no replies at the time, but expects some to be available later. Typically the HSM reports HostReadCount in two places: the number under ModuleJobStats counts a deferred read twice, once when it is initially deferred, and once when it finally returns some data. The number under ModulePCIStats counts this as one operation.

HostReadDeferred

The number of times a read operation to the HSM was suspended because it was waiting for more replies to become available. When the HSM is working at full capacity, a sizeable proportion of the total reads are likely to be deferred.

HostReadEmpty

The number of times a read from the HSM returned no data because there were no commands waiting for completion. In general, this only happens infrequently during HSM startup or reset. It can also happen if PauseForNotifications is disabled.

HostReadErrors

The number of times a read to an HSM failed because the parameters supplied with the read were incorrect. A nonzero value here typically indicates some problem with the host interface or device driver.

HostReadTerminated

The number of times an HSM had to cancel a read operation which has been deferred. This normally happens only if the clear key is pressed while the HSM is executing commands. Otherwise it might indicate a device driver, interface, or firmware problem.

HostReadUnderruns

Not currently reported by the HSM.

HostUnhandledIRQs

On PCI HSMs, the number of unidentified interrupts from the host. If this is nonzero, a driver or PCI bus problem is likely.

HostReadReconnect

On PCI HSMs, the number of deferred reads that have now completed. This should be the same as HostReadDeferred, or one less if a read is currently deferred.

HostWriteBadData

Not currently reported by the HSM.
Attempts to write bad data to the HSM are reflected in HostWriteErrors.

HostWriteCount

The number of write operations (used to submit new commands) that have been received by the HSM from the host machine. One write operation may contain more than one command block. The operation is most efficient when this is the case.

HostWriteErrors

The number of times the HSM rejected the write data from the host. A nonzero value may indicate that data is being corrupted in transfer, or that the hardserver/device driver has got out of sync with the HSM’s interface.

HostWriteNoMemory

Not currently reported by the HSM. Write failures due to a lack of memory are reflected in HostWriteErrors.

HostWriteOverruns

Not currently reported by the HSM. Write overruns are reflected in HostWriteErrors.

LongOutstanding

The number of LongJobs sent by the specified client that are currently executing on one or more HSMs. When an HSM accepts a LongJobs command from a client, this number increases by 1 and QOutstanding decreases by 1. Commands that are processed purely by the server are never included in this count.

MaxClients

The maximum number of client connections ever in use simultaneously to the hardserver.
This gives an indication of the peak load experienced so far by the server.

MaxCryptoClients

Only in network-attached HSMs
The maximum number of client connections permitted by license.

MaxTempC

The maximum temperature recorded by the HSM’s temperature sensor. This is stored in non-volatile memory, which is cleared only when the HSM is initialized. First-generation HSMs do not have a temperature sensor and do not return temperature statistics.

MemAllocKernel

PCIe HSMs
not supported
Connect network-attached HSMs
The total amount of RAM allocated for kernel (that is, non-SEE) use in an HSM. This is principally used for the object store (keys, logical tokens, and similar) and for big-number buffers.
nShield 5c and later network-attached HSMs
Obsolete, retained only for backwards compatibility. Shows the same value as MemTotal.

MemAllocUser

PCIe HSMs
not supported
Connect network-attached HSMs
The total amount of RAM allocated for user-mode processes in the HSM (0 for non-SEE use). This includes the size of the SEE Machine image, and the total heap space available to it.
nShield 5c and later network-attached HSMs
Obsolete, retained only for backwards compatibility. Shows the same value as MemTotal.

MemTotal

The total amount of RAM (both allocated and free) available to the HSM. This is the installed RAM size minus various fixed overheads.

MinTempC

The minimum temperature recorded by the HSM’s temperature sensor. This is stored in non-volatile memory, which is cleared only when the HSM is initialized. First-generation HSMs do not have a temperature sensor and do not return temperature statistics.

NVMFreeSpace

The total amount of free space in the NVRAM of the HSM, in bytes.

NVMWearLevel

The wear level of the HSM’s NVRAM, expressed as a percentage of the ratio between the erase count and the endurance.

NVMWornBlocks

The percentage of worn blocks in the NVRAM of the HSM.

ObjectsCreated

The number of times a new object has been put into the object store. This appears under the HSM’s ModuleObjStats node.

ObjectsDestroyed

The number of items in the HSM’s object store that have been deleted and their corresponding memory released.

ObjectCount

The current number of objects (keys, logical tokens, buffers, SEE Worlds) in the object store. This is equal to ObjectsCreated minus ObjectsDestroyed. An empty HSM contains a small number of objects that are always present.

PFNCompleted

The number of PauseForNotifications commands that have been completed by the HSM. Normally, this is one less than the PFNIssued figure because there is normally one such command outstanding.

PFNIssued

The number of PauseForNotifications commands accepted by the HSM from the hardserver. This normally increases at a rate of roughly one every two seconds. If the hardserver has this facility disabled (or a very early version), this does not occur.

PFNRejected

The number of PauseForNotifications commands rejected by the HSM when received from the hardserver. This can happen during HSM startup or reset, but not in normal use. It indicates a hardserver bug or configuration problem.

QOutstanding

The number of commands waiting for an HSM to become available on the specified client connection. When an HSM accepts a command from a client, this number decreases by 1 and DevOutstanding increases by 1. Commands that are processed purely by the server are never included in this count.

RemoteIPAddress

The remote IP address of a client who has this connection. A local client has the address 0.0.0.0.

ReplyCount

The total number of replies returned from server to client, or from HSM to server.

ReplyBytes

The total length of all the reply blocks received after completion.

ReplyMarshalErrors

The number of times a reply was not understood when it was received. A nonzero value indicates either that the parties at each end of a connection have mismatched version numbers (for example, a more recent hardserver has sent a command to a less recent HSM that the HSM does not understand), or that the data transfer mechanism is faulty.

SystemFans

The fan speed (RPM) for each fan in the HSM.

Uptime

The length of time (in seconds) since an HSM was last reset, the hardserver was started, or a client connection was made.

ModuleDriverStats fields

DriverIRQs

Total number of interrupts

DriverReadIRQs

Read interrupts

DriverWriteIRQs

Write interrupts

DriverWriteFails

Write failures

DriverWriteBlocks

Blocks written

DriverWriteBytes

Bytes written

DriverReadFails

Read failures

DriverReadBlocks

Blocks read

DriverReadBytes

Bytes read

DriverEnsureFail

Read request failures

DriverEnsure

Read requests