Install and configure the WSOP server

[mongodb-enterprise-7.0]
name=MongoDB Enterprise Repository
baseurl=https://repo.mongodb.com/yum/redhat/$releasever/mongodb-enterprise/7.0/$basearch/
gpgcheck=1
enabled=1
gpgkey=https://www.mongodb.org/static/pgp/server-7.0.asc

% sudo yum install -y mongodb-enterprise

% sudo mkdir ~/wsop_install

% cd ~/wsop_install
% sudo tar -xzf ~ /Downloads/wsop-p11-3.3.0-697-4bfd3ef.tar.gz

% cd /
% sudo tar zxvf ~/wsop_install/corecrypto.tar.gz
% sudo tar zxvf ~/wsop_install/wsop-common.tar.gz
% sudo tar zxvf ~/wsop_install/dbmt.tar.gz

% cd /opt/nfast/webservices/dbmt-2.2.0/
% sudo ./install.sh

% sudo cp /etc/mongod.conf /etc/mongod.conf.example

# mongod.conf

# for documentation of all options, see:
#   http://docs.mongodb.org/manual/reference/configuration-options/

# where to write logging data.
systemLog:
  destination: file
  logAppend: true
  path: /var/log/mongodb/mongod.log

# Where and how to store data.
storage:
  dbPath: /var/lib/mongo

# how the process runs
processManagement:
  fork: true  # fork and run in background
  pidFilePath: /var/run/mongodb/mongod.pid  # location of pidfile
  timeZoneInfo: /usr/share/zoneinfo

# network interfaces
net:
  port: 27017
  bindIp: 0.0.0.0  # Enter 0.0.0.0,:: to bind to all IPv4 and IPv6 addresses or, alternatively, use the net.bindIpAll setting.
  tls:
    mode: requireTLS
    certificateKeyFile: /etc/ssl/mongodb/db_server.pem
    CAFile: /etc/ssl/mongodb/myCA.pem

security:
  clusterAuthMode: x509

#operationProfiling:

replication:
  replSetName: rs1

#sharding:

# Enterprise-Only Options

#auditLog:

% mongosh

Current Mongosh Log ID: 664cf54837141997c3a26a12
Connecting to:          mongodb://127.0.0.1:27017/?directConnection=true&serverSelectionTimeoutMS=2000&appName=mongosh+2.2.6
Using MongoDB:          7.0.9
Using Mongosh:          2.2.6

For mongosh info see: https://docs.mongodb.com/mongodb-shell/


To help improve our products, anonymous usage data is collected and sent to MongoDB periodically (https://www.mongodb.com/legal/privacy-policy).
You can opt-out by running the disableTelemetry() command.

------
   The server generated these startup warnings when booting
   2024-05-21T15:25:01.616-04:00: Access control is not enabled for the database. Read and write access to data and configuration is unrestricted
   2024-05-21T15:25:01.616-04:00: You are running this process as the root user, which is not recommended
   2024-05-21T15:25:01.617-04:00: /sys/kernel/mm/transparent_hugepage/enabled is 'always'. We suggest setting it to 'never' in this binary version
   2024-05-21T15:25:01.617-04:00: Soft rlimits for open file descriptors too low

Enterprise test>

Enterprise test> rs.initiate()

{
  info2: 'no configuration specified. Using a default configuration for the set',
  me: 'wsopserver-redhat-8:27017',
  ok: 1
}
Enterprise rs1 [direct: other] test>

Enterprise rs1 [direct: other] test> use admin

switched to db admin

db.createUser(
    {
	user: "mAdmin",
	pwd: "admin",
	roles: [ { role: "userAdminAnyDatabase", db: "admin" },
	         { role: "dbAdminAnyDatabase", db: "admin" },
		 { role: "readWriteAnyDatabase", db: "admin" } ]
    }
)

db.createUser(
...     {
...         user: "mAdmin",
...         pwd: "admin",
...         roles: [ { role: "userAdminAnyDatabase", db: "admin" },
...                  { role: "dbAdminAnyDatabase", db: "admin" },
...                  { role: "readWriteAnyDatabase", db: "admin" } ]
...     }
... )
{
  ok: 1,
  '$clusterTime': {
    clusterTime: Timestamp({ t: 1716319876, i: 4 }),
    signature: {
      hash: Binary.createFromBase64('AAAAAAAAAAAAAAAAAAAAAAAAAAA=', 0),
      keyId: Long('0')
    }
  },
  operationTime: Timestamp({ t: 1716319876, i: 4 })
}
Enterprise rs1 [direct: primary] admin> exit

% sudo chown -R mongod:mongod /var/lib/mongo
% sudo chown -R mongod:mongod /var/lib/mongo/WiredTiger.turtle
% sudo chown -R mongod:mongod /var/lib/mongo/journal/

% sudo mkdir -p /data/db/
% sudo chown -R mongod:mongod /data/db/

% sudo service mongod start

% sudo systemctl enable mongod

% sudo service mongod status

Redirecting to /bin/systemctl status mongod.service
● mongod.service - MongoDB Database Server
   Loaded: loaded (/usr/lib/systemd/system/mongod.service; enabled; vendor preset: disabled)
   Active: active (running) since Tue 2024-05-21 15:39:03 EDT; 36s ago
     Docs: https://docs.mongodb.org/manual
 Main PID: 263321 (mongod)
   Memory: 191.6M
   CGroup: /system.slice/mongod.service
           └─263321 /usr/bin/mongod -f /etc/mongod.conf

May 21 15:39:03 wsopserver-redhat-8 systemd[1]: Started MongoDB Database Server.
May 21 15:39:03 wsopserver-redhat-8 mongod[263321]: {"t":{"$date":"2024-05-21T19:39:03.390Z"},"s":"I",  "c":"CONTROL",  "id":7484500, "ctx":"main","msg":"Environment variable MONGOD…k\" to false"}
Hint: Some lines were ellipsized, use -l to show in full.

% sudo cp /opt/nfast/webservices/corecrypto/conf/config.yaml.example /opt/nfast/webservices/corecrypto/conf/config.yaml

% nfkminfo | grep hknso

hknso 0adead5baac6c31d69dd964e00309829601fcd05

# Server Authentication TLS (corecrypto server) Options
external_tls:
    # Host for corecrypto to listen on
    host: 0.0.0.0

    # Port for corecrypto to listen on
    port: 18001

    # TLS Certificate
    cert_file: /opt/nfast/webservices/corecrypto/tls/db/wsop_server.pem

    # Appname:Ident of the key used to protect the TLS private key
    # key_appname_ident: appname:ident

    # Exclude TLS private key from queries
    # exclude_tls_key: true

    # TLS private key file - note only required if not using HSM protection
    key_file: /opt/nfast/webservices/corecrypto/tls/db/wsop_server.key

    # TLS CA Certificate for Mutual Authentication
    ca_certificate_file: /opt/nfast/webservices/corecrypto/tls/db/myCA.pem

    # Enable TLS Client Authentication
    # Warning: we suggest not disabling client_auth_enabled as this will allow unauthenticated client requests
    client_auth_enabled: true

    # API Gateway
    # This flag is set to true when webservices are expected to work behind an API Gateway
    api_gateway: false

    # Headers
    # This option is only used when webservices work behind an API Gateway
    # By default webservices will look into Authorization headers passed by the gateway to identify its clients by
    # issuer and subject.
    # If Authorization headers are not provided, then webservices will identify a clients' issuer and subject values
    # from the request headers defined in issuer_header and subject_header config fields
    # headers:
    #     issuer_header:
    #     subject_header:

    # Directory where to look for the CRL files, these need to have extension .crl or.pem to be loaded
    crl_directory: /opt/nfast/webservices/corecrypto/tls/external/crls

    # Interval at which to poll the CRL directory for changes to CRL files
    crl_poll_interval: 1m

    # Preferred Cipher Suites for external clients.
    # The full list of supported cipher suites can be found in the nShield Web Services Option Pack User Guide
    # The default provided in this file is a list of recommended cipher suites.
    cipher_suites:
        - ECDHE-ECDSA-AES128-GCM-SHA256
        - ECDHE-RSA-AES128-GCM-SHA256
        - ECDHE-ECDSA-AES256-GCM-SHA384
        - ECDHE-RSA-AES256-GCM-SHA384
        - ECDHE-ECDSA-CHACHA20-POLY1305
        - ECDHE-RSA-CHACHA20-POLY1305

logging:
    # The loglevel
    # Valid values: Trace, Debug, Info, Warning, Error
    loglevel: Warning

    # Logging to console (stdout or stderr)
    console:
        # The console level output
        # Valid values: stderr, stdout, discard or empty
        output: discard

    # Logging to a file
    file:
        # Enable logging to a file
        enabled: true

        # The absolute path of the file that logs should be written to
        filepath: /opt/nfast/log/corecrypto.log

    # Logging to syslog
    syslog:
        # Enable logging to a configured syslog server
        enabled: false

        # The networking protocol to be used to send logs to syslog
        # Optional - if syslog is enabled but network is not configured, the local syslog daemon will be used
        # Valid values: udp
        network: udp

        # The IP/Hostname of the machine hosting syslog and the port to access syslog on in the format hostname:port
        # Optional - if syslog is enabled but host is not configured, the local syslog daemon will be used.
        # Valid values: localhost, and IPv4 addresses in the format x.x.x.x where x is a value between 0 and 255
        host: localhost:514

# Health Check Options
health:
    # Interval in seconds between estate health check
    # Estate means the modules which belong of the security world
    estate_check_interval: 5s

    # Period in seconds after which the estate health check will timeout
    estate_check_timeout: 4s

    # Interval in seconds between each security world check
    # Its value is a multiple of the estate_check_interval value
    # Note: each time a security world check takes place, an estate
    # health check also takes place
    sworld_check_interval: 300s

    # Interval in seconds between each database health check
    database_check_interval: 5s

    # Period in seconds after which a database health check will timeout
    database_check_timeout: 30s

    # Allow unauthenticated clients to probe the health check endpoint.
    # Only applicable when tls.client_auth_enabled is true
    allow_unauthenticated_clients: true

    # Extend health check status to include whether the service can acquire FIPS-authentication
    include_fips_ready_check: false

# World
world:
    # Hash of the security world Officer
    # It needs to match the world in the database and needs to be set before starting corecrypto.
    hknso: 0adead5baac6c31d69dd964e00309829601fcd05

# Concurrent Request Limiter Options
# Too many concurrent requests can lead to a denial of service
concurrent_request_limiter:
    # Maximum number of concurrent requests that the server supports
    outstanding_requests_limit: 500
    # Maximum number of concurrent requests for generating or importing RSA keys
    rsa_key_requests_limit: 10
    # Maximum number of concurrent requests for generating or importing ECDSA keys
    ecdsa_key_requests_limit: 30

# Caching Options
cache:
    # Period of inactivity in minutes after which a key manager will be closed
    # Only applicable when tls.client_auth_enabled is true
    # Setting of 0 is used to disable closing an inactive key manager.
    key_manager_inactivity: 1440m

    # Capacity of the key cache. When the number of keys in the cache reaches this
    # capacity, then the keys are evicted based on a least recently used (LRU) policy.
    # Minimum value 100
    key_cache_capacity: 30000

    # Period of time for which a key stays in the cache before it is evicted
    # Minimum value 1m
    key_TTL_period: 60m

    # Capacity of the group cache: when the number of groups in the cache reaches the
    # capacity, then the groups are evicted based on least recently used (LRU) policy
    # Minimum value 100
    group_cache_capacity: 20000

    # Time to live value for a group in the cache
    # Minimum value 1m
    group_TTL_period: 120m

    # Maximum number of protection domains which are stored in the corecrypto cache
    # Minimum value 100
    max_number_of_active_protection_domains: 10000

    # Time to live value for protection domains in the cache
    # Minimum value 1m
    domain_TTL_period: 720m

# Database Options
database:
    # List of database hosts
    hosts:
        - localhost:27017

    # The loglevel of the database driver component
    # Valid values: Trace, Debug, Info, Warning, Error
    loglevel: Info

    # Time before a database request should fail
    timeout: 5s

    # Maximum returned keys when listing, large queries can hurt the service.
    # This limit affects the list keys endpoint.
    # If you wish to retrieve more than the maximum limit then you may make multiple API
    # requests and combine the results within your application using the offset and limit.
    # listkeys_max_limit: 300000

    # Name of the database
    db_name: nshield-corecrypto

    # Webservices Corecrypto Segregation (WCS) Options
    # The segregation database must be defined in order
    # to enable WCS. When enabled, corecrypto objects
    # will be segregated based on the mappings defined in the collection.
    # If the segregation database is not defined,
    # WCS is disabled.
    #
    # segregations_db_name: segregation_db
    # segregations_collection_name: segregations

    # Authentication method with database. Valid values: [none, pwd, tls]
    # none - no authentication
    # pwd  - username and password authentication using mongodb SCRAM
    # tls  - x509 authentication
    auth_type: tls

    # If 'auth_type' is pwd, 'auth_username_file' and 'auth_password_file'
    # options define the location of a secure file containing the username
    # and passphrase to use for authentication.
    # 'auth_source' is a database-type specific identifier for what to authenticate
    # against. For mongodb this is the name of the authentication database.
    #
    # auth_username_file: /opt/nfast/webservices/corecrypto/pwd-auth/config-username-auth
    # auth_password_file: /opt/nfast/webservices/corecrypto/pwd-auth/config-password-auth
    # auth_source: userdb

    # Transport Layer Security. Default is false.
    disable_tls: false

    # Path to the mongoDB TLS certificate
    db_ca_file: /opt/nfast/webservices/corecrypto/tls/db/myCA.pem

    # Path to the corecrypto client certificate (used when Mutual Authentication is enabled)
    db_cert_file: /opt/nfast/webservices/corecrypto/tls/db/wsop_server.pem

    # Path to the corecrypto client private key (used when Mutual Authentication is enabled)
    db_key_file: /opt/nfast/webservices/corecrypto/tls/db/wsop_server.key

    # Type of database. Supported values: mongodb
    db: mongodb

    # Disable this option when WSOP is running with keySafe5 instance.
    is_WSOP_standalone: true

    mongodb:
        # Name of the Replication Set
        replica_set: rs1

        # Timeout for connection to the database server
        connect_timeout: 5s
        # Timeout for selecting a connection from the pool
        selection_timeout: 5s
        # Timeout waiting for read/write in the socket
        socket_timeout: 5s

        # Minimum and maximum connections to use in mongodb's connection pool
        min_pool_size: 1
        max_pool_size: 100

#group options
group:
    # Allow all group deletions.
    # Disable this option to prevent the deletion of the reserved groups belonging
    # to either the Module Protection or Well-Known Key Protection domains.
    # A reserved group is defined as any public segregated group which has the same name as
    # the protection domain it belongs to.
    allow_all_group_deletions: true

% sudo /opt/nfast/python3/bin/dbmt db-init --config /opt/nfast/webservices/dbmt-2.2.0/config.yaml

Starting initialisation of the database...
Establishing connection to hardserver
TLS enabled
X509 Authentication enabled
Establishing mongo connection to: localhost:27017
Creating database with name: nshield-corecrypto
Setting indices on collections: nshield-corecrypto
Adding to database: Security World: Security World identifier: 57f6ab61-5547-4ed3-a249-4a9c1710c167
Migrating the module certificates
Migrating "module_BD10-03E0-D947" module file
Adding to database: module certificates: esn: BD10-03E0-D947, hkml: 1dd6a3890c4ec65010466ee3ba5eaef3d6a61777
Migrating cardsets
Migrating cards
getting protection domain by uuid
Adding to database: domain: name: Module Protection type: Module id: 5ee26b95-7e90-53c3-ae75-2e5ddea011bc
Adding to database: group_id: 5ee26b95-7e90-53c3-ae75-2e5ddea011bc name: Module Protection domain_id: 5ee26b95-7e90-53c3-ae75-2e5ddea011bc domain_type Module
getting protection domain by uuid
Adding to database: domain: name: Well-Known Key Protection type: WellKnown id: 2bd40730-85b1-5deb-8417-fb78a7735743
Adding to database: group_id: 2bd40730-85b1-5deb-8417-fb78a7735743 name: Well-Known Key Protection domain_id: 2bd40730-85b1-5deb-8417-fb78a7735743 domain_type WellKnown
Initialisation of the database completed.
Finished

% sudo /opt/nfast/webservices/sbin/install

-- Running install fragment corecrypto
Creating wsopd group.
Checking for user 'wsopd'
Creating wsopd user.
useradd: warning: the home directory already exists.
Not copying any file from skel directory into it.
Checking user 'wsopd' is in correct group 'wsopd'
users created correctly
Installing startup scripts for 'corecrypto'.
Enabling the systemd service unit
Adding and enabling a systemd unit
Created symlink /etc/systemd/system/multi-user.target.wants/nc_corecrypto.service → /etc/systemd/system/nc_corecrypto.service.
Note: Forwarding request to 'systemctl enable nc_corecrypto.service'.
Starting nCipher 'corecrypto' server process.
Job for nc_corecrypto.service failed because the control process exited with error code.
See "systemctl status nc_corecrypto.service" and "journalctl -xe" for details.
● nc_corecrypto.service - nFast corecrypto
   Loaded: loaded (/etc/systemd/system/nc_corecrypto.service; enabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Tue 2024-05-21 16:02:03 EDT; 30ms ago
     Docs: https://nshielddocs.entrust.com/
  Process: 295512 ExecStopPost=/bin/chgrp $WSOPD_GROUP corecrypto.log (code=exited, status=0/SUCCESS)
  Process: 295510 ExecStopPost=/bin/chown $WSOPD_USER corecrypto.log (code=exited, status=0/SUCCESS)
  Process: 295508 ExecStopPost=/bin/bash -c echo corecrypto quit results: $SERVICE_RESULT $EXIT_CODE $EXIT_STATUS >> corecrypto.log (code=exited, status=0/SUCCESS)
  Process: 295506 ExecStopPost=/bin/bash -c echo "'corecrypto' shut down" >>corecrypto.log (code=exited, status=0/SUCCESS)
  Process: 295503 ExecStopPost=/bin/bash -c date >>corecrypto.log (code=exited, status=0/SUCCESS)
  Process: 295501 ExecStopPost=/bin/rm -f corecrypto.pid (code=exited, status=0/SUCCESS)
  Process: 295352 ExecStartPost=/bin/bash -c if [ -f "/opt/nfast/scripts/startup/wait-for-corecrypto" ]; then exec "/opt/nfast/scripts/startup/wait-for-corecrypto"; fi (code=exited, status=1/FAILURE)
  Process: 295351 ExecStart=/bin/bash -c if [ -f /etc/nfast.conf ]; then . /etc/nfast.conf; fi; exec "/opt/nfast/sbin/crypto"  >>corecrypto.log 2>>corecrypto.log (code=exited, status=1/FAILURE)
  Process: 295349 ExecStartPre=/bin/bash -c if [ -f /etc/nfast.conf ]; then owrite=$(stat -c%A /etc/nfast.conf | awk '{print substr($0,length-1,1)}'); if [ "$owrite" != "-" ]; then echo "/etc/nfast.conf can be written to by non-root users" >> corecrypto.log; exi\
t 1; fi; fi (code=exited, status=0/SUCCESS)
  Process: 295347 ExecStartPre=/bin/bash -c if [ -f /etc/nfast.conf ]; then owner=$(stat -c%u:%g /etc/nfast.conf); if [ "$owner" != "0:0" ]; then echo "/etc/nfast.conf is not fully owned by root" >> corecrypto.log; exit 1; fi; fi (code=exited, status=0/SUCCESS)
  Process: 295345 ExecStartPre=/bin/chgrp $WSOPD_GROUP corecrypto.log (code=exited, status=0/SUCCESS)
  Process: 295343 ExecStartPre=/bin/chown $WSOPD_USER corecrypto.log (code=exited, status=0/SUCCESS)
  Process: 295341 ExecStartPre=/bin/touch corecrypto.log (code=exited, status=0/SUCCESS)
  Process: 295339 ExecStartPre=/bin/chgrp $WSOPD_GROUP corecrypto.pid (code=exited, status=0/SUCCESS)
  Process: 295337 ExecStartPre=/bin/chown $WSOPD_USER corecrypto.pid (code=exited, status=0/SUCCESS)
  Process: 295335 ExecStartPre=/bin/touch corecrypto.pid (code=exited, status=0/SUCCESS)
 Main PID: 295351 (code=exited, status=1/FAILURE)

May 21 16:01:58 wsopserver-redhat-8 systemd[1]: Starting nFast corecrypto...
May 21 16:01:58 wsopserver-redhat-8 systemd[1]: nc_corecrypto.service: Main process exited, code=exited, status=1/FAILURE
May 21 16:02:03 wsopserver-redhat-8 bash[295352]: waiting for 'corecrypto'
May 21 16:02:03 wsopserver-redhat-8 bash[295352]: 'corecrypto' did not start; see /opt/nfast/log/corecrypto.log.
May 21 16:02:03 wsopserver-redhat-8 systemd[1]: nc_corecrypto.service: Control process exited, code=exited status=1
May 21 16:02:03 wsopserver-redhat-8 systemd[1]: nc_corecrypto.service: Failed with result 'exit-code'.
May 21 16:02:03 wsopserver-redhat-8 systemd[1]: Failed to start nFast corecrypto.
2024-05-21 16:01:58.416 [INFO] [DBADAPTER] [295351] TLS CA key loaded
2024-05-21 16:01:58.416 [ERROR] [DBADAPTER] [295351] TLS configuration failed: open /opt/nfast/webservices/corecrypto/tls/db/wsop_server.key: permission denied
2024-05-21 16:01:58.416 [FATAL] [WSOP] [295351] [server] NewWSOPServer: cannot initialise database invalid adapter configuration
Tue May 21 16:02:03 EDT 2024
'corecrypto' shut down
corecrypto quit results: exit-code exited

% sudo chown -R root:wsopd /opt/nfast/webservices/corecrypto/tls
% sudo chmod -R 750 /opt/nfast/webservices/corecrypto/tls

xxx.xxx.xxx.xxx wsopserver-redhat-8

% curl -k 'https://xxx.xxx.xxx.xxx:18001/health' | jq


  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    52  100    52    0     0   4333      0 --:--:-- --:--:-- --:--:--  4333
{
  "releaseId": "1.5.0",
  "status": "pass",
  "version": "1"
}

% sudo curl -X GET \
  --cacert /opt/nfast/webservices/corecrypto/tls/db/myCA.pem \
  --cert /opt/nfast/webservices/corecrypto/tls/external/wsop_client.pem \
  --key /opt/nfast/webservices/corecrypto/tls/external/wsop_client.key \
  --header 'Content-Type: application/json' \
  --header 'Accept: application/json' 'https://xxx.xxx.xxx.xxx:18001/health' | jq

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    52  100    52    0     0   5200      0 --:--:-- --:--:-- --:--:--  5200
{
  "releaseId": "1.5.0",
  "status": "pass",
  "version": "1"
}

% /opt/nfast/sbin/crypto --version

crypto, 2.3.0-714-5306bc4