Virtual Partitioning

Introduction

Virtual Partitioning is a new option that allows a WSOP administrator to manage the access to database records by controlling their visibility on a client-by-client basis. An X.509 certificate is used to identify the client and defines the virtual partition label, which is used to filter the client’s view of the database. Multiple clients can be assigned to share the same virtual partition label and records can also be assigned to a common, visible to all users label.

Corecrypto configuration file

Virtual Partitioning database and the mapping collection need to specified be in the corecrypto config.yaml.

Virtual Partitioning database

The segregation_db_name field specifies the name of Virtual Partitioning database.

    # WSOP Corecrypto Segregation (WCS) Options
    # The segregation database and collection must be defined in order
    # to enable WCS. When enabled, corecrypto objects
    # will be segregated based on the mappings defined in the collection.
    # If the segregation database and collection are not defined,
    # WCS is disabled.
    #
    segregations_db_name: segregation_db

Virtual Partitioning collection

segregations_collection_name field specifies the collection inside Virtual Partitioning database that contains the mappings of the virtual partitions

    # WSOP Corecrypto Segregation (WCS) Options
    # The segregation database and collection must be defined in order
    # to enable WCS. When enabled, corecrypto objects
    # will be segregated based on the mappings defined in the collection.
    # If the segregation database and collection are not defined,
    # WCS is disabled.
    #
    segregations_db_name: segregation_db
    segregations_collection_name: segregations
Virtual Partitioning is enabled once both fields, segregation_db_name and segregations_collection_name, are defined. If one of the fields is commented out or undefined, Virtual Partitioning is disabled.

Virtual Partitioning database mapping

The mapping inside Virtual Partitioning collection should contain the following fields:

  • ISSUER

  • SUBJECT

  • label

The mappings will be created by the database administrator as WSOP’s rights on Virtual Partitioning database are readonly.

Issuer

The "ISSUER" field contains relative distinguished names of the X.509 certificate issuer for the client that is mapped to a specific virtual partition.

Subject

The "SUBJECT" field contains relative distinguished names of the X.509 certificate subject for the client that is mapped to a specific virtual partition.

Relative distinguished names

Only the following relative distinguished names should be used in the "ISSUER" and "SUBJECT" fields:

  • CN for common name

  • OU for organizational unit

  • O for organization name

  • PC for postal code / zip code

  • STREET for street / first line of address

  • L for locality name

  • ST (or SP or S) for state or province name

  • C for country

The order of these names matters and they should be listed in the order defined above. If a name is missing, it should not be listed in the "ISSUER"/"SUBJECT" field.

Label

The "label" field contains the virtual partition name of the client. If the "label" field does not exist or is empty "label":"", the virtual partition will be set to common.

The common virtual partition can be accessed and modified by all clients that are mapped in Virtual Partitioning collection.
If the client in use does not have a mapping in Virtual Partitioning database, all operations performed by that client will give a 403:Forbidden error.
{
  "error": "Forbidden",
  "message": "unknown issuer and subject in segregation database",
  "path": "/km/v1/groups",
  "status": "403",
  "timestamp": "2021-12-15T16:48:21Z"
}
A client should not have the X.509’s issuer and subject mapped to more the one label in the database.

Index

Subject and issuer index must be set as case insensitive and unique.

An example using mongo shell for virtual partitioning database set to "segregation_db" and collection set to "segregations":

use segregation_db

db.createCollection("segregations", { collation: { locale: 'en', strength: 2 } } )

db.segregations.createIndex({ "issuer": 1, "subject": 1 }, { unique: true } )
locale must be set to the correct language: https://docs.mongodb.com/manual/reference/collation-locales-defaults/

Example of a valid mapping record

An example of a valid mapping record:

{
  "_id": {
    "$oid": "documentID"
  },
  "label": "VirtualPartitionLabel",
  "subject": "CN=CommonName,OU=OrganizationalUnit,O=Organization,C=Country",
  "issuer": "CN=CommonName,OU=OrganizationalUnit,O=Organization,L=Locality,C=Country"
}

Example of setting a mapping record using mongo shell

use segregation_db

db.segregations.insert({
  "label": "VirtualPartitionLabel",
  "subject": "CN=CommonName,OU=OrganizationalUnit,O=Organization,C=Country",
  "issuer": "CN=CommonName,OU=OrganizationalUnit,O=Organization,L=Locality,C=Country"
})

Migration

Virtual Partitioning is supported by the Database Management Tool. For more information about the migration of a partitioned RFS, see the Database Management Tool section of this User Guide.

Logs and errors corecrypto usage

Virtual Partitioning disabled

This is an example output where at least one of the fields, segregation_db_name and segregations_collection_name, is commented out or undefined, Virtual Partitioning is disabled.

[INFO] [DBADAPTER] [1129] TLS CA key loaded
[INFO] [DBADAPTER] [1129] TLS client keys loaded
[INFO] [DBADAPTER] [1129] NewMongoAdapter: Created new MongoDB Adapter client - Hosts: [mongo1:30001 mongo2:30002 mongo3:30003] ReplicaSet: rs1 Database name: nshield-corecrypto Segregation: disabled
[INFO] [WSOP] [1129] [server] configureAPI: database initialised
[DEBUG] [WSOP] [1129] [DB] Adapter[ncore]: databaseHealthCheck: started
This is not an error, the service will run with Virtual Partitioning disabled

Virtual Partitioning enabled

This is an example output where both fields, segregation_db_name and segregations_collection_name, are defined, Virtual Partitioning is enabled.

[INFO] [WSOP] [1180] [config] loglevel set to: debug
[INFO] [DBADAPTER] [1180] TLS CA key loaded
[INFO] [DBADAPTER] [1180] TLS client keys loaded
[INFO] [DBADAPTER] [1180] NewMongoAdapter: Created new MongoDB Adapter client - Hosts: [mongo1:30001 mongo2:30002 mongo3:30003] ReplicaSet: rs1 Database name: nshield-corecrypto Segregation: enabled. Using mappings from segregation_db.segregations
[INFO] [WSOP] [1180] [server] configureAPI: database initialised
[DEBUG] [WSOP] [1180] [DB] Adapter[ncore]: databaseHealthCheck: started

Virtual Partitioning enabled and the client is not mapped

This is an example output where a client is not mapped.

[ERROR] [DBADAPTER] [1180] getOneRecord: record not found where [{issuer CN=CommonName,OU=OrganizationalUnit,O=Organization,L=Locality,C=Country} {subject CN=CommonName,OU=OrganizationalUnit,O=Organization,C=Country}]
[ERROR] [DBADAPTER] [1180] GetSegregationLabel: database record not found
[ERROR] [WSOP] [1180] [KM] NCoreKeyManager[c66a5fcc-be23-5d3b-aa7a-997ec27e3b71]: NewNCoreKeyManager: cannot get the segregation label: database record not found
[ERROR] [WSOP] [1180] [KM] NCoreKeyManager[c66a5fcc-be23-5d3b-aa7a-997ec27e3b71]: NewNCoreKeyManager: cannot get the segregation label: database record not found
[ERROR] [WSOP] [1180] [request] [9e36eedd-acc9-430b-abec-7e0b54a8fd4b] &models.RestErrorObject{Error:"Forbidden", Message:"unknown issuer and subject in segregation database", Path:"/km/v1/groups", Status:"403", Timestamp:"2021-12-15T16:48:21Z"}
[INFO] [WSOP] [1180] [request] [9e36eedd-acc9-430b-abec-7e0b54a8fd4b] IPaddress - "GET /km/v1/groups HTTP/2.0" 403

Example script for Virtual Partitioning database generation

The following is an example python script that demonstrates how the segregation db and collection is created and populated with clients and their corresponding virtual partitions.

The segregation_map dictionary defines the label for each certificate. The Subject and Issuer fields are extracted from the certificate and this is inserted into the database with the corresponding label.

#!/usr/bin/python3
"""
This example script adds segregation mappings for TLS certs as defined in
segregation_map
"""
from pymongo import MongoClient
from pymongo.collation import Collation
from cryptography import x509
from os import path

config = {
    "MONGO_HOST": "mongo1",
    "MONGO_PORT": 30001,
    "SEGREGATION_DB": "segregation_db",
    "SEGREGATION_COLLECTION": "segregations",
}

# When no label is provided, the client belongs to the common virtual partition
segregation_map = {
    # client certificate : partition label
    "client1.crt": "Label_A",
    "client2.crt": "Label_A",
    "client3.crt": "Label_B",
    "client4.crt": "Label_C",
    "client5.crt": None,
    "client6.crt": None
}

try:
    # Create the Segregation database and collection
    with MongoClient(host=config["MONGO_HOST"], port=config["MONGO_PORT"]) as mongoClient:
        mongoClient[config["SEGREGATION_DB"]
                    ][config["SEGREGATION_COLLECTION"]].drop()
        # The index of Subject and Issuer is case insensitive
        collection = mongoClient[config["SEGREGATION_DB"]].create_collection(
            config["SEGREGATION_COLLECTION"], collation=Collation(locale='en', strength=2))
        collection.create_index([("issuer", 1), ("subject", 1)], unique=True)

        documents_to_add = []

        # Process certificates
        for certificate in segregation_map:
            filename = path.join(path.dirname(
                path.abspath(__file__)), certificate)
            with open(filename, 'rb') as pfile:
                cert = x509.load_pem_x509_certificate(pfile.read())
                
                subject = []

                subject_serial_number = cert.subject.get_attributes_for_oid(
                    x509.NameOID.SERIAL_NUMBER)
                if subject_serial_number:
                    subject.append(subject_serial_number[0].rfc4514_string())

                subject_common_name = cert.subject.get_attributes_for_oid(
                    x509.NameOID.COMMON_NAME)
                if subject_common_name:
                    subject.append(subject_common_name[0].rfc4514_string())

                subject_organization_unit_name = cert.subject.get_attributes_for_oid(
                    x509.NameOID.ORGANIZATIONAL_UNIT_NAME)
                if subject_organization_unit_name:
                    subject.append(
                        subject_organization_unit_name[0].rfc4514_string())

                subject_organization_name = cert.subject.get_attributes_for_oid(
                    x509.NameOID.ORGANIZATION_NAME)
                if subject_organization_name:
                    subject.append(
                        subject_organization_name[0].rfc4514_string())

                subject_postal_code = cert.subject.get_attributes_for_oid(
                    x509.NameOID.POSTAL_CODE)
                if subject_postal_code:
                    subject.append(subject_postal_code[0].rfc4514_string())

                subject_street_address = cert.subject.get_attributes_for_oid(
                    x509.NameOID.STREET_ADDRESS)
                if subject_street_address:
                    subject.append(subject_street_address[0].rfc4514_string())

                subject_locality_name = cert.subject.get_attributes_for_oid(
                    x509.NameOID.LOCALITY_NAME)
                if subject_locality_name:
                    subject.append(subject_locality_name[0].rfc4514_string())

                subject_province_name = cert.subject.get_attributes_for_oid(
                    x509.NameOID.STATE_OR_PROVINCE_NAME)
                if subject_province_name:
                    subject.append(subject_province_name[0].rfc4514_string())

                subject_country_name = cert.subject.get_attributes_for_oid(
                    x509.NameOID.COUNTRY_NAME)
                if subject_country_name:
                    subject.append(subject_country_name[0].rfc4514_string())

                subject = ','.join(subject)

                issuer = []

                issuer_serial_number = cert.issuer.get_attributes_for_oid(
                    x509.NameOID.SERIAL_NUMBER)
                if issuer_serial_number:
                    issuer.append(issuer_serial_number[0].rfc4514_string())

                issuer_common_name = cert.issuer.get_attributes_for_oid(
                    x509.NameOID.COMMON_NAME)
                if issuer_common_name:
                    issuer.append(issuer_common_name[0].rfc4514_string())

                issuer_organization_unit_name = cert.issuer.get_attributes_for_oid(
                    x509.NameOID.ORGANIZATIONAL_UNIT_NAME)
                if issuer_organization_unit_name:
                    issuer.append(
                        issuer_organization_unit_name[0].rfc4514_string())

                issuer_organization_name = cert.issuer.get_attributes_for_oid(
                    x509.NameOID.ORGANIZATION_NAME)
                if issuer_organization_name:
                    issuer.append(issuer_organization_name[0].rfc4514_string())

                issuer_postal_code = cert.issuer.get_attributes_for_oid(
                    x509.NameOID.POSTAL_CODE)
                if issuer_postal_code:
                    issuer.append(issuer_postal_code[0].rfc4514_string())

                issuer_street_address = cert.issuer.get_attributes_for_oid(
                    x509.NameOID.STREET_ADDRESS)
                if issuer_street_address:
                    issuer.append(issuer_street_address[0].rfc4514_string())

                issuer_locality_name = cert.issuer.get_attributes_for_oid(
                    x509.NameOID.LOCALITY_NAME)
                if issuer_locality_name:
                    issuer.append(issuer_locality_name[0].rfc4514_string())

                issuer_province_name = cert.issuer.get_attributes_for_oid(
                    x509.NameOID.STATE_OR_PROVINCE_NAME)
                if issuer_province_name:
                    issuer.append(issuer_province_name[0].rfc4514_string())

                issuer_country_name = cert.issuer.get_attributes_for_oid(
                    x509.NameOID.COUNTRY_NAME)
                if issuer_country_name:
                    issuer.append(issuer_country_name[0].rfc4514_string())

                issuer = ','.join(issuer)

            record = {}
            record["subject"] = subject
            record["issuer"] = issuer
            if segregation_map[certificate]:
                record["label"] = segregation_map[certificate]
            else:
                # No label value indicates the common partition
                pass

            documents_to_add.append(record)

        collection.insert_many(documents_to_add)

        # Reporting
        for i in collection.find({}):
            # display 'null' when no label
            if 'label' not in i:
                i['label'] = 'null'
            print("Added label: '{0}' subject: '{1}', issuer: '{2}'".format(
                i['label'], i['subject'], i['issuer']))

except Exception as ex:
    print(ex)
    import sys
    sys.exit(1)