Virtualization Remote Server

The nShield Solo XC is compatible with the leading server virtualization and hypervisor management platforms, including:

Virtualization provides an environment where multiple operating systems can run at the same time on one physical computer. Each virtual machine is an isolated, virtualized computer system that can run its own operating system.

Microsoft Hyper-V, a role in Windows Server used to create and manage a virtualized server computing environment
VMware vSphere / ESXi, a robust, bare-metal hypervisor that installs directly onto your physical server.

All vSphere management functions are performed through remote management tools.
Citrix XenServer - includes the XenCenter management console.

PCI passthrough is configured using the XenCenter software with command line tools and utilities. PCI passthrough allows a VM client direct access to the nShield Solo XC.

The operating system that runs within a virtual machine is referred to as a guest operating system.

nShield software includes the nShield hardserver applications. These applications enable applications running on multiple virtual guest operating systems to all share nShield Solo XC hardware.

Hardserver processing services can be shared among multiple virtual operating system instances as long as each instance has hardserver installed. Inside of the operating system, hardservers can communicate with other hardservers.

Virtualization and Hyper-V

The host hardserver is configured to run on the Parent/Dom0 operating system. The Parent/Dom0 operating system has privileged access to the Solo XC hardware over the PCI bus.

Instead of using a physical network for communication between the VM guest instances running on the same physical system, most hypervisors provide the capability to instantiate some form of virtual switch which allows the network communication to take place between the VMs entirely within the hypervisor software. This means that nCore data does not need to be routed outside of the server hardware.

Virtualization and XenServer/VMware vSphere hypervisor, ESXi

ESXi and XenServer do not use the concept of a Parent/Dom0 VM. Instead, an additional VM is defined in the system as the host with passthrough permissions to enable access to the nShield Solo XC.

ESXi environment

After installing VMware ESXI, the VM guest can be remotely managed and the PCI passthrough of the Solo module configured using vSphere. PCI passthrough allows a VM guest direct access to the nShield Solo XC.

Set up a basic single-node vCenter server instance

Follow the steps below to use the vCenter Simple Install to set up a basic single-node vCenter Server instance. You will install the vSphere Web Client and use its in-browser interface to add ESXi hosts to your vSphere inventory.

Log on the system as administrator and start at least one ESXi host.
Install ESXi using the vCenter Simple Install option using the instructions provided in the VMware vSphere documentation (https://docs.vmware.com/en/VMware-vSphere/index.html).
Install the vSphere Web Client using the instructions provided in the VMware vSphere documentation.

Configure passthrough devices on a host

Follow the steps below to add ESXi hosts to the vCenter Server inventory, in order to create a vSphere environment and use vSphere features.

Enter the IP address, username and root password of your host created when you installed ESXi.
Select Login, the Getting Started page will be displayed.
Select the Configuration tab.
Select Advanced Settings.
Select Configure Passthrough. The Passthrough Configuration page is displayed listing all available passthrough devices.
Select Edit.
Select the check box to mark the endpoint for passthrough.

For example, the check mark box for 02:00.0 will be Freescale Semiconductor Inc <class> Power PC.
Select OK.

ESXi will now be successfully installed and the Solo PCIe module has been configured for passthrough.

Create the VM guest instance

VMware ESXi provides the capability of PCI passthrough and it is a bare metal Hypervisor. This requires the creation of two or more guests which communicate via Vswitch. One of the guest will act as the primary guest and will be configured as described below utilizing the PCI card connected via passthrough. The second and subsequent guests can be composed of the identical configuration with the exception of the PCI passthrough connection.

To create the VM guest instance:

Navigate to File > New > Virtual Machine in the vSphere Client. A wizard will prompt you through each of the settings displayed in the working pane.
Select Typical Configuration and then select Next.
Enter a name and select Next.
Select a storage device for the VM files.
Select a Guest Operating System (OS) and an OS version from the drop down menu.
Select Next.
Configure the network connections as follows:
1. How many NICs do you want to connect? 1.
2. Network: VM Network.
3. Adapter: VMXNET 3.
4. Connect at Power On: ✓.
Select Next.
Configure the virtual disk size for the guest VM as follows:

It is important to select the same network configuration for both the guest primary VM and the guest secondary VM, as it is a requirement for IP communication between the two.
1. Datastore: <datastore1>.
2. Available space (GB): <357.3>.
3. Virtual disk size: 50 GB.
4. Select Thick Provisioned Lazy Zeroed.
Select Next.
Select Edit the virtual machine settings before completion.
Select Continue.
Select Add.
Select PCID.
Select Next.
Select the configured PCI passthrough device.

For example, 02:00.0 will be Freescale Semiconductor Inc <class> Power PC.
Select Next.
Select Finish.

XenServer environments

Install the XenServer, follow the instructions in the Citrix XenServer Quick Start Guide, see https://docs.citrix.com/en-us/xenserver.

Configure the XenCenter client

To remotely manage VM guests and configure PCI passthrough of the nShield Solo XC:

Enter the XenServer web client IP address.
Select XenCenter installer. The XenCenter software will auto install.
Select the XenServer that you want to connect to and manage from the Resources pane. A connection is established providing access to all the VMs installed on the server.

Select the Console tab from the Properties tabs pane.

Dom0 is the initial domain started by the Xen hypervisor on boot. Dom0 runs the Xen management toolstack and is has direct access to the hardware. Dom0 provides Xen virtual disks and network access for VM guests, each VM guest is referred to as a DomU (that is, an unprivileged domain).

Run the command lspci.

A detailed list of all the PCI buses and devices in the system is displayed, for example:
```
02:00.0 Power PC:  Freescale Semiconductor Inc Device 082c (rev11)02:00:0
```
represents the nShield Solo XC card endpoint.
Open the file /boot/extlinux.conf and scroll to the dom0 linux kernel append section. Add the PCI slot as shown below with the following command:
```
pciback.hide=(02:00.0)
```
Newer versions of Citrix XenServer utilize:

xen-pciback.hide=(xx:xx:x)
Scroll to the end of the file.

Run the command:

pciback.hide=<NG solo card endpoint>

This command enters the PCI slot, for example:

pciback.hide=(02:00.0) --- /boot/initrd-fallback.img

Save and close the file.
Run the command:
```
extlinux -I /boot
```
Run the command:
```
reboot
```
Run the command:
```
xe vm-list
```
Locate the uuid using the VI Editor for the VM that you want to assign the PCI passthrough to.

Run the command:

xe vm-param-set other-config:pci=0/0000:<endpoint of the NG solo card> uuid: <uuid>

This command adds the PCI device to the selected VM, for example:

xe vm-param-set other-config:pci=0/0000:02:00.0 uuid: 4a4ab965-a91d-70e7-2ec-a4c0004e1e8d

If a PCI passthrough needs to be removed from a specific guest VM, run the command:

xe vm-param-clear param-name=other-config uuid=<vm uuid>

When the installation of XenCenter has completed, you can access [https://( XENSERVERIP)] to acquire the corresponding XenCenter Client Remote management interface.

Create a XenServer guest instance and hardserver configuration

The XenServer is a bare metal Hypervisor that provides the PCI passthrough capability. As part of this process, you must create two Dom U guests that communicate through the Vswitch. One guest acts as the primary guest and is configured as described below utilizing the PCI card connected via passthrough. The second guest can be composed of the identical configuration with the exception of the PCI passthrough connection.

To create the first DomU guest VM:

Select the server from the Resources pane, right-click and select New VM from the dropdown menu.
Select a Template.
Select an Operating system for the first DomU guest VM.
Select Next.
Select Name.
Enter a name and select Next.

The DomU guest VM name will also be displayed in the XenCenter’s Resources pane. You can change the name at any time.
Select Installation Media.
Select Install from ISO library or DVD drive and then select the appropriate media from the drop down menu.
Select Next.
Select Home Server.
Select Place the VM on this server and then select a home server from the drop down list of available servers.
Select Next.
Select CPU & Memory and enter the number of CPUs, chose your topology and enter an amount for memory.
Select Next.
Select Storage.
Select Use these virtual disks: and select a virtual disk from the display.
Select Next.
Select Networking and select the virtual network interface.
Select Finish.

If the guest VM is configured to have a PCI module via passthrough and the module is not connected to the VM instance, the guest VM instance will fail to power on. Verify that the Solo XC card is located on the same slot that was selected for the passthrough to the guest VM.

Hyper-V environment

The instructions assume there is a single nShield Solo XC module in the system.

The commands starting with PS C:\> should be run in PowerShell in elevated mode.

Set up

Install Hyper-V on the server

Follow the instructions in the Windows documentation for Hyper-V, see https://docs.microsoft.com/en-us/windows-server/virtualization/hyper-v/.

Add the Hyper-V role to the server

To add the Hyper-V role in Windows server:

Log in as Administrator.
Open Server Manager.
Select Manage.
Select Add Roles and Features.
Select Next.
Select Role-based or feature-based installation.
Select Next.
Select Select a server from the server pool.
Select a server that has Windows 2016 installed. You will be adding Hyper-V to this server.
Select Next.
Select Hyper-V.
Select Next.
Reboot the system.

Once rebooted, Hyper-V will be supported by the Server 2016 instance.

Prepare the server

Enable the Input Output Memory Management Unit (IOMMU) policy on the server. This policy controls whether the Hyper-V server uses an IOMMU. To enable it, run the command:
```
bcdedit /set hypervisoriommupolicy enable
```
Check no devices are already set up for VM. Run the command:
```
PS C:\> Get-VMHostAssignableDevice
```

Prepare the device

Display the device address. Run the command:

PS C:\> (Get-PnpDevice -PresentOnly).Where{ $_.InstanceId -like '*VEN_1957*' } | Format-Table -autosize

Disable the device. Run the command:

PS C:\> Disable-PnpDevice -Verbose -InstanceId $instanceId -Confrm:$false

To find the $instanceId run the command:

PS C:\> $instanceId = (Get-PnpDevice -PresentOnly).Where{ $_.InstanceId -like '*VEN_1957*' } | select -expand InstanceId

Dismount the device. Run the command:

PS C:\> $locationPath = Dismount-VmHostAssignableDevice -LocationPath $locationPath -Force -Verbose

To find the $locationPath run the command:

PS C:\> $locationPath = (Get-PnpDeviceProperty -KeyName DEVPKEY_Device_LocationPaths -InstanceId $instanceId).Data[0]

Verify that the device is disabled and dismounted. Run the command:
```
PS C:\> Get-VMHostAssignableDevice
```

Install the Security World software

Install the Security World software suite into the operating system of the guest VM. Once the suite is installed, you can initialize the hardserver and then configure the guest VMs.

Insert the DVD-ROM containing the Security World software. The Security World software will auto install.
Run the enquiry utility to check that the module is working correctly. See Checking the installation.

Create the VM guest instance on the server

Open the Hyper-V Manager within your Windows 2016 server.
Log in as Administrator.
Navigate to Action > New > Virtual Machine.
Select Next (to create a virtual machine with a custom configuration).
Enter a name for the new guest VM instance.

Use the default location setting.
Select Next.
Select the OS generation to be installed on the new guest VM instance.

For example, Generation 2 is selected. Generation 2 is valid for products such as Windows 8 and beyond and with Windows Server 2016.
Select Next.
Select an amount of memory for allocation to this guest VM instance.
Select Next.
Select Next.
Select Create a virtual hard disk.
Enter Name, Location and Size.
Select Next.
Select one of the following options:
- Install an operating system later, if you have a disk.
- Install an operating system from a bootable image file, if you have the ISO path.
Select Next.
Select Finish.

Configure the VM guest instance on the server

Stop and select the VM guest instance. Run the commands:

PS C:\> $vmName = 'ws2016'

PS C:\> Stop-VM -VMName $vmName

Turn off the Automatic Stop Action. Run the command:

PS C:\> Set-VM -VMName $vm Name -AutomaticStopAction TurnOff

Make sure the memory minimum bytes match the memory startup bytes. Run the command:

PS C:\> Set-VM -VM $vm -DynamicMemory -MemoryMinimumBytes 4096MB -MemoryMaximumBytes 16384MB -MemoryStartupBytes 4096MB

Assign a device to the VM guest instance. Run the commands:

PS C:\> Add-VMAssignableDevice -VM $vmName -LocationPath $locationPath -Verbose

PS C:\> Start-VM -VMName $vmName

To find the $locationPath run the command:

PS C:\> $locationPath = (Get-PnpDeviceProperty -KeyName DEVPKEY_Device_LocationPaths -InstanceId $instanceId).Data[0]

It is possible to assign the same device to a single VM guest instance multiple times. In this case the VM will not start. To check currently assigned devices, run the command below. To remove an assigned device see Remove a device from the VM guest instance.

PS C:\> Get-VMAssignableDevice -VMName $vmName

Remove a device from the VM guest instance

Remove a device from the VM. Run the commands:

PS C:\> $vmName = “ws2016”

PS C:\> Remove-VMAssignableDevice -Verbose -VMName $vmName}

Undo passthrough

Mount a single device. Run the command:

Mount-VMHostAssignableDevice -Verbose -LocationPath $locationPath

To find the $locationPath run the command:

PS C:\> $locationPath = (Get-PnpDeviceProperty -KeyName DEVPKEY_Device_LocationPaths -InstanceId $instanceId).Data[0]

Enable a single device in device manager. Run the command:

Enable-PnpDevice -Confirm:$false -Verbose -InstanceId $instanceId

To find the $locationPath run the command:

PS C:\> $locationPath = (Get-PnpDeviceProperty -KeyName DEVPKEY_Device_LocationPaths -InstanceId $instanceId).Data[0]