DPACC Metadata Revised 2016/4/6

Table of Contents Motivation Information Elements Data representation Convergence discussion for IFA004

Motivation Metadata Static IM definition for dpacc resource DPACC mgmt implementation Dynamic Info exchanges via various interfaces s-API: Acc-agent Nfvi-vim: Agent-VIM VIM-MANO interface

Information Model Discussion 1/2 dev_idDevice IDM a unique device ID for each physical accelerator (either implemented via HW or SW) macMAC addressO MAC address for some of the HW accelerators, e.g. iNIC numa_nodeNUMA FlagM Whether the accelerator is NUMA attaching CPU, default to False. device_typeHW/SW typeM The type of HW/SW device of the accelerator (either physical device or virtual function). Typical types include: GPU, FPGA, SW, ASIC, DSP etc. (Provide a reference, or define them here.) Note: SoC is not a valid “device_type”, as it is actually composed of a collection of accelerators, each should be associated with its own device_id, device_type, etc. acc_funcFunctionalityM The acceleration functionality the accelerator offers. Typical functionalities include: CRYPTO, IPSec, PDCP, FFT, etc. acc_capabilit y Capability Capacity? MThe acceleration capability of the given acceleration functionality. E.g. in case of crypto, the algorithms supported, the expected performance metrics, etc. Capacity is defined in the context of specific configuration setting, while capability is defined as the collection of the capabilities. MigratableAllows migrate O Whether the acceleration resource supports live migration or not.

Information Model Discussion 2/2 queue_num Channel_num Queue Channel number Optional Number of concurrent channels supported by the accelerator (either physical or virtual). queue_type Channel_type Queue TypeOptional The acceleration functionality of a loaded queuechannel on a programmable accelerator or multi-purpose accelerator. address Accessible Address Mandatory for HW accelerator Accessible Address for HW accelerator. Bus address for bus attached accelerators, or assigned system address for integrated accelerator or virtual address for a virtual function. For example, Bar address for PCI device (via IO configuration). Need further examples for non-PCI device (e.g. mem address). pf_address Physical Address Optional attributes to “address” Physical address of a HW accelerator that supports virtualization (i.e. the physical accelerator can be sliced into a number of virtual functions, which appear to be an independent physical device to its assignees), otherwise the same with Accessible Address. vendor_idVendor IDMandatory Globally unique vendor ID, assigned by XXX, retrieved from the HW device driver. Default to 0 for SW acc. ?For non-PCI device type. product_idProduct IDMandatory Internally unique product ID, assigned by the vendor, retrieved from the HW device driver. Default to 0 for SW acc. meta_version Version of spec Mandatory The version of the Metadata spec, as defined by Dpacc and implemented by NOMAD project. acc_versionVersion of acc MandatoryThe version of the HW/SW accelerator, as defined by the vendor for the product.

Data Model Discussion 1/2 dev_idString “pci_”+ PFA for pci device,.e.g. pci_0000_05_00_1 What about other types of device? macString The Hexadecimal (in hexadecimal) Representation of LAN MAC addresses have been defined in ISO/IEC E.g., 52:54:00:BA:E2:90 numa_nodeBooleanTrue(1) or False(0) device_typeEnumerationGPU, FPGA, SW, ASIC, DSP acc_funcEnumeration CRYPTO, IPSec, PDCP, FFT EMPTY - special value for non-initialized multi-purpose or re-programmable accelerators acc_capabilityStructure A array of Capable Operations, e.g. “encryption”, “decryption”. For each Capable Operation, identify supported configuration sets, e.g. algorithms like “aes”, “3DES” etc. For each Capable Operation under specific configuration setting, specify the capacity of the accelerator. For instance, for AES encryption operation, given 128 bytes input, the throughput metric is 10Gbps. meta_versionStringNeed input for versioning convention acc_versionStringAs defined by the vendor

Data Model Discussion 2/2 channel_numINTEGERPositive integer. channel_typeEnumerationSame as acc_func addressString Unique PFA (PCI Function Address), bus number ’:’ device number ’:’ function number. E.g. 0000:05:00.1 What about other types of device? pf_addressStringSame as address vendor_idString “pci_” + PCI vendor_id (32-bit Hexadecimal (in hexadecimal) Representation) as defined in ) What about other types of device? product_idString“pci_” + PCI vendor_id (32-bit Hexadecimal (in hexadecimal) Representation) as defined in ) What about other types of device?

ETSI NFV IFA004 Resource Feature Requirements NumberingFeatureFunctional requirements description ACC_RES_FEATURE.001 (device_id)UIDEach acceleration resource shall have a unique identifier. ACC_RES_FEATURE.002 (acc_version) Version ( Added as a Mandatory element) Each acceleration resource shall specify the version of its accelerator. ACC_RES_FEATURE.003 (acc_functionality)TypeEach acceleration resource shall have a clear type (e.g. Crypto, FFT, IPSec, etc.) ACC_RES_FEATURE.004 (acc_capability)CapabilitiesEach acceleration resource shall indicate its acceleration specific capabilities. ACC_RES_FEATURE.005 (queue_number)Number of ChannelsEach acceleration resource shall indicate how many channels it supports. ACC_RES_FEATURE.006 Number of Contexts (Need further clarification) Each acceleration resource shall indicate how many contexts it supports. ACC_RES_FEATURE.007 (migratable) Allows Migration (Added as an Optional element) Each acceleration resource shall indicate if it supports live migration capabilities. ACC_RES_FEATURE.008 (acc_capability)QoSEach acceleration resource shall indicate the quality of service level it supports. ACC_RES_FEATURE.009 Data Format (Need further clarification) Each acceleration resource shall indicate the data format they operate on. ACC_RES_FEATURE.010 (acc_type)Re-Programmability Each acceleration resource shall indicate whether it requires a hardware image to be programmed with before it can operate. ACC_RES_FEATURE.011Resource Availability (needed exchange but not metadata) Each acceleration resource shall indicate the level or amount of availability that are currently unused and can be allocated. Table : Functional requirements for acceleration resources features

Todos Start a draft on metadata and call for online review Potentially combined with the DPACC mgmt API discussion Include an example workthrough for selected use-case Element definition and format specification for non-PCI HW acc Convergence with IFA004

Clarifications and List Discussion Context and Channels Acceleration resource context: This identifies a specific environment within the acceleration resource such as configuration and state. Acceleration resource contexts allow an acceleration resource to provide state isolation and protection to multiple users sharing the same acceleration resource and allow users to be migrated between different compatible acceleration resources. Whereas a channel is a path to the accelerator that is flow controlled and could have a specific level of service assigned to it. Examples of channels include virtio vrings and SR-IOV DMA rings, etc. On the IPSEC side, we are moving forward with the virtio based device. Since, many accelerators don’t support virto interface, VMM (hypervisor) is being used as a proxy between virtio-ipsec-device and actual hardware accelerator today. Since, vrings are software resources now, there is not a big limitation on number of vrings. When the hardware accelerator itself supports virtio interface, then the number of vrings (channels) supported by hardware would be limited. In those cases, advertising the number of channels (Rings) supported by hardware to Cloud operating systems (such as Openstack controller) is needed. When an acceleration context is allocated to a VM, the “Cloud Controller” can choose the number of rings to be assigned to that acceleration context. The case described by Srini can be generalized with any situation where channel resources to an accelerator are part of the accelerator domain (i.e. non-CPU domain) – i.e. SR-IOV DMA rings, etc. The capability of supporting contexts – form of accelerator “slicing” can be another view – can be seen as important if VNFs share acceleration blocks and the use of available accelerators must be optimized as much as possible.

Clarifications and List Discussion Data Format Data format is an indicator to what format the accelerator expects the data to be in before it could process it and what format the accelerator will generate. This is mainly important when VNFs are completely offloaded to hardware accelerators and in case of Chaining (when the output of Accelerator A is to be chained to Accelerator B). The decision of whether those accelerators can be chained “directly” or not will take this into account. As far as the “Data format” is concerned : I am assuming that Rabi meant the command/response data format between VMs and Accelerators. At this time, IPSec virtio specifications assumes that the accelerators support any host endian- ness. Since hardware accelerators are supposed to work with both little endian and big endian processors, IPSec virtio specification gives guidance on the host byte order to the accelerator as part of virtio device capability negotiation. It is up to the accelerator to do any conversion.