Accessing network hardware The Network Interface Controllers are part of a larger scheme used in modern PCs for device control

Memory-mapped I/O Non-Intel processor architectures typically will dedicate a set of memory-addresses to performing communication with hardware devices which are installed in the system Such architectures allow software to read device-status or write device-commands using ordinary CPU instruction-opcodes, such as ‘mov’, ‘test’, ‘add’, ‘xor’, et cetra

I/O ports But Intel’s original x86 architecture used a different approach in which a separate set of addresses and ‘special’ instructions are employed for accessing system devices The device addresses are called I/O ports, and the opcodes are named ‘in’ and ‘out’: –The ‘in’ instruction reads a device’s ‘status’ –The ‘out’ instruction writes device ‘command’

Range of I/O ports For devices integrated into a PC system’s motherboard, the port-numbers used were 8-bit values and could be specified within an I/O instruction, as ‘immediate’ data Example:in $0x64, %al# read port 0x64 For optional devices plugged into ‘slots’ on the motherboard, a port-number could be a 16-bit value, located in the DX register Example: in %dx, %al# indirect address

Two separate address-spaces memory address-space (4GB) I/O address-space (64KB) accessed only by using the special ‘in’ and ‘out’ instructions accessible by using a wide variety of general-purpose arithmetical and logical instructions

Early PCs Peripheral devices in the early PCs used fixed i/o-ports and fixed memory-addresses, e.g.: –Video memory address-range: 0xA0000-0xBFFFF –Programmable timer i/o-ports: 0x40-0x43 –Keyboard and mouse i/o-ports: 0x60-0x64 –Real-Time Clock’s i/o-ports:0x70-0x71 –Hard Disk controller’s i/o-ports: 0x01F0-01F7 –Graphics controller’s i/o-ports: 0x03C0-0x3CF –Serial-port controller’s i/o-ports: 0x03F8-0x03FF –Parallel-port controller’s i/o-ports: 0x0378-0x037A

The PC’s evolution It became clear in the 1990s that there would be contention among equipment vendors for ‘fixed’ resource-addresses, which of course were in limited supply Among the goals that motivated the PCI Specification was the creation of a more flexible scheme for allocating addresses that future peripheral devices could use

PCI Configuration Space PCI Configuration Space Body (48 doublewords – variable format) 64 doublewords PCI Configuration Space Header (16 doublewords – fixed format) A non-volatile parameter-storage area for each PCI device-function

PCI Configuration Header Status Register Command Register Device ID Vendor ID BIST Cache Line Size Class Code Class/SubClass/ProgIF Revision ID Base Address 0 Subsystem Device ID Subsystem Vendor ID CardBus CIS Pointer reserved capabilities pointer Expansion ROM Base Address Minimum Grant Interrupt Pin reserved Latency Timer Header Type Base Address 1 Base Address 2Base Address 3 Base Address 4Base Address 5 Interrupt Line Maximum Latency doublewords Dwords

Three x86 address-spaces memory space (4GB) i/o space (64KB) PCI configuration space (16MB) accessed using a large variety of processor instructions (mov, add, or, shr, push, etc.) and virtual-to-physical address-translation accessed only by using the processor’s special ‘in’ and ‘out’ instructions (without any translation of port-addresses) i/o-ports 0x0CF8-0x0CFF dedicated to accessing PCI Configuration Space

reserved Interface to PCI Configuration Space CONFADD ( 0x0CF8) CONFDAT ( 0x0CFC) ENEN bus (8-bits) device (5-bits) doubleword (6-bits) function (3-bits) 00 PCI Configuration Space Address Port (32-bits) PCI Configuration Space Data Port (32-bits) 31 0 Enable Configuration Space Mapping (1=yes, 0=no)

Reading PCI Configuration Data Step one: Output the desired longword’s address (bus, device, function, and dword) with bit 31 set to 1 (to enable access) to the Configuration-Space Address-Port Step two: Read the designated data from the Configuration-Space Data-Port: # read the PCI Header-Type field (byte 2 of dword 3) for bus=0, device=0, function=0 movl$0x C, %eax# setup address in EAX movw$0x0CF8, %dx# setup port-number in DX outl%eax, %dx# output address to port mov$0x0CFC, %dx# setup port-number in DX inl%dx, %eax# input configuration longword shr$16, %eax# shift word 2 into AL register movb%al, header_type# store Header Type in variable

Demo Program We created a short Linux utility that searches for and reports all of your system’s PCI devices It’s named “pciprobe.cpp” on our class website It uses some C++ macros that expand to Intel input/output instructions -- which normally are ‘privileged’ instructions that a Linux application- program is not allowed to execute (segfault!) But using ‘sudo’ you can escalate the privilege- level at which this utility-program will be run: $ sudo./pciprobe

Example: network interface We can identify the network interface controllers in our classroom’s PC’s by class-code 0x02 Then the subclass-code 0x00 is for ‘Ethernet’ We can identify the NIC from its VENDOR and DEVICE identification-numbers: VENDOR_ID = 0x8086// for Intel Corporation DEVICE_ID = 0x109A// for 82573L controller Our second demo-program (i.e, ‘82573pci.cpp’) shows this nic’s full PCI Configuration Space

nic Typical NIC TX FIFO RX FIFO transceiver LAN cable BUSBUS main memory packet buffer CPU

Intel’s Pro1000 nics These network controllers implement a large number of 32-bit device registers Two ways are implemented for accessing these numerous registers: –Using a small range of I/O port-addresses –Using a large range of ‘memory-mapped’ register-addresses

PCI Base-Address registers The key to accessing our NIC’s various registers is provided by values we find in certain PCI Configuration Space locations port-address 32-bit Base-Address Register layout for I/O space 1 RSVRSV memory-address 32-bit Base-Address Register layout for Memory space 0 PREFPREF unused RSVRSV 00=32-bit decode, 01=reserved, 10=64-bit decode, 11=reserved

Output from ‘83573pci.cpp’ executed on ‘anchor01’ BAR0 (at 0x10): 0xE (Physical memory-address for NIC’s mapped register-bank) BAR1 (at 0x14): 0x BAR2 (at 0x18): 0x (I/O port-address for NIC’s ‘multiplexing’ register-bank) BAR3 (at 0x1C): 0x BAR4 (at 0x20): 0x BAR5 (at 0x24): 0x

‘mapped’ advantage For high-performance (and ‘thread-safe’) access to the NIC’s bank of registers, the memory-mapped option offers very clear advantages over multiplexed I/O access: –Access is faster to memory than to I/O ports –Access to memory is ‘atomic’, whereas the multiplexed access via I/O ports is a 2-step operation, giving rise to ‘race conditions’

I/O advantage But during the system’s ‘startup’ process (before the Operating System is loaded), there’s no alternative but to use the I/O access-method, because the memory- mapped registers are at high addresses which the processor cannot yet reach For example, in cases where an OS needs to be loaded via a BOOTP network-server

Clarification Don’t confuse PCI Configuration Space registers with our NIC’s device registers! CPU’s I/O-port address-space NIC’s PCI Configuration Space registers window System’s PCI Configuration Space NIC device-registers Network Interface Controller CPU’s memory address-space BAR2BAR0

In-class exercise Copy our ‘82573pci.cpp’ source-module to your own directory, then rename it as ‘audio.cpp’ Your assignment is to modify it so that it will display the PCI Configuration Space registers in our classroom workstation’s sound-card (a ‘multimedia’ class device)