What’s needed to transmit? A look at the minimum steps required for programming our anchor nic’s to send packets

Access to PRO/1000 registers Device registers are hardware mapped to a range of addresses in physical memory You obtain the location (and the length) of this memory-range from a BAR register in the nic device’s PCI Configuration Space Then you request the Linux kernel to setup an I/O ‘remapping’ of this memory-range to ‘virtual’ addresses within kernel-space

i/o-memory remapping dynamic ram nic registers vram IO-APIC Local-APIC user space APIC registers kernel code/data nic registers vram ‘virtual’ address-spacephysical address-space 1-GB 3-GB

Enhanced ‘82573.c’ module Originally we created this module to view the PCI Configuration Space for our nic Today we’ve added two character-mode device-driver ‘methods’ allowing us to see values in all of the nic’s internal registers You can use our ‘fileview’ utility to ‘read’ and ‘seek’ in the ‘/dev/82573’ device-file (or you could write your own application)

An observation We notice that the 82573L device retains the values in many of its internal registers This fact reduces the programming steps that will be required to operate our nic on the anchor cluster machines, since Intel’s own Linux device driver (‘e1000.ko’) has already initialized many nic registers But we WILL need to bring ‘eth1’ down!

Use ‘/sbin/ifconfig’ You can use the ‘/sbin/ifconfig’ command to find out whether the ‘eth1’ interface has been turned off: $ /sbin/ifconfig eth1 If it is still operating, you can turn it off with the (privileged) command: $ sudo /sbin/ifconfig eth1 down

Our ‘nicping.c’ demo We created this module to help us identify all of the programming steps that would be needed for our nic to ‘transmit’ a packet The action of displaying this module’s pseudo-file (e.g., with the ‘cat’ command) will trigger transmission of a ‘broadcast’ message to all the other stations on our anchor cluster’s LAN (if they are ‘listening’)

Tx-Desc Ring-Buffer Circular buffer (128-bytes minimum) TDBA base-address TDLEN (in bytes) TDH (head) TDT (tail) = owned by hardware (nic) = owned by software (cpu) 0x00 0x10 0x20 0x30 0x40 0x50 0x60 0x70 0x80

How ‘transmit’ normally works descriptor0 descriptor1 descriptor2 descriptor Buffer0 Buffer1 Buffer2 Buffer3 List of Buffer-Descriptors We setup each data-packets that we want to be transmitted in a ‘Buffer’ area in ram We also create a list of buffer-descriptors and inform the NIC of its location and size Then, when ready, we tell the NIC to ‘Go!’ (i.e., start transmitting), but let us know when these transmissions are ‘Done’ Random Access Memory

How ‘transmit’ works here descriptor0 descriptor1 descriptor2 descriptor3 descriptor4 descriptor5 descriptor6 descriptor7 Packet Buffer List of Buffer-Descriptors We setup just one packet-buffer, and we let all of our descriptors pointing to it We retain ‘ownership’ of all our descriptors, so nothing gets transmitted, until we have prepared all our data in the packet-buffer Then, when ready, we tell the NIC to ‘Go!’ (i.e., start transmitting), by giving the nic ‘ownership’ of only the next descriptor Random Access Memory TDH TDT

Essential ‘transmit’ registers enum{ E1000_CTRL0x0000,// Device Control E1000_STATUS0x0008,// Device Status E1000_IMC0x00D8,// Interrupt Mask Clear E1000_TCRL0x0400,// Transmit Control E1000_TDBAL0x3800,// Tx Descriptor Base Address Low E1000_TDBAH0x3804,// Tx Descriptor Base Address High E1000_TDLEN0x3808,// Tx Descriptor Length E1000_TDH0x3810,// Tx Descriptor Head E1000_TDT0X3818,// Tx Descriptor Tail E1000_RA0x5400,// Receive address-filter Array };

Programming challenge Can you see how use the code shown in this example-module to implement a TX capability in our earlier ‘netframe.c’ driver?