A network driver ‘framework’ We construct a ‘skeleton’ module showing just the essential pieces of a Linux network device driver

Overview hardware device driver module Linux operating system Kernel networking subsystem application program standard runtime libraries user space kernel space

Source-code layout #include … typedef struct { /* driver’s private data */ } MY_DRIVERDATA; char modname[ ] = “netframe”; struct net_device *netdev; netframe.c my_initmy_exit The mandatory module- administration functions my_open() my_stop() my_hard_start_xmit() my_isr() my_get_info() The network driver’s “payload” functions

module_init() This function will execute when the driver is installed in the kernel (‘/sbin/insmod’) Its role is to allocate and partially initialize a ‘struct net_device’ object for our network interface controller (i.e., hardware device), then “register” that object with the kernel For ethernet NICs there exists a kernel helper-function that drivers can utilize

The ‘key’ statements… typedefstruct { /* the driver’s private data */ } MY_DRIVERDATA; struct net_device *netdev; static int __init my_init( void ) { netdev = alloc_etherdev( sizeof( MY_DRIVERDATA ) ); if ( !netdev ) return –ENOMEM; netdev->open= my_open; netdev->stop= my_stop; netdev->hard_start_xmit = my_hard_start_xmit; returnregister_netdev( netdev ); }

module_exit() This function will execute when the driver is removed from the kernel (‘/sbin/rmmod’) Its role is to “unregister” the ‘net_device’ structure and free the memory that was allocated during the module’s initialization

The ‘key’ statements… struct net_device *netdev; static void __exit my_exit( void ) { unregister_netdev( netdev ); free_netdev( netdev ); }

open() The kernel will call this function when the system administrator “configures” the NIC (e.g., with the ‘/sbin/ifconfig’ command) to assign an IP-address to the interface and and bring it UP Thus the role of ‘open()’ would be to reset the hardware to a known working state and initiate packet-queueing by the kernel

The ‘key’ statements… int my_open( struct net_device *dev ) { /* initialize any remaining ‘private’ data */ /* prepare the hardware for operation */ /* install an Interrupt Service Routine */ /* enable the NIC to generate interrupts */ netif_start_queue( netdev ); return0; //SUCCESS }

stop() The kernel will call this function when the NIC is brought DOWN (i.e., to turn off its transmission and reception of packets) This could occur because of a command (such as ‘/sbin/ifconfig’) executed by the System Administrator, or because a user is removing the driver-module from the kernel (with the ‘/sbin/rmmod’ command)

The ‘key’ statements… int my_stop( struct net_device *dev ) { netif_stop_queue( netdev ); /* kill any previously scheduled ‘tasklets’ (or other deferred work) */ /* turn off the NIC’s transmit and receive engines */ /* disable the NIC’s ability to generate interrupts */ /* delete the NIC’s Interrupt Service Routine */ return0; //SUCCESS }

hard_start_xmit() The kernel will call this function whenever it has data that it wants the NIC to transmit The kernel will supply the address for a socket-buffer (‘struct sk_buff’) that holds the packet-data that is to be transmitted So this function’s duties are: to initiate transmission, update relevant statistics, and then release that ‘sk_buff’ structure

The ‘key’ statements… int my_hard_start_xmit( struct sk_buff *skb, struct net_device *dev ) { /* code goes here to initiate transmission by the hardware */ dev->trans_start= jiffies; dev->stats.tx_packets += 1; dev->stats.tx_bytes += skb->len; dev_kfree_skb( skb ); return0; //SUCCESS }

What about reception? The NIC hardware receives data-packets asynchronously – not at a time of its own choosing – and we don’t want our system to be ‘stalled’ doing ‘busy-waiting’ Thus an interrupt handler is normally used to detect and arrange for received packets to be validated and dispatched to upper layers in the kernel’s network subsystem

Simulating an interrupt Our network device-driver ‘framework’ was only designed for demonstration purposes; it does not work with any actual hardware But we can use a ‘software interrupt’ that will trigger the execution of our ISR To implement this scheme, we’ll need to employ an otherwise unused IRQ-number, along with its associated ‘Interrupt-ID’

Advanced Programmable Interrupt Controller Multi-CORE CPU CPU 0 CPU 1 I/O APIC LOCAL APIC LOCAL APIC ●●●●●● IRQ0 IRQ1 IRQ2 IRQ3 IRQ23 The I/O APIC component is programmable – its 24 inputs can be assigned to interrupt ID-numbers in the range 0x20..0xFF (lower numbers are reserved by Intel for the CPU’s exception-vectors) The I/O-APIC’s 24 Redirection Table registers determine these assignments

The I/O APIC in our classroom machines supports 24 Interrupt-Request input-lines Its 24 programmable registers determine how interrupt-signals get routed to CPUs Two-dozen IRQs Redirection-table

Redirection Table Entry reserved interrupt vector ID L/PL/P STATUSSTATUS H/LH/L RIRRRIRR E/LE/L MASKMASK extended destination destination delivery mode 000 = Fixed 001 = Lowest Priority 010 = SMI 011 = (reserved) 100 = NMI 101 = INIT 110 = (reserved) 111 = ExtINT Trigger-Mode (1=Edge-triggered, 0=Level-triggered) Remote IRR (for Level-Triggered only) 0 = Reset when EOI received from Local-APIC 1 = Set when Local-APICs accept Level-Interrupt sent by IO-APIC Interrupt Input-pin Polarity (1=Active-High, 0=Active-Low) Destination-Mode (1=Logical, 0=Physical) Delivery-Status (1=Pending, 0=Idle)

Our ‘ioapic.c’ module Last semester we created a module that will show us which IRQ-numbers are not currently being used by our system, and the Interrupt-IDs those IRQ-signals were assigned to by Linux during ‘startup’ Timeout for an in-class demonstration

my_isr() We created a “dummy” Interrupt Service Routine for our ‘netframe.c’ demo-module #define IRQ4// temporarily unused (normally for serial-UART #define intID0x49// our I/O-APIC has assigned this ID to to IRQ 4 irqreturn_t my_isr( int irq, void *my_netdev_addr ) { struct net_device *dev = (struct net_device *)my_netdev_addr; MY_DRIVERDATA*priv = dev->priv; // we do processing of the received packet in our “bottom half” tasklet_schedule( &priv->my_rxtasklet ); returnIRQ_HANDLED; }

Installing and removing an ISR if ( request_irq( IRQ, my_isr, IRQF_SHARED, dev->name, dev ) < 0 ) return –EBUSY; IRQ’s signal-number entry-point for interrupt-handler option-flagsname for display ISR data-argument free_irq( IRQ, dev ); This statement would go in the driver’s ‘open()’ function… …and this statement would go in the driver’s ‘stop()’ function Here ‘dev’ is the address of the interface’s ‘struct net_device’ object

Processing a received packet When the NIC notifies our driver that it has received a new ethernet-packet, our driver must allocate a socket-buffer structure for the received data, initialize the ‘sk_buff’ with that data and supporting parameters, then pass that socket-buffer upward to the kernel’s network subsystem for delivery to the appropriate application-program that is listening for it

The ‘key’ statements… void my_rxhandler( unsigned long data ) { struct net_device*dev = (struct net_device *)data; struct sk_buff*skb; intrxbytes = 60;// just an artificial value here skb = dev_alloc_skb( rxbytes + 2 ); skb->dev = dev; skb->protocol = eth_type_trans( skb, dev ); skb->ip_summed = CHECKSUM_NONE; dev->stats.rx_packets += 1; dev->stats.rx_bytes += rxbytes; netif_rx( skb ); }

Triggering the interrupt… We allow a user to trigger execution of our interrupt-handler (for testing purposes), by reading from a pseudo-file that our driver creates during module-initialization, whose ‘get_info()’ function includes execution of a software-interrupt instruction: ‘int $0x49’ This inline assembly language instruction is produced via the GNU ‘asm’ construct

Using the ‘asm-construct’ #define intID0x49 asm(“ int %0 “ : : “i” (intID) ); assembly language opcode parameter indicator parameter type (“i” = immediate data) parameter-value (symbolic) statement keyword This example shows how a symbolic constant’s value, defined in the high-level C programming language using a ‘#define’ preprocessor directive, is able to be referenced by an “inline” assembly language statement within a C code-module

Testing our ‘framework’ You can download, compile, and install our ‘netframe.c’ network driver module It doesn’t do anything with real hardware, but it does illustrate essential interactions of a network device driver with the Linux operating system’s networking subsystem

In-class exercise #1 Use the ‘/sbin/ifconfig’ command to assign an IP-address to the ‘struct net_device’ object that our framework-module creates You can discover the interface’s name by using our earlier ‘netdevs.c’ module You should use a ‘private’ IP-address EXAMPLE (for station ‘hrn23501’): $ sudo /sbin/ifconfig eth up

In-class exercise #2 Use ‘ifconfig’ to confirm the IP-address, the IRQ, and the interface’s status: $ /sbin/ifconfig eth1 Use ‘ifconfig’ to examine the interface’s statistics (packets transmitted/received)

In-class exercise #3 Use the ‘cat’ command to simulate an interrupt from your device’s interface Verify that your interrupt-handler did get executed, by looking at the statistics, and by displaying the output of a pseudo-file Linux creates (named ‘/proc/interrupts’) $ cat /proc/interrupts

In-class exercise #4 Try removing the ‘netframe.ko’ module (with the ‘/sbin/rmmod’ command), then use the ‘dmesg’ command to see your system’s log-file messages