i/o multiplexing On adding a ‘poll()’ method to our character-mode device-driver for an 82573L network controller

An application idea We want to create an application program that would allow two users to conduct an on-line chat-session while working from two separate nodes on our ‘anchor’ cluster Whatever either user decides to type will be displayed on the both users’ screens To keep their two input-streams visually separated we want to use two ‘windows’

‘netchat.cpp’ Hello, Susan Hi, Paul _ Hi, Paul Hello, Susan _ anchor01anchor02 Whatever a user types will be displayed in the lower window on that user’s screen -- and in the upper window on the other user’s screen

Multiplexed input-streams In order to implement the foregoing idea, it’s necessary for the ‘netchat’ program to accept input from TWO hardware devices: –the keyboard (i.e., ‘standard input’ device) –the ethernet controller (i.e., ‘/dev/nic’) Such a situation is a fairly common one in UNIX/Linux application programming, and is referred to as ‘i/o multiplexing’

Device-driver’s role Special software support is needed within each of the device-drivers in order for any application-program to do ‘i/o multiplexing’ in a way that avoids wasting of CPU time (For an excellent discussion of the various approaches that can be taken to deal with this basic programming issue, see Richard Stevens: Advanced Programming in the UNIX Environment, Chapter 12)

The basic problem Normally when an application ‘reads’ from a device-file, that process will ‘sleep’ until some data is available from that device So if data becomes available on another device, it will not get processed because the application is ‘blocked’ from being given any CPU time by the OS scheduler This would spoil our ‘netchat’ application

‘read()’ causes ‘blocking’ ‘netchat’ application terminal Ethernet controller Whichever device the application attempts to read from, it will get ‘blocked’ until that device has some data to deliver read write

Do multiprocessing? One idea for getting around this ‘blocking’ problem would be to just use the ‘fork()’ system-call to create a separate process for reading from the different device-files Each process can sleep, and whichever process receives any new data will be awakened and scheduled for execution No changes needed to device-driver code

Different processes do ‘read()’ ‘netchat’ parent- process terminal Ethernet controller ‘netchat’ child-process read write Using multiple processes can overcome the ‘blocking-read’ problem, but complicates the code for program termination

Non-blocking ‘read’ It is possible for the application to request ‘non-blocking’ read-operations – i.e., any ‘read()’ calls will immediately return with 0 as return-value in case no data is available The standard-input device-driver already has support for this non-blocking option, and it can be easily added to the ‘read()’ function in network controller’s driver

Code-modification ssize_t my_read( struct file *file, char *buf, size_t len, loff_t *pos ) { static intrxhead = 0; // in case no new data has been received, then either // return immediately if non-blocking mode is in effect // or else sleep until some new data arrives (or until // the user hits -C to cancel execution) if ( rxhead == ioread32( io + E1000_RDH ) { if ( file->f_flags & O_NONBLOCK ) return 0; if ( wait_event_interruptible( wq_rx, rxring != ioread32( io + E1000_RDH ) ) return –EINTR; } …

Uses ‘busy-waiting’ loop ‘netchat’ application terminal Ethernet controller read write Using the ‘nonblocking-read’ option overcomes the problem of a sleeping task, but it wastefully consumes the CPU time write

The ‘elegant’ solution The ‘select()’ system-call provides a very general scheme for doing i/o-multiplexing in a manner that avoids wasting CPU time or making the program-code complicated But it does require adding an extra driver ‘method’ – the so-called ‘poll()’ function

The ‘select()’ arguments Using ‘select()’ requires an application to setup an ‘fd_set’ object, which defines the set of file-descriptors whose activity needs to be monitored by the Linux kernel (in our ‘netchat’ application this would be just the two device-files: the console keyboard and the gigabit ethernet network controller) This ‘fd_set’ object becomes an argument

Using ‘select()’ in ‘netchat’ intkbd = STDIN_FILENO;// keyboard ID intaux = open( “/dev/nic”, O_RDWR );// device-file ID fd_set permset;// create an ‘fd_set’ object FD_ZERO( &permset );// initialize it to ‘empty’ FD_SET( kbd, &permset );// add keyboard to set FD_SET( aux, &permset );// and add the nic to set while (1){ fd_setreadset = permset; if ( select( 1+aux, &readset, NULL, NULL, NULL ) < 0 ) break; if ( FD_ISSET( kbd, &readset ) ) { /* process keyboard input */ } if ( FD_ISSET( aux, &readset ) ) { /* process network input */ } }

How it works The ‘readset’ argument to the ‘select()’ system-call lets the kernel know which device-drivers should have their ‘poll()’ method invoked Then each device-driver’s ‘poll()’ method will perform a test to determine if any new data is ready to be read from that device So the application calls ‘read()’ only when a device is ready with data immediately

struct file_operations We need to include the function-pointer to our implementation for the ‘poll()’ method: struct file_operations my_fops ={ owner:THIS_MODULE, read:my_read, write:my_write, ioctl:my_ioctl, poll:my_poll, };

Our driver’s ‘poll()’ method Linux provides helper-functions to do most of the supporting work for use of ‘select()’ #include // for the ‘poll_wait()’ helper-function unsigned int my_poll( struct file *file, struct poll_table_struct *wait ) { unsigned intmask = 0; poll_wait( file, &wq_recv, wait ); if ( (ioread32( io + E1000_RDH ) != rxhead ) mask |= POLLIN | POLLRDNORM; returnmask; }

The ‘ncurses’ library Our ‘netchat’ application uses the Linux implementation for the UNIX ‘curses’ API This function-library includes routines that not only can manage the console’s cursor, but also implements ‘raw’ keyboard input (for instantaneous keystroke processing) and allows drawing of window-borders

Compiling ‘netchat’ The ‘g++’ compiler can compile-and-link in a single command: $ g++ netchat.cpp -l ncurses -o netchat The C++ source-file Lowercase letter ‘L’ (for library to link with) Name of the library Name for the compiler’s output-file

Try it out We can try running the ‘netchat’ program on any pair of our anchor-cluster stations Of course, we will first need to install our ‘nicpoll.ko’ device-driver module on each of those ‘anchor’ machines Then a user at either machine can type in any messages, and every character typed will be visible immediately on both screens

But… There is a slight problem with using the present version of our ‘nicpoll.c’ module All of the network packets are ‘broadcast’ to every station on the ‘anchor’ cluster! So any third-party who is simultaneously trying to use our network will be receiving all the packets that are being broadcast

In-class exercise Based on what you have learned from our past lessons, and from your reading of the Intel “Open Source Software Developer’s Manual” for its PCIe GbE Controllers, can you propose any ideas for using the NIC’s hardware capabilities so as to allow a pair of anchor-stations to use ‘netchat’ without any other anchor-stations being involved?