1. Interrupt Handling Sarah Diesburg COP 5641

2. Interrupt Handling One big responsibility of an operating system is to handle hardware connected to the machine  Hard drives, keyboards and mice, wireless radios, and karaoke microphones Processors are usually much faster than devices

3. Interrupts Prevent CPUs from busy waiting A signal that the hardware can send when it wants the CPU’s attention Need to pay attention to concurrency issues

4. Topics Interrupt handling  Overview  Registration of handlers  Interaction with hardware  Limitations of handlers  Deregistration  Tasklets and bottom halves  Interrupt sharing

5. Overview of Interrupts

6. Preparing the Parallel Port LDD3 illustrates interrupt handling with the short module Setting bit 4 of port 2 ( 0x37a or 0x27a ) enables interrupt reporting (via outb call) Once enabled, the parallel interface generates an interrupt whenever the electrical signal at pin 10 (ACK bit) changes from low to high

7. Preparing the Parallel Port Without a printer, one can connect pins 9 and 10 of the parallel connector  Pin 9 is the most significant bit of the parallel data byte  Writing ASCII to /dev/short0 will not generate any interrupts  Writing binary data will generate several interrupts  Use D-25 connector

8. Installing an Interrupt Handler Without a interrupt handler installed for an interrupt, Linux simply acks and ignores it Since interrupt lines are few (typically 16), sharing is expected  Typically, only “crufty” ISA drivers do not share

9. Installing an Interrupt Handler If not sharing interrupt line (almost never)  Initialize interrupt handler when the device is first opened  Call free_irq in the last close If sharing interrupt line (almost always)  Initialize interrupt handler during module load  Call free_irq in module unload

10. Installing an Interrupt Handler To register an interrupt handler, call #include int request_irq(unsigned int irq, irq_handler_t handler, unsigned long flags, const char *name, void *dev);  irq : the requested interrupt number  handler : the interrupt handler function pointer  flags : various interrupt handler flags  name : for /proc/interrupts  dev : pointer/unique cookie for shared interrupt lines (can be set to NULL if not shared)

11. Installing an Interrupt Handler flags  IRQF_DISABLED indicates a “fast” interrupt handler If set, interrupts are disabled on the current processor If unset, interrupt handlers run with all interrupts enabled except their own (usual setting)  Never have to worry about own interrupt handler being nested  IRQF_SHARED signals that the interrupt can be shared  IRQF_SAMPLE_RANDOM indicates that the generated interrupts can contribute to generate random numbers (used by / dev/random and / dev/urandom )

12. Installing an Interrupt Handler On success, request_irq returns zero  Common error is –EBUSY, which denotes given interrupt line is already in use and no sharing request_irq can sleep  Calls kmalloc() later on… Make sure device is completely set up before calling request_irq  Don’t want to start servicing interrupts before device is ready

13. Installing an Interrupt Handler The short example if (short_irq >= 0) { result = request_irq(short_irq, short_interrupt, IRQF_DISABLED, "short", NULL); if (result) { printk(KERN_INFO "short: can't get assigned irq %i\n", short_irq); short_irq = -1; } else { /* enable it -- assume this *is* a parallel port */ outb(0x10,short_base+2); }

14. The /proc Interface /proc/interrupts shows interrupts with installed handlers CPU0 CPU1 0: 4848108 34 IO-APIC-edge timer 2: 0 0 XT-PIC cascade 8: 3 1 IO-APIC-edge rtc 10: 4335 1 IO-APIC-level aic7xxx 11: 8903 0 IO-APIC-level uhci_hcd 12: 49 1 IO-APIC-edge i8042 NMI: 0 0 LOC: 4848187 4848186 ERR: 0 MIS: 0 Linux handles interrupts on the first CPU to maximize cache locality Device names Programmable interrupt controllers

15. The /proc Interface /proc/stat shows number of interrupts received since system boot  Architecture dependent file format  Look for the intr string intr 5167833 5154006 2 0 2 4907 0 2 68 4 0 4406 9291 50 0 0 Total number Interrupt number 4 used 4907 times

16. Implementing a Handler Cannot transfer data to and from user space Cannot sleep  Cannot call schedule, wait_event, down  Can only use GFP_ATOMIC to allocate memory Might need to clear a bit on the interface board  Allows subsequent interrupts to be received

17. Implementing a Handler Wakes up processes waiting for the interrupt  The network card example Must process packets while waiting for more packets The interrupt handler copies new networking packets into main memory and readies network card for more packets The handler needs to execute in a minimum amount of time  Uses bottom half (typically tasklet or workqueue) to schedule computation later E.g. network card – to sort packets and send them to correct application

18. Implementing a Handler The short example irqreturn_t short_interrupt(int irq, void *dev_id { struct timeval tv; int written; do_gettimeofday(&tv); written = sprintf((char *)short_head,"%08u.%06u\n", (int)(tv.tv_sec % 100000000), (int)(tv.tv_usec)); short_incr_bp(&short_head, written); /* bp = buffer pointer */ wake_up_interruptible(&short_queue); return IRQ_HANDLED; }

19. Implementing a Handler static inline void short_incr_bp(volatile unsigned long *index, int delta) { unsigned long new = *index + delta; barrier(); /* Don't optimize these two together */ *index = (new >= (short_buffer + PAGE_SIZE)) ? short_buffer : new; } Without barrier… static inline void short_incr_bp(volatile unsigned long *index, int delta) { *index = *index + delta; /* could expose an incorrect value */ if (*index >= (short_buffer + PAGE_SIZE)) *index = short_buffer; } Variable can be accessed externally at any time

20. Implementing a Handler To read the buffer, use /dev/shortint ssize_t short_i_read(struct file *filp, char __user *buf, size_t count, loff_t *f_pos) { int count0; DEFINE_WAIT(wait); while (short_head == short_tail) { prepare_to_wait(&short_queue, &wait, TASK_INTERRUPTIBLE); if (short_head == short_tail) { schedule(); } finish_wait(&short_queue, &wait); if (signal_pending(current)) /* a signal arrived */ return -ERESTARTSYS; /* tell the fs layer to handle it */ }

21. Implementing a Handler /* count0 is the number of readable data bytes */ count0 = short_head - short_tail; if (count0 < 0) {/* wrapped */ count0 = short_buffer + PAGE_SIZE - short_tail; } if (count0 < count) { count = count0; } if (copy_to_user(buf, (char *)short_tail, count)) { return -EFAULT; } short_incr_bp(&short_tail, count); /* wrap the tail pointer */ return count; }

22. Implementing a Handler To raise interrupts  Connect pins 9 and 10 of the parallel connector  Write to /dev/shortint Which alternately writes 0x00 and 0xff to the parallel port An interrupt is raised whenever the electrical signal at pin 10 (ACK bit) changes from low to high

23. Implementing a Handler To write to /dev/shortint ssize_t short_i_write(struct file *filp, const char __user *buf, size_t count, loff_t *f_pos) { int written = 0, odd = *f_pos & 1; unsigned long port = short_base; void *address = (void *) short_base; if (use_mem) { /* memory-mapped */ while (written < count) iowrite8(0xff*((++written + odd) & 1), address); } else { while (written < count) outb(0xff*((++written + odd) & 1), port); } *f_pos += count; return written; }

24. Implementing a Handler Without connecting pins 9 and 10  Use /dev/shortprint to drive a printer  Write implementation uses a circular buffer to store data to be printed  Read implementation is same as shown

25. Handler Arguments and Return Value Typical use of the argument in an interrupt handler static irqreturn_t sample_interrupt(int irq, void *dev_id) { struct sample_dev *dev = dev_id; /* now `dev' points to the right hardware item */ /*.... */ }  irq : for printk  dev_id : for finding out which instance of device is in charge of the current interrupt event

26. Handler Arguments and Return Value  Returns IRQ_HANDLED if the device needs attention; otherwise, returns IRQ_NONE Kernel can detect “spurious interrupts” if all interrupts on line return IRQ_NONE Typical open code static void sample_open(struct inode *inode, struct file *filp) { struct sample_dev *dev = hwinfo + MINOR(inode->i_rdev); request_irq(dev->irq, sample_interrupt, 0 /* flags */, "sample", dev /* dev_id */); /*....*/ return 0; }

27. Enabling and Disabling Interrupts Interfaces for manipulating state of interrupts  Disable interrupt system for current processor  Mask out interrupt line for entire machine  and Why?  Synchronization  Common scenario Obtain lock to prevent another processor from accessing shared data Disabling interrupts provides protection against concurrent access from a possible interrupt handler

28. Enabling and Disabling Interrupts For current processor only  local_irq_disable() disables interrupts Dangerous to call if interrupts were already disabled prior to invocation  local_irq_enable() enables interrupts Unconditionally enables interrupts

29. Enabling and Disabling Interrupts Sometimes a mechanism is needed to restore interrupts to a previous state  E.g. common code path can be reached both with and without interrupts enabled Since kernel is complex, much safer to restore to previous state using flags  local_irq_save(flags)  local_irq_restore(flags)

30. Disabling a Single Interrupt Can disable (mask out) a specific interrupt line for an entire system  E.g. disable delivery of a device’s interrupts before manipulating its state  Use of functions are discouraged, and cannot disable shared interrupt lines (think ISA) void disable_irq(int irq); void disable_irq_nosync(int irq); void enable_irq(int irq);

31. Disabling a Single Interrupt Calls can be nested  If disable_irq is called twice, two enable_irq calls are required to reenable the IRQ The calling thread of the disable_irq should not hold resource needed by the current interrupt to complete disable_irq_nosync returns immediately  Need to handle potential race conditions

32. Checking Interrupt Status Macro irqs_disabled() returns nonzero if the interrupt system on the local processor is disabled Checking current context  in_interrupt() Returns nonzero if in interrupt handler or bottom half  in_irq() Returns nonzero only if in interrupt handler

33. Top and Bottom Halves Interrupt handling sometimes needs to perform lengthy tasks  This problem is resolved by splitting the interrupt handler into two halves Top half responds to the interrupt  The one registered to request_irq  Saves data to device-specific buffer and schedules the bottom half Bottom half is scheduled by the top half to execute later  With all interrupts enabled  Wakes up processes, starts I/O operations, etc.

34. Top and Bottom Halves Two mechanisms may be used to implement bottom halves  Tasklets No sleep  Workqueues Can sleep Short module can be loaded to use either tasklet or workqueue

35. Tasklets Cannot run in parallel with itself  Scheduling is not cumulative Can run in parallel with other tasklets on SMP systems Guaranteed to run on the same CPU that first scheduled them  Can be sure that tasklet does not begin executing before the handler has completed

36. Tasklets In the short example, use tasklet=1 to install the tasklet-based interrupt handler void short_do_tasklet(unsigned long); DECLARE_TASKLET(short_tasklet, short_do_tasklet, 0); irqreturn_t short_tl_interrupt(int irq, void *dev_id) { /* cast to stop 'volatile' warning */ do_gettimeofday((struct timeval *) tv_head); short_incr_tv(&tv_head); tasklet_schedule(&short_tasklet); short_wq_count++; /* record that an interrupt arrived */ return IRQ_HANDLED; }

37. Tasklets void short_do_tasklet (unsigned long unused) { int savecount = short_wq_count, written; short_wq_count = 0; /* number of interrupts before this call */ written = sprintf((char *)short_head, "bh after %6i\n",savecount); short_incr_bp(&short_head, written); do { /* write the time values */ written = sprintf((char *)short_head,"%08u.%06u\n", (int)(tv_tail->tv_sec % 100000000), (int)(tv_tail->tv_usec)); short_incr_bp(&short_head, written); short_incr_tv(&tv_tail); } while (tv_tail != tv_head); wake_up_interruptible(&short_queue); }

38. Workqueues Invoke a function at some future time in the context of a special worker process Can sleep Cannot copy data to and from user space

39. Workqueues In the short example, set wq=1 to install the workqueue-based interrupt handler static struct work_struct short_wq; /* this line is in the short_init() to initialize a work_struct*/ INIT_WORK(&short_wq, (typeof(short_wq.func)) short_do_tasklet, NULL); irqreturn_t short_wq_interrupt(int irq, void *dev_id) { do_gettimeofday((struct timeval *) tv_head); short_incr_tv(&tv_head); schedule_work(&short_wq); short_wq_count++; /* record that an interrupt arrived */ return IRQ_HANDLED; }

40. Interrupt Sharing Installing a shared handler  Set IRQF_SHARED flag when requesting interrupt  The dev_id must be unique Cannot be NULL  Returns IRQ_NONE if the handler is not the target handler request_irq() succeeds if  The interrupt line is free  All handlers registered agree to share

41. Interrupt Sharing When an interrupt arrives, the kernel invokes every handler registered for that interrupt  The handler must be able to recognize its own interrupts No probing function is available for shared handlers  Most hardware designed for interrupt sharing can tell the CPU which interrupt it is using No need for explicit probing

42. Interrupt Sharing free_irq needs the correct dev_id Watch out for enable_irq and disable_irq  Not a good idea to disable other devices’ interrupts

43. Running a Handler In the short example, use shared=1 to install a shared interrupted handler irqreturn_t short_sh_interrupt(int irq, void *dev_id, struct pt_regs *regs) { int value, written; struct timeval tv; /* If it wasn't short, return immediately */ value = inb(short_base); if (!(value & 0x80)) return IRQ_NONE; /* clear the interrupting bit */ outb(value & 0x7F, short_base); Check the most significant bit

44. Running a Handler /* the rest is unchanged */ do_gettimeofday(&tv); written = sprintf((char *)short_head,"%08u.%06u\n", (int)(tv.tv_sec % 100000000), (int)(tv.tv_usec)); short_incr_bp(&short_head, written); wake_up_interruptible(&short_queue); return IRQ_HANDLED; } Assumes that pins 9 and 10 are connected The example would not work for printers, since the printer protocol disallow sharing

45. The /proc Interface and Shared Interrupts Check /proc/interrupts CPU0 0: 892335412 XT-PIC timer 1: 453971 XT-PIC i8042 2: 0 XT-PIC cascade 5: 0 XT-PIC libata, ehci_hcd 8: 0 XT-PIC rtc 9: 0 XT-PIC acpi 10: 11365067 XT-PIC ide2, uhci_hcd, uhci_hcd, SysKonnect 11: 4391962 XT-PIC uhci_hcd, uhci_hcd 12: 224 XT-PIC i8042 14: 2787721 XT-PIC ide0 15: 203048 XT-PIC ide1 NMI: 41234

46. Interrupt-Driven I/O Buffering improves performance  Also leads to interrupt-driven I/O Input buffer is filled at interrupt time  Emptied by the read processes Output buffer is filled by write processes  Emptied at interrupt time Hardware generates interrupts when  New data has arrives and is ready for retrieval  When it is ready to accept new data or to acknowledge a successful data transfer

47. A Write-Buffering Example Shortprint module implements output- oriented driver for the parallel port The write function calls shortp_write()  Calls shortp_start_output() Schedules a timer that calls shortp_timeout()  Calls either shortp_timeout() or shortp_interrupt() Schedules shortp_do_work()  Calls shortp_do_write() to write individual characters The printer calls shortp_interrupt() Schedules shortp_do_work()

48. A Write-Buffering Example printer write()

49. A Write-Buffering Example The shortprint example maintains a one- page circular output buffer  A write system call only writes data to the buffer  The actual write is scheduled later static size_t shortp_write(struct file *filp, const char __user *buf, size_t count, loff_t *f_pos) { int space, written = 0; unsigned long flags; if (down_interruptible(&shortp_out_sem)) return –ERESTARTSYS;

50. A Write-Buffering Example while (written < count) { /* Hang out until some buffer space is available. */ space = shortp_out_space(); if (space <= 0) { if (wait_event_interruptible(shortp_out_queue, (space = shortp_out_space()) > 0)) goto out; }...

51. A Write-Buffering Example /* Move data into the buffer. */ if ((space + written) > count) space = count - written; if (copy_from_user((char *) shortp_out_head, buf, space)) { up(&shortp_out_sem); return -EFAULT; } shortp_incr_out_bp(&shortp_out_head, space); buf += space; written += space;...

52. A Write-Buffering Example /* If no output is active, make it active. */ spin_lock_irqsave(&shortp_out_lock, flags); if (!shortp_output_active) shortp_start_output(); spin_unlock_irqrestore(&shortp_out_lock, flags); } out: *f_pos += written; up(&shortp_out_sem); return written; }

53. shortp_start_output static DECLARE_WORK(shortp_work, shortp_do_work, NULL); static struct workqueue struct *shortp_workqueue; static void shortp_start_output(void) { if (shortp_output_active) /* Should never happen */ return; /* Set up a timer to handle occasionally missed interrupts */ shortp_output_active = 1; shortp_timer.expires = jiffies + TIMEOUT; add_timer(&shortp_timer); /* calls shortp_timeout */ /* And get the process going. */ queue_work(shortp_workqueue, &shortp_work); }

54. shortp_do_work static void shortp_do_work(void *unused) { int written; unsigned long flags; shortp_wait(); /* wait until the device is ready */ spin_lock_irqsave(&shortp_out_lock, flags); /* Have we written everything? */ if (shortp_out_head == shortp_out_tail) { /* empty */ shortp_output_active = 0; wake_up_interruptible(&shortp_empty_queue); del_timer(&shortp_timer); } else /* Nope, write another byte */ shortp_do_write();

55. shortp_do_work /* If somebody's waiting, wake them up if enough space. */ if (((PAGE_SIZE + shortp_out_tail - shortp_out_head) % PAGE_SIZE) > SP_MIN_SPACE) { wake_up_interruptible(&shortp_out_queue); } spin_unlock_irqrestore(&shortp_out_lock, flags); }

56. shortp_do_write static void shortp_do_write(void) { unsigned char cr = inb(shortp_base + SP_CONTROL); /* Reset the timer */ mod_timer(&shortp_timer, jiffies + TIMEOUT); /* Strobe a byte out to the device */ outb_p(*shortp_out_tail, shortp_base+SP_DATA); shortp_incr_out_bp(&shortp_out_tail, 1); if (shortp_delay) udelay(shortp_delay); outb_p(cr | SP_CR_STROBE, shortp_base+SP_CONTROL); if (shortp_delay) udelay(shortp_delay); outb_p(cr & ~SP_CR_STROBE, shortp_base+SP_CONTROL); }

57. shortp_interrupt static irqreturn_t shortp_interrupt(int irq, void *dev_id, struct pt_regs *regs) { if (!shortp_output_active) return IRQ_NONE; /* Remember the time, and farm off the rest to the workqueue function */ do_gettimeofday(&shortp_tv); queue_work(shortp_workqueue, &shortp_work); return IRQ_HANDLED; }

58. shortp_timtout static void shortp_timeout(unsigned long unused) { unsigned long flags; unsigned char status; if (!shortp_output_active) return; spin_lock_irqsave(&shortp_out_lock, flags); status = inb(shortp_base + SP_STATUS);

59. shortp_timtout /* If the printer is still busy we just reset the timer */ if ((status & SP_SR_BUSY) == 0 || (status & SP_SR_ACK)) { shortp_timer.expires = jiffies + TIMEOUT; add_timer(&shortp_timer); spin_unlock_irqrestore(&shortp_out_lock, flags); return; } /* Otherwise we must have dropped an interrupt. */ spin_unlock_irqrestore(&shortp_out_lock, flags); shortp_interrupt(shortp_irq, NULL, NULL); }