Outline for Today Objectives: Announcements Interrupts (continued) Lottery Scheduling Announcements BRING CARDS TO SHUFFLE

Role of Interrupts in I/O So, the program needs to access an I/O device… Start an I/O operation (special instructions or memory-mapped I/O) Device controller performs the operation asynchronously (in parallel with) CPU processing (between controller's buffer & device). If DMA, data transferred between controller's buffer and memory without CPU involvement. Interrupt signals I/O completion when device is done.

do_IRQ() handle_IRQ_event Ret_from_intr() CPU handles Interrupt Device raises interrupt line, CPU detects this, CPU stops current operation, disables interrupts, enters kernel mode, saves current program counter, jumps to predefined location, saves other processor state needed to continue at interrupted instruction. For each interrupt number, jumps to address of appropriate interrupt service routine. [do_IRQ()] Interrupt context Kernel stack of whatever was interrupted Can not sleep Handlers on this line do what needs to be done. [handle_IRQ_event()] Unless SA_INTERRUPT specified when registered, re-enable interrupts during handler execution If line is shared, loop through all handlers Restores saved state at interrupted instruction [ret_from_intr()], returns to user mode. ld add st mul beq sub bne do_IRQ() handle_IRQ_event Ret_from_intr() User Program Interrupt Handling

Interrupt Control local_irq_disable() and local_irq_enable() -- affecting all interrupts for this processor local_irq_save(…) and local_irq_restore(…) – save and disable interrupts on this processor and restore previous interrupt state disable_irq(irq), disable_irq_nosynch(irq), enable_irq(irq) – affecting particular interrupt line Informational: irqs_disabled() – local interrupts disabled? in_interrupt() – in interrupt context or process context? In_irq() – executing an interrupt handler?

Bottom Half Processing Deferring work that is too heavyweight for interrupt handling Mechanisms: Softirqs “soft interrupts”, statically defined (32 max.) action functions that can run concurrently on SMP pending when bit set in 32-bitmask (usually set in associated interrupt handler [raise_softirq()] run with interrupts enabled, proper locking required Tasklets dynamically created functions that are built upon softirqs, two of the same type can not run concurrently lists of tasklet_struct hooked to 2 of the softirqs Workqueue implemented as kernel-based “worker” threads with process context of their own -- thus allowed to sleep

Lottery Scheduling Waldspurger and Weihl (OSDI 94)

Claims Goal: responsive control over the relative rates of computation Support for modular resource management Generalizable to diverse resources Efficient implementation of proportional-share resource management: consumption rates of resources by active computations are proportional to relative shares allocated

Basic Idea Resource rights are represented by lottery tickets abstract, relative (vary dynamically wrt contention), uniform (handle heterogeneity) responsiveness: adjusting relative # tickets gets immediately reflected in next lottery At allocation time: hold a lottery; Resource goes to the computation holding the winning ticket.

Fairness Expected resource allocation is proportional to # tickets held - actual allocation becomes closer over time. Throughput – Expected number of lotteries won by client E[w] = n p where p = t/T Response time -- # lotteries to wait for first win E[n] = 1/p No starvation w # wins t # tickets T total # tickets n # lotteries Number of lotteries won – binomial distribution Response time – geometric distribution

Example List-based Lottery 10 2 5 1 2 Summing: 10 12 17 Random(0, 19) = 15

Bells and Whistles Ticket transfers - objects that can be explicitly passed in messages Can be used to solve priority inversions Ticket inflation Create more - used among mutually trusting clients to dynamically adjust ticket allocations Currencies - “local” control, exchange rates Compensation tickets - to maintain share with I/O use only f of quantum, ticket inflated by 1/f in next

Kernel Objects Backing tickets Currency name amount 1000 base C_name Active amount 300 ticket Issued tickets

base alice bob task1 task3 task2 thread1 thread3 thread4 thread2 3000 1000 base 2000 base 1 bob = 20 base 1 alice = 5 base alice bob 200 100 100 bob 200 alice 100 alice task1 1 task2= .4 alice = 2 base task3 100 task2 500 100 task1 300 task2 100 task3 200 task2 thread1 thread3 thread4 thread2

base alice bob task1 task3 task2 thread1 thread3 thread4 thread2 3000 1000 base 2000 base 1 bob = 20 base 1 alice = 3.33 base alice bob 300 100 100 bob 200 alice 100 alice task1 1 task2= .4 alice = 1.33 base task3 100 100 task2 500 100 task1 300 task2 100 task3 200 task2 thread1 thread3 thread4 thread2

Example List-based Lottery 1 base 2bob 5 task3 2bob 10 task2 Random(0, 2999) = 1500

Compensation A holds 400 base, B holds 400 base A runs full 100msec quantum, B yields at 20msec B uses 1/5 allotted time Gets 400/(1/5) = 2000 base at each subsequent lottery for the rest of this quantum a compensation ticket valued at 2000 - 400

Ticket Transfer Synchronous RPC between client and server create ticket in client’s currency and send to server to fund it’s currency on reply, the transfer ticket is destroyed

Control Scenarios Dynamic Control Conditionally and dynamically grant tickets Adaptability Resource abstraction barriers supported by currencies. Insulate tasks.

UI mktkt, rmtkt, mkcur, rmcur fund, unfund lstkt, lscur, fundx (shell)

Relative Rate Accuracy Figure 4: Relative Rate Accuracy. For each allocated ratio, the observed ratio is plotted for each of three 60 second runs. The gray line indicates the ideal where the two ratios are identical.

Fairness Over Time 8 second time windows over 200 sec. Execution Figure 5: Fairness Over Time. Two tasks executing the Dhry-stone benchmark with a 2 : 1 ticket allocation. Averaged over the entire run, the two tasks executed 25378 and 12619 iterations/sec., for an actual ratio of 2.01 : 1.

Client-Server Query Processing Rates Figure 7: Query Processing Rates. Three clients with an 8 : 3 : 1 ticket allocation compete for service from a multithreaded database server. The observed throughput and response time ratios closely match this allocation.

Controlling Video Rates Figure 8: Controlling Video Rates. Three MPEG viewers are given an initial A: B : C = 3 : 2 : 1 allocation, which is changed to 3 : 1 : 2 at the time indicated by the arrow. The total number of frames displayed is plotted for each viewer. The actual frame rate ratios were 1.92 : 1.50 : 1 and 1.92 : 1 : 1.53, respectively, due to distortions caused by the X server.

Insulation Figure 9: Currencies Insulate Loads. Currencies A and B are identically funded. Tasks A1 andA2 are respectively allocated tickets worth 100:A and 200:A.TasksB1 andB2 are respectively allocated tickets worth 100:B and 200:B. Halfway through the experiment, task B3 is started with an allocation of 300:B.The resulting inflation is locally contained within currency B,andaf-fects neither the progress of tasks in currency A, nor the aggregate A: B progress ratio.

Other Kinds of Resources Claim: can be used for any resource where queuing is used Control relative waiting times for mutex locks. Mutex currency funded out of currencies of waiting threads Holder gets inheritance ticket in addition to its own funding, passed on to next holder (resulting from lottery) on release. Space sharing - inverse lottery, loser is victim (e.g. in page replacement decision, processor node preemption in MP partitioning)

Lock Funding Waiting thread 1 Waiting thread 1 lock 1 holding thread 1 bt holding thread 1

Lock Funding 1 Waiting thread 1 lock 1 1 t bt New holding thread Old holding thread 1

Mutex Waiting Times Figure 11: Mutex Waiting Times. Eight threads compete to acquire a lottery-scheduled mutex. The threads are divided into two groups (A, B) of four threads each, with the ticket allocation A: B = 2 : 1. For each histogram, the solid line indicates the mean (); the dashed lines indicate one standard deviation about the mean ( ÿ). The ratio of average waiting times is A: B = 1 : 2.11; the mutex acquisition ratio is 1.80 : 1.