MP2: Rate-Monotonic CPU Scheduling University of Illinois at Urbana-Champaign Department of Computer Science CS423 Raoul Rivas Revised and Presented by Keun Soo Yim

Extending Linux kernel to support RM scheduler Goal Extending Linux kernel to support RM scheduler Understand real-time scheduling concepts Design a real-time scheduler (i.e., RM) and its admission controller in OS Implement the RM scheduler and admission controller as a Linux kernel module Learn how to use the kernel-lv. API of the Linux scheduler, timer, and proc file system Test the kernel-level real-time scheduler by implementing a user-level periodic task CS423 MP2

Overview Implements a real-time scheduler in a kernel module Uses the Proc file system as an interface instead of system calls Periodic task model Register a task (or process) Execute real-time Loop  Rate-Monotonic sched. De-registration Simplicity by separation of policy and mechanism Reuses the Linux kernel API for low-level operations Linux Scheduler Scheduler Operations Proc Filesystem Periodic App. Proc FS Write RMS Kernel Module Proc FS Read List of Processes User Space Kernel Space CS423 MP2

Thread (High RT Priority) Design Processing Time Deadline of Job 1 1. Register Process (or Task) Job completion 4. Yield … 2. Check 5. Deregister 3. Job Arrival Period User-Level Periodic Process 1. Register Process Proc FS Write Op. RM Admission Controller 2. List of RM Processes Proc FS Read Op. Linked List for RM Tasks 4. Yield Yield Func. Handler Dispatching Thread (High RT Priority) wakeup() 5. Deregister Linux Scheduler Timer User Space Kernel Space CS423 MP2

Thread (High RT Priority) Design Processing Time Preemption Point? Deadline of Job 1 1. Register Process (or Task) Job completion 4. Yield … 2. Check 5. Deregister 3. Job Arrival Period Where are the preemption points? How the scheduling policy and mechanism are separated? No Process is RUNNING but there is a process READY to be scheduled No Process is READY but there is a RUNNING process to be preempted There is a process RUNNING to be preempted and there is a process READY to be scheduled User-Level Periodic Process 1. Register Process Proc FS Write Op. RM Admission Controller 2. List of RM Processes Proc FS Read Op. Linked List for RM Tasks 4. Yield Yield Func. Handler Dispatching Thread (High RT Priority) wakeup() 5. Deregister Linux Scheduler Timer User Space Kernel Space CS423 MP2

Scheduling Points schedule() is the entry point of the scheduler and invoked by: scheduler_tick() if a reschedule is marked. (task_struct→needs_resched) yield() is called by an application in user space schedule() in kernel or process context in kernel space Can schedule() be called inside interrupt handler? Timer 1/HZ tick_periodic add_timer(1 jiffy) jiffies++ scheduler_tick() schedule() Process Context Kernel Context yield() CS423 MP2

Two Halves / Deferring Work Kernel timers are used to create timed events They use jiffies to measure the system time Timers are interrupts We can't sleep or hog CPU in them! Solution: Divide the work into two parts Uses the timer handler to signal a thread. (TOP HALF) Let the kernel thread do the real job. (BOTTOM HALF) TOP HALF Timer Interrupt context Timer Handler: wake_up(thread); Kernel context Thread: While(1) { Do work(); Schedule(); } BOTTOM HALF CS423 MP2

Linux Kernel Scheduler Process Priority-based time-sharing scheduler Defines a function f(P) for a process P to calculate an effective priority R The highest effective priority process is always scheduled. Fairness is realized since the effective priorities change over time depending on dynamic process state information. Such process-specific dynamic information is stored in the Process Control Block (PCB) (Sched. Parameters, Process State Info.) f Goodness value Scheduler CS423 MP2

Process Control Block PCB is defined by the task_struct structure PCBs are managed by a circular doubly linked list task_struct *current points to PCB of the currently running process. PCB contains the process state: Running or ready: TASK_RUNNING Waiting: TASK_INTERRUPTIBLE or TASK_UNINTERRUPTIBLE Other: TASK_TRACED, TASK_STOPPED task_struct task_struct Current task_struct task_struct CS423 MP2

Process Control Block mm: the memory descriptor Additionally PCB contains: priority: the static priority rt_priority: the real-time priority counter: the number of clock ticks left in the scheduling budget of the process policy: the scheduling policy SCHED_NORMAL: Normal SCHED_FIFO or SCHED_RR mm: the memory descriptor task_struct priority rt_priority counter policy state need_ resched CS423 MP2

Priority Level Process P1 P2 SCHED FIFO NORMAL f rt_priority (99, 0) Goodness value Scheduler SCHED FIFO P2 NORMAL Prio 1 Prio 140 Context Switch Pick Process 1 rt_priority (99, 0) Nice (-20, +19) CS423 MP2

Goodness Pseudo code: p: process to evaluate, prev: previous process if (p->policy != SCHED_OTHER) return 1000 + p->rt_priority; if (p->counter == 0) return 0; if (p->mm == prev->mm) return p->counter + p->priority + 1; return p->counter + p->priority; CS423 MP2

O(1) Scheduler v2.6 – before v2.6.23 Problem Scalability (e.g., Java) A task used all of its time slice is moved to the expired RQ. During the move, the time slice and priority are updated. If no tasks exist on the active RQ for a given priority, the pointers are swapped. v2.6 – before v2.6.23 http://www.ibm.com/developerworks/linux/library/l-scheduler/ CS423 MP2

Bitmap 31 … … … … … … … … … … … … … … … … 63 … … … … … … … … … … … … … … .. … 32 95 … … … … … … … … … … … … … … .. … 64 127 … … … … … … … … … … … … … … .. … 96 159 … … … … … … … … … … … … … … .. … 128 int sched_find_first_bit(unsigned long *b){ if (b[0]) return __ffs(b[0]); if (b[1]) return __ffs(b[1]) + 32; if (b[2]) return __ffs(b[2]) + 64; return __ffs(b[3]) + 96; } __ffs(): bit scan forward instruction returns the index of the least significant set bit in x86. CS423 MP2

CFS – Completely Fair Sched. CFS from kernel v2.6.23 for normal process (not real-time process) to improve fairness. Maintains virtual runtime (amount of time given to a task) and supports sleeper fairness to ensure that tasks not currently runnable (e.g., due to I/O) receive a comparable share later when it is needed A time-ordered red-black tree is used as a RQ data structure, which is self-balancing (longest path <= 2 x shortest path) and has O(log n) operation time. CS423 MP2

Scheduler API schedule(): triggers rescheduling Runs after changing the state of any process wake_up_process(): wakes a process This can be called in the interrupt context. sched_setscheduler(): sets sched. parameters e.g., priority & scheduling policy set_current_state(): changes the state used to put the running context to sleep. set_task_state(): changes the state of another CS423 MP2

Implementation Proc file system (FS) Linked list Critical section by mutex Kernel thread Timer Caution: Source code provided in this slide give examples of how such mechanisms are implemented and thus are simplified and incomplete, while the MP1 solution and reference materials provide the complete implementation. CS423 MP2

PROC FS [MP1 CODE] static struct proc_dir_entry *mp1_proc_dir; int __init my_module_init(void) { mp1_proc_dir=proc_mkdir("mp1",NULL); register_task_file=create_proc_entry("status", 0666, mp1_proc_dir); register_task_file->read_proc = proc_registration_read; register_task_file->write_proc = proc_registration_write; … } void __exit my_module_exit(void){ remove_proc_entry("status", mp1_proc_dir); remove_proc_entry("mp1", NULL); CS423 MP2

PROC FS [MP1 CODE] int proc_registration_write(struct file *file, const char *buffer, unsigned long count, void *data){ char *proc_buffer; unsigned int pid; proc_buffer=kmalloc(count, GFP_KERNEL); copy_from_user(proc_buffer, buffer, count); … kfree(proc_buffer); return count; } CS423 MP2

PROC FS [MP1 CODE] int proc_registration_read(char *page, char **start, off_t off, int count, int* eof, void* data){ off_t i; struct list_head *pos; i=0; if (off == 0) { memcpy(page, “hello world”, 11); i = 11; *eof=1; } return i; CS423 MP2

LINKED LIST [MP1 CODE] LIST_HEAD(mp1_task_list); static DEFINE_MUTEX(mp1_mutex); mutex_lock(&mp1_mutex); list_add_tail(&t->task_node, &mp1_task_list); mutex_unlock(&mp1_mutex); CS423 MP2

LINKED LIST [MP1 CODE] struct list_head *pos; mutex_lock(&mp1_mutex); list_for_each(pos, &mp1_task_list) { p = list_entry(pos, struct mp1_task_stats, task_node); /* use of p->pid and p->cpu_use */ } mutex_unlock(&mp1_mutex); CS423 MP2

KERNEL THREAD [MP1 CODE] struct task_struct* update_kthread; int __init my_module_init(void){ update_kthread=kthread_create(scheduled_update, NULL,"kmp1"); } void __exit my_module_exit(void){ kthread_stop(update_kthread); CS423 MP2

KERNEL THREAD [MP1 CODE] int scheduled_update(void *data){ while(1) { if (stop_thread==1) break; … set_current_state(TASK_INTERRUPTIBLE); schedule(); set_current_state(TASK_RUNNING); } return 0; wake_up_process(update_kthread); CS423 MP2

TIMER [MP1 CODE] inline void timer_init(struct timer_list *timer, void (*function)(unsigned long)){ init_timer(timer); timer->function=function; timer->data=(unsigned long) timer; } int __init my_module_init(void){ timer_init(&up_timer, up_handler); void __exit my_module_exit(void){ del_timer_sync(&up_timer); CS423 MP2

TIMER [MP1 CODE] void up_handler(unsigned long ptr){ wake_up_process(update_kthread); } int scheduled_update(void *data){ while(1) { … set_timer(&up_timer,UPDATE_TIME); CS423 MP2

Conclusion Linux uses Priority based Scheduling Priorities are adjusted based on the state of the process and its scheduling policy The PCB (task_struct) contains the information about the state of a process Rescheduling in Linux follows the Two-Halves approach. Reschedule can occur in process context or kernel context only. We can use the Scheduler API to build our own policy on top of the Linux scheduler. Our RMS scheduler will have a scheduler thread and will be invoked after a Job Release and after a Job Completion CS423 MP2