1. MP2: Rate-Monotonic CPU Scheduling
University of Illinois at Urbana-Champaign
Department of Computer Science
CS423
Raoul Rivas
Revised and Presented by
Keun Soo Yim

2. Extending Linux kernel to support RM scheduler
Goal
Extending Linux kernel to support RM scheduler
Understand real-time scheduling concepts
Design a real-time scheduler (i.e., RM) and its admission controller in OS
Implement the RM scheduler and admission controller as a Linux kernel module
Learn how to use the kernel-lv. API of the Linux scheduler, timer, and proc file system
Test the kernel-level real-time scheduler by implementing a user-level periodic task
CS423 MP2

3. Overview Implements a real-time scheduler in a kernel module
Uses the Proc file system as an interface instead of system calls
Periodic task model
Register a task (or process)
Execute real-time Loop  Rate-Monotonic sched.
De-registration
Simplicity by separation of policy and mechanism
Reuses the Linux kernel API for low-level operations
Linux
Scheduler
Scheduler
Operations
Proc
Filesystem
Periodic
App.
Proc FS
Write
RMS
Kernel
Module
Proc FS
Read
List of
Processes
User Space
Kernel Space
CS423 MP2

4. Thread (High RT Priority)
Design
Processing
Time
Deadline of Job 1
1. Register
Process
(or Task)
Job completion 4. Yield …
2. Check
5. Deregister
3. Job Arrival
Period
User-Level
Periodic
Process
1. Register Process
Proc FS Write Op.
RM Admission Controller
2. List of RM Processes
Proc FS
Read Op.
Linked List
for RM Tasks
4. Yield
Yield Func. Handler
Dispatching
Thread (High RT Priority)
wakeup()
5. Deregister
Linux
Scheduler
Timer
User Space
Kernel Space
CS423 MP2

5. Thread (High RT Priority)
Design
Processing
Time
Preemption Point?
Deadline of Job 1
1. Register
Process
(or Task)
Job completion 4. Yield …
2. Check
5. Deregister
3. Job Arrival
Period
Where are the preemption points?
How the scheduling policy and mechanism are separated?
No Process is RUNNING but there is a process READY to be scheduled
No Process is READY but there is a RUNNING process to be preempted
There is a process RUNNING to be preempted and there is a process READY to be scheduled
User-Level
Periodic
Process
1. Register Process
Proc FS Write Op.
RM Admission Controller
2. List of RM Processes
Proc FS
Read Op.
Linked List
for RM Tasks
4. Yield
Yield Func. Handler
Dispatching
Thread (High RT Priority)
wakeup()
5. Deregister
Linux
Scheduler
Timer
User Space
Kernel Space
CS423 MP2

6. Scheduling Points
schedule() is the entry point of the scheduler and invoked by:
scheduler_tick() if a reschedule is marked. (task_struct→needs_resched)
yield() is called by an application in user space
schedule() in kernel or process context in kernel space
Can schedule() be called inside interrupt handler?
Timer
1/HZ
tick_periodic
add_timer(1 jiffy)
jiffies++
scheduler_tick()
schedule()
Process Context
Kernel Context
yield()
CS423 MP2

7. Two Halves / Deferring Work
Kernel timers are used to create timed events
They use jiffies to measure the system time
Timers are interrupts
We can't sleep or hog CPU in them!
Solution:
Divide the work into two parts
Uses the timer handler to signal a thread. (TOP HALF)
Let the kernel thread do the real job. (BOTTOM HALF)
TOP HALF
Timer
Interrupt
context
Timer Handler:
wake_up(thread);
Kernel
context
Thread:
While(1) {
Do work();
Schedule(); }
BOTTOM HALF
CS423 MP2

8. Linux Kernel Scheduler
Process
Priority-based time-sharing scheduler
Defines a function f(P) for a process P to calculate an effective priority R
The highest effective priority process is always scheduled.
Fairness is realized since the effective priorities change over time depending on dynamic process state information.
Such process-specific dynamic information is stored in the Process Control Block (PCB)
(Sched. Parameters, Process State Info.) f
Goodness
value
Scheduler
CS423 MP2

9. Process Control Block PCB is defined by the task_struct structure
PCBs are managed by a circular doubly linked list
task_struct *current points to PCB of the currently running process.
PCB contains the process state:
Running or ready: TASK_RUNNING
Waiting: TASK_INTERRUPTIBLE or TASK_UNINTERRUPTIBLE
Other: TASK_TRACED, TASK_STOPPED
task_struct
task_struct
Current
task_struct
task_struct
CS423 MP2

10. Process Control Block mm: the memory descriptor
Additionally PCB contains:
priority: the static priority
rt_priority: the real-time priority
counter: the number of clock ticks left in the scheduling budget of the process
policy: the scheduling policy
SCHED_NORMAL: Normal
SCHED_FIFO or SCHED_RR
mm: the memory descriptor
task_struct
priority
rt_priority
counter
policy
state
need_ resched
CS423 MP2

11. Priority Level Process P1 P2 SCHED FIFO NORMAL f rt_priority (99, 0)
Goodness
value
Scheduler
SCHED
FIFO P2
NORMAL
Prio 1
Prio 140
Context Switch
Pick Process 1
rt_priority
(99, 0)
Nice
(-20, +19)
CS423 MP2

12. Goodness
Pseudo code: p: process to evaluate, prev: previous process if (p->policy != SCHED_OTHER) return p->rt_priority; if (p->counter == 0) return 0; if (p->mm == prev->mm) return p->counter + p->priority + 1; return p->counter + p->priority;
CS423 MP2

13. O(1) Scheduler v2.6 – before v2.6.23 Problem
Scalability (e.g., Java)
A task used all of its time slice is moved to the expired RQ.
During the move, the time slice and priority are updated.
If no tasks exist on the active RQ for a given priority, the pointers are swapped.
v2.6 – before v2.6.23
CS423 MP2

14. Bitmap 31 … … … … … … … … … … … … … … … … 63 … … … … … … … … … … … … … … .. … 32 95 … … … … … … … … … … … … … … .. … 64
127 … … … … … … … … … … … … … … .. … 96
159 … … … … … … … … … … … … … … .. …
128
int sched_find_first_bit(unsigned long *b){ if (b[0]) return __ffs(b[0]); if (b[1]) return __ffs(b[1]) + 32; if (b[2]) return __ffs(b[2]) + 64; return __ffs(b[3]) + 96; } __ffs(): bit scan forward instruction returns the index of the least significant set bit in x86.
CS423 MP2

15. CFS – Completely Fair Sched.
CFS from kernel v for normal process (not real-time process) to improve fairness.
Maintains virtual runtime (amount of time given to a task) and supports sleeper fairness to ensure that tasks not currently runnable (e.g., due to I/O) receive a comparable share later when it is needed
A time-ordered red-black tree is used as a RQ data structure, which is self-balancing (longest path <= 2 x shortest path) and has O(log n) operation time.
CS423 MP2

16. Scheduler API schedule(): triggers rescheduling
Runs after changing the state of any process
wake_up_process(): wakes a process
This can be called in the interrupt context.
sched_setscheduler(): sets sched. parameters
e.g., priority & scheduling policy
set_current_state(): changes the state
used to put the running context to sleep.
set_task_state(): changes the state of another
CS423 MP2

17. Implementation Proc file system (FS) Linked list
Critical section by mutex
Kernel thread
Timer
Caution: Source code provided in this slide give examples of how such mechanisms are implemented and thus are simplified and incomplete, while the MP1 solution and reference materials provide the complete implementation.
CS423 MP2

18. PROC FS [MP1 CODE]
static struct proc_dir_entry *mp1_proc_dir; int __init my_module_init(void) { mp1_proc_dir=proc_mkdir("mp1",NULL); register_task_file=create_proc_entry("status", 0666, mp1_proc_dir); register_task_file->read_proc = proc_registration_read; register_task_file->write_proc = proc_registration_write; … } void __exit my_module_exit(void){ remove_proc_entry("status", mp1_proc_dir); remove_proc_entry("mp1", NULL);
CS423 MP2

19. PROC FS [MP1 CODE]
int proc_registration_write(struct file *file, const char *buffer, unsigned long count, void *data){ char *proc_buffer; unsigned int pid; proc_buffer=kmalloc(count, GFP_KERNEL); copy_from_user(proc_buffer, buffer, count); … kfree(proc_buffer); return count; }
CS423 MP2

20. PROC FS [MP1 CODE]
int proc_registration_read(char *page, char **start, off_t off, int count, int* eof, void* data){ off_t i; struct list_head *pos; i=0; if (off == 0) { memcpy(page, “hello world”, 11); i = 11; *eof=1; } return i;
CS423 MP2

21. LINKED LIST [MP1 CODE]
LIST_HEAD(mp1_task_list); static DEFINE_MUTEX(mp1_mutex); mutex_lock(&mp1_mutex); list_add_tail(&t->task_node, &mp1_task_list); mutex_unlock(&mp1_mutex);
CS423 MP2

22. LINKED LIST [MP1 CODE]
struct list_head *pos; mutex_lock(&mp1_mutex); list_for_each(pos, &mp1_task_list) { p = list_entry(pos, struct mp1_task_stats, task_node); /* use of p->pid and p->cpu_use */ } mutex_unlock(&mp1_mutex);
CS423 MP2

23. KERNEL THREAD [MP1 CODE]
struct task_struct* update_kthread; int __init my_module_init(void){ update_kthread=kthread_create(scheduled_update, NULL,"kmp1"); } void __exit my_module_exit(void){ kthread_stop(update_kthread);
CS423 MP2

24. KERNEL THREAD [MP1 CODE]
int scheduled_update(void *data){
while(1) {
if (stop_thread==1) break; …
set_current_state(TASK_INTERRUPTIBLE);
schedule();
set_current_state(TASK_RUNNING); }
return 0;
wake_up_process(update_kthread);
CS423 MP2

25. TIMER [MP1 CODE]
inline void timer_init(struct timer_list *timer, void (*function)(unsigned long)){ init_timer(timer); timer->function=function; timer->data=(unsigned long) timer; } int __init my_module_init(void){ timer_init(&up_timer, up_handler); void __exit my_module_exit(void){ del_timer_sync(&up_timer);
CS423 MP2

26. TIMER [MP1 CODE]
void up_handler(unsigned long ptr){ wake_up_process(update_kthread); } int scheduled_update(void *data){ while(1) { … set_timer(&up_timer,UPDATE_TIME);
CS423 MP2

27. Conclusion Linux uses Priority based Scheduling
Priorities are adjusted based on the state of the process and its scheduling policy
The PCB (task_struct) contains the information about the state of a process
Rescheduling in Linux follows the Two-Halves approach. Reschedule can occur in process context or kernel context only.
We can use the Scheduler API to build our own policy on top of the Linux scheduler.
Our RMS scheduler will have a scheduler thread and will be invoked after a Job Release and after a Job Completion
CS423 MP2