1. Linux Realtime Preemption and Its Impact on ULDD* - Progress Report -
[*:User Level Device Driver]
Katsuya Matsubara　　　Hitomi Takahashi
IGEL Co. Ltd
funded by Renesas Solutions Corp.
2006/10/27
CELF Tech. Jamboree #11
[translated by ikoma]

2. User Level Device Driver
Attempt to Implement a Device Driver as a User Task
Kernel programming is not easy for beginners
It is said that device drivers make kernel unstable
As scheduler has been improved on 2.6, platform is ready?
O(1) scheduler
NPTL
Kernel preemption
Some drivers such as USB have already been implemented in user level
2006/10/27
CELF Tech. Jamboree #11

3. Current Work
Evaluating and Measuring the Impact (Effect) of Ingo Molner's Linux Realtime Preemption Patch (PREEMPT_RT) on ULDD
Understanding behavior of PREEMPT_RT
Adapting PREEMPT_RT to SuperH architecture
Experimenting and measuring the impact on ULDD
2006/10/27
CELF Tech. Jamboree #11

4. Realtime Support of Linux
Complete Realtime
Guarantee deadlines of all tasks -> Difficult with grafting on existing (non-realtime) OSes
Nested OS, Dual OS/Dual Core, ART-Linux
Partial Realtime
Guarantee best efforts for deadlines of some tasks
Most of realtime tasks utilize limited resources only
=> Minimize latency = preemption
=> Further less latency = realtime preemption
CONFIG_PREEMPT, CONFIG_PREEMPT_RT
2006/10/27
CELF Tech. Jamboree #11

5. Latency in Interrupt Processing
Latency of Waking Up Interrupt Handler
Between generation of interrupt and handler wakeup
=> Depend on interrupt disabled regions
Latency of Task Wakeup
Between task wakup indication by interrupt handler and actual wakeup
=> Depend on preemption disabled regions (incl. interrupt disabled regions)
=> Depend on scheduling policy
2006/10/27
CELF Tech. Jamboree #11

6. Preemption in Linux Timing of Preemption
at return from interrupt handling
at return from system call
when task sleeps voluntarily
⇒リターン時に別タスクのスタックポインタをセットし、pop
=> set stack pointer of aother task at return, and pop
On 2.4/2.6 without CONFIG_PREEMPT, Preemption (Task Switching) does NOT Occur in Kernel Mode (Interrupt Handling Does)
at return from interrupt in kernel mode, control returns to the original point (because the code does not support preemption)
2006/10/27
CELF Tech. Jamboree #11

7. 2.4 / 2.6 without CONFIG_PREEMPT
Task A
Task B
user
kernel
Interrupt Processing
Data wait (sleep)
H/W
Interrupt
Interrupt disabled region
Wakes up
Task A
2006/10/27
CELF Tech. Jamboree #11

8. CONFIG_PREEMPT Task A Task B user kernel Interrupt Processeing
Data wait
(sleep)
H/W
interrupt
Interrupt disabled region
Wakes up Task B
2006/10/27
CELF Tech. Jamboree #11

9. PREEMPT_RT Task A Task B user kernel Interrupt Processing Data wait
(sleep)
H/W
Interrupt
Interrupt disabled region
Wakes up Task B
2006/10/27
CELF Tech. Jamboree #11

10. Features Added/Modified by PREEMPT_RT
Preemption in Critical Region
Preemption in Interrupt Processing
Preemption in Interrupt Disabled Code Region
Priority Inheritance of Spinlock and Semaphore
Processing of Preemption Disabled Code which Requires Spinlock
Miscellaneous Optimization
2006/10/27
CELF Tech. Jamboree #11

11. i. Preemption in Critical Region
spinlock_t, rwlock_t, rcu_read_lock() and rcu_read_unlock() have been Modified to Allow Preemption in Semaphore Protected Regions
_irq(), such as spin_lock_irqsave(), does not Actually Inhibit Hardware Interrupt
Spinlocks so far are kept Usable with raw_spinlock_t
2006/10/27
CELF Tech. Jamboree #11

12. ii. Preemption in Interrupt Processing
Interrupt Processing is Executed in Process Context
Redirect processing to irqd with redirect_hardirq()
Preemptable
Only Interrupt Processing Declared as SA_NODELAY is Executed in Interrupt Context
e.g. CPU timer interrupt, fpu interrupt, etc.
Wakeup latency may be influenced
Careful coding required = Better not to use
2006/10/27
CELF Tech. Jamboree #11

13. iii. Preemption in Interrupt Disabled Code Region
Preemption while “Interrupt Disabled” !?
With SMP, concurrent execution of interrupt processing is possible by more than one CPU
spin_lock_irqsave() does not Inhibit Preemption
No problem?
=> Even if other interrupt processing started, it would be blocked to get the same lock = Critical region can be set
local_irq_save() Inhibits Preemption
No corresponding lock available
Using locks rather reduces latency. But SMP performance may be deteriorated
2006/10/27
CELF Tech. Jamboree #11

14. iv. Priority Inheritance of Spinlock and Semaphore in Kernel
Boost Priority of Lock Holding Task accordintg to High Priority Task which Requires Lock
Priority is Inherited Repeatedly (Transitive)
It is Possible to Set “Preemption Point” and to Release Lock there (e.g. JBD Journal Layer)
Priority Inheritance from Writer to Reader is Difficult
In PREEMPT_RT, read lock is restricted to hold only one task at a time
Semaphore (not Lock) Utilizes Event Mechanism and Priority Inheritance is Difficult
Can not specify posting task
compat_semaphore() and compat_rw_semaphore() are provided (not studied further; sorry...)
2006/10/27
CELF Tech. Jamboree #11

15. Porting of PREEMPT_RT Currently Supported Architectures
Intel x86
PowerPC
ARM
MIPS
etc.
Supporting New Architectures
Modification in arch/ARCHITECTURE, include/asm- ARCHITECTURE
add “raw-” and “compat_” to locks and semaphores where inturrput or preemption should be inhibited
Specify SA_NODELAY at registration of interrupt handlers which should be executed in interrupt context (Timer handler etc.)
And add architecture specific processing (if defined(ARCHITECTURE))
2006/10/27
CELF Tech. Jamboree #11

16. ToDo to Support New Architecture
Just to Support Easily, Add “_raw” to all Locks and local_irqsave, and “_compat” to Semaphores (It should work)
Opportunity of preemption does not inclrease = no meaning
At each Critical Regions and Interrupt Processing, the Necesity to Inhibit Preemption should be Analyzed
2006/10/27
CELF Tech. Jamboree #11

17. Impact on ULDD Up to Here Today, Sorry. Prediction
Less interrupt disabled region
Less preemption disabled region
=> More chance of preemption wll make less scheduling latency
Lock with priority inheritance
Less jitter of latency expected, due to no more priority inversion
Mesuring acturelly, ...
Up to Here Today, Sorry.
2006/10/27
CELF Tech. Jamboree #11

18. Status Linux 2.6.16 + linuxsh patch + Patch for RTS7751R2D
+ patch rt29
Added raw_, compat_ , etc: Works (Thanks to Lineo)
Now working on tuning (removing unnecessary raw_ and compat_)
ULDD
Have implemented the driver notifying H/W interrupt to user task
Have implemented SM501 UART driver on RTS7751R2D
Choosing disturbance processes (can not see the effect of Linux preemption with lat_proc in Lmbench)
2006/10/27
CELF Tech. Jamboree #11

19. TODO Porting of PREEMPT_RT Support for SH4 Transition to 2.6.18 ?
Comparison with Intel x86
Implementation of ULDD targeted H/W devices on Intel x86
Design of ULDD v2
Extend so that basic processing such as interrupt determination and interrupt inhibition etc. is executed on kernel side
Reducing difference of API with kernel drivers
Referring to mechanisms used in drivers such as wlan, libusb, etc. which have already realized as ULDD
2006/10/27
CELF Tech. Jamboree #11