Linux Realtime Preemption and Its Impact on ULDD* - Progress Report - [*:User Level Device Driver] Katsuya Matsubara　　　Hitomi Takahashi IGEL Co. Ltd funded by Renesas Solutions Corp. 2006/10/27 CELF Tech. Jamboree #11 [translated by ikoma]

User Level Device Driver Attempt to Implement a Device Driver as a User Task Kernel programming is not easy for beginners It is said that device drivers make kernel unstable As scheduler has been improved on 2.6, platform is ready? O(1) scheduler NPTL Kernel preemption Some drivers such as USB have already been implemented in user level 2006/10/27 CELF Tech. Jamboree #11

Current Work Evaluating and Measuring the Impact (Effect) of Ingo Molner's Linux Realtime Preemption Patch (PREEMPT_RT) on ULDD Understanding behavior of PREEMPT_RT Adapting PREEMPT_RT to SuperH architecture Experimenting and measuring the impact on ULDD 2006/10/27 CELF Tech. Jamboree #11

Realtime Support of Linux Complete Realtime Guarantee deadlines of all tasks -> Difficult with grafting on existing (non-realtime) OSes Nested OS, Dual OS/Dual Core, ART-Linux Partial Realtime Guarantee best efforts for deadlines of some tasks Most of realtime tasks utilize limited resources only => Minimize latency = preemption => Further less latency = realtime preemption CONFIG_PREEMPT, CONFIG_PREEMPT_RT 2006/10/27 CELF Tech. Jamboree #11

Latency in Interrupt Processing Latency of Waking Up Interrupt Handler Between generation of interrupt and handler wakeup => Depend on interrupt disabled regions Latency of Task Wakeup Between task wakup indication by interrupt handler and actual wakeup => Depend on preemption disabled regions (incl. interrupt disabled regions) => Depend on scheduling policy 2006/10/27 CELF Tech. Jamboree #11

Preemption in Linux Timing of Preemption at return from interrupt handling at return from system call when task sleeps voluntarily ⇒リターン時に別タスクのスタックポインタをセットし、pop => set stack pointer of aother task at return, and pop On 2.4/2.6 without CONFIG_PREEMPT, Preemption (Task Switching) does NOT Occur in Kernel Mode (Interrupt Handling Does) at return from interrupt in kernel mode, control returns to the original point (because the code does not support preemption) 2006/10/27 CELF Tech. Jamboree #11

2.4 / 2.6 without CONFIG_PREEMPT Task A Task B user kernel Interrupt Processing Data wait (sleep) H/W Interrupt Interrupt disabled region Wakes up Task A 2006/10/27 CELF Tech. Jamboree #11

CONFIG_PREEMPT Task A Task B user kernel Interrupt Processeing Data wait (sleep) H/W interrupt Interrupt disabled region Wakes up Task B 2006/10/27 CELF Tech. Jamboree #11

PREEMPT_RT Task A Task B user kernel Interrupt Processing Data wait (sleep) H/W Interrupt Interrupt disabled region Wakes up Task B 2006/10/27 CELF Tech. Jamboree #11

Features Added/Modified by PREEMPT_RT Preemption in Critical Region Preemption in Interrupt Processing Preemption in Interrupt Disabled Code Region Priority Inheritance of Spinlock and Semaphore Processing of Preemption Disabled Code which Requires Spinlock Miscellaneous Optimization 2006/10/27 CELF Tech. Jamboree #11

i. Preemption in Critical Region spinlock_t, rwlock_t, rcu_read_lock() and rcu_read_unlock() have been Modified to Allow Preemption in Semaphore Protected Regions _irq(), such as spin_lock_irqsave(), does not Actually Inhibit Hardware Interrupt Spinlocks so far are kept Usable with raw_spinlock_t 2006/10/27 CELF Tech. Jamboree #11

ii. Preemption in Interrupt Processing Interrupt Processing is Executed in Process Context Redirect processing to irqd with redirect_hardirq() Preemptable Only Interrupt Processing Declared as SA_NODELAY is Executed in Interrupt Context e.g. CPU timer interrupt, fpu interrupt, etc. Wakeup latency may be influenced Careful coding required = Better not to use 2006/10/27 CELF Tech. Jamboree #11

iii. Preemption in Interrupt Disabled Code Region Preemption while “Interrupt Disabled” !? With SMP, concurrent execution of interrupt processing is possible by more than one CPU spin_lock_irqsave() does not Inhibit Preemption No problem? => Even if other interrupt processing started, it would be blocked to get the same lock = Critical region can be set local_irq_save() Inhibits Preemption No corresponding lock available Using locks rather reduces latency. But SMP performance may be deteriorated 2006/10/27 CELF Tech. Jamboree #11

iv. Priority Inheritance of Spinlock and Semaphore in Kernel Boost Priority of Lock Holding Task accordintg to High Priority Task which Requires Lock Priority is Inherited Repeatedly (Transitive) It is Possible to Set “Preemption Point” and to Release Lock there (e.g. JBD Journal Layer) Priority Inheritance from Writer to Reader is Difficult In PREEMPT_RT, read lock is restricted to hold only one task at a time Semaphore (not Lock) Utilizes Event Mechanism and Priority Inheritance is Difficult Can not specify posting task compat_semaphore() and compat_rw_semaphore() are provided (not studied further; sorry...) 2006/10/27 CELF Tech. Jamboree #11

Porting of PREEMPT_RT Currently Supported Architectures Intel x86 PowerPC ARM MIPS etc. Supporting New Architectures Modification in arch/ARCHITECTURE, include/asm- ARCHITECTURE add “raw-” and “compat_” to locks and semaphores where inturrput or preemption should be inhibited Specify SA_NODELAY at registration of interrupt handlers which should be executed in interrupt context (Timer handler etc.) And add architecture specific processing (if defined(ARCHITECTURE)) 2006/10/27 CELF Tech. Jamboree #11

ToDo to Support New Architecture Just to Support Easily, Add “_raw” to all Locks and local_irqsave, and “_compat” to Semaphores (It should work) Opportunity of preemption does not inclrease = no meaning At each Critical Regions and Interrupt Processing, the Necesity to Inhibit Preemption should be Analyzed 2006/10/27 CELF Tech. Jamboree #11

Impact on ULDD Up to Here Today, Sorry. Prediction Less interrupt disabled region Less preemption disabled region => More chance of preemption wll make less scheduling latency Lock with priority inheritance Less jitter of latency expected, due to no more priority inversion Mesuring acturelly, ... Up to Here Today, Sorry. 2006/10/27 CELF Tech. Jamboree #11

Status Linux 2.6.16 + linuxsh patch + Patch for RTS7751R2D + patch-2.6.16-rt29 Added raw_, compat_ , etc: Works (Thanks to Lineo) Now working on tuning (removing unnecessary raw_ and compat_) ULDD Have implemented the driver notifying H/W interrupt to user task Have implemented SM501 UART driver on RTS7751R2D Choosing disturbance processes (can not see the effect of Linux preemption with lat_proc in Lmbench) 2006/10/27 CELF Tech. Jamboree #11

TODO Porting of PREEMPT_RT Support for SH4 Transition to 2.6.18 ? Comparison with Intel x86 Implementation of ULDD targeted H/W devices on Intel x86 Design of ULDD v2 Extend so that basic processing such as interrupt determination and interrupt inhibition etc. is executed on kernel side Reducing difference of API with kernel drivers Referring to mechanisms used in drivers such as wlan, libusb, etc. which have already realized as ULDD 2006/10/27 CELF Tech. Jamboree #11