1. 1 Dynamic CPU Management for Real-Time, Middleware-Based Systems Eric EideTim Stack Eric EideTim Stack John RegehrJay Lepreau University of Utah, School of Computing May 27, 2004

2. 2 Monoliths → Modules Old: Monolithic RT systems Old: Monolithic RT systems Made by one organization Made by one organization One-off products One-off products Closed and static Closed and static New: Modular RT systems New: Modular RT systems Made by many organizations Made by many organizations Product lines and families Product lines and families Open and dynamic Open and dynamic Hard to extend and integrate Hard to compose and control Tasks + RTOS RTOS Task

3. 3 Modularity: Middleware Common backplane Common backplane Relieve developers of tedious low-level detail Relieve developers of tedious low-level detail Fewer bugs, shorter time to market Fewer bugs, shorter time to market Reusable across multiple products, product families Reusable across multiple products, product families RT Middleware RTOS TaskRTTaskRTTaskRT

4. 4 Modularity: Separate RT Separate application logic from RT logic Separate application logic from RT logic Reuse of task parts Reuse of task parts Modularize RT parts for understandability Modularize RT parts for understandability Separation enables remodularization Separation enables remodularization But... how? But... how? RTOS Task RT Middleware Task RT

5. 5 Modularity: Specifying RT Need composability Need composability Reservations are composable, but not enough Reservations are composable, but not enough Need adaptability Need adaptability …for complex parts …for complex parts data-, mode-, or configuration-dependent demand data-, mode-, or configuration-dependent demand …for open systems …for open systems unknown agents and resources before deployment unknown agents and resources before deployment …because prediction is hard …because prediction is hard

6. 6 Idea: “CPU Broker” Service for managing processor resources on a host Service for managing processor resources on a host Mediates between RT tasks and RT OS Mediates between RT tasks and RT OS Broker is a negotiator, not a scheduler Broker is a negotiator, not a scheduler Task CPU Broker RTOS

7. 7 Idea 1: Separate Concerns Separate application logic from RT logic Separate application logic from RT logic Separate per-task and global decision makers Separate per-task and global decision makers Separate negotiation from scheduling Separate negotiation from scheduling Manage both middleware-based and other applications Manage both middleware-based and other applications Task RTOS RT Policy

8. 8 Idea 2: Dynamic/Adaptive Dynamic monitoring of applications Dynamic monitoring of applications Changing task sets Changing task sets External inputs to the broker at run time External inputs to the broker at run time Set and change broker configuration at run time Set and change broker configuration at run time Task RTOS RT Policy

9. 9 Idea 3: Open Framework Parts are objects Parts are objects Plug-in per-task “advocates” Plug-in per-task “advocates” Plug-in global policies Plug-in global policies Connect to other QoS management systems Connect to other QoS management systems Task RTOS RT Policy RT

10. 10 Contributions Architecture addressing critical software engineering challenges Architecture addressing critical software engineering challenges …moving from reservations to negotiations …moving from reservations to negotiations …supporting more modular RT abstractions …supporting more modular RT abstractions Implementation on COTS MW and RTOS Implementation on COTS MW and RTOS Evaluation Evaluation …with synthetic RT applications …with synthetic RT applications …with a distributed RT military application …with a distributed RT military application

11. 11 Related Work Our focus: address SE challenges Our focus: address SE challenges Build on previous work Build on previous work Feedback-driven scheduling Feedback-driven scheduling QoS Manager [Abeni and Buttazzo, ’99] QoS Manager [Abeni and Buttazzo, ’99] RT middleware RT middleware RT CORBA [OMG], feedback [Lu et al., ’03] RT CORBA [OMG], feedback [Lu et al., ’03] MW-based QoS architectures MW-based QoS architectures DQM [Brandt et al., ’98], QuO [Zinky et al., ’97] DQM [Brandt et al., ’98], QuO [Zinky et al., ’97]

12. 12 Design Overview Task RTOS Advocate Policy Scheduler Proxy Policy Within the broker: Within the broker: Advocates Advocates Policies Policies Scheduler proxy Scheduler proxy

13. 13 Advocates Per-application adaptation Per-application adaptation Request CPU resources for a task Request CPU resources for a task Chain to build up complex behaviors Chain to build up complex behaviors Goal: match expected task demand Goal: match expected task demand Advocate Policy Scheduler Proxy Policy

14. 14 Topmost Advocate Input CPU consumed “status” Output request for a periodic reservation “advice” (C, P) Predict future need …library of advocates …or, write your own can collocate with task Task data (e.g., mode) OS data (e.g., getrusage) Task Advocate status status and advice …

15. 15 Subsequent Advocates Modify advice or perform side effect cap request (min/max) talk to other MW watchdog timer Chaining allows reuse of advocates dynamic configuration … and reconfiguration … Advocate status, new advice … Advocate status, advice Other MW, e.g. Load Predictor

16. 16 Policies Global adaptation Global adaptation Collect advice from advocates Collect advice from advocates Request reservations Request reservations Propagate information back to advocates Propagate information back to advocates Goal: negotiate to resolve contention Goal: negotiate to resolve contention Advocate Policy Scheduler Proxy Policy

17. 17 Policy Objects Encapsulate negotiation …hard or soft RT Multiple policies via partitions Setting up policies …use library policies …or, write your own Set/reset policy data at run time, e.g. …importances, weights resv. requests … Policy statuses, advices Policy ……… … … reservation requests Partition

18. 18 Putting It All Together Tasks: status Tasks: status Advocates: predictions Advocates: predictions Policies: allocations Policies: allocations Scheduler proxy Scheduler proxy RTOS: reservations RTOS: reservations Propagate changes Propagate changes Task RTOS Advocate Policy Scheduler Proxy Policy

19. 19 Implementation COTS RT scheduling and accounting COTS RT scheduling and accounting TimeSys Linux TimeSys Linux Open architecture, dynamic control Open architecture, dynamic control CORBA CORBA Separate application and RT logic Separate application and RT logic QuO QuO TimeSys Linux TimeSys Linux

20. 20 TimeSys Linux Core services Core services Reservation-based scheduling Reservation-based scheduling High-resolution CPU usage timers High-resolution CPU usage timers Good abstractions Good abstractions Multiple threads can share one reservation Multiple threads can share one reservation Reservations can be inherited Reservations can be inherited Thread→reservation associations can be created and manipulated “externally” Thread→reservation associations can be created and manipulated “externally” …allow easy monitoring and manipulation …allow easy monitoring and manipulation

21. 21 CORBA Advocates and policies: CORBA objects Advocates and policies: CORBA objects Easy to assemble Easy to assemble Easy to invoke at run time Easy to invoke at run time In one process or multiple processes In one process or multiple processes Basis for custom advocates and policies Basis for custom advocates and policies Dynamic loader for custom objects Dynamic loader for custom objects Basis for connecting to MW-based tasks Basis for connecting to MW-based tasks …allows open and dynamic architecture …allows open and dynamic architecture

22. 22 QuO Apps not designed for the CPU Broker Apps not designed for the CPU Broker Use QuO delegates Use QuO delegates Improved integration with MW-based tasks Improved integration with MW-based tasks Sync’ed with task cycle Sync’ed with task cycle Customizable Customizable … but, small source changes required … but, small source changes required Real-Time App Object Implementation QuO Delegate CPU Broker Periodic CORBA requests

23. 23 Process Advocate Need to manage non- MW applications, too Need to manage non- MW applications, too Use “proc_advocate” Use “proc_advocate” Transparent to task Transparent to task No MW required No MW required Entirely reusable Entirely reusable …but, less access to task state …but, less access to task state E.g., period E.g., period CPU Broker Real-Time App proc_advocate TimeSys Linux

24. 24 Using the CPU Broker Scripted, interactively Scripted, interactively Command-line tool Command-line tool “set priority of task mplayer to 5” “set priority of task mplayer to 5” Programmatically Programmatically E.g., with QARMA [Gillen et al., 2004] E.g., with QARMA [Gillen et al., 2004] Task RTOS Adv. Policy cbhey QARMA

25. 25 Evaluation Overhead Overhead Synthetic applications (correct operation) Synthetic applications (correct operation) UAV application UAV application Experiments performed in Utah’s Emulab Experiments performed in Utah’s Emulab 850 MHz Pentium III, 512 MB RAM 850 MHz Pentium III, 512 MB RAM TimeSys Linux/NET 3.1.214, Red Hat 7.3 TimeSys Linux/NET 3.1.214, Red Hat 7.3

26. 26 Measured Overhead Reasonable and small for many RT applications Further optimizations possible Configuration Monitor+Broker CPU Time (μs) Monitor Only Real Time (μs) 2-way QuO delegate 17421587 1-way QuO delegate 1716660 In-broker proc. advocate 400400 “load”“latency”

27. 27 UAV Military Simulation Distributed application Distributed application Soft real-time Soft real-time Broker applied at ATR Broker applied at ATR Java process Java process Multi-threaded Multi-threaded Irregular CPU demand Irregular CPU demand Goals Goals Ensure ATR meets deadlines Ensure ATR meets deadlines Allow high system utility Allow high system utility UAV Dist. ATR 2 fps → ←2 fps alerts →

28. 28 Broker Extension Custom advocate for ATR Custom advocate for ATR Predict GC cycles from recent CPU demand Predict GC cycles from recent CPU demand Other broker objects from our library Other broker objects from our library Recv Instr. Load RTOS Adv. Policy Advocate

29. 29 Utility: Custom Advocate Custom advocate accurately predicts demand, allowing high system utility Load Pred.

30. 30 Resilience to CPU Load Metric Unloaded, Baseline CPU Load CPU Load, With Broker Frames processed 432320432 Avg. FPS 1.841.321.81 Min. FPS 1.670.451.11 Max. FPS 2.002.011.99 Std. Dev. 0.090.340.09 Alerts received 765076 Avg. latency 127.671560.44325.72 Min. latency (ms) 101.00362.00145.00 Max. latency (ms) 193.003478.00933.00 Std. Dev. 33.46961.62153.60

31. 31 Conclusions Dynamic RT systems face critical design- time and run-time challenges Dynamic RT systems face critical design- time and run-time challenges Our CPU Broker successfully addresses many of these challenges Our CPU Broker successfully addresses many of these challenges specifications: separated and consolidated specifications: separated and consolidated dynamic: negotiations atop reservations dynamic: negotiations atop reservations open framework: extension and integration open framework: extension and integration →More modular and understandable RT →More modular and understandable RT

32. 32 Open Source CPU Broker available online CPU Broker available online http://www.cs.utah.edu/flux/alchemy/ http://www.cs.utah.edu/flux/alchemy/