RTDroid: toward building dynamic real-time systems ? Lukasz Ziarek University at Buffalo Dagstuhl Seminar: Resource Bound Analysis http://rtdroid.cse.buffalo.edu

Collaborators Ethan Blanton – Fiji Systems Inc Jan Vitek – Northeastern Karthik Dantu – UB Steve Ko - UB Yin Yan (Ph.D. Student) Many masters students http://rtdroid.cse.buffalo.edu

Motivations Open source project Well-defined APIs Rich on-device peripherals How can we write a real-time application on Android, while still leveraging Android’s existing functionality? http://rtdroid.cse.buffalo.edu

Cochlear Implant Device Cochlear implant as a motivating example An external device for audio recording and processing Implanted electrodes stimulate the cochlea http://rtdroid.cse.buffalo.edu

Architecture of Cochlear Implant Application on Android UI Activity Audio Recorder (Service) Periodic Audio Recording Task Volume Receiver Audio Processor (Service) Button Listener MsgHandler Periodic Audio Processing Task 8 ms UI Handler Output Simulator (Broadcast Receiver)

Sound Processing Latency Duration of 12% of audio processing releases is longer than 8 ms. Cochlear Implant: Audio Processing Duration

What Is Missing in Android? Lack of real-time expressiveness Can’t prioritize the execution of application components Can’t specify real-time configuration Lack of real-time communication No priority awareness for message/intent delivery No protection mechanism for cross-context communication Lack of memory guarantees Non Real-time components Real-time components UI Activity Audio Recorder (Service) Periodic Audio Recording Task Volume Receiver Audio Processor (Service) Button Listener MsgHandler Periodic Audio Processing Task UI Handler Output Simulator (Broadcast Receiver)

A View Of Android Applications No Real-time Expressiveness FIFO Message Passing Fair System Services Application Framework Constructs And APIs System Services In order to see why Android is not predictable, we have to examine each layer in Android stack. In bottom layer, it is well-known. Many features are not predictable, for example …. In nutshell, there is not predictability in kernel layer In VM Layer, it is also well-known that it is not predictable. In framework, our research show it is not predictable. There layers are known no predictable, The Android Linux Kernel is developed to achieving full CPU utilization, the predictability of the system were not taken into account. VM: Dalvik was design for mobile devices and consider more about the memory optimization, battery lift and low frequent CPU. Runtime Unpredictable Garbage Collection No Support for RT POSIX Extension Bionic Dalvik/ART Fair Scheduling Out of Memory Killer Kernel Forced GC Power Management Linux with Android Modifications Android Stack

Proposed Architectures for Real-time Android Applications RT Apps Applications RT Apps Application Framework Application Framework Libraries Core Libraries Core Dalvik VM Dalvik VM Linux Kernel Linux Real-time Real-time Hypervisor Completely Isolated

Proposed Architectures for Real-time Android Applications Applications Application Framework Application Framework Libraries Core RT JVM Libraries Core Dalvik VM Extended Dalvik VM Linux Real-time Linux Real-time Lower Latencies Best Effort Interactions Unclear ?

RTDroid Architecture Programing Model Validation Mechanisms Applications Application Framework Redesigned Components Redesigned Message Passing Redesigned System Services Runtime Bionic Extended Bionic Real-time Garbage Collection POSIX RT Extension RTSJ Libraries Dalvik/ART Fiji MVM Linux with Android Modifications Priority Based Scheduling PIP Locks RT Linux or RTEMS RTDroid Stack http://rtdroid.cse.buffalo.edu

Components and System Services http://rtdroid.cse.buffalo.edu

Redesigned Components Application Framework Constructs and APIs Runtime System Services RT Handler RT AlarmManager RTSJ RT SensorManager RTDroid RT SoundManager

Why Looper and Handler? One of the central message-passing mechanisms. All Android apps unknowingly use Looper-Handler. Most of the system services use Looper-Handler. http://rtdroid.cse.buffalo.edu

Looper and Handler … Looper is a per-thread message loop. Handler processes messages. A thread uses Handler.send() to send a message. Receiver Thread Sender Thread Looper Message Queue Handler Handler Ref (provides send()) … Message Message

What’s Wrong with Looper/Handler in a Real-time Context? Message Queue Looper … RT msg msg N msg 1 Messages are processed FIFO or by client-specified time Handler No guarantee on message delivery RT client thread Non-RT client thread Non-RT client thread http://rtdroid.cse.buffalo.edu

RTLooper and RTHandler Msg Queue Msg Queue RT Looper (RT Thread) Looper (Non-RT Thread) msg1 msg1 … … msg n msg n RT Handler Handler instance Handler instance RT client thread Non-RT client thread Non-RT client thread … http://rtdroid.cse.buffalo.edu

Android Sensor Manager APIs Application Enable/disable hardware Accelerometer Sensor Event Listener Configure sampling rate Gyroscope Subscribe to sensor events … Magnetometer http://rtdroid.cse.buffalo.edu

How Does Android’s Sensor Manager Work? Input Event HAL Sensor Service Sensor Manager Sensor Event Listener Sensor Thread Native SensorManager Sensor Fusion Sensor Thread Event Queue Kernel System Runtime Framework Application http://rtdroid.cse.buffalo.edu

Shared delivery path and unbound delivery time What Happens In a Real-Time Context? One application is listening on two sensors: Accelerometer with higher priority Gyroscope with lower priority Application Android Framework Kernel Accelerometer Data Gyroscope Data Shared delivery path and unbound delivery time http://rtdroid.cse.buffalo.edu

RT Sensor Manager Event-driven architecture Prioritize the delivery procedure of sensor events Multiple listeners with different sampling rates http://rtdroid.cse.buffalo.edu

Orientation Polling Thread RT Sensor Manager Listener 1 Listener 2 Listener 3 Listener 4 Delivery Thread 1 Delivery Thread 2 Delivery Thread 3 Delivery Thread 4 Listener’s rate and priority Accel Events Gyro Events Orientation Event The highest priority of its listeners Sensor Fusion Accel Polling Thread Gyro Polling Thread Orientation Polling Thread /dev/input/ event6 /dev/input/ event1

Remaining Questions Multi-app execution Real-time Android Programming model Static Memory Guarantees http://rtdroid.cse.buffalo.edu

Multi-app Execution http://rtdroid.cse.buffalo.edu

Android: Application Isolation & Communication One VM per application Virtual Memory Each application has its own PID, UID for isolation App A App B System Server App Components App Components ActivityManager Activity Thread ART Activity Thread ART ServiceManager Linux Kernel Binder Driver

Multi-Tasking VM (MVM) All applications run in a single VM. The VM runs each application in an isolation unit called a partition, but all applications run as a single process (the same memory space). How does MVM provide isolation? Separated Class loading (Fiji uses lightweight VMs) Dynamically manage Memory boundaries Statically Configured Heap Memory http://rtdroid.cse.buffalo.edu

Time Partitions in MVM MVM slice manager: Fixed-length time slice for each partition Partition initiation, suspension, resume, termination Two approaches for thread scheduling: Direct OS scheduling vs partition dispatcher with a thread pool

Communication in MVM Shared address space provides flexibility Wait free pair transactions One way communication channel over primitives CAS and replicas Priority Rollback Protocol Inspired by transactional memory Similar tracking structures write buffers, undo log The lock itself provides concurrency control http://rtdroid.cse.buffalo.edu

Programming Model http://rtdroid.cse.buffalo.edu

Can we write an App with RTDroid? Applications RT-Looper and RT-Handler RT Alarm Manager RT Sensor Manager What else is missing? Application Framework Runtime Extended Bionic Fiji VM RT Linux or RTEMS Linux with Android Modifications http://rtdroid.cse.buffalo.edu

Android Linux Kernel with RT Patch Programming Model Android Apps RT app Android Applications RTDroid Applications Declarative real-time components Real-time communication channel Memory guarantees region-based allocation Design principles: Retains a similar style of Android Minimizes code complexity in application Enables application validation with real-time configuration prior to execution Android Framework RTDroid Framework Dalvik / ART Fiji Java VM Android Linux Kernel with RT Patch

Declarative Real-Time Components <service name="pkg.ProcessingService” priority="79"> <memSizes persistent="1M” release="1M" /> <periodic-task name="procTask“ priority="79"> <release start="0ms" periodic="8ms" /> </periodic-task> … </service> Service + onStartCommand() + onBind() + onCreate() + onDestroy() BroadReceiver + onReceive(…) Real-time components RT service RT broadcast receiver Periodic task Application manifest Priority Timing parameters Memory bounds RealtimeReceiver + onClean() RealtimeService + onPause() PeriodicTask + onRelease()

Real-Time Communication Channels <channel name=“msghandler” type=“msg- passing”> <!-- message delivery and processing policies --> </ channel> Four types of channels RT message passing channel RT intent broadcasting channel RT bulk data transfer channel Cross-context channel Communication policies Message delivery order Component invocation Message constraints <service name="pkg.RecordingService" priority=“78"> <intent-filter count="2" role=“publisher"> <action name="msghandler"/> </intent-filter> </service> <service name=" pkg.ProcessingService " priority=“79"> <intent-filter count="2" role=“subscriber"> <action name="msghandler"/> </intent-filter> </service>

Memory Guarantees in RTDroid Region-based memory management with constant cost for object allocation and reclamation. Based on the concept of the scoped memory in RTSJ The complexity of region management is hidden in our program model The lifetime management of scope is associated with the lifetime of each application component. http://rtdroid.cse.buffalo.edu

Memory Guarantees in RTDroid Immortal scope Objects that have the same lifetime as the application Persistent scope Objects that have the same lifetime as its associated component Release scope Temporary objects for the execution of callback functions http://rtdroid.cse.buffalo.edu

Region-Based Memory Management in Cochlear Implant Processing Service Persistent Release Output Receiver Heap Memory (FijiVM) Heap Memory (ART/Dalvik) Android Adaptor UI Activities Immortal Scope RT Channels RTDroid Proxy Real-time Runtime in RTDroid Android Runtime

A Fully Dynamic System http://rtdroid.cse.buffalo.edu

Dynamic Real-Time Systems Ability to install / remove / restart applications Non real-time Real-time JIT as a service … for validation? Validation of system wide constraints Modification of an existing application Adding / removing capabilities http://rtdroid.cse.buffalo.edu

System wide memory management Can we manage memory more efficiently / predictably if we take a whole system view? Hussein et al. showed for non RT http://rtdroid.cse.buffalo.edu

Validating Interactions Session Types Mechanisms for validating interactions Communication protocols “match” Participants are duals of one another No notion of rate / resources Partial failures Error handling without restarting a session http://rtdroid.cse.buffalo.edu

Reasoning about power? Power management is incredibly important for mobile We currently do not do anything interesting Lots of interesting work to explore Energy types Power modeling + power management http://rtdroid.cse.buffalo.edu

Validating Timeliness? What is feasible to validate at runtime? Can package static results with the app Bytecode based WCET (Bates et al.) Potential starting point? JIT based specialization / validation? Online Schedulability analysis? http://rtdroid.cse.buffalo.edu

Evaluation http://rtdroid.cse.buffalo.edu

Platform Categories RTEMS Soft Real-time Soft Real-time Desktop Smartphone Soft Real-time Desktop Soft Real-time Embedded http://rtdroid.cse.buffalo.edu

Cochlear Implant Sensing Processing … Real-Time Processing Non-Real-Time UI Real-Time Processing … http://rtdroid.cse.buffalo.edu

Cochlear Implant http://rtdroid.cse.buffalo.edu

Evaluation on jPapabench http://rtdroid.cse.buffalo.edu

jPapaBench (UAV flight control) http://rtdroid.cse.buffalo.edu

Wind Farm Blade Monitoring http://rtdroid.cse.buffalo.edu

Wind Farm Blade Monitoring http://rtdroid.cse.buffalo.edu

Conclusion RTDroid A real-time mobile system with Android-like programming interfaces. RT kernel + RT JVM + RT framework layer RTDroid runs multi-apps in a single Multi-tasking VM with isolation. RTDroid provides a real-time programming model with memory guarantees. http://rtdroid.cse.buffalo.edu

Porting jPapaBench into RTDroid Fly-By-Wired (FBW) Simulation Autopilot TestPPMTask Handler SimulatorFlight ModelTaskHandler Navigation TaskHandler SendDataTo Autopolit SimulatorIR TaskHandler AltitudeControl TaskHandler Data injection SimulatorGPS TaskHandler … CheckFailsafe TaskHandler Data subscription SensorManager CheckMega128 ValuesTaskHandler Stablization TaskHandler

Base line performance on Base line performance on LEON3 Evaluation for Java Autopilot RTEMS The separation is caused by the interval between the time sensor data are injected into buffer in sensor manager, and the time the polling thread reads data. Base line performance on Nexus S Base line performance on LEON3

RT SensorManager stress tests on LEON3 Evaluation for Java Autopilot RTEMS Message buffering dominates the overall latency, Which means the interval between the time sensor data are injected into buffer in sensor manager, and the time the polling thread reads data. The polling rate for IR sensor is bounded to 25ms. This is the largest interval in Polling. We obtain a few latencies that are 30 ms in the listener extensive experiment. the combination of the largest interval in polling with the preemption cost. This overhead is mainly additional to additionally buffering costs as the data is replicated for each priority level and the delivered to corresponding handlers. The overhead is proportional to the number of low priority listeners and is the combination of the largest interval in polling including the preemption cost. There is little difference between 30 memory and 30 computational threads. There is enough slack in the system for the GC to keep up with memory allocations performed in the memory noise generating threads. RT SensorManager stress tests on LEON3

RTService Code Example <service name=“.AutopilotService” persistent=“1M” release=“1M”> <rt-android:priority priority=90> <SubService id=“stabilization-task” release= “1M”> <rt-android:priority priority=29> <rt-android:periodic ms=“50”, ns=“0”> </SubService> <SubReceiver id=“sf-recvr” release=“1M”> <intent-filter count=4> <action name=“sim-status”> <data size=“50K” mimetype=“app/octet-stream”> </itent-filter> </SubReceiver> </service> Class AutopilotService extents RTService{ FlightModel fModel; AutopilotModel aptModel; public final SubReceiver sfRecvr = new SubReceiver(){ pubic void onReciver(Content t, Intent i){ read intent msg; fModel.process(i); } }; public void onCreate(){ //flightModel, autopilotModel instantiation stabTask = initSubService( new StabRunnable(aptModel, fModel)); registerRTSubReceiver(sfRecvr, ”sf-recvr”);

AutopilotService Persistent Scope (1M) RTService Code Example Class AutopilotService extents RTService{ FlightModel fModel; AutopilotModel aptModel; public final SubReceiver sfRecvr = new SubReceiver(){ pubic void onReciver(Content t, Intent i){ read intent msg; fModel.process(i); } }; public void onCreate(){ //flightModel, autopilotModel instantiation stabTask = initSubService( new StabRunnable(aptModel, fModel)); registerRTSubReceiver(sfRecvr, ”sf-recvr”); AutopilotService Persistent Scope (1M) AutopilotService Release (1M) stabTask Release (1M) sfRecvr Release (1M)