1. Performance Evaluation of Real-Time Operating Systems
RTEMS, RTLinux, eCos

2. Real-Time Systems
Systems that interact predictably with events in the outside world
Examples:
Flight control systems
Collision avoidance systems
Satellite guidance systems
Patient Monitoring System
Control Systems for synchrotrons

3. Real-Time Operating Systems
Free the applications programmer from writing code for task scheduling and dispatching.
Manage time-sharing of a processor for a number of different tasks and interrupt sources, while adhering to strict time constraints.
Widely used in all kinds of applications both on PCs and in embedded systems.
Generally operate unattended -- consequences of failure can be catastrophic.
The function of a real-time operating system is to manage time-sharing of a processor for a number of different tasks and interrupt sources, while adhering to strict time constraints.
Real-time operating systems are widely used in all kinds of applications both on PCs and in embedded systems where the processor is completely encapsulated within the hardware it is controlling.
Real-time systems must interact predictably with devices in the outside world.
“They generally operate unattended, and the consequences of any kind of failure range from severe to unthinkable.”

4. What is available? Many RTOSs have been developed:
Commercial
Open source
RTOS vendors and developers publish performance metrics to showcase their products. However, their evaluations are:
not performed on a common platform
not comprehensive
Many commercial RTOSs are available.
Several open source, royalty-free packages are also available or under development.
RTOS vendors and developers publish performance metrics to show the superiority of their products.
However, these metrics are not performed on a common platform and so are like comparing apples to oranges.
They are also not necessarily comprehensive.

5. Evaluation of an RTOS
How would you choose an appropriate RTOS for your application?
Need for an impartial evaluation.
Several such projects have been conducted.
Only evaluated some performance characteristics
Only looked at a few commercial operating systems
Many are now out of date
‘Real-Time Consult’ introduced a more comprehensive project.
Developed a thorough test methodology
Evaluated several commercial RTOSs
Have not evaluated open source RTOSs
An extensive project was started about three years ago, by a group called “Real-Time Consult”, to qualify and compare real-time operating systems.
they have developed a methodology and started producing evaluation reports for several commercially available RTOSs.
However, we have not found any suitable comparison for open source RTOSs.
There are many benefits to using an open source RTOS, such as:
the cost benefit (royalty-free) and, more importantly,
the performance can be more thoroughly tested than if the source code is not available
code can be modified or modules written for your own particular application.

6. Research Objective
Performance evaluation, on a common platform, of three open source real-time operating systems:
RTEMS
RTLinux
eCos
I will get each RTOS running on a PC, then do performance tests on each one.

7. Real-Time Operating Systems
Overview
RTOS
Task
Scheduling
Task
Dispatching
Task
Synchronization
The function of a real-time operating system is to manage time-sharing of a processor for a number of different tasks and interrupt sources, while adhering to strict time constraints.

8. Task Scheduling Time Slicing Last Arrival Order First Time Task 3

9. Task Scheduling Preemptive Priority High Priority Low Time Task 3
The function of a real-time operating system is to manage time-sharing of a processor for a number of different tasks and interrupt sources, while adhering to strict time constraints.
Task 1
Task 1

10. Task Dispatching
Task-control blocks is the most popular method of identifying and managing tasks.
Task-Control Block (TCB) contains:
a context (e.g. program counter and register contents)
an identification string
a status, such as ready,
executing or blocked
a priority (if applicable)
A Typical
Task Control Block
Program Counter
Task Status
Task ID Number
Pointer to next TCB
Other Context
Priority :
Register Contents

11. Task Synchronization
Events
Messages
Semaphores
Mutexes

12. Desired Performance Characteristics
Tasks should run with:
minimal event latency (the time between the triggering of a task and its start)
minimal jitter (the variation in running times of a task that is supposed to run at a fixed period).

13. Performance Evaluation
Throughput – speed at which the system executes instructions
Responsiveness – how fast it starts to handle in interrupt request
Determinism – how it reacts under load
how long does it take to finish what it is doing and start handling the interrupt request

14. Performance Metrics Low Level Tests Context switching
Interrupt latency
Exclusion objects
Semaphores
Mutexes
Synchronization events

15. Context Switch Time Task A Start Task B : : Context Switch Time
Switch to Task B
Save context
of Task A
Load context
of Task B
Task A
Start
Task B PC
Register 0 :
Other Context PC
Register 0 :
Other Context
Time
Context Switch Time

16. Other Considerations Not an isolated event
Affected by number of tasks pending
Depends on priorities of pending tasks
Amount of context to be saved

17. Interrupt Latency Interrupt and Interrupt Dispatch Latencies Task Task
ISR
First Instruction in
Interrupt Service Routine
(ISR) is executed
Interrupt
Occurs
Task
resumes
ISR ends
Task
Time
Interrupt
Latency
Interrupt
Dispatch Latency

18. Multiple Interrupt Latencies
Interrupt-to-Task Run
Interrupt
Interrupt Dispatch Time
Interrupt Service Routine
Other Interrupt
Pre-emption Disabled
More than one interrupt is pending:
response time for second interrupt depends on the length of the first interrupt. You have to finish serving the first interrupt which means you have to execute the Interrupt Service Routine. This includes:
ISR must, at a minimum, send a synchronization event to a task.
Other processing code might also be included in the ISR and must also finish. This is why it is recommended that this kind of processing be done outside the ISR if at all possible.
Return from interrupt and enable next interrupt
Scheduling
Context Switch
Return from System Call
Task Run

19. Interrupt Task Response Time
Real response to an interrupt often occurs in a task synchronized by, but outside of, the actual ISR.
Task response also depends on whether the kernel is preemptable. If not, then kernel call must be completed before ISR.
More than one interrupt is pending:
response time for second interrupt depends on the length of the first interrupt. You have to finish serving the first interrupt which means you have to execute the Interrupt Service Routine. This includes:
ISR must, at a minimum, send a synchronization event to a task.
Other processing code might also be included in the ISR and must also finish. This is why it is recommended that this kind of processing be done outside the ISR if at all possible.
Return from interrupt and enable next interrupt

20. Performance Metrics
High Level Tests
Network throughput
Stress tests

21. Real-Time Consult Project
Evaluation and comparison of RTOS performance
Commercial systems only, to date
Evaluations available for:
Intime 1.20
RTX 4.2
Hyperkernel 4.3
VxWorks/x
pSOSystem/x
QNX 6.1
CE 3.0

22. Real-Time Consult Project
Test Method: timing is measured using an external PCI bus analyzer.
Two types of tests:
Performance Tests
Stress Tests
Qualitative Evaluation (API richness, etc.)

23. Real-Time Consult Project
Performance Tests (context switch & latencies):
Thread switch latency
time required to preempt current thread
Interrupt latency
preempt current thread and start interrupt handler
Interrupt dispatch latency
switch from interrupt context to context of interrupted thread or next thread in queue
Thread creation and deletion

24. Task Switch Latency Test
Create tasks
TSK1, …, TSKN
Task start
TSK0
Delete tasks
Write trace on PCI
Bus
TRC_TSK0_TS0
Yield
Processor
Yes No
End of Test?
Task end
Each Task
Write trace on PCI
Bus
TRC_TSKN_TSN
Yield
Processor
Task start
TSKN

25. Real-Time Consult Project
Performance Tests (objects, file, network):
Synchronization objects
time to create and delete a synchronization object
Exclusion objects
time to create and delete an exclusion object
File system operations
creating and deleting files, reading from files and writing to files in synchronous mode
Network stack
performance of TCP/IP stack
bandwidth for various packet sizes, and CPU usage

26. Real-Time Consult Project
Stress Tests:
Two simultaneous interrupts
Interrupt nesting
maximum sustainable interrupt frequency
maximum number of objects
memory leaks

27. Focus of my Research
Evaluation of three Open Source Real-Time Operating Systems:
RTEMS
Real Time Executive for Multiprocessor Systems
RTLinux
Real Time Linux
eCos
Embedded Configurable Operating System
There are many benefits to using an open source RTOS, such as:
the cost benefit (royalty-free) and, more importantly,
the performance and reliability can be more thoroughly tested than if the source code is not available
code can be modified or modules written for your own particular application.

28. Application Dependent Software Standard Application Components
RTEMS
Developed by the U.S. military as alternative to using commercial RTOS
Small, easy to port
High level of user configurability
Kernel is preemptible
Application Dependent Software
Standard Application Components
Device Drivers
R T E M S
Target Hardware

29. RTLinux
Abstraction layer between the hardware and the standard Linux kernel
Appears as actual hardware to standard Linux kernel
Lowest priority is assigned to standard Linux kernel, which then runs as a independent task.
RTLinux executive is nonpreemptible
Leaves Linux kernel essentially untouched so it doesn’t hinder future Linux development

30. ECOS Targeted at high-volume applications: Configurable: Portable
consumer electronics, telecommunications, automotive
other deeply embedded applications.
Configurable:
lets developer configure a system that best matches the needs of the application.
Typical configuration options:
type of scheduler
number of priority levels.
current release of the system has over 200 options
Portable
Fully preemptable

31. Typical Application
Control system for the Canadian Light Source (CLS) uses an open source real-time operating system and control software:
RTOS  RTEMS
Control Software  EPICS
(Experimental Physics and Industrial Control System)
Such distributed control systems typically comprise tens or even hundreds of computers, networked together to allow communication between them and to provide control and feedback of the various parts of the device from a central control room, or even remotely over the internet.
Commonly used in distributed control systems to operate devices such as:
Particle Accelerators
Large Experiments
Major Telescopes.