1. PTIDES Project Overview
Slobodan Matic, Jia Zou
Edward Lee, John Eidson
ActionWebs, 09/29/10

2. Orchestrating networked computational resources with physical systems
Transportation
(Air traffic control at SFO)
Avionics
Building Systems
Telecommunications
Automotive
Instrumentation
(Soleil Synchrotron)
E-Corner, Siemens
Factory automation
Power generation and distribution
Daimler-Chrysler
Military systems:
Deterministic Implementations of Event-driven Software
Courtesy of
General Electric
Courtesy of Doug Schmidt
Courtesy of Kuka Robotics Corp.

3. Current Practice: Printing Press
High-speed, high precision
Speed: 1 inch/ms
Precision: 0.01 inch
-> Time accuracy < 10us
Application aspects
local (control)
distributed (coordination)
global (modes)
Open standards (Ethernet)
Synchronous, Time-Triggered
IEEE time-sync protocol
Bosch-Rexroth

4. For distributed cyber-physical systems,
Timing needs to be a part of the network semantics,
not a side effect of the implementation.
Technologies needed:
Time synchronization
Bounds on latency
Time-aware fault isolation and recovery

5. A Programming Model for Distributed Cyber-Physical Systems
The question we address:
Given a common notion of time shared to some known precision across a network, and given bounded network latencies, can we design better distributed embedded software?
Our answer (PTIDES):
Use discrete-event (DE) models for specification of systems, bind model time to real time only exactly where this is needed.

6. Outline Introduction PTIDES Basics and Workflow
PTIDES Runtime Framework
Single CPU: Tunneling Ball Device
Multiple CPU: Power Distribution Subsystem
Summary

7. PTIDES Basics
Programming Temporally Integrated Distributed Embedded Systems
Based on Discrete-Event model of computation
Event processing in time-stamp order
Deterministic under simple causality conditions
Fixed-point semantics
Can be extended to super-dense form
Timing constraints for interaction with environment

8. Model vs. Physical Time At sensors and actuators
Relate model time (τ) to physical time (t)
t ≥ τ
t ≤ τ τ1 do i4
model time τ1 τ4
physical time t1 t4

9. Single Processor PTIDES Example
Bounded delay between any two hardware components
Bounded sensor latency (d0)
t ≥ τ , t ≤ τ + do
t ≤ τ τ1 do i4 τ2
model time τ2
physical time t2
e2 at i2

10. Single Processor PTIDES Example
t ≥ τ , t ≤ τ + do
t ≤ τ τ1 do i4 τ2
model time τ2
physical time t2
τ2+d0
e2 safe to process if t > τ2 + do

11. Single Processor PTIDES Example
t ≥ τ , t ≤ τ + do
t ≤ τ τ1 do i4 τ2
model time τ1
physical time t2
τ1+d0
e2 safe to process if t > τ2 + do

12. Distributed PTIDES Example
Local event processing decisions:
Bounded communication latency (d0)
Distributed platforms time-synchronized with bounded error (e)
τ is safe to process if: t > τ + do2 + e - d 2
d 1 τ1 τ2
d01
Sensor
Merge τ
Actuator
d 2 τ3 τ4
do2
Network Interface o3

13. Distributed PTIDES Example
Local event processing decisions:
Bounded communication latency (d0)
Distributed platforms time-synchronized with bounded error (e)
τ is safe to process if: t > τ + do2 + e - d 2
d 1
d01
Sensor τ1
Merge τ
Actuator
d 2 τ3 τ4
do2
Network Interface o3

14. What Did We Gain? e1 = (v1, τ1) δ e1, e2, … e2 = (v2, τ2)
First Point: Ensures deterministic data outputs
Merge
e1 = (v1, τ1) δ
e1, e2, …
safe to process analysis for e
safe to process analysis for e
e2 = (v2, τ2)
Second Point: Ensures deterministic timing delay from Sensor to Actuator
t ≤ τ + do
t ≤ τ i4 do τ1 τ2
Now that I defined the GES, let’s take a step back and

15. PTIDES Program Design Workflow
HW Platform
PTIDES run-time

16. PTIDES Runtime Framework
Software components (actors) are “glued together” by a code generator into an executable
Lightweight real-time kernel (5KB)
Actors are state machines, each with an event queue and context
Asynchronous delivery: producer posts to event queue
Framework handles thread-safe event exchange and queuing  
Run-to-completion : No blocking code
No busy-waiting, semaphore, mailbox
Preemptive
-> Single-stack implementation
Encapsulate each resource inside a dedicated active object
rest of app shares the dedicated active object via events 
Fast and deterministic memory allocation
Event pools:  fixed-size memory blocks

17. PTIDES Scheduler Two layer execution engine
Event coordination (safe-to-process)
Event scheduling (prioritize safe events)
Earliest Deadline First foundation
Deadline based on path from input port to actuator
Schedulability analysis requires WCET of software components + event models
PTIDES implementation delivers DE semantics even w/o WCET
Even if deadlines missed, events are determinate
PTIDES repeatable even if WCET wrong

18. PTIDES Workflow HW Platform Software Component Library Ptides Model
Code Generator
PTIDES run-time
Code
Plant Model
Network Model
HW in the Loop Simulator
Causality Analysis
Program Analysis
Schedulability Analysis
Analysis
Mixed Simulator

19. Schedulability Analysis
Three cases:
Zero event processing time assumption (feasibility test)
if P fails, P will not satisfy constraints on any hardware
No resource sharing assumption (an event is processed as soon it is safe)
if P fails, P may still satisfy constraints on other hardware
Resource sharing (a safe event is processed according to a scheduling algorithm)
if P fails, P does not satisfy this implementation (and algorithm)

20. Feasibility Test Feasibility assumptions
Zero event processing time
Bounds on event communication latency (d0)
If test fails, no HW satisfies constraints
detect flows

21. Feasibility Test
For each input port i on a sensor-actuator path

22. Single CPU - Proof of Concept
Tunneling Ball Device
sense ball
track disk
adjust trajectory

23. Single CPU Proof of Concept
Event sequences
periodic
quasi-periodic
sporadic

24. Tunneling Ball Device – 10 rps

25. Tunneling Ball Device – 1 rps

26. Distributed Testbed Objectives
Safe event processing
Offset attribute of a port
Check: real time vs model time + offset
Bounded latency between hardware components
Distribution: bounded-delay networking protocol
Scheduling
Deadlines defined by sensor – actuator model delays
Single CPU: use Earliest Deadline First
Distribution: local from end-to-end deadlines
Expressiveness of interaction
Complexity of real sensor, actuator and network interfaces
Timing support
External to microprocessor
National Semiconductor DP83640: IEEE 1588 enabled PHY chip

27. Synchrophasor-based measurement and control
Power swing and
Unstability detection

28. Experiment Diagram Grid emulator built with National Instruments PXI
‘PMU’s built with Renesas demo boards with DP83640
Ethernet bridge or 1588 boundary/transparent clock
‘SVP’ built with Renesas demo board with DP83640

29. Basic PTIDES Timing Testing
Vary phasor data independently
Freq. and phases w.r.t. global time
Sample voltage and current
Signal processing
Send phasor data
Local control: on/off breaker
Wireshark monitoring of network events
Detect discrepancies
If unstable region send on/off command

30. Single platform PTIDES approximation model

31. Multi-platform PTIDES approximation model

32. Summary
The networking problem requires timing to be a correctness property rather than a quality of service
PTIDES model of computation offers a possible programming model for distributed CPS
Correct deterministic DE implementations
Better match for reactive state-machine systems  than traditional multi-threaded     or     super loop     programming models
Time stamps provides runtime information for detecting and reacting to variety of timing faults
Goal: scalability, robustness, fault-tolerance