1 Maintaining Logical and Temporal Consistency in RT Embedded Database Systems Krithi Ramamritham

2 Overview Motivation Problems in real-time database systems (RTDBs) Contributions Proposed solutions for Real-Time Databases: –Maintaining temporal consistency –A novel principle to assign deadlines & periods for real-time update transactions Conclusions Other contributions & future work

3 Motivation Embedded applications need access to time varying data – e.g., temperature, pressure, etc. Need sufficiently fresh data for informed decisions, especially for real- time applications. A RTDB provides necessary support –executes transactions within deadlines –maintains temporal consistency of data  data values valid only for limited time A RTDB is a core component of real-time applications –Air traffic control –Avionics –Process control –Intelligent networks

4 RTDB Model for Maintaining Temporal Validity of Real-Time Data Real-Time Databases Network Sensor 1 Sensor 2 Sensor N.. A real-time object in RTDBs models a real world entity, e.g., position of an aircraft Values are sampled by sensors, and propagated to RTDBs (propagation delay >= 0) Real-time data in RTDBs must remain fresh in order to react to abnormal situations timely Transactions may be triggered to deal with abnormal situations

5 What is Data Temporal Consistency in RTDBs? Temporal Consistency: keep data valid relative to real world Time Value X Real-time data values change continuously Data values are sampled periodically A validity interval is associated with a data value Within validity interval, a data value is temporally valid – deviation from real world is acceptable

6 Applications with Temporal Consistency in RTDBs Air traffic control: –Real-time data: aircraft position, speed, direction, altitude, etc. 20,000 data entities validity intervals of 1 ~ 10 seconds Process control: – Real-time data: pressure, temperature, flow rate, etc. Intelligent network: network services databases –Real-time data: network traffic management data e.g., bandwidth utilization, buffer usage

7 Our focus: real-time data & their impact on transactions Traditional RTDB work focuses on transactions with deadlines Commercial RTDBs –EagleSpeed: meeting timing constraints –ClustRa: providing real-time response for telecom applications

8 Commercial RTDB Examples EagleSpeed TM – meeting timing constraints (Lockheed Martin Corp. Ocean, Radar & Sensor Systems) –Quick, efficient and deterministic processing –Predictable response time –Concurrency control and deadlock prevention –Priority inheritance for transactions –Fault tolerance Applications –Navy’s AN/BSY-2 combat system –The AN/SQQ-89 (V) 14 and (V) 15 surface ship systems –The C130 AEWC mission systems For more info:

9 Commercial RTDB Examples, Continued ClustRa -- supporting commercial telecom applications Real-time requirements: –Real-time response: <100 ms –High throughput: > 1000 transactions/second –High availability: < 2 minutes unavailable per year –High scalability: worldwide telephony network Applications: –Next generation network service (Telcordia) –Global mobility management (Inmarsat) –Intelligent telecom network (Oddvar Hesjedal) –Real-time network management For more info:

10 Maintaining Temporal consistency -- Strict Transaction Correctness in RTDBs Observation: Validity intervals of accessed data may expire if a transaction does not complete in time Transaction correctness criterion: A transaction can commit if and only if –it is logically consistent, –it meets its deadline, and –at the commit time, the objects it has read are still temporally consistent.

11 Data-Deadline in RTDBs Valid X Valid Y Deadline T Time T : Transaction X,Y : Data t0t0 Begin T t1t1 R T (Y) t2t2 t5t5 Our novel concept: Data-Deadline (DDL) – derived from data validity and transaction deadline DDL t (T) = min (deadline(T), min(End_Valid(X))) – T: a transaction – t: time – X: real-time data read by T by time t – End_Valid(X): end of validity interval of X DDL t2 (T) = t5 DDL t3 (T) = t4 R T (X) t3t3 t4t4 Abort T

12 Novel Transaction Scheduling Algorithms: Data-Deadline based Scheduling Data-Deadline based scheduling – Data-Deadline based Least Slack First (DDLSF) slack t (T) = DDL t (T) - t - Estimated Remaining Execution Time t (T) – Earliest Data-Deadline First (EDDF) DDL t (T) t slack t (T) E Time

13 Novel Transaction Scheduling Algorithms: Forced Wait If T can commit before X becomes invalid, T reads X. Otherwise, T waits for the next version of X. Begin T Wait ? Valid X Deadline T Time T : Transaction X : Data t1t1 t2t2 t4t4 t3t3 Valid X R T (X) t5t5 Commit T

14 Novel Transaction Scheduling Algorithms: Similarity Begin T Valid X Deadline T Time T : Transaction X : Data t1t1 t2t2 t3t3 Valid X R T (X) t5t5 Commit ? t4t4 If a version read is similar to the current version, the transaction can commit

15 Novel Transaction Scheduling Algorithms: Forced Wait + Similarity Valid X Valid Y Deadline T Time T : Transaction X,Y : Data t0t0 Begin T t1t1 R T (Y) t2t2 t4t4 R T (X) t5t5 t3t3 Wait? Commit? t6t6 Valid X Valid Y Forced wait is employed when a data is read Similarity is employed at commit time

16 Summary of Experimental Results: Temporal Consistency Maintenance Data-Deadline based policies alone do not improve performance significantly Forced Wait improves performance significantly (reduces missed deadline percentage up to 50% ) Similarity improves performance (reduces missed deadline percentage up to 30%) When combined with Similarity, Forced Wait plays a dominant role in enhancing performance

17 Proposed Solutions & Results Maintaining Temporal Consistency Assign Deadlines & Periods to RT Update Transactions

18 Assign Periods & Deadlines -- Problem & Goals Problem domain: –maintaining temporal validity of real-time data by periodic update transactions Goals: assigning periods and deadlines s.t. –update transactions can be guaranteed to complete by their deadlines –the imposed workload is minimized Problems with traditional approaches Proposed solution and performance results Summary

19 Problem : Maintaining Temporal Validity of Real-Time Data V t+V t V : Validity length t’+V t’ V Real-time data refreshed by periodic update sensor transactions – X has to be refreshed before its validity interval expires – validity duration updated upon refresh How to maintain the validity of data while minimizing the workloads due to update transactions ?

20 Traditional Approach: Half-Half -- Sample at twice the rate of change: P = D = V/2 Definition: X : Real-Time Data V : Validity Interval Length T : Trans Updating X P : Period of T D : Deadline of T C : Computation Time of T V Problem : Imposes unnecessarily high workload t P=D t+V/2 t +V t Observation : Data validity can be guaranteed if Period + Deadline <= Validity Length Workload : C / P = 2C / V P=D D t t+V/2 t +V t P P more than V/2 & D less than V/2 !

21 Deriving Deadlines and Periods: Intuition of More-Less Principle Data validity can be guaranteed if  Period + Relative Deadline <= Validity Length (1) To reduce the workload (C/P) imposed by T without violating (1) :  Increase period to be more than half of validity length  Decrease relative deadline to be less than half of validity length If relative deadline <= period,  deadline monotonic scheduling is an optimal fixed priority scheduling alg

22 For a set of transactions {T i } 1 <= i <= m Validity Constraint (to ensure data validity) : Period + Deadline <= Validity Length More-Less Principle: Definition Deadline Constraint (to reduce workload) : Computation Time <= Deadline <= Period Schedulability Constraint (by deadline monotonic) : Response time of the 1 st instance <= Deadline Note: 1 st instance response time is the longest response time if all transactions start at same time Question: Is More-Less always better than Half-Half ?

23 More-Less Principle: P & D T1 T2 C V Parameters Half-Half T1 T2 D P Utilization : 1/ /10 = More-Less (priority order: T1 > T2) T1 T2 D P Utilization : 1/2 + 2/16 = < Determining deadline and period of a transaction in More-Less: Deadline: D = Response time of the 1st instance; Period : P = Validity Length - Deadline; Does the priority order T2 > T1 produce same P and D? Is more-less always better than half-half ?

24 More-Less Better than Half-Half Theorem: {T i } can be scheduled by (Half-Half, any fixed priority scheduling alg)  {T i } can be scheduled by (More-Less, Deadline Monotonic scheduling) The reverse is not true Question: How to determine transaction priority order s.t. load is minimized under More-Less ?

25 Shortest Validity First Shortest Validity First (SVF) –assign orders to transactions in the inverse order of validity interval length resolve ties in favor of a transaction with less slack (V - C) –is optimal under certain restrictions

26 Shortest Validity First: Summary Restrictions: 1)  C i <= min (V j /2) 2) C 2 - C 1 <= 2(V 2 - V 1 ) (i.e., the increase of computation time is less than twice the increase in validity interval length), If restrictions (1) & (2) hold, SVF is optimal If only restriction (1) holds, SVF is near optimal In general, SVF is a good heuristic (shown in experiments)

27 Applications & Experimental Results Application of More-Less: Avionics Systems Experimental Parameters : C i = [5,15] ms & V i = [4000,8000] ms Experimental results show that More-Less provides lower CPU workload and better schedulability than Half-Half No. of Sensor Transactions CPU Workload More-Less can deal with 30% more load !!!

28 Experiments: Coscheduling of Mixed Workloads -- Performance of Sensor Transactions No. of Sensor Transactions Missed Deadline Ratio Mixed Workloads: Sensor Transaction Class &Triggered Transaction Class Transaction Scheduling: High priority class: Sensor -- Deadline monotonic scheduling Low priority class:Triggered -- Earliest deadline first scheduling

29 Experiments: Coscheduling of Mixed Workloads -- Performance of Triggered Transactions No. of Sensor Transactions (Triggered Trans Arrival Rate = 10 trans/s) Missed Deadline Ratio More-Less improves perf. of triggered transactions by 80%!

30 Conclusions Development and evaluation of efficient and novel approaches to temporal consistency maintenance –concept of data-deadline and related scheduling algorithms –forced-wait –similarity Development, analysis, and evaluation of More-Less, an efficient principle for maintaining real-time data validity