1. Real-Time Databases and Data Services Krithi Ramamritham, Sang Son, Lissa Dipippo

2. Satisfying QoS/QoD requirements Transaction exec times & data access patterns are not known a priori, but vary dynamically Transaction timeliness & data freshness may pose conflicting requirements

3. Motivation Increasing amount of sensor data Agile manufacturing, target tracking, surveillance, structual monitoring, weather forecast, traffic control,... Wireless sensors push the limit Desired real-time data service Timely query/transaction processing using fresh data Existing databases based on closed-world assumption No notion of sensor data service Timing guarantee or data freshness not considered

4. Real-time data Service...... User Requests Poor QoS Real-Time Transaction Processing Real-Time Transaction Processing Overload Data Conflicts Overload Data Conflicts QoS Guarantees? Update Streams

5. Standford real-time information processor (STRIP) Addressed the problem of balancing between the freshness & timeliness requirements Soft real-time: Maximize #transactions finishing within the deadlines using fresh data Apreriodic updates Freshness metrics MA (Max Age): Similar to avi Unapplied Updates: Be optimistic & assume a data is fresh unless an update has been received but not installed yet

6. STRIP Four scheduling algorithms Update first Transaction first Split Updates Update to high importance data will be installed upon arrival; otherwise, scheduled after transactions On Demand Transaction has precedence, but searches update queue upon accessing a stale data

7. QMF (QoS-aware Miss ratio and Freshness guarantees) Guaranteed Quality of real-time data service Timing guarantee Support desired deadline miss ratio or response time Home-land security problem, traffic jam,... Quality of Data Data freshness (temporal consistency) Reflect the time-varying real-world status

8. Challenges Unpredictable workloads, access patterns Sudden increase of service requests Severe data contention Conflicts between timing & freshness requirements Some deadline misses/freshness violations are inevitable

9. Key Ideas Feedback control Robustness against unpredictable workloads Widely applied to software peformance management Feedback Control of Computing Systems, Joseph L. Hellerstein et., Wiley Interscience Cost-effective freshness management Novel freshness metrics Cost-benefit model Adaptive update policy Admission control Tardy commits are worse than dropping

10. QMF Architecture U. Thresh. Manager U. Thresh. Manager MR/Util. Controllers MR/Util. Controllers QoD Manager … … Admission Control Admission Control User Trans. Update Streams Ready Queue Trans. Handler Trans. Handler … Dispatch Abort/Restart Block Queue MR, Util. UU  U new

11. Real-Time Database Model Main memory database model TimesTen, STRIP, DataBlitz, Polyhedra Firm deadlines Sensor data updates Periodic Jitter? User transactions Arithmetic/logical operations considering the current real-world state

12. QoS Metrics: Miss Ratio Admitted transactions Average Transient Overshoot (V) Settling time (Ts) V Ts

13. QoS Metrics: Data Freshness Database Freshness: Set of temporal data Perceived Freshness: Set of temporal data accessed by timely transactions RTDB

14. Feedback-Based Miss Ratio/Utilization Control MR Controller MR Controller RTDB Error Current MR Threshold WW + _ Transactions

15. Feedback Control Details PI controllers Sampling period Settling time, overshoot 5sec Overhead QoD fluctuations Think time Tuning Root Locus method in Matlab

16. Cost-Effective Updates Access Update Ratio (AUR) AUR[i] = Access Freq[i] / Update Freq[i] Access Frequency Benefit Update Frequency Cost

17. Cost-Benefit Model Hot Data Cold Data AUR = 1

18. Adaptive Update Policy D = D imm D imm AUR < 1 AUR =1 D od Underutilized StateModerately loaded StateOverloaded State

19. Future Work More flexible QoD metrics and adaptive QoD management schemes More effective feedback control & QoS/QoD management approaches RTDB testbed Apply RTDB techniques to e-commerce applications 3-tier systems: Web server, application server & back-end database Mobile, hand-held RTDB

20. Real-time data service in embedded applications Data needs of embedded applications become more complicated Traffic control Weather forecast Put RT data server in the frontline Collect info rather than raw data and disseminate in a timely manner Key issues such as power/energy management should be reconsidered considering real-time data service semantics

21. Wireless Sensor Networks View WSN as a distributed DB (???) TinyDB, Cougar, SINA,... I don’t agree, but you can have different opinions... Real-time event detection Real-time routing Data aggregation QoD in WSNs – anything more than freshness? data values? accuracy? SNR? Security & Trustworthiness Scalability of depandability Resilience of availability

22. Vision (Example) Real-time traffic control using sensor data and weather information

23. Questions?