1. Network Computing Laboratory HiFi Systems: Network-Centric Query Processing for the Physical World Michael J. Franklin, Shawn R. Jeffrey, et al UC Berkeley TelegraphCQ Team 2 nd CIDR Conf. 2005

2. Korea Advanced Institute of Science and Technology Table of Contents One line Comment Motivating Scenario HiFi System with CSAVA processing stage Internal Architecture of HiFi Node Critiques New Idea -1,2

3. Korea Advanced Institute of Science and Technology One line Comment It’s a preliminary work describing the group’s vision to distribute their TelegraphCQ system to a hierarchical network

4. Korea Advanced Institute of Science and Technology Motivating Scenario – Supply Chain Management “Smart Shelves” continuously monitor item addition and removal. Info is sent back through the supply chain.

5. Korea Advanced Institute of Science and Technology Hi Fan-In system Ursa-Minor (TinyDB-based) Ursa-Major (TelegraphCQ w/Archiving) Mid-tier Stargate Mid-tier Processing Node

6. Korea Advanced Institute of Science and Technology Characteristics of HiFi Systems High Fan-In, globally-distributed architecture Large data volumes generated at edges Filtering and cleaning must be done there Successive aggregation as you move inwards Summaries/anomalies continually, details later Strong temporal focus Strong spatial/geographic focus Streaming data and stored data Integration within and across enterprises

7. Korea Advanced Institute of Science and Technology A View on this example Filtering, Cleaning, Alerts Monitoring, Time-series Data mining (recent history) Archiving (provenance and schema evolution) Geographic Scope local global Several Readers Regional Centers Central Office

8. Korea Advanced Institute of Science and Technology Headquarters Regional Centers Warehouse Warehouse Doors Receptor High fan-in system levels with associated CSAVA processing stages RFID Clean Remove Anomalies Smooth Interpolate for lost/garbled readings Arbitrate Remove duplicates Validate Correlate with business rules Analyze Tactical decision support

9. Korea Advanced Institute of Science and Technology Internal Architecture of a HiFi node Metadata Repository Data Stream Processor Cache Manager Data Listener Resource Manager Query Dispatcher Local View Manager Query Placement Service Query Listener Control Manager Data Disseminator Query Planner DSP Manager Archive Manager Logical Query Planner Physical Query Planner HiFi Glue Data Flow Query Flow Control Flow

10. Korea Advanced Institute of Science and Technology Critiques Strong Point They classify and formulate five distinct data processing stage They develop the prototype system (in VLDB 05) Weak Point Designing MDR is critical but no initial effort is done No new system requirement Solutions are not technically deep

11. Korea Advanced Institute of Science and Technology New Idea - 1 Data SourceCQ engineWeb Server SP Accel Clients Filtered out By-passing Buffering

12. Korea Advanced Institute of Science and Technology New Idea – related to SPAccel Designing front-end component (Cache??) Filtering out unwanted input data By-passing data matching query predicates Buffering data for windowed queries (views) or distributed queries Buffering Query Results

13. Korea Advanced Institute of Science and Technology Issues expected Cache replacement mechanism How to index cached elements What to cache? How much?

14. Korea Advanced Institute of Science and Technology New Idea -2 processing stream data for OLAP queries OLTPOLAP Users Clerk, IT professionalKnowledge worker Function Day to day operationsdecision support DB design application-orientedsubject-oriented Data current, up-to-datehistorical, summarized detailed, flat relational multidimensional isolatedintegrated, consolidated Usage repetitivead-hoc Access read/write, lots of scans index/hash on prim. key Unit of work short, simple transactioncomplex query #Records accessed tensmillions #Usersthousandshundreds DB size100MB-GB100GB-TB Metrictransaction throughput query throughput/response

15. Korea Advanced Institute of Science and Technology A Sample Data Cube sum USA Canada Mexico CountryCountry Date Product CD video camera 1Q2Q3Q4Q

16. Korea Advanced Institute of Science and Technology New Idea - 2 Stream data in terms of OLAP domain OLAP queries are Inherently multidimensional Spans a long time Need data from multiple sources Processing OLAP queries are Memory intensive Computation intensive

17. Korea Advanced Institute of Science and Technology Naïve Solution Pre-computing popular computation path

18. Korea Advanced Institute of Science and Technology Supplementary Silde Cleaning CREATE VIEW cleaned_rfid_stream AS ( SELECT receptor_id, tag_id FROM rfid_stream rs WHERE read_strength >= strength_T) Smoothing CREATE VIEW smoothed_rfid_stream AS ( SELECT receptor_id, tag_id FROM cleaned_rfid_stream GROUP BY receptor_id, tag_id HAVING count(*) >= count_T)