SWiM Panel on Engine Implementation Jennifer Widom.

Slides:



Advertisements
Similar presentations
CPSCG: Constructive Platform for Specialized Computing Grid Institute of High Performance Computing Department of Computer Science Tsinghua University.
Advertisements

Files Used to transfer data to and from disk. Opening an Output File Stream #include // File stream library. ofstream outfile;// Declare file stream variable.
Starfish: A Self-tuning System for Big Data Analytics.
Categories of I/O Devices
P3- Represent how data flows around a computer system
MapReduce Online Created by: Rajesh Gadipuuri Modified by: Ying Lu.
1 11. Streaming Data Management Chapter 18 Current Issues: Streaming Data and Cloud Computing The 3rd edition of the textbook.
What’s New in BMC ProactiveNet 9.5?
WS-VLAM: Towards a Scalable Workflow System on the Grid V. Korkhov, D. Vasyunin, A. Wibisono, V. Guevara-Masis, A. Belloum Institute.
Fjording the Stream: An Architecture for Queries over Streaming Sensor Data Samuel Madden, Michael J. Franklin University of California, Berkeley Proceedings.
Threads - Definition - Advantages using Threads - User and Kernel Threads - Multithreading Models - Java and Solaris Threads - Examples - Definition -
Building a Data Stream Management System Prof. Jennifer Widom Joint project with Prof. Rajeev Motwani and a team of graduate studentshttp://www-db.stanford.edu/stream.
1 PODS 2002 Motivation. 2 PODS 2002 Data Streams data sets Traditional DBMS – data stored in finite, persistent data sets data streams New Applications.
Oracle 10g Database Administrator: Implementation and Administration Chapter 14 Proactive Maintenance.
The Stanford Data Streams Research Project Profs. Rajeev Motwani & Jennifer Widom And a cast of full- and part-time students: Arvind Arasu, Brian Babcock,
© 2004, D. J. Foreman 2-1 Concurrency, Processes and Threads.
SWIM 1/9/20031 QoS in Data Stream Systems Rajeev Motwani Stanford University.
Resource Management Reading: “A Resource Management Architecture for Metacomputing Systems”
October 30, 2008 Extensible Workflow Management for Simmod ESUG32, Frankfurt, Oct 30, 2008 Alexander Scharnweber (DLR) October 30, 2008 Slide 1 > Extensible.
Stream Handling Streams - means flow of data to and from program variables. - We declare the variables in our C++ for holding data temporarily in the memory.
1 DAN FARRAR SQL ANYWHERE ENGINEERING JUNE 7, 2010 SCHEMA-DRIVEN EXPERIMENT MANAGEMENT DECLARATIVE TESTING WITH “DEXTERITY”
Efficient Scheduling of Heterogeneous Continuous Queries Mohamed A. Sharaf Panos K. Chrysanthis Alexandros Labrinidis Kirk Pruhs Advanced Data Management.
Self Adaptivity in Grid Computing Reporter : Po - Jen Lo Sathish S. Vadhiyar and Jack J. Dongarra.
Upgrade to Real Time Linux Target: A MATLAB-Based Graphical Control Environment Thesis Defense by Hai Xu CLEMSON U N I V E R S I T Y Department of Electrical.
OracleAS Reports Services. Problem Statement To simplify the process of managing, creating and execution of Oracle Reports.
An Integration Framework for Sensor Networks and Data Stream Management Systems.
Data Stream Systems Reynold Cheng 12 th July, 2002 Based on slides by B. Babcock et.al, “Models and Issues in Data Stream Systems”, PODS’02.
1 Dynamically Adaptive Distributed System for Processing CompleX Continuous Queries Bin Liu, Yali Zhu, Mariana Jbantova, Brad Momberger, and Elke A. Rundensteiner.
The τ - Synopses System Yossi Matias Leon Portman Tel Aviv University.
CS848 Class Project: A Survey on QoS for Multi-tier Web Systems Huaning(Mike) Nie
Marcelo R.N. Mendes. What is FINCoS? A Java-based set of tools for data generation, load submission, and performance measurement of event processing systems;
CHT Project Progress Report 10/07 Simon. CHT Project Develop a resource management scheduling algorithm for CHT datacenter. ◦ Two types of jobs, interactive/latency-
1 STREAM: The Stanford Data Stream Management System STanfordstREamdatAManager 陳盈君 吳哲維 林冠良.
Giuseppe Codispoti INFN - Bologna Egee User ForumMarch 2th BOSS: the CMS interface for job summission, monitoring and bookkeeping W. Bacchi, P.
Chair for Computer Science 6 (Data Management) Friedrich-Alexander-University of Erlangen-Nuremberg Michael Daum, Frank Lauterwald, Philipp Baumgärtel,
Data Stream Management Systems
Chapter 4 – Threads (Pgs 153 – 174). Threads  A "Basic Unit of CPU Utilization"  A technique that assists in performing parallel computation by setting.
Introduction to Databases
MROrder: Flexible Job Ordering Optimization for Online MapReduce Workloads School of Computer Engineering Nanyang Technological University 30 th Aug 2013.
Memory Hierarchy Adaptivity An Architectural Perspective Alex Veidenbaum AMRM Project sponsored by DARPA/ITO.
CS4432: Database Systems II Query Processing- Part 2.
Shell Interface Shell Interface Functions Data. Graphical Interface Graphical Interface Command-line Interface Command-line Interface Experiments Private.
Marcelo R.N. Mendes. What is FINCoS? A set of tools for data generation, load submission, and performance measurement of CEP systems; Main Characteristics:
Lecture on Central Process Unit (CPU)
1 University of Maryland Runtime Program Evolution Jeff Hollingsworth © Copyright 2000, Jeffrey K. Hollingsworth, All Rights Reserved. University of Maryland.
Parameter Sweep and Resources Scaling Automation in Scalarm Data Farming Platform J. Liput, M. Paciorek, M. Wrona, M. Orzechowski, R. Slota, and J. Kitowski.
Chapter 2 Data Manipulation © 2007 Pearson Addison-Wesley. All rights reserved.
CPU-GPU Collaboration for Output Quality Monitoring Mehrzad Samadi and Scott Mahlke University of Michigan March 2014 Compilers creating custom processors.
I/O: Input-Output By: Tommy Zeng. What is I/O? I/O – short for “Input – Output” How a computer interacts with its users Input – gets information from.
Marcelo R.N. Mendes. What is FINCoS? A Java-based set of tools for data generation, load submission, and performance measurement of event processing systems;
Joint Institute for Nuclear Research Synthesis of the simulation and monitoring processes for the data storage and big data processing development in physical.
Holding slide prior to starting show. Scheduling Parametric Jobs on the Grid Jonathan Giddy
10/2/20161 Operating Systems Design (CS 423) Elsa L Gunter 2112 SC, UIUC Based on slides by Roy Campbell, Sam King,
WORKING OF SCHEDULER IN OS
Processes and threads.
BOSS: the CMS interface for job summission, monitoring and bookkeeping
BOSS: the CMS interface for job summission, monitoring and bookkeeping
Wayne Wolf Dept. of EE Princeton University
课程名 编译原理 Compiling Techniques
Scalability Tests With CMS, Boss and R-GMA
Process Virtualization. Process Process is a program that has initiated its execution. A program is a passive entity; whereas a process is an active entity.
Data Stream Management System (DSMS)
ColdFusion Performance Troubleshooting and Tuning
Software maintenance.
Lecture Topics: 11/1 General Operating System Concepts Processes
GATES: A Grid-Based Middleware for Processing Distributed Data Streams
Concurrency, Processes and Threads
Adaptive Query Processing (Background)
Overview of Computer system
CS Introduction to Operating Systems
Presentation transcript:

SWiM Panel on Engine Implementation Jennifer Widom

Query Processing Architecture Input Data Streams Users issue continuous and ad-hoc queries Administrator can monitor query execution and adjust run-time parameters Applications register continuous queries Output Stream   X X Waiting Op Ready Op Running Op Synopses Query Plans Scheduler

Issues Memory versus disk for intermediate state More generally: More generally: Is there a clear latency-accuracy-storage tradeoff? What is the objective of the scheduler? More generally: More generally: What are the performance metrics for continuous queries?

Issues (cont’d) How important is a built-in approach to approximation? More generally: More generally: Will data streams really be all that fast? If so, can’t we expect the source to filter? How adaptive does execution really need to be? More generally: More generally: Will data streams really be all that variable?

Issues (cont’d) What does the API for a Data Stream Management System look like? –For registering queries –For seeing results –For understanding approximation –For monitoring the system