1 Stream Applications By Michael Stonebraker Adjunct Professor, MIT

2 M.I.T. PurposePurpose  Are we whistling Dixie  Anything that can’t be done by a conventional app server and a DBMS  Reality check on functionality  Anybody need QoS?  Are we whistling Dixie  Anything that can’t be done by a conventional app server and a DBMS  Reality check on functionality  Anybody need QoS?

3 M.I.T. WhoWho  Me  Dennis Shasha  Ted Johnson  Me  Dennis Shasha  Ted Johnson

4 M.I.T. CategorizationCategorization  “The real time enterprise”  No sensors  Current sensor apps  There currently exists an interested user  Future sensor apps  Wishful thinking  “The real time enterprise”  No sensors  Current sensor apps  There currently exists an interested user  Future sensor apps  Wishful thinking

5 M.I.T. Traditional View of the World Operational system Operational system Data Warehouse Periodic data movement Actions Data mining Transactions Bulk data mover Bulk data mover

6 M.I.T. The Traditional View The Traditional View  Oriented toward identifying “pearls of wisdom” that will change strategic long-term business practices  Not toward tactical (real time) business analysis and tuning  To do that, you need real time stream coming out of operational system  Oriented toward identifying “pearls of wisdom” that will change strategic long-term business practices  Not toward tactical (real time) business analysis and tuning  To do that, you need real time stream coming out of operational system

7 M.I.T. ExamplesExamples  Schwab  Has a system to compute the commission on each trade in real time  300K of rule-based code (home brew)  No aggregates; presumably legacy DBMS lookup  Schwab  Has a system to compute the commission on each trade in real time  300K of rule-based code (home brew)  No aggregates; presumably legacy DBMS lookup

8 M.I.T. ExamplesExamples  Fidelity  Trades originate in lots of systems  Get executed in several system  Has a system to route trades in real time  Complains it is too slow (not quite sure why)  Wants prioritization of trades  Fidelity  Trades originate in lots of systems  Get executed in several system  Has a system to route trades in real time  Complains it is too slow (not quite sure why)  Wants prioritization of trades

9 M.I.T. Current Sensor-based Applications  US military (Mitre)  Batallion monitoring  “20 queries” on web site  Mix of history and CQ  Some queries are pretty hard (tell me when 3 of my 4 tanks cross the front line)  US military (Mitre)  Batallion monitoring  “20 queries” on web site  Mix of history and CQ  Some queries are pretty hard (tell me when 3 of my 4 tanks cross the front line)

10 M.I.T. Current Sensor-based Applications  US military (Mitre)  QoS crucial (drop everything to work on incoming missile)  Main bottleneck – network bandwidth  But military apps are not politically correct  US military (Mitre)  QoS crucial (drop everything to work on incoming missile)  Main bottleneck – network bandwidth  But military apps are not politically correct

11 M.I.T. Current Sensor-based Applications  Environment monitoring  Fish pills  To monitor mercury level in water  Whistling Dixie  Environment monitoring  Fish pills  To monitor mercury level in water  Whistling Dixie

12 M.I.T. Future Sensor-based Applications – electronic property stickers  MIT library  Mis-shelved books  And stolen books  Whistling Dixie  MIT library  Mis-shelved books  And stolen books  Whistling Dixie

13 M.I.T. Future Sensor-based Applications – electronic property stickers  MIL/LCS  Lost projectors  Stolen laptops  Whistling Dixie  MIL/LCS  Lost projectors  Stolen laptops  Whistling Dixie

14 M.I.T. Future Sensor-based Applications – supply chain management  Gilette  Bought 10m “dimes”  Presumably to put on retail bundles  For theft?  For supply chain replenishment  May be whistling Dixie  Gilette  Bought 10m “dimes”  Presumably to put on retail bundles  For theft?  For supply chain replenishment  May be whistling Dixie