1. 2006-05-18 10:00 – 14:00 @102A1 Esaki Laboratory Pre-Workshop Discussion

2. Chapter Introduction (22 pages) –Overview –Evaluation –Characteristics Accident (3 pages) –Accidents already occurred –Accidents we must assume Discussion (18 pages)

3. INTRODUCTION Historical System (weather.hongo) –Overview –Evaluation –Characteristics New Approach (live-e.hongo) –Concept (Sensor Abstraction) –System Overview –Characteristics

4. Live E! Project Overview A Sensor Network Infrastructure E stands for three meanings –Earth –Environment –Electronics Application –Public Service (such as for Natural Disaster) –Business –Education

5. INTRODUCTION Historical System (weather.hongo) –Overview –Evaluation –Characteristics New Approach (live-e.hongo) –Concept (Sensor Abstraction) –System Overview –Characteristics

6. Historical System weather.hongo.wide.ad.jp Windows 2003 Server MSSQL APP Upload with SOAP Vaisala Sensors Web Service(SOAP/XML) WD WDDBLog Web Interface

7. Vaisala Sensor(WXT510) Measure Item –Temperature –Humidity –Pressure (Barometer) –RainFall –WindDir –WindSpeed –(CO2) Serial Interface iLon100 WXT510 RS232C Lonworks Internet weather.hongo SOAP WXT510s are connected to weather.hongo through Lonworks.

8. Behavior of Data Upload neuron_idWNIIDTimeTemperatureHumidityPressureRainFallWindDirWindSpeed TABLE WD TABLE WDDBLog CO2 neuron_idTimeTemperatureHumidityPressureRainFallWindDirWindSpeedCO2 WeatherDataWrite Update WDInsert WDDBLog Vaisala Sensor Data Current Data Table Archive Data Web Service Each sensor has multiple sensor type DATA MODEL Inside weather.hongo

9. API (weather.hongo) neuron_idWNIIDTimeTemperatureHumidityPressureRainFallWindDirWindSpeed TABLE WD TABLE WDDBLog CO2 neuron_idTimeTemperatureHumidityPressureRainFallWindDirWindSpeedCO2 Current Data Table Archive Data WeatherDataReadAllWDReadLog Web Service Select WDSelect WDDBLog XML The Latest Sensor Data Archived Sensor Data

10. Evaluation (weather.hongo) The Number of Sensor –About 50. Upload Frequency –Temperature, Humidity, Pressure, RainFall Every 5 minutes –WindDir, WindSpeed Every 20 seconds WDDBLog Table Size –Increases about 250,000 rows per day. –Current Row Size is 39,863,725 (2006-05-16 10:12:00) –A Query takes about 40 seconds.

11. Characteristics (weather.hongo) Designed by Echelon. What the Sensor is: –Vaisala sensor (at least, sensor on Lonworks) –Each sensor has multiple sensor type. –Sensors can be identified by NeuronID. No sensor meta-information –Sensor meta-information (sensor profile) must be provided by other systems. Centralized –No scalability, No robustness

12. INTRODUCTION Historical System (weather.hongo) –Overview –Evaluation –Characteristics New Approach (live-e.hongo) –Concept (Sensor Abstraction) –System Overview –Characteristics

13. New Approach (live-e.hongo) What the Sensor is: (Sensor Abstraction) –Sensor ID of Live E! NeuronID is base on Lonworks system. We must make a new ID system. Globally-unique –A single sensor type. Each sensor must have only one value. A Multiple sensor must be divided to single sensors. –Profile Meta-information that describes sensor.

14. Sensor ID sensor_id –Globally-unique ID –Format (Syntax Rule) Domain Name + /SensorModel/Location/SensorType –Example live-e.org/WXT510/0300000431b2/Temperature live-e.unl.im.dendai.ac.jp/WM918/kanda/12/roof/Temperature live-e.unl.im.dendai.ac.jp/WM918/kanda/12/roof/Humidity live-e.unl.im.dendai.ac.jp/WM918/kanda/12/roof/Pressure mew.co.jp/FS-Va-01/aoyama_elementary_school/Temperature Semantics –No meaning –Just a String

15. Sensor Data Each sensor has a single value. –The value can be determined by “sensor_id” and sampled “time”. value = F( sensor_id, time) Ex. The result of GetCurrentData(sensor_id); 23.5

16. (Major) Sensor Profile sensor_id (Ex. live-e.org/WXT5 ---/Temperature) sensor_model (Ex. WXT510) sensor_type (Ex. Temperature) latitude (Ex. 35.712194) longitude (Ex. 139.76775) location (Ex. 東大情報基盤センター ) address (Ex. 東京都文京区弥生 2-11-16)

17. Sensor Profile Example GetProfile(“live-e.org/WXT510/ --- /Temperature”); live-e.org/WXT510/03000005c3a2/Temperature vaisala WXT510 Temperature 139.76775 35.712194 東大情報基盤センター 東京都文京区弥生 2-11-16

18. API (live-e.hongo) String GetCurrentDataAll(); String GetCurrentData(String sensor_id); String GetCurrentDataByAreaRect(double,double,double,double); String GetCurrentDataByType(String sensor_type); String GetCurrentDataOf(String sensor_id_like); String GetDataByTimespan(String,Date,Date); String GetProfileAll(); String GetProfile(String sensor_id); String GetProfileByAreaRect(double,double,double,double); String GetProfileByType(String sensor_type);

19. live-e.hongo.wide.ad.jp weather.hongo APP Web Service(SOAP/XML) Debian Etch PostgreSQL8.1 Java+Axis WM918 FS-Va-01 WXT510 live-e.hongo

20. Weather Station(WM918) Measure Item –Temperature –Humidity –Pressure –RainFall –WindDir –WindSpeed WM918 PC RS232C Internet live-e.hongo SOAP

21. Field Server (FS-Va-01) Measure Item –Temperature –Humidity –Pressure –RainFall –WindDir –WindSpeed –Solar_Radiation –Soil_Moisture –Soil_Temperature –Photo FS-Va-01 Internet live-e.hongo SOAP DataCenter Matsushita-Denko

22. Data Upload Protocol Protocol –SOAP Data Format –XML-like format live-e.unl.im.dendai.ac.jp/WM918/kanda/12/roof/Temperature 2006-05-12T04:34:42.0000+09:00 23.1 ・・・ continues

23. Behavior of Data Upload sensor_idtimevalue TABLE sensor_data_currentTABLE sensor_data_log sensor_idtimevalue DB_INSERT Update sensor_data_current Sensors Data Web Service data_index Insert sensor_data_log

24. Characteristics (live-e.hongo) Improved –Sensor Abstraction –Profile Management The same as weather.hongo –Directly connected to the Database Upload Message Flow Query Processing –Centralized All the sensors are managed in one database No scalability

25. Accident Accidents Already Occurred –The Accident of New Measure Item –The Accident of Software Update Accidents we must assume. –Server Crush –Rigid System Difficulty of introducing new architecture or new model. –Database-oriented –Sensors are deployed without managing

26. The Accident of New Measure Item There is another data archive system. http://www.cnl.k.u-tokyo.ac.jp/~koba/live_e/index.php –This system collects data by parsing HTML of web interface of weather.hongo. http://weather.hongo.wide.ad.jp/WDShow/WDShow.aspx One day, a new measure item “DayRainFall” was added. –Of course, the HTML structure was changed. The archive system went into out of order.

27. The Accident of Software Update Sensor data upload software had a bug. –It had been already published. –Isi removed the bug. Published again Announced One of us deployed the old version. –The fact was found out by human check. –live-e.hongo could not detect it. WM918 PC RS232C Internet live-e.hongo SOAP

28. Discussion

29. Discussion Agenda 1.Evaluate the system in many aspects –Scalability, Robustness,..., etc. 2.Introduce related work. 3.Determine what we focus on... pub-sub? or range query? or any other? 4.Design prototype system 5.Determine what to do at the workshop

30. Evaluate Point Scalability Robustness Flexibility Reliability Efficiency Functionality Availability Feasibility Heterogeneity Management Automation Simplicity

31. Scalability What is O(N)? N: The Number of Node. Resource Capacity VS O(N). –R>>O(N) at N=1,000,000 → Scale –R<O(N) at N=1,000,000 → Does Not Scale Current System (live-e.hongo) –Does not scale!! What do we need to introduce to obtain scalability?

32. Robustness (Tolerance) What would happen if... ? –A server crushed. –Disconnected. Current System –It must be a disaster. Lose the data archived Out of Service Cannot collect sensor data Take time to comeback What do we have to do to obtain tolerance?

33. Flexibility Is it easy to extend... ? – easy to add some functions ? – easy to add resources ? (such as disk space or CPU) Current System –Rigid. –Each parts are tightly dependent. –We need lots of energy to remodel. How do we design a flexible system?

34. Reliability Is the data correct? –Can we trust the value of sensors? –How deep can we detect irregular behavior? –Who has the responsibility of the data? –Certification Current System –Trust all the sensor even if one of them said -324 ℃ What do we need to improve reliability?

35. Efficiency Low cost? –Time, Money, Power, Space... –It is said... The cost is: big central > small distributed Current System –Unknown It is important to think of efficiency when we design system.

36. Functionality Complex Query OK? –Range query Current system –Good How do we obtain functionality within the system?

37. Availability Is Live E! system opened? Example of open system –Internet –DHT Current System –Not opend –Only the system manager can control the system. How do we provide availability?

38. Feasibility Can we execute the plan? –Human Resource –Money –Law We have to consider whether we can do it or not.

39. Heterogeneity Management How do we manage sensors uniformly? –Abstraction? –Namespace Management?

40. Automation What do we need to design system to work automatically? –When software updated. –When resource added.

41. Simplicity Simpler, More User Few would like to learn complex access methods.

42. RELATED WORK Content-Based Network –Publish-Subscribe Service DHT (Distributed Hash Table) –i3 (publish-subscribe on DHT) –PHT (range query on DHT) Caching Live Update Sensor Abstraction

43. What we focus on... Publish - Subscribe Model Range Query Data Mining The Abstraction of Sensor Live Update

44. Prototype System Depends on what we focus on... Talk about these: –Architecture –Interface and Protocol –Evaluation –Who

45. At the Workshop Evaluate the system. –What do you want to do? –I want to do something aggressive. Ex. Increase virtual sensors until the system fails.