1. Sensor Systems and Real World Data S. Muthukrishnan David Rosenbluth

2. Outline Theme: –Use sensors for behavioral and physiological measurements. –Collect data continuously for a significant period of time in natural environments. (minimally intrusive) –Clean data and analyze for patterns. –Build applications based on sensor data. How this course will be organized.

3. Sensors: Our Focus Our Focus: Measuring Individual Behavior and physiology in natural situations (mobility, robustness) using multiple sensors. What is NOT our focus: Not measuring temp, pressure etc using motes at geographic locations. Not about intrusions. Not fire sensors in buildings. Not sensors in airplane motors or buildings.

4. Applications We will look following areas: –Rehabilitative medicine –Behavioral Risk factors in Disease –Epidemiology: Disease Transmission –Sociology: Measuring Social Networks –Behavioral Predictors of Seizure in Epilepsy –Diagnostics for ADD We will discuss a variety of applications.

5. Sensors: Types Electromagnetic –Photosensors, RF –Current, voltage – EKG, EMG, EEG, … Mechanical –Physical pressure/force –accelerometer –Sound –Heat Chemical –Smell –Taste –pH –glucometer Biological –DNA –T cell count Nuclear

6. Sensors: sizes Huge. Examples: –biosensors in airports, Medium: –cleveland medical Tiny: –motes. Stationary, mobile with people, automatic mobile,…

7. Nature of sensor data Measurement Accuracy –Samples per Second: Streaming sampling rate vs. stored sampling rate –Bits per Sample Channels per sample * bits per channel –Small: core body temp –Medium: audio –Large: images Bandwidth requirements/Storage requirements

8. Difficulties of Real-world Sensor Data Sensor Signals are a complex mixture of influences from different sources and different types of phenomena. Applications have diverse requirements for sensor data: accuracy, timeliness, representation. Must be able to cope with large quantities of data. Interesting events may be sparse.

9. How to collect sensor data? In some of the cases, sensor may be connected by wire to IP network (camera at home for security), but in our personal space data applications, wireless connectivity. What wireless environments? Cellular (GPRS, CDMA) or LAN (802.11) or bluetooth. NOT ultra wideband sensors, directional antenna? RFIDs? (active vs passive sensing). Challenges: intermittent connectivity in natural situations; noisy environments.

10. Architecture for sensor data collection Sensor -> bluesentry -> iPAQ -> Server? By bluetooth or 802.11 or cellular or special radio or … Dataflow: Grab, compress, stream or periodic upload? Omitting the idea of processing data at sensors such as summarizing, computing FFT, etc. Not networking sensors, ie, sensors do NOT talk to each other.Not (YET) doing query processing on sensor networks, for example. Bluetooth networking?

11. MIMO Sensor Architectures Accelerometer 802.11 Signal Strength EKG Audio EMG Content Synthesis Data Cleaning Data Transformation Data Chunking Cross Linking Streams Stream Query Processing Resource Management Location Activity Heart Rhythm Identity

12. Adding Meta-Data to Sensor Streams Annotation Indexing Linking Chunking/Fractionation …|X -4 | X -3 | X -2 | X -1 | X 0 | X 1 | X 2 | X 3 | X 4 | X 5 | X 6 | X 7 | X 8 | X 9 | X 10 | X 11 | X 12 | … Walking …|X -4 | X -3 | X -2 | X -1 | X 0 | X 1 | X 2 | X 3 | X 4 | X 5 | X 6 | X 7 | X 8 | X 9 | X 10 | X 11 | X 12 | … …|t -4 | t -3 | t -2 | t -1 | t 0 | t 1 | t 2 | t 3 | t 4 | t 5 | t 6 | t 7 | t 8 | t 9 | t 10 | t 11 | t 12 | … …|X -4 | X -3 | X -2 | X -1 | X 0 | X 1 | X 2 | X 3 | X 4 | X 5 | X 6 | X 7 | X 8 | X 9 | X 10 | X 11 | X 12 | … …|Y -4 | Y -3 | Y -2 | Y -1 | Y 0 | Y 1 | Y 2 | Y 3 | Y 4 | Y 5 | Y 6 | Y 7 | Y 8 | Y 9 | Y 10 | Y 11 | Y 12 | … …|X -4 | X -3 | X -2 | X -1 | X 0 | X 1 | X 2 | X 3 | X 4 | X 5 | X 6 | X 7 | X 8 | X 9 | X 10 | X 11 | X 12 | …

13. Nature of Sensor Data A week of continuous data.: how does it look? Engineering problems: –How to find and interpret events in large quantities of sensor data? –How to annotate data? –How to synchronize timestamps across data streams? –Data quality problems.

14. Applications: Biometrics Gesture Passwords

15. Accel x Accel y Accel z Wireless x Wireless y Wireless Speed Localization

16. Course Groups of people Each group gets a sensor system: sensors, wireless connectivity, iPAQ etc. Each group chooses an application domain and builds infrastructure for collecting continuous data, cleaning, analyzing and building an application. In addition, everyone reads about sensors, applications and shares what they have learned, sometimes presenting papers. We will also have expert speakers.

17. Timeline 1 st Lecture: Muthu + David on intro. 2 nd Lecture: Amit on accelerometer and location applications. 3 rd Lecture: Su on audio measurements, application to annotation, challenges and weaknesses. 4 th Lecture: We begin implementations.

18. Homework 1 [1] Find example(s) of sensors, their manufacturers, cost, type etc not discussed in the lecture. [2] Discuss different architectures for sensor data gathering. [3] Find interesting new applications for sensors, focusing on application areas we listed here. [4] Take one sensor and discuss the entire lifecycle of data from gathering to quality problems to applications with attention to issues that are unique/specific to that sensor. Email: muthu@cs.rutgers.edu, drosenbl@telcordia.com