1. 모바일 센서 네트워킹 소개 및 최근 연구 동향 이의진 KAIST 지식서비스공학과

2. Smart Phone/Pad Sensor Applications Resizing screen/tilt Environment adjustment of apps for user’s comfort – Adjustment in cinema, movement prediction Gaming Augmented Reality (AR) – AR Gaming – AR Navigation Bar codes Geo–tagging, recommendations.. Network of objects, locations and people, 3D social networking? Distributed sensor systems – Noise mapping, traffic information, etc. And anything you can imagine…

3. 목차 스마트폰 센싱 기초 – 스마트폰 센서 및 프로그래밍 – 위치 센서 – 무선네트워킹 모바일 센싱 응용 – Vehicular Sensing – People Sensing

4. I. 스마트폰 센싱 기초 이의진 KAIST 지식서비스공학과 Slides based on: Sensor Fusion on Android Devices – A Revolution in Motion Processing, David Sachs, Feb. 2010 Android Sensors, Stefan Varga, Michal Kostic, Oct. 2010

5. Contents Elements of mobile sensor networking Programming Android sensor/location service – Setting up sensor/location service – Processing events How do sensors work? – Accelerometer, Magnetometer, Gyroscope, Light Sensor, Proximity Sensor, GPS, Microphone, etc. How do location services work? – GPS, cell-tower location, Wi-Fi positioning

6. Elements of Mobile Sensor Networking network infrastructure Mobile sensors r vehicles, smartphones r run sensing applications r may be stationary (non- mobile) or mobile

7. Elements of Mobile Sensor Networking network infrastructure Wireless nets r mobile to base station r base station: e.g., cell towers, 802.11 access points; connected to a wired network r (also, mobile-to-mobile: Bluetooth, Wi-Fi ad hoc) r multiple access protocol coordinates link access r various data rates, transmission distance Bluetooth, WiFi Ad Hoc Wi-Fi, 3G/LTE/WiMax

8. Elements of Mobile Sensor Networking network infrastructure Computing infra r back-end platform for data processing and web services r could be part of network infra (or independent) r centralized or de- centralized infra

9. Wireless Networking Standards Indoor 10-30m Outdoor 50-200m Mid-range outdoor 200m – 4 Km Long-range outdoor 5Km – 20 Km.056.384 1 4 5-11 54 GSM, IS-95, CDMA 2G UMTS/WCDMA, CDMA2000 3G 802.15 802.11b 802.11a,g UMTS/WCDMA-HSPDA, CDMA2000-1xEVDO 3G cellular enhanced LTE, 802.16 (WiMAX) 200 802.11n Data rate (Mbps) All IP Wi-Fi Bluetooth 4G

10. Contents Elements of mobile sensor networking Programming Android sensor/location service – Setting up sensor/location service – Processing events How do sensors work? – Accelerometer, Magnetometer, Gyroscope, Light Sensor, Proximity Sensor, GPS, Microphone, etc. How do location services work? – GPS, cell-tower location, Wi-Fi positioning

11. Smart Phone/Pad Sensors Nexus OneNexus SiPhone4 Samsung Galaxy S HTC Incredible Galaxy Tab/ iPad2 Accelerometer OOOOOO Magnetometer OOOOOO Gyroscope OO?O Light OOOOOO Proximity OOOOOO Camera OOOOOO Voice OOOOOO GPS OOOOOO

12. Android System Architecture

13. Android API (I) Package: android.hardware Classes: – SensorManager – Android sensor service Sensor – specific sensor SensorEventListener – SensorEvent – specific event of the sensor = data – LocationManager – Android location service LocationProvider (e.g., WPS, Network, GPS) LocationListener – Processing updates for location (Location) or provider (LocationProvider)

14. Android API Most sensors interfaced through SensorManager or LocationManager – Obtain pointer to android service using Context.getSystemService(name) – For name, use constant defined by Context class SENSOR_SERVICE for SensorManager LOCATION_SERVICE for LocationManager Check for available sensors using List mgr.getSensorList(int type) – Type constants provided in Sensor class documentation Check for available location providers using List providers = mgr.getAllProviders();

15. SensorManager Use getDefaultSensor(int type) to get a pointer to the default sensor for a particular type Sensor accel = sensorManager.getDefaultSensor( Sensor.TYPE_ACCELEROMETER); Register for updates of sensor values using registerListener(SensorEventListener, Sensor, rate) – Rate is an int, using one of the following 4 constants SENSOR_DELAY_NORMAL (delay: 200ms) SENSOR_DELAY_UI (delay: 60ms) SENSOR_DELAY_GAME (delay: 20ms) SENSOR_DELAY_FASTEST (delay: 0ms) – Use the lowest rate necessary to reduce power usage Registration will power up sensor: mSensorService.enableSensor(l, name, handle, delay);

16. SensorManager Unregister for sensor events using unregisterListener(SensorEventListener, Sensor) or unregisterListener(SensorEventListener) Undregistering will power down sensors: mSensorService.enableSensor(l, name, handle, SENSOR_DISABLE) Perform register in OnResume() and unregister in OnPause() to prevent using resources while your activity is not visible SensorListener is deprecated, use SensorEventListener instead – See documentation for Sensor, SensorManager, SensorEvent and SensorEventListener

17. SensorEventListener Must implement two methods onAccuracyChanged(Sensor sensor, int accuracy) onSensorChanged(SensorEvent event) SensorEvent – int accuracy – Sensor sensor – long timestamp Time in nanoseconds at which event happened – float[] values Length and content of values depends on sensor type

18. API – Setup public class MainActivity extends Activity implements SensorEventListener {.. private SensorManager sm = null; … public void onCreate (Bundle savedInstanceState) {.. sm = (SensorManager) getSystemService(SENSOR_SERVICE); } protected void onResume () {.. List typedSensors = sm.getSensorList(Sensor.TYPE_LIGHT); // also: TYPE_ALL if (typedSensors == null || typedSensors.size() <= 0) … error… sm.registerListener(this, typedSensors.get(0), SensorManager.SENSOR_DELAY_GAME); // Rates: SENSOR_DELAY_FASTEST, SENSOR_DELAY_GAME, // SENSOR_DELAY_NORMAL, SENSOR_DELAY_UI }

19. API – Processing Events It is recommended not to update UI directly! public class MainActivity extends Activity implements SensorEventListener {.. private float currentValue; private long lastUpdate; … public void onSensorChanged(SensorEvent event) { currentValue = event.values[0]; lastUpdate = event.timestamp; }.. }

20. API – Cleanup public class MainActivity extends Activity implements SensorEventListener { … protected void onPause () { … sm. unregisterListener (this); } … protected void onStop () { … sm. unregisterListener (this); }.. }

21. LocationManager This class provides access to the system location services These services allow applications – To obtain periodic updates of the device's geographical location, – To fire an application-specified Intent when the device enters the proximity of a given geographical location. Obtain pointer to Android service using Context.getSystemService (Context.LOCATION_SERVICE)

22. LocationProvider Providers – GPS_PROVIDER GPS: 1-10 meters – NETWORK_PROVIDER Wifi: 10-100 meters Cellular: 1 km – PASSIVE_PROVIDER Piggy-back on other updates Each provider has trade-off of power usage, delay, and accuracy Use minimum accuracy necessary for application to reduce power usage

23. Location Class A class representing a geographic location sensed at a particular time (a "fix"). A location consists of a latitude and longitude, a UTC timestamp and optionally information on altitude, speed, and bearing. Information specific to a particular provider or class of providers may be communicated to the application using getExtras, which returns a Bundle of key/value pairs. Each provider will only provide those entries for which information is available.

24. LocationListener Class Used for receiving notifications from the LocationManager when the location has changed. These methods are called if the LocationListener has been registered with the location manager service using the method: – requestLocationUpdates (Provider, minTime, minDistance, LocationListener) Name of the provider with which to register Minimum time interval for notifications, in milliseconds Minimum distance interval for notifications, in meters LocationListener whose onLocationChanged(Location) method will be called for each location update onLocationChanged(Location)

25. LocationListener Class Event handling functions onLocationChanged (Location location) Called when the location has changed. onProviderDisabled (String provider) Called when the provider is disabled by the user. onProviderEnabled (String provider) Called when the provider is enabled by the user. onStatusChanged (String provider, int status, Bundle extras) Called when the provider status changes.

26. API – Setup public class MainActivity extends Activity implements LocationListener{.. private LocationManager lm = null; … public void onCreate (Bundle savedInstanceState) {.. lm = (LocationManager) getSystemService(LOCATION_SERVICE); Criteria criteria = new Criteria(); best = lm.getBestProvider(criteria, true); } protected void onResume () {.. lm.requestLocationUpdates(LocationManager.GPS_PROVIDER, 60000, 1, this); lm.requestLocationUpdates(best, 1000, 1, this); }

27. API – Processing Events It is recommended not to update UI directly! public class MainActivity extends Activity implements LocationListener{.. private Location curlocation; … public void onLocationChanged(Location location) { if (location == null) log("\nLocation[unknown]"); else { log("\n" + location.toString()); curlocation = location; }

28. API – Cleanup public class MainActivity extends Activity implements LocationListener{ … protected void onPause () { … lm.removeUpdates(this); } … protected void onStop () { … lm.removeUpdates(this); }.. }

29. Contents Elements of mobile sensor networking Programming Android sensor/location service – Setting up sensor/location service – Processing events How do sensors work? – Accelerometer, Magnetometer, Gyroscope, Light Sensor, Proximity Sensor, GPS, Microphone, etc. How do location services work? – GPS, cell-tower location, Wi-Fi positioning

30. Accelerometer Mass on spring GravityFree FallLinear AccelerationLinear Acceleration plus gravity 1g = 9.8m/s 2 -1g 1g

31. Accelerometer STMicroelectronics STM331DLH three-axis accelerometer (iPhone4)

32. Accelerometer Sensor.TYPE_ACCELEROMETER Values[3] = m/s 2, measure the acceleration applied to the phone minus the force of gravity (x, y, z) SensorManager’s constants – GRAVITY_EARTH, GRAVITY_JUPITER, – GRAVITY_MARS, GRAVITY_MERCURY, – GRAVITY_MOON, GRAVITY_NEPTUNE

33. Compass Magnetic field sensor (magnetometer) Z X Y X Y Z 3-Axis Compass? Magnetic inclination Horizontal Gravity Magnetic field vector Magnetic declination Magnetic north Geographic north

34. Compass Hall Effect 3-Axis Compass

35. Compass Sensor.TYPE_MAGNETIC_FIELD values[3] = in micro-Tesla (uT), magnetic field in the X, Y and Z axis SensorManager’s constants – MAGNETIC_FIELD_EARTH_MAX: 60.0 – MAGNETIC_FIELD_EARTH_MIN: 30.0

36. Orientation Sensor Sensor.TYPE_ORIENTATION – A fictitious sensor: orientation is acquired by using accelerometer and compass Deprecated – Now use getOrientation (float[] R, float[] result) Values[3] – (Azimuth, Pitch, Roll) – Azimuth, rotation around the Z axis 0 <= azimuth <= 360, 0 = North, 90 = East, 180 = South, 270 = West – Pitch, rotation around the X axis -180 <= pitch <= 180 0 = sunny side up 180, -180 = up side down -90 = head up (perpendicular) 90 = head down (perpendicular) – Roll, rotation around the Y axis -90<=roll <= 90 Positive values when the z-axis moves toward the x-axis.

37. Orientation: Why Both Sensors? We need two vectors to fix its orientation! (gravity and magnetic field vectors) Tutorial: http://cache.freescale.com/files/sensors/doc/app_note/AN4248.pdfhttp://cache.freescale.com/files/sensors/doc/app_note/AN4248.pdf

38. Gyroscope Angular velocity sensor – Coriolis effect – “fictitious force” that acts upon a freely moving object as observed from a rotating frame of reference

39. Gyroscope

40. Sensor.TYPE_GYROSCOPE Measure the angular velocity of a device Detect all rotations, but few phones have it – iPhone4, iPad2, Samsung Galaxy S, Nexus S – Values[] – iPhone 4 gives radians/sec, and makes it possible to get the rotation matrix

41. Accelerometer vs. Gyroscope Accelerometer – Senses linear movement, but worse rotations, good for tilt detection, – Does not know difference between gravity and linear movement Shaking, jitter can be filtered out, but the delay is added Gyroscope – Measure all types of rotations – Not movement – Does not amplify hand jitter A+G = both rotation and movement tracking possible

42. Other Sensors Light: – Sensor.TYPE_LIGHT – values[0] = ambient light level in SI lux units Proximity: – Sensor.TYPE_PROXIMITY – values[0]: Proximity sensor distance measured in centimeters (sometimes binary near-far) Temperature: – Sensor.TYPE_TEMPERATURE – values[0] = temperature Pressure: – Sensor.TYPE_PRESSURE – values[0] = pressure

43. Contents Elements of mobile sensor networking Programming Android sensor/location service – Setting up sensor/location service – Processing events How do sensors work? – Accelerometer, Magnetometer, Gyroscope, Light Sensor, Proximity Sensor, GPS, Microphone, etc. How do location services work? – GPS, cell-tower location, Wi-Fi positioning

44. Global Positioning System (GPS) 27 satellite constellation Powered by solar energy Originally developed for US military Each carries a 4 rubidium atomic clocks – locally averaged to maintain accuracy – updated daily by US Air Force Ground control – Satellites are precisely synchronized with each other The orbits are arranged so that at any time, anywhere on Earth, there are at least four satellites "visible" in the sky. A GPS receiver's job is to locate three or more of these satellites, figure out the distance to each, and use this information to deduce its own location. This operation is based on a mathematical principle called tri-lateration

45. Tri-lateration Imagine you are somewhere in Korea and you are TOTALLY lost -- for whatever reason, you have absolutely no clue where you are. You find a friendly local and ask, "Where am I?" He says, "You are 70 km miles from 대전 “ You ask somebody else where you are, and she says, "You are 60 km from 대구 " Now you have two circles that intersect. You now know that you must be at one of these two intersection points. If a third person tells you that you are 100 km from 광주, you can eliminate one of the possibilities. You now know exactly where you are 70km 100km 60km

46. Global Positioning System (GPS) 3D-Trilateration

47. Global Positioning System (GPS) 3D-Trilateration

48. Global Positioning System (GPS) 3D-Trilateration

49. Assisted-GPS: GPS + Network Reduce satellite search space by focusing on where the signal is expected to be Other assistance data from cellular nets – Time sync – Frequency – Visible satellites – Local oscillator, etc.. MS-based vs. assisted GPS

50. GPS Errors (in Smartphones) In-vehicle signal attenuation Smartphone’s inferior antenna (worse!) – PND uses Spiral helix; Microstrip antenna vs. Galaxy S (single wire) low GPS reading under high speed environments – 4800bps (600 B/s) – http://www.hadaller.com/dave /research/papers/MitigatingG PSError- UWTechReport08.pdf http://www.hadaller.com/dave /research/papers/MitigatingG PSError- UWTechReport08.pdf Galaxy S GPS Antenna PND GPS antennas V.S.

51. Network Positioning Method Cell-tower localization with tri-lateration Cell-tower

52. Wi-Fi Positioning System Fingerprinting (e.g., RADAR, Skyhook) Training phase (building a fingerprint table): for each location, collect signal strength samples from towers, and keep the average for each location Positioning phase: – Calculate the distance in signal strength space between the measured signal strength and the fingerprint DB – Select k fingers with the smallest distance, and use arithmetic average as the estimated location RSSI (x, y, z) = (-20, -10, -15) (-15, -12, 18) …………  L1=avg(x, y, z) = (xx, yy, zz) RSSI (x, y, z) = (-21, -40, -18) (-16, -42, 12) …………  L2=avg(x, y, z) = (xx’, yy’, zz’) RSSI: Received Signal Strength Indicator

53. II. 모바일 센싱 응용 이의진 KAIST 지식서비스공학과

54. Contents Vehicular sensing – MobEyes – CarTel – Pothole patrol People sensing – PIER – SoundSense

55. Vehicular Sensor Network (VSN) Onboard sensors (e.g., video, chemical, pollution monitoring sensors) Large storage and processing capabilities (no power limit) Inter-vehicular communications via 802.11p (IEEE WAVE) Smartphones

56. Vehicular Sensing Applications Vehicular safety warning – Forward collision, ice on the road Traffic engineering – Street-level traffic pattern, congestion analysis Ride quality monitoring – Cracks, potholes Environment monitoring – Urban environment pollution monitoring Civic and homeland security – Forensic accident or crime site investigations – Terrorist alerts Location-aware micro-blogging Potholes

57. Vehicular Sensing Architecture V2V vs. Infrastructure-based approaches V2V ApproachInfrastructure-based Approach MobEyes CarTel

58. MobEyes: Smart Mobs for Urban Monitoring with Vehicular Sensor Networks* Uichin Lee, Eugenio Magistretti, Mario Gerla, Paolo Bellavista, Antonio Corradi Network Research Lab CS, UCLA * Uichin Lee, Eugenio Magistretti, Biao Zhou, Mario Gerla, Paolo Bellavista, Antonio Corradi "MobEyes: Smart Mobs for Urban Monitoring with a Vehicular Sensor Network," IEEE Wireless Communications, 2006

59. MobEyes: V2V Storage and Retrieval Private Cars: – Continuously collect images on the street (store data locally) – Process the data and detect an event (if possible) – Create meta-data of sensed Data -- Summary (Type, Option, Location, Vehicle ID, …) – Post it on the distributed index The police build an index and access data from distributed storage

60. Problem Description VSN challenges – Mobile storage with a “sheer” amount of data – Large scale up to hundreds of thousands of nodes Goal: developing efficient meta-data harvesting/data retrieval protocols for mobile sensor platforms – Posting (meta-data dissemination) [Private Cars] – Harvesting (building an index) [Police] – Accessing (retrieve actual data) [Police]

61. Searching on Mobile Storage - Building a Distributed Index Major tasks: Posting / Harvesting Naïve approach: “Flooding” – Not scalable to thousands of nodes (network collapse) – Network can be partitioned (data loss) Design considerations – Non-intrusive: must not disrupt other critical services such as inter-vehicle alerts – Scalable: must be scalable to thousands of nodes – Disruption or delay tolerant: even with network partition, must be able to post & harvest “meta-data”

62. Mobility-assist Meta-data Diffusion/Harvesting Let’s exploit “mobility” to disseminate meta-data! Mobile nodes are periodically broadcasting meta- data of sensed data to their neighbors – Data “owner” advertises only “his” own meta-data to his neighbors – Neighbors listen to advertisements and store them into their local storage A mobile agent (the police) harvests a set of “missing” meta-data from mobile nodes by actively querying mobile nodes (via. Bloom filter)

63. Mobility-assist Meta-data Diffusion/Harvesting + Broadcasting meta-data to neighbors + Listen/store received meta-data Periodical meta-data broadcasting Agent harvests a set of missing meta-data from neighbors HREQ HREP

64. Meta-data Harvesting Results Meta-data harvesting: agent actively harvests meta-data Impact of node density (#nodes), speed, mobility – Higher speed, faster harvesting – Higher density, faster harvesting (more # of meta-data from neighbors) – Less restricted mobility, faster harvesting (Man>Westwood) Manhattan MobilityReal-track Mobility Time (s) Fraction of actively harvested meta-data

65. Trajectory Tracking Agent re-constructs the trajectory of a node

66. CarTel: A Distributed Mobile Sensor Computing System Bret Hull, Vladimir Bychkovsky, Kevin Chen, Michel Goraczko, Allen Miu, Eugene Shih, Yang Zhang, Hari Balakrishnan, and Samuel Madden Sensys 2006

67. CarTel System Architecture Open wireless Access Point User’s wireless Access Point Adapters log GPS, OBD, camera data Data sent via prioritized continuous queries ICEDB Remote Delay-tolerant relay via WiFi CafNet PortalClients Prioritizes data Answers local snapshot queries Logs continuous query results ICEDB Server

68. CarTel S/W Architecture Portal Data Visualization CafNet Stack CQ Web Server ICEDB Server OBD-II WiFi Monitor Camera Traffic Speed/ Delay Portal Applications Portal Streaming Sensor Data Cont. queries + adaptors

69. Intermittently connected DB (ICEDB) ICEDB server – Maintains a list of continuous queries submitted by applications – Queries are pushed to mobile nodes using CafNets – Results from ICEDB clients are stored in the RDBMS at the portal ICEDB client – Process the sensed data and return the query results using CafNet (carry-and-forward net) – Prioritize the result streams in the order of importance

70. Intermittently connected DB (ICEDB) Queries in ICEDB are written in SQL with several extensions for continuous queries and prioritization – EX)SELECT carid, traceid, time, location FROM gps WHERE gps.time BETWEEN now()-1 mins AND now() RATE 5 mins Prioritization is required since delivering data in FIFO order is suboptimal in bandwidth- constrained network – Intermittent connectivity due to high speed mobility and restricted mobility patterns

71. Portal Users navigate sensor data in CarTel using a web- based interface Main components of the portal – Portal framework – ICEDB server to retrieve sensor data – Data visualization library to display geo-coded data Trace: all sensor data collected during a single trip (i.e., between ignition “on” and “off”)

72. The Pothole Patrol: Using a Mobile Sensor Network for Road Surface Monitoring Jakob Eriksson, Lewis Girod, Bret Hull, Ryan Newton, Samuel Madden, Hari Balakrishnan MIT Computer Science and Artificial Intelligence Laboratory Mobisys 2008

73. Mobile Road Surface Monitoring Potholes: hazardous to drivers and increasing repair costs due to vehicle damage Determine “which” roads need to be fixed Static sensors will not do well – requires mobility! Challenges: – Differentiate potholes from other road anomalies (railroad crossings, expansion joints) – Coping with variations in detecting the same pothole (speed, sensor orientation)

74. Pothole Patrol: Architecture Pothole Record Clustering Cab 1 GPS 3 Axis Accelerometer Location Interpolator Pothole Detector Cab 2 GPS 3 Axis Accelerometer Location Interpolator Pothole Detector Central Server

75. Sample Traces Smooth RoadRail Crossing PotholeExpansion Joint

76. Detection Algorithm Features of accelerometer data High energy events are potholes? – Not really! – Rail road crossings, expansion joints, door slamming are high energy events Accelerometer data is processed by embedded computer – 256-sample windows – Pass through 5 different filters

77. Detection Algorithm Speed – Car is not moving or moving slowly – Rejects door slam and curb ramp events IN Windows of all event classes SpeedHigh-passz-peak xz-ratio speed vs. z ratio OUT Pothole Detections

78. Detection Algorithm High-Pass – Removes low-freq components in X- and Z-axes – Filters out events like turning, veering, braking. IN Windows of all event classes SpeedHigh-passz-peak xz-ratio speed vs. z ratio OUT Pothole Detections

79. Detection Algorithm z-peak – Prime characteristic for minor anomalies – Rejects all windows with absolute z-acceleration < t z IN Windows of all event classes SpeedHigh-passz-peak xz-ratio speed vs. z ratio OUT Pothole Detections tztz

80. Detection Algorithm xz- ratio – Assumes potholes impact only side of the vehicle – Identifies anomalies that span width of the road (rail crossings, speed bumps) high z-peak + low x-peak vs. high z-peak + high x-peak (potholes) – x peak (within Δw (=32) samples from z peak ) < t x * z peak IN Windows of all event classes SpeedHigh-passz-peak xz-ratio speed vs. z ratio OUT Pothole Detections tztz txtx

81. Detection Algorithm speed vs. z ratio – At high speeds, small anomalies cause high peak accelerations – Rejects windows where Z peak < t s X speed IN Windows of all event classes SpeedHigh-passz-peak xz-ratio speed vs. z ratio OUT Pothole Detections tztz txtx tsts

82. Detection Algorithm Filtering parameter threshold tuning via training – Thresholds: z-peak, xz-ratio, and speed vs. z ratio – Perform exhaustive search over a range of values Improving detection accuracy with clustering – Cluster of at least k events must happen in the same location with small margin of error(Δd) – Clustering algorithm Place each detection in Δd X Δd grid. Compute pair-wise distances in same or neighboring grid cells Iteratively merge pairs of distances in order of distance Max intra cluster distance < Δt Reported location is the centroid of the locations

83. Performance on labeled data – Randomly divided into training set and test set Cf: loosely labeled data (extra information about manholes, bridges, etc.) Performance ClassHand Labeledw/ Loosely Labeled Pothole88.9%92.4% Manhole0.3%0.0% Expansion joints2.7%0.3% Railroad Crossing8.1%7.3%

84. Contents Vehicular sensing – MobEyes – CarTel – Pothole patrol People sensing – PIER – SoundSense

85. PIER: Personal Environmental Impact Report, as a Platform for Participatory Sensing Systems Research Min Mun, Sasank Reddy, Katie Shilton, Nathan Yau, Jeff Burke, Deborah Estrin, Mark Hansen, Eric Howard, Ruth West, Peter Boda UCLA, Nokia Mobisys 2009

86. Participatory Sensing Distributed data collection and analysis at the personal, urban, and global scale Individuals and communities make decisions about when and how to Capture  Store  Access  Analyze  Share Participants use mobile phones to gather data and web services to aggregate and interpret the assembled information.

87. Health and Wellness: Personal Environmental Impact Report CO2 emissionsFast food exposure PM2.5

88. Personal Environmental Impact Report Carbon Impact: a measure of transportation- related carbon footprint Sensitive Site Impact: a user’s transportation related airborne particulate matter emissions near sites with populations sensitive to it Smog Exposure: a user’s transportation-related exposure to particulate matter emissions Fast Food Exposure: the time integral of proximity to fast-food eating establishment Ultimately we want people to take notice of impact and exposure and be able to start conversations. The absolute numbers are not what’s key, but instead, trends over time. How can one reduce impact and minimize exposure?

89. PEIR as a Participatory Sensing System "Sensing Pollution without Pollution-Sensors" Existing Infrastructure Annotation /Inferences Scientific Models Activity Classification e.g., staying, walking, driving GIS Data Annotation e.g. weather, traffic Impact and Exposure Calculation Data Aggregation Tracklog format School,hospital,fast food restaurant locations Weather, traffic data User profile

90. PEIR as a Participatory Sensing System

92. SoundSense: Scalable Sound Sensing for People-Centric Applications on Mobile Phones Hong Lu, Wei Pan, Nicholas D. Lane, Tanzeem Choudhury, Andrew T. Campbell Dept. of Computer Science, Dartmouth College Mobisys 2009

93. SoundSense allows us to sense people and the environment

94. SoundSense Architecture Admission Control Acoustic Features Decision Tree Classifier Markov Model Recognizer Ambient Sound Learning Voice Analysis Music Analysis Sound Waveform Framing & Admission Control Feature Extraction Coarse Category Classification Intra-Category Classification

95. Admission Control Filter out silence and non-informative frames Criteria: – Energy & Spectral Entropy

96. Discriminative Acoustic Features Frequency Spectrum based features: – Robust to distance/muffling – Computationally very cheap after initial FFT

97. Discriminative Acoustic Features Spectral Rolloff for Music and Voice Spectral Rolloff Time

98. Coarse Category Classification Acoustic Features Decision Tree Classifier Ambient Sound Voice Music S1S2 S3 S1S2 S3 S1S2 S3 Markov Model Recognizer Music, Music, Voice, Music, Voice, Music, Music Music, Music, Music, Music, Music, Music, Music Confusion matrix for the decision tree classifier with Markov model smoothing

99. Learning Significant Sound Unique to the individual Not practical to build supervised classifiers for all the potential significant sounds – Unsupervised learning! Examples: – Mystery Sound 1  Car (turning signal) – Mystery Sound 2  Vacuum Cleaner Incrementally build models of encountered sounds Rank them by order of “importance” Human-in-the-loop labeling

100. Evaluation Friday Saturday

101. 관련 토픽 프라이버시 – 센서데이터를 개인 단말에서 서버로 전송시 – 관련 논문 : Privacy by Design - Principles of Privacy-Aware Ubiquitous Systems, Marc Langheinrich, Ubicomp 2001 Location Disclosure to Social Relations: Why, When, & What People Want to Share, Consolvo et al., CHI'05 스마트폰 에너지 – 센서 duty-cycling – 통신 에너지 사용 trade-offs (WiFi vs. 3G) – 관련 논문 : Decomposing power measurements for mobile devices, Andrew Rice and Simon Hay, Percom 2010 Coolspots: Reducing the power consumption of wireless mobile devices with multiple radio interfaces, Pering et al., Mobisys 2006 Activity classification – 가속도 센서를 이용한 activity classification – 관련논문 : Activity Recognition from User-Annotated Acceleration Data, Pervasive 2004

102. 강좌요약 network infra 스마트폰 센싱 기초 센서, 위치, 무선통신 모바일 센싱 응용 Vehicular Sensing People Sensing

103. 모바일 센서 네트워킹 관련 주요 학회 MobiSys: http://www.sigmobile.org/mobisyshttp://www.sigmobile.org/mobisys Ubicomp: http://www.ubicomp.orghttp://www.ubicomp.org Pervasive: http://pervasiveconference.orghttp://pervasiveconference.org PerCom: http://www.percom.org/http://www.percom.org/ SenSys: http://sensys.acm.orghttp://sensys.acm.org MobiCom: http://www.sigmobile.org/mobicomhttp://www.sigmobile.org/mobicom