1. MultiScale Sensing: A new paradigm for actuated sensing of dynamic phenomena Diane Budzik 9-8-06 Electrical Engineering Department Center for Embedded Networked Sensing University of California Los Angeles

2. Outline Environmental Sensing and Applications Environmental Phenomena Classification MultiScale Sampling Methodology Results: Simulation and using NIMS 3D Conclusion Future Directions

3. Environmental Sensing Environmental phenomena are dynamic –Solar light radiation Varying light patterns on forest floor –C0 2 flux –Humidity Dynamic phenomena –High spatial and temporal variation –Requires high spatial sampling rate to achieve desired fidelity Spatial sampling rate of 10 samples/m 2 over a transect exceeding 1000 m 2 –Requires actuated sensing (mobile robot)

4. Bracken Ferns: James Reserve World-wide distribution, occurring in multiple habitats from cold temperate to tropical forests Fern fronds are carcinogenic and young fronds may release hydrogen cyanide when they are damaged –Threat to livestock if eat bracken ferns –Threat to humans when toxins are passed via milk from affected cows Study growth patterns of bracken ferns Photosynthesis Light distribution

5. Spatial Frequency Temporal Frequency quasi-static, smooth fields quasi-static, smooth fields Environmental Phenomena Classification Qualitative classification –Spatiotemporal frequency distribution

6. Environmental Phenomena Classification Spatial Frequency Temporal Frequency terrain, plant distribution terrain, plant distribution

7. Environmental Phenomena Classification Spatial Frequency Temporal Frequency raster scan sampling

8. Environmental Phenomena Classification Spatial Frequency Temporal Frequency quasi-static, smooth fields dynamic amplitude variation of smooth field

9. Environmental Phenomena Classification Spatial Frequency Temporal Frequency quasi-static, smooth fields dynamic amplitude variation of smooth field

10. Environmental Phenomena Classification Spatial Frequency Temporal Frequency quasi-static, smooth fields dynamic amplitude variation of smooth field

11. Environmental Phenomena Classification Spatial Frequency Temporal Frequency quasi-static, smooth fields dynamic amplitude variation of smooth field

12. Environmental Phenomena Classification Spatial Frequency Temporal Frequency terrain, plant distribution terrain, plant distribution contaminant distribution

13. Environmental Phenomena Classification Spatial Frequency Temporal Frequency static network sampling raster scan sampling

14. Environmental Phenomena Classification Spatial Frequency Temporal Frequency quasi-static, smooth fields dynamic amplitude variation of smooth field quasi-static, “rough”, complex fields

15. Environmental Phenomena Classification Spatial Frequency Temporal Frequency quasi-static water flow distribution terrain, plant distribution terrain, plant distribution contaminant distribution

16. Environmental Phenomena Classification Spatial Frequency Temporal Frequency adaptive sampling static network sampling raster scan sampling

17. Environmental Phenomena Classification Spatial Frequency Temporal Frequency quasi-static, smooth fields dynamic amplitude variation of smooth field quasi-static, “rough”, complex fields dynamic phenomena

18. Environmental Phenomena Classification Spatial Frequency Temporal Frequency quasi-static, smooth fields dynamic amplitude variation of smooth field quasi-static, “rough”, complex fields dynamic phenomena

19. Environmental Phenomena Classification Spatial Frequency Temporal Frequency quasi-static, smooth fields dynamic amplitude variation of smooth field quasi-static, “rough”, complex fields dynamic phenomena

20. Environmental Phenomena Classification Spatial Frequency Temporal Frequency quasi-static, smooth fields dynamic amplitude variation of smooth field quasi-static, “rough”, complex fields dynamic phenomena

21. Environmental Phenomena Classification Spatial Frequency Temporal Frequency general example, numerous applications quasi-static water flow distribution terrain, plant distribution terrain, plant distribution contaminant distribution

22. Environmental Phenomena Classification Spatial Frequency Temporal Frequency adaptive sampling static network sampling raster scan sampling multiscale sampling

23. Environmental Phenomena Classification Spatial Frequency Temporal Frequency adaptive sampling static network sampling raster scan sampling multiscale sampling

24. Outline Environmental Sensing and Applications Environmental Phenomena Classification MultiScale Sampling Methodology Results: Simulation and using NIMS 3D Conclusion Future Directions

25. Sensing Light Distribution Under a Forest Canopy August 12, 2005 James San Jacinto Mountains Reserve

26. Sensing Light Distribution Under a Forest Canopy James San Jacinto Mountains Reserve August 12, 2005

27. Sensing Light Distribution Under a Forest Canopy James San Jacinto Mountains Reserve August 12, 2005

28. Sensing Light Distribution Under a Forest Canopy James San Jacinto Mountains Reserve August 12, 2005

29. Sensing Light Distribution Under a Forest Canopy James San Jacinto Mountains Reserve August 12, 2005

30. MultiScale is a method for phenomena sampling that uses a hierarchy of resources of varying sensing and mobility modalities A two-tier MultiScale approach: –High resolution, low fidelity sensor (imager) provides a global view of the environment –Low resolution, high fidelity sensor (PAR sensor) used for actuated sampling MultiScale Sampling (MSS) Methodology

31. Image Acquisition and Processing Binary Segmented Image Actual Image from JR

32. Task Prioritization Multi Robot Task Allocation is the problem of assigning robots to tasks Suppose at a given period of time the system maintains a set of tasks and a set of robots Tasks are prioritized based on one of two heuristics: –Largest area first Service fewer tasks, tasks are larger in area –Least service time first Service many tasks, tasks are smaller in area Highest priority task is assigned to the closest robot Commitment policy

33. Simulation Snapshot Largest area first –Selects one large task to sample –Several minutes to complete task –Use when no prior information about phenomena Lowest service time first –Selects many small tasks sample –Several seconds to complete all tasks –Use when model of phenomena exists Phenomena Heuristic: largest area first Heuristic: lowest service time first

34. Outline Environmental Sensing and Applications Environmental Phenomena Classification MultiScale Sampling Methodology Results: Simulation and using NIMS 3D Conclusion Future Directions

35. Simulation Results for Area and Time Heuristics Sampling density (s) Normalized sampled area Sampling time: 0.1 sec Lower sampling density decreases sampling time and intra-task travel (during sampling) time Higher speed decreases intra-task travel time and decreases inter-task travel time Normalized sampled area Total information available during 1 hour of images Took the difference between consecutive images and counted number of white pixels in the differenced images

36. Simulation Results: Area and Time Heuristics Normalized number of tasks sampled Sampling density (s) Normalized number of tasks sampled Total number of tasks during 1 hour of images For consecutive images, counted number new tasks

37. Simulation Results: Reconstruction Error Sampling density = 4Sampling density = 10 Sampling density (s) Normalized reconstruction error Count number of pixels in sampled image that are different from input image Normalized with total amount of information available in sampled region

38. Simulation Results: Multiple Robots Sampling density = 6 Speed = 60 cm/s Maximum possible normalized sampled area = 0.59 –Total maximum amount of information possible = 1.0 –Simulation artifact: centralized task allocation implementation results in some skipped tasks Normalized sampled area Number of mobile robots

39. NIMS 3D Results 1.Pulley Attachment 2.Node Platform 3.Motor Control Box Normalized sampled area Sampling density (s) 3 2 1 1 1 1 PAR Sensor

40. Conclusion Environmental phenomena are dynamic –Solar light radiation (varying sunlight patterns on forest floor) Static sensor sampling, raster scan sampling, and adaptive sampling are not adequate for sampling dynamic phenomena MultiScale is a method for phenomena sampling that uses a hierarchy of resources of varying sensing and mobility modalities –High resolution, low fidelity sensor (imager) provides a global view of the environment –Low resolution, high fidelity sensor (PAR sensor) used for actuated sampling Task Prioritization –Largest area first –Lowest service time first Simulation results and using NIMS 3D –MultiScale is adequate for sampling dynamic phenomena

41. Future Directions Online learning for phenomena modeling and sampling optimization One approach –Learn spatiotemporal statistics of tasks –Task is a way to cluster phenomena into entities that are interesting to a scientist

42. Thank you! ?

43. Image Processing: Task Extraction Actual Image from JR Segmented Image Binary Segmented Image

44. MultiScale Sampling vs Adaptive Sampling System Characteristics: –Speed = 50 cm/s –Sampling time = 0.1 sec –Area of the environment: Length = 768 cm Width = 480 cm –Sampling density: Equivalent number of sampling locations for AS and MSS –Phenomena speed = 12 pixels/min (12 cm/min) –Linear Interpolation used in reconstruction –Varying phenomena size

45. Performance Comparison: AS and MSS Large size phenomenon Dynamic Scene Static Scene

46. Conclusions RSS and SNS are efficient when the phenomena is static and spatial distribution of the phenomena is known AS is efficient when no prior information is available and when the phenomena has high spatial variability –Sampling latency inherent in AS renders it inadequate for achieving high fidelity sampling of dynamic (high spatiotemporal variation) phenomena MSS is efficient for dynamic phenomena –Sampling latency is reduced because of real-time information provided by a high resolution, low fidelity sensor (imager or network of static sensors)