1. 1 Toward Autonomous Kilometer-Scale Site Survey by Planetary Rovers David R. Thompson drt@ri.cmu.edu Robotics Institute Carnegie Mellon University NASA ASTEP NNG0-4GB66G

2. 2 Agenda Autonomous Site Survey  Samples to environments  Experimental design Component technologies  Feature detection  Spatial models  Adaptive collection/return Future challenges

3. 3 1m 10m 10 2 m 10 3 m 10 4 m 1975 2003 1997 ? 2009 Optimal daily range single site multiple-site over-the-horizon

4. 4 Exploration bottlenecks / challenges Sparse samples Cannot target features in advance Limited bandwidth Labor-intensive data analysis

5. 5 Artificial geologic intelligence? Recognize and classify interesting features Acquire data autonomously Prioritize data products for downlink Adapt to trends and anomalies

6. 6 Previous work Marsokhod Rover Tests (Gulick et al., 2001)‏ Antarctic Meteorite Search (Pedersen et al., 2001)‏ Single Cycle Instrument Placement (Pedersen et al., 2004, Madison 2006, etc.)‏ OASIS, WATCH systems (Castaño et al., 2005, 2006, 2007, etc.)‏ EO-1 Agent, Sensorweb (Chien et al., 2005, etc.)‏

7. 7 Image Courtesy NASA / JPL-Caltech / Cornell

8. 8 Science value is context sensitive Courtesy USGS Courtesy NASA

9. 9 An experimental design approach 1. Define geologic parameters of interest 2. Model correlations between parameters and data 3. Seek data that minimizes expected posterior entropy P(  )‏ P(  |Data) ‏ H(Θ) = - ∫ P(Θ) log P(Θ) dΘ

10. 10 Spatial inference Cross-sensor inference ? ? Statistical models exploit correlations

11. 11 Agenda Autonomous site survey  Motivation  An experimental design approach Component technologies  Feature detection  Spatial models  Adaptive collection/return Future challenges

12. 12 Amboy Crater, Mojave Desert, CA feature detection data model action selection

13. 13 z

14. 14 1. Feature detection / classification 2. Spatial model (map)‏ 3. Action selection Surface images, spectra Navigation, data collection Selective downlink Remote sensing Component technologies

15. 15 Remote sensing multispectral pixel intensities feature detection data model action selection

16. 16 Rock detection feature detection data model action selection a b

17. 17 Rock detection feature detection data model action selection slow and accurate quick and dirty

18. 18 Rock detection: quick and dirty Filter Cascade A (Left-lit)‏ detected rocks input image feature detection data model action selection max Filter Cascade B (Right-lit)‏

19. 19 Rock detection: quick and dirty feature detection data model action selection

20. 20 Reflectance Spectroscopy feature detection data model action selection

21. 21 Spectrum collection procedure 1.Calibrate spectrometer 2.Kinematic pointing 3.Visual servoing: 1.Detect rocks in image 2.Match across images with SIFT (Lowe 2001)‏ 3.Stereo depth -> desired pixel (u,v)‏ 4.Pan-tilt servo update feature detection data model action selection

22. 22 Amboy Crater spectrometer tests feature detection data model action selection 4x40 minute traverses through a field of rocks Rover constructed a map of unique rocks and associated spectra

23. 23 Amboy Crater spectrometer tests feature detection data model action selection

24. 24 Amboy Crater spectrometer tests feature detection data model action selection Targeted spectroscopy averaged 10-15 unique rocks per run Fixed pointing never found isolated rocks Performance was sensitive to lighting 50m

25. 25 Classification via unsupervised learning feature detection data model action selection wavelength (nm)‏ 0.0 1.0 0.8 0.6 0.4 0.2 85013501850 2350 350 reflectance basalt sediment

26. 26 1. Feature detection / classification 2. Spatial model (map)‏ 3. Action selection Surface images, spectra Navigation, data collection Selective downlink Orbital data Component technologies

27. 27 Spatial models: Gaussian processes feature detection data model action selection Solves a regression problem: [location, remote sensing] class Learns the appropriate degree of spatial and cross-sensor correlation C(x i,x j ) ∝ exp {-∑ d w d |x i d - x j d |} spatial / cross-sensor structure

28. 28 Amboy Crater tests feature detection data model action selection Built maps of eroded basalt mounds Over 100 distinct trials  10 to 60 minutes  100m to 1.0 km Geologic class from fixed-angle spectroscopy

29. 29 Navigation sequence

30. 30 Inference result

31. 31 Amboy Crater tests feature detection data model action selection

32. 32 1. Feature detection / classification 2. Spatial model (map)‏ 3. Action selection Surface images, spectra Navigation, data collection Selective downlink Orbital data Component technologies

33. 33 “Corridor exploration” scenario Restricted planning task Robot moves forward to reach end-of-day goal Periodic sampling ~500m DbDb DaDa ~100m

34. 34 Informative path planning feature detection data model action selection Recursive path planning (Chackuri / Pal 2005)‏ Predict entropy of future observations Maximize expected information gain Respect time and resource constraints

35. 35 Informative path planning

36. 36 Recovery from Navigation Error

37. 37 collection returned data collected data environment downlink min H( environment | returned data ) feature detection data model action selection min H( environment | returned data )

38. 38 min X,X' H(Θ|X') collection,Θ,Θ downlink XX' environment collected datareturned data feature detection data model action selection min X,X' H(Θ|X')

39. 39 Data Collection Maximum Entropy Sampling (Sebastiani and Wynn, 2000)‏ Data Return Combinatorial Optimization collection,Θ,Θ downlink XX' environment collected datareturned data feature detection data model action selection Objective Function min X,X' H(Θ|X') ∝ -H(,X) + H(X|X')‏

40. 40 Selective data return feature detection data model action selection

41. 41 Agenda Autonomous site survey  Motivation  An experimental design approach Component technologies  Feature detection  Spatial models  Adaptive collection/return Future challenges

42. 42 Short term:  Analyze Amboy results  Integrate targeted spectroscopy and mapping Long term:  better feature analysis and image descriptors  more sophisticated models  free scheduling of science activities  balance navigability and science objectives  more application domains Future Challenges

43. 43 “Take-home messages” Science value is context-sensitive Remote exploration as adaptive experimental design Progress toward autonomous kilometer- scale site survey

44. 44 Thanks! David Wettergreen, Francisco Calderón, Dom Jonak, Ron Greeley, Shelby Cave, Phil Christensen, Steve Chien, Steven Flores, James Teza, Jeff Schneider, Heather Dunlop

45. 45 Rock detection: slow and accurate feature detection data model action selection sun angle superpixels high-scoring solution segmentation search texton map initial segmentation

46. 46 Rock detection: slow and accurate feature detection data model action selection Classification accuracy by feature Shape Shading Texture 93.8% 80.5% 94.7% 95.6% 91.2% 97.3% 91.2% None 53.1%

47. 47 Rock detection: slow and accurate feature detection data model action selection

48. 48 Classification via “unsupervised learning”... 0.0, 0.4, 0.5... feature detection data model action selection

49. 49 feature detection data model action selection Principal Component Analysis