1. Students Aaron Roney, Albert Soto, Brian Kuehner, David Taylor, Mark Hibbeler, Nicholas Logan, Stephanie Herd Tele-Operation and Vision System for Moon Exploration NASA JSC Mentors Dr. Bob Savely Mike Goza Project Mentor Dr. Giovanni Giardini Project Advisor Prof. Tamás Kalmár-Nagy

2. Project Members  Nuclear Engineering  Mechanical Engineering  Aerospace Engineering  Mechanical Engineering  Computer Engineering  Aaron Roney  Albert Soto  Brian Kuehner  David Taylor  Mark Hibbeler  Nicholas Logan  Stephanie Herd  Sophomore  Senior  Sophomore  Freshman

3. Motivations  Lunar surface exploration  Human perspective  In safety  With low risk  3D environment reconstruction  Self location with artificial vision system

4. Objectives  Vision System  Ego-Motion estimation  Environment reconstruction Tele-Operation System with Visual Feedback  Tele-Operation System  Remote Vehicle Control  Hardware and Mechanical Implementation

5. Visual System (onboard the Vehicle) Ground Station Vehicle Hardware WIRELESS NETWORK WIRELESS NETWORK

6. Visual System (onboard the Vehicle) Ground Station Vehicle Hardware WIRELESS NETWORK WIRELESS NETWORK

7. Theory Left image Right image u left p u right p v left p v right p u left p  It is impossible to compute the 3D coordinates of an object with a single image  Solution: Stereo Cameras  Disparity computation 3D reconstruction Image

8.  Main Goal: digital environment 3D reconstruction  Object detection (i.e. obstacles)  High level planning  Self localization Use Stereo Cameras to generate 3D environment Environment Reconstruction

9.  Disparity map computation:  Given 2 images, it is a collection of pixel disparities  Point distances can be calculated from disparities Environment can be reconstructed from disparity map Left ImageRight ImageDisparity Map Environment Reconstruction

10. Perspective Projection Equation  Main goal: evaluate the motion (translation and rotation) of the vehicle from sequences of images Ego-Motion Estimation  Solving will give velocities of the vehicle Optical Flow Example  Optical Flow is related to vehicle movement through the  Least Square solution

11. Reference Motion [mm]Detected Motion [mm] TxTx 04.5 TyTy 0-0.9 TzTz 5045.3 ΩxΩx 0-0.1 ΩyΩy 0-0.2 ΩzΩz 00 Ego-Motion: Example Optical Flow Left ImageOptical Flow Right Image

12. Visual System (onboard the Vehicle) Ground Station Vehicle Hardware WIRELESS NETWORK WIRELESS NETWORK

13. Calibration and Filtering  Calibration: removes image distortion  Filtering Process:  Improves image quality  Increases the robustness of the vision system

14. Visual System (onboard the Vehicle) Ground Station Vehicle Hardware WIRELESS NETWORK WIRELESS NETWORK

15. Tele-Operations Laptop on TAMUBOT TAMUBOT Control SystemWireless Router Control PC Tropos Router Picture PC

16. Vehicle Vehicle Courtesy of Prof. Dezhen Song Baseline D L FOV 1 FOV 2 α Horizontal View  Camera support system  3-DOF mechanical neck:  Panoramic rotation  Tilt rotation  Telescopic capability  Controlled height and baseline length

17. Conclusions and Future Work  Demonstrated:  Ego-motion estimation  Environment Reconstruction  Vehicle control and movement  Future Developments:  System integration  Filtering and improving results

18.  Thanks to: –Prof. Tamás Kalmár-Nagy –Dr. Giovanni Giardini –Prof. Dezhen Song –Change Young Kim –Magda Lagoudas –Tarek Elgohary –Pedro Davalos Acknowledgements