Students Aaron Roney, Albert Soto, Brian Kuehner, David Taylor, Mark Hibbeler, Nicholas Logan, Stephanie Herd Tele-Operation and Vision System for Moon Exploration NASA JSC Mentors Dr. Bob Savely Mike Goza Project Mentor Dr. Giovanni Giardini Project Advisor Prof. Tamás Kalmár-Nagy

Project Members  Nuclear Engineering  Mechanical Engineering  Aerospace Engineering  Mechanical Engineering  Computer Engineering  Aaron Roney  Albert Soto  Brian Kuehner  David Taylor  Mark Hibbeler  Nicholas Logan  Stephanie Herd  Sophomore  Senior  Sophomore  Freshman

Motivations  Lunar surface exploration  Human perspective  In safety  With low risk  3D environment reconstruction  Self location with artificial vision system

Objectives  Vision System  Ego-Motion estimation  Environment reconstruction Tele-Operation System with Visual Feedback  Tele-Operation System  Remote Vehicle Control  Hardware and Mechanical Implementation

Visual System (onboard the Vehicle) Ground Station Vehicle Hardware WIRELESS NETWORK WIRELESS NETWORK

Visual System (onboard the Vehicle) Ground Station Vehicle Hardware WIRELESS NETWORK WIRELESS NETWORK

Theory Left image Right image u left p u right p v left p v right p u left p  It is impossible to compute the 3D coordinates of an object with a single image  Solution: Stereo Cameras  Disparity computation 3D reconstruction Image

 Main Goal: digital environment 3D reconstruction  Object detection (i.e. obstacles)  High level planning  Self localization Use Stereo Cameras to generate 3D environment Environment Reconstruction

 Disparity map computation:  Given 2 images, it is a collection of pixel disparities  Point distances can be calculated from disparities Environment can be reconstructed from disparity map Left ImageRight ImageDisparity Map Environment Reconstruction

Perspective Projection Equation  Main goal: evaluate the motion (translation and rotation) of the vehicle from sequences of images Ego-Motion Estimation  Solving will give velocities of the vehicle Optical Flow Example  Optical Flow is related to vehicle movement through the  Least Square solution

Reference Motion [mm]Detected Motion [mm] TxTx 04.5 TyTy TzTz ΩxΩx ΩyΩy ΩzΩz 00 Ego-Motion: Example Optical Flow Left ImageOptical Flow Right Image

Visual System (onboard the Vehicle) Ground Station Vehicle Hardware WIRELESS NETWORK WIRELESS NETWORK

Calibration and Filtering  Calibration: removes image distortion  Filtering Process:  Improves image quality  Increases the robustness of the vision system

Visual System (onboard the Vehicle) Ground Station Vehicle Hardware WIRELESS NETWORK WIRELESS NETWORK

Tele-Operations Laptop on TAMUBOT TAMUBOT Control SystemWireless Router Control PC Tropos Router Picture PC

Vehicle Vehicle Courtesy of Prof. Dezhen Song Baseline D L FOV 1 FOV 2 α Horizontal View  Camera support system  3-DOF mechanical neck:  Panoramic rotation  Tilt rotation  Telescopic capability  Controlled height and baseline length

Conclusions and Future Work  Demonstrated:  Ego-motion estimation  Environment Reconstruction  Vehicle control and movement  Future Developments:  System integration  Filtering and improving results

 Thanks to: –Prof. Tamás Kalmár-Nagy –Dr. Giovanni Giardini –Prof. Dezhen Song –Change Young Kim –Magda Lagoudas –Tarek Elgohary –Pedro Davalos Acknowledgements