Depth Analysis With Stereo Cameras Timo Hohn and Leo Nickerson Group 5

Functionality Motivation Take stereo images of a scene and calculate the distance objects are away from the two cameras. This distance will be represented on an output image as a color gradient that is displayed on an external VGA monitor. Motivation 3D depth analysis for remote robot imagery, where more information of the scene is necessary. Object detection and avoidance applications for robot navigation where IR systems are not available.

Hardware Block Diagram

Software Flow Diagram Start (User Initiates Analysis) Collect JPEGS from Both Cameras Trim Images To Overlap Pixel Recognition Depth Analysis of Matched Pixels Update output image Modify Video Buffer to Include Output Image as Overlay

Design Challenges Interfacing the Cameras` NTSC video in and RS232 communication to the DE2 board Adapting the camera communication library to work well with the DE2 board Using the onboard TV decoder to collect frame buffers from one of the two cameras Sending manipulated frames to VGA out. Application Notes are available from previous projects Pixel Recognition Open source libraries from the OpenCV project will be used for matching pixels May perform image subtraction to reduce number of pixels required to analyze

Included FPGA and External Components Cyclone II Processor (currently economy version) VGA and TV decoder accessed as Avalon-MM slaves Use of onboard SRAM modules and the possibility of larger SDRAM storage depending on memory use GPIO pins (Tx, Rx, and GND pins for each camera, which use 3.3V logic) Additional +5V power supply for camera modules (either USB wall plug or battery packs) A single onboard push button for initiating image capture

Example C++ Module For Finding Distances #include "DistFind.h" #include <cmath> DistFind::DistFind(int x_res, int fov_degree, double width){ x_res = x_res; fov_rad = (double)fov_degree*PI/180; rad_per_pixel = fov_rad/x_res; width = width; } double DistFind::find(int x_camera1, int x_camera2){ double offset1 = rad_per_pixel/2 + abs(x_camera1-320)*rad_per_pixel; double offset2 = rad_per_pixel/2 + abs(x_camera2-320)*rad_per_pixel; … … (x_camera1>320) ? offset1*=-1 : ; (x_camera2<320) ? offset2*=-1 : ; double phi1 = PI/2 + offset1; double phi2 = PI/2 + offset2; double angle_c = PI - phi1 - phi2; double sine_ratio = width/sin(angle_c); double side1 = sine_ratio*sin(phi2); double side2 = sine_ratio*sin(phi1); return (phi1<phi2) ? sin(phi1)*side1 : sin(phi2)*side2; }

Unit Testing Integration Testing Using stored images to test depth and pixel recognition tasks Sending an image out the VGA port to test that interface Collect raw data from NTSC video and observe its accuracy Take raw video input from one camera and route it directly to the VGA port Starting with a basic, high contrast environment with a single object. Work our way up to more complex environments involving overlapping objects.

Optional Features Additional Features Secondary Features Performing depth calculations on real time video (additional hardware required) Automation and optimization of frequent image captures (eliminate the need for user initialization) Secondary Features Removal of the NTSC video overlay and only display still images Simplification of analysis environment if pixel recognition algorithms deem inaccurate

Questions?