Presentations Computer Vision Research Lab F Relevant UMass ResearchEdward Riseman F Image Warping for Image RegistrationHoward Schultz F 3D Panoramic ImagingHoward Schultz F 3D Terrain and Site ModelingEdward Riseman F Aided Search and Target CueingGary Whitten F Knowledge-Based and Adaptive Image Processing Allen Hanson Considerable overlap of the 5 topics Interaction between subproblems and solutions

Computer Vision Personnel F Allen R. Hanson and Edward M. Riseman l Professors, Computer Science Department l Co-Directors, Computer Vision Research Lab F Howard Schultz l Research Professor, UMass since 1989 l JPL, Member of the Technical Staff, l Bendix Aerospace, Research Engineer, l ERIM, Research Scientist, F Gary Whitten l Research Professor, UMass since 1996 l University of Maryland, Research Faculty,  Co-PI with Azierl Rosenfeild, ONR ATR program l Martin-Marietta, Senior Scientist, l Fairchild Western, Senior Engineer,

Presentations Computer Vision Research Lab F Relevant UMass ResearchEdward Riseman F Image Warping for Image RegistrationHoward Schultz F 3D Panoramic ImagingHoward Schultz F 3D Terrain and Site ModelingEdward Riseman F Aided Search and Target CueingGary Whitten F Knowledge-Based and Adaptive Image Processing Allen Hanson

Relevant UMass Research F RADIUS Program DARPA/ORD (Research and Development in Image Understanding Systems)  5 Universities and 2 Corporations F UMass Ascender I  Multi-view reconstruction of buildings  Focused on accurate 3D acquisition of flat-roof structures  Produces textured-mapped CAD models  Fly-through visualization

Ascender I System for Automatic Site Modeling Multiple ImagesGeometric Models rooftop detection line extraction corner detection perceptual grouping epipolar matching multi-image triangulation geometric constraints precise photogrammetry extrusion to ground plane texture mapping

Relevant UMass Research F APGD Program DARPA/NIMA (Automatic Population of Geospatial Databases) F UMass Ascender II  Context-sensitive reconstruction of cultural sites  Knowledge-based strategies for intelligently invoking algorithms  Reconstruction of many types of complex buildings  Identification of buildings, roads, parking lots, etc.  Reconstruction from SAR and IFSAR images

Video Clip 1 Ascender I

3D Accuracy vs. Number of Views Number of views used vs. 3-D reconstruction accuracy in meters Distance (meters)

3D Reconstruction Accuracy Median planimetric and altimetric errors (in meters) between reconstructed 3-D polygon vertices and ground truth roof vertices. Image IV Planimetric IV Altimetric

Relevant UMass Research F ONR Basic Research (Recovering the Small Scale Structure of the Ocean Surface) F Terrest (Terrain Reconstruction System) l Produces a digital elevation map from pairs of aerial images l Adapted from oceanographic to land-based applications l Utilized in DARPA-Army Unmanned Ground Vehicle (UGV) program  Path planning  Visibility analysis  Battlefield awareness and visualization

Terrest Terrain Reconstruction System F Goals l Rapid and accurate generation of elevation maps l Multiple Images/Sensors l Oblique Images l Widely Separated Images l Large base-to-height ratios l Subpixel precision l Capable of using a-priori information (DTED, etc) l Capable of running on distributed systems

ISPRS ‘Flat Scene’ Left Image Right Image Reconstruction

Video Clip 2 Terrest

Relevant UMass Research F Fusion of Information from the above programs l 2D lines, corners, polygons from images l 3D lines, corners, surfaces from DEMs l Grouping to construct buildings l Applying constraints during stereo analysis to improve DEM generation

Results: Denver, Co. Texture-Mapped Rendered Elevation

DEM + Optical 1 Results from Ascender and Terrest 2D building rooftops DEM Building Extraction Wireframe DEM 

DEM + Optical 1 Combined Results DEM Alone

Kirtland AFB Optical w/ 2D polygons from Ascender Registered IFSAR Reconstruction Median ElevationProject to IFSAR

Approach: 2D Polygons + DEM Registered Optical and DEM Parameterized Model Library Site Model Model Indexing

Video Clip 3 Data fusion

Relevant UMass Research F NSF Environmental Monitoring project l Global forest management using aerial and satellite images l Terrain classification using real-time interactive decision tree classification l Video sequence registration and analysis l Automatic aerial photogrammetry

Vegetation classification and image registration Hypercluster image with GPS-logged points shown in yellow Dual camera video strip 0.4km 30m

Aerial Image Interpretation and classification for environmental monitoring

Video Clip 4 Interactive Teacher-Learner Classifier

Relevant UMass Research F 1998-? DARPA SAFER Mobile Robot Project Interdisciplinary - vision, robotics, AI, software engineering l Cooperative mobile robots in search and rescue l Multiple mobile robots and sensor configurations l 3D obstacle detection and avoidance l Object (human) recognition l 3D reconstruction from Panoramic images

Robotic Panoramic Sensor for 3D reconstruction F Current UMass projects with similar panoramic sensor F Change in azimuth and scale of an object determines its location in space

Presentations Computer Vision Research Lab F Relevant UMass ResearchEdward Riseman F Image Warping for Image RegistrationHoward Schultz F 3D Panoramic ImagingHoward Schultz F 3D Terrain and Site ModelingEdward Riseman F Aided Search and Target CueingGary Whitten F Knowledge-Based and Adaptive Image Processing Allen Hanson

F Precise image registration depends on l Sensor projection model l Internal model parameters (focal length, distortion,...) l Sensor physics (IR, EO, Radar,...) l External parameters l Scene geometry Image Warping for Image Registration

Context Sensitive Image Warping F Objects at infinity F Distant objects not at infinity “billboard objects” F Close objects F No depth information required F Depends no distance to target F Some level of 3D reconstruction required Scene GeometryWarping Function

F To register objects at infinity l No 3D information l Requires calibration parameters measured at assembly Sensor models Focal length Lens distortion Relative orientation between sensors Image Registration

F Factors for Registering objects not at infinity l Calibration parameters, plus l Distance to target  radar  sonar  parallax F Factors for Registering close objects l Calibration parameters plus l Full or partial 3D reconstruction  model matching  stereo Image Registration

F UMass calibration lab designed for l video and small format cameras l Geometric accuracy to ~ 0.001° l Radiometric accuracy to 1:1,000 Calibration

F Relevant UMass ResearchEdward Riseman F Image Warping for Image RegistrationHoward Schultz F 3D Panoramic ImagingHoward Schultz F 3D Terrain and Site ModelingEdward Riseman F Aided Search and Target CueingGary Whitten F Knowledge-Based and Adaptive Image Processing Allen Hanson Presentations Computer Vision Research Lab

F From a sequence of overlapping digital l Create a mosaic (composite) image l Individual images are seamlessly patched l New images overwrite old images F For most applications the mosaic grows indefinitely F For panoramic applications the mosaic image wraps around Panoramic Image Stabilization

F Motivated by DARPA VSAM program (Video Surveillance and Monitoring) l Processing video sequences for  motion detection  stabilization  object tracking  activity recognition l Relevant factors for submarine domain  unmodeled sensor motion  small temporal interval  small image displacement Approach

Mosaic from an image sequence

0o0o 360 o Panoramic Mosaic from an image sequence

F With the current configuration l Operator sees only one frame at a time l Maximum scan rate is 6 seconds F A stabilized panoramic view enables l The operator to see the entire field-of-view l Separates scene acquisition and viewing procedures l Much faster, more reliable observations  A full panoramic view will take ~2 seconds to generate l Computation of the distance to an object from multiple panoramic views 3D Panoramic Imaging

F From two stabilized panoramic views l determine the location of targets l track objects 3D Panoramic Imaging Time 1Time 2 Motion Parallax

Robotic Panoramic Sensor for 3D reconstruction F Current UMass projects with similar panoramic sensor F Change in azimuth and scale of an object determines its location in space

Presentations Computer Vision Research Lab F Relevant UMass ResearchEdward Riseman F Image Warping for Image RegistrationHoward Schultz F 3D Panoramic ImagingHoward Schultz F 3D Terrain and Site ModelingEdward Riseman F Aided Search and Target CueingGary Whitten F Knowledge-Based and Adaptive Image Processing Allen Hanson

3D Terrain and Site Modeling Naval Scenarios F DOD Joint Program Office (JPO) l UGV - Unmanned Ground Vehicle Program l UAV - Unmanned Air Vehicle Program F UAV Images could be used by UGVs for l 3D terrain modeling l mission rehearsal l path planning l ATD and ATR

3D Terrain and Site Modeling Naval Scenarios F Naval UAV images for l 3D site modeling of coastal regions l Register periscope images with 3D terrain models l Visualization fly-through l Training and mission planing l ATD and ATR

3D Terrain and Site Modeling Naval Scenarios F Motion Sequences from Periscope Images l 3D reconstruction from ocean level views l Frontal view of coast allows accurate range estimates  cannot view building roofs  obscure terrain (e.g., behind buildings) l Panoramic views enables reconstruction of the complete surrounding naval environment

Presentations Computer Vision Research Lab F Relevant UMass ResearchEdward Riseman F Image Warping for Image RegistrationHoward Schultz F 3D Panoramic ImagingHoward Schultz F 3D Terrain and Site ModelingEdward Riseman F Aided Search and Target CueingGary Whitten F Knowledge-Based and Adaptive Image Processing Allen Hanson

Presentations Computer Vision Research Lab F Relevant UMass ResearchEdward Riseman F Image Warping for Image RegistrationHoward Schultz F 3D Panoramic ImagingHoward Schultz F 3D Terrain and Site ModelingEdward Riseman F Aided Search and Target CueingGary Whitten F Knowledge-Based and Adaptive Image Processing Allen Hanson