2. Presentations Computer Vision Research Lab F Relevant UMass ResearchEdward Riseman F Image Warping for Image RegistrationHoward Schultz F 3D Panoramic ImagingHoward Schultz F 3D Terrain and Site ModelingEdward Riseman F Aided Search and Target CueingGary Whitten F Knowledge-Based and Adaptive Image Processing Allen Hanson Considerable overlap of the 5 topics Interaction between subproblems and solutions

3. Computer Vision Personnel F Allen R. Hanson and Edward M. Riseman l Professors, Computer Science Department l Co-Directors, Computer Vision Research Lab F Howard Schultz l Research Professor, UMass since 1989 l JPL, Member of the Technical Staff, 1984-86 l Bendix Aerospace, Research Engineer, 1982-84 l ERIM, Research Scientist, 1978-82 F Gary Whitten l Research Professor, UMass since 1996 l University of Maryland, Research Faculty, 1995-96  Co-PI with Azierl Rosenfeild, ONR ATR program l Martin-Marietta, Senior Scientist, 1990-95 l Fairchild Western, Senior Engineer, 1985-90

4. Presentations Computer Vision Research Lab F Relevant UMass ResearchEdward Riseman F Image Warping for Image RegistrationHoward Schultz F 3D Panoramic ImagingHoward Schultz F 3D Terrain and Site ModelingEdward Riseman F Aided Search and Target CueingGary Whitten F Knowledge-Based and Adaptive Image Processing Allen Hanson

5. Relevant UMass Research F 1993-1996 RADIUS Program DARPA/ORD (Research and Development in Image Understanding Systems)  5 Universities and 2 Corporations F UMass Ascender I  Multi-view reconstruction of buildings  Focused on accurate 3D acquisition of flat-roof structures  Produces textured-mapped CAD models  Fly-through visualization

6. Ascender I System for Automatic Site Modeling Multiple ImagesGeometric Models rooftop detection line extraction corner detection perceptual grouping epipolar matching multi-image triangulation geometric constraints precise photogrammetry extrusion to ground plane texture mapping

7. Relevant UMass Research F 1996-1999 APGD Program DARPA/NIMA (Automatic Population of Geospatial Databases) F UMass Ascender II  Context-sensitive reconstruction of cultural sites  Knowledge-based strategies for intelligently invoking algorithms  Reconstruction of many types of complex buildings  Identification of buildings, roads, parking lots, etc.  Reconstruction from SAR and IFSAR images

8. Video Clip 1 Ascender I

9. 3D Accuracy vs. Number of Views Number of views used vs. 3-D reconstruction accuracy in meters Distance (meters).25.50.75 1.0 2 3 4 5 6

10. 3D Reconstruction Accuracy Median planimetric and altimetric errors (in meters) between reconstructed 3-D polygon vertices and ground truth roof vertices. Image711713525927 IV Planimetric0.680.731.090.89 IV Altimetric0.510.550.900.61

11. Relevant UMass Research F 1989-1995 ONR Basic Research (Recovering the Small Scale Structure of the Ocean Surface) F Terrest (Terrain Reconstruction System) l Produces a digital elevation map from pairs of aerial images l Adapted from oceanographic to land-based applications l Utilized in DARPA-Army Unmanned Ground Vehicle (UGV) program  Path planning  Visibility analysis  Battlefield awareness and visualization

12. Terrest Terrain Reconstruction System F Goals l Rapid and accurate generation of elevation maps l Multiple Images/Sensors l Oblique Images l Widely Separated Images l Large base-to-height ratios l Subpixel precision l Capable of using a-priori information (DTED, etc) l Capable of running on distributed systems

13. ISPRS ‘Flat Scene’ Left Image Right Image Reconstruction

14. Video Clip 2 Terrest

15. Relevant UMass Research F Fusion of Information from the above programs l 2D lines, corners, polygons from images l 3D lines, corners, surfaces from DEMs l Grouping to construct buildings l Applying constraints during stereo analysis to improve DEM generation

16. Results: Denver, Co. Texture-Mapped Rendered Elevation

17. DEM + Optical 1 Results from Ascender and Terrest 2D building rooftops DEM Building Extraction Wireframe DEM 

18. DEM + Optical 1 Combined Results DEM Alone

19. Kirtland AFB Optical w/ 2D polygons from Ascender Registered IFSAR Reconstruction Median ElevationProject to IFSAR

20. Approach: 2D Polygons + DEM Registered Optical and DEM Parameterized Model Library Site Model Model Indexing

21. Video Clip 3 Data fusion

22. Relevant UMass Research F 1998-2001 NSF Environmental Monitoring project l Global forest management using aerial and satellite images l Terrain classification using real-time interactive decision tree classification l Video sequence registration and analysis l Automatic aerial photogrammetry

23. Vegetation classification and image registration Hypercluster image with GPS-logged points shown in yellow Dual camera video strip 0.4km 30m

24. Aerial Image Interpretation and classification for environmental monitoring

25. Video Clip 4 Interactive Teacher-Learner Classifier

26. Relevant UMass Research F 1998-? DARPA SAFER Mobile Robot Project Interdisciplinary - vision, robotics, AI, software engineering l Cooperative mobile robots in search and rescue l Multiple mobile robots and sensor configurations l 3D obstacle detection and avoidance l Object (human) recognition l 3D reconstruction from Panoramic images

27. Robotic Panoramic Sensor for 3D reconstruction F Current UMass projects with similar panoramic sensor F Change in azimuth and scale of an object determines its location in space

28. Presentations Computer Vision Research Lab F Relevant UMass ResearchEdward Riseman F Image Warping for Image RegistrationHoward Schultz F 3D Panoramic ImagingHoward Schultz F 3D Terrain and Site ModelingEdward Riseman F Aided Search and Target CueingGary Whitten F Knowledge-Based and Adaptive Image Processing Allen Hanson

29. F Precise image registration depends on l Sensor projection model l Internal model parameters (focal length, distortion,...) l Sensor physics (IR, EO, Radar,...) l External parameters l Scene geometry Image Warping for Image Registration

30. Context Sensitive Image Warping F Objects at infinity F Distant objects not at infinity “billboard objects” F Close objects F No depth information required F Depends no distance to target F Some level of 3D reconstruction required Scene GeometryWarping Function

31. F To register objects at infinity l No 3D information l Requires calibration parameters measured at assembly Sensor models Focal length Lens distortion Relative orientation between sensors Image Registration

32. F Factors for Registering objects not at infinity l Calibration parameters, plus l Distance to target  radar  sonar  parallax F Factors for Registering close objects l Calibration parameters plus l Full or partial 3D reconstruction  model matching  stereo Image Registration

33. F UMass calibration lab designed for l video and small format cameras l Geometric accuracy to ~ 0.001° l Radiometric accuracy to 1:1,000 Calibration

34. F Relevant UMass ResearchEdward Riseman F Image Warping for Image RegistrationHoward Schultz F 3D Panoramic ImagingHoward Schultz F 3D Terrain and Site ModelingEdward Riseman F Aided Search and Target CueingGary Whitten F Knowledge-Based and Adaptive Image Processing Allen Hanson Presentations Computer Vision Research Lab

35. F From a sequence of overlapping digital l Create a mosaic (composite) image l Individual images are seamlessly patched l New images overwrite old images F For most applications the mosaic grows indefinitely F For panoramic applications the mosaic image wraps around Panoramic Image Stabilization

36. F Motivated by DARPA VSAM program (Video Surveillance and Monitoring) l Processing video sequences for  motion detection  stabilization  object tracking  activity recognition l Relevant factors for submarine domain  unmodeled sensor motion  small temporal interval  small image displacement Approach

37. Mosaic from an image sequence

38. 0o0o 360 o Panoramic Mosaic from an image sequence

39. F With the current configuration l Operator sees only one frame at a time l Maximum scan rate is 6 seconds F A stabilized panoramic view enables l The operator to see the entire field-of-view l Separates scene acquisition and viewing procedures l Much faster, more reliable observations  A full panoramic view will take ~2 seconds to generate l Computation of the distance to an object from multiple panoramic views 3D Panoramic Imaging

40. F From two stabilized panoramic views l determine the location of targets l track objects 3D Panoramic Imaging Time 1Time 2 Motion Parallax

41. Robotic Panoramic Sensor for 3D reconstruction F Current UMass projects with similar panoramic sensor F Change in azimuth and scale of an object determines its location in space

42. Presentations Computer Vision Research Lab F Relevant UMass ResearchEdward Riseman F Image Warping for Image RegistrationHoward Schultz F 3D Panoramic ImagingHoward Schultz F 3D Terrain and Site ModelingEdward Riseman F Aided Search and Target CueingGary Whitten F Knowledge-Based and Adaptive Image Processing Allen Hanson

43. 3D Terrain and Site Modeling Naval Scenarios F DOD Joint Program Office (JPO) l UGV - Unmanned Ground Vehicle Program l UAV - Unmanned Air Vehicle Program F UAV Images could be used by UGVs for l 3D terrain modeling l mission rehearsal l path planning l ATD and ATR

44. 3D Terrain and Site Modeling Naval Scenarios F Naval UAV images for l 3D site modeling of coastal regions l Register periscope images with 3D terrain models l Visualization fly-through l Training and mission planing l ATD and ATR

45. 3D Terrain and Site Modeling Naval Scenarios F Motion Sequences from Periscope Images l 3D reconstruction from ocean level views l Frontal view of coast allows accurate range estimates  cannot view building roofs  obscure terrain (e.g., behind buildings) l Panoramic views enables reconstruction of the complete surrounding naval environment

46. Presentations Computer Vision Research Lab F Relevant UMass ResearchEdward Riseman F Image Warping for Image RegistrationHoward Schultz F 3D Panoramic ImagingHoward Schultz F 3D Terrain and Site ModelingEdward Riseman F Aided Search and Target CueingGary Whitten F Knowledge-Based and Adaptive Image Processing Allen Hanson

47. Presentations Computer Vision Research Lab F Relevant UMass ResearchEdward Riseman F Image Warping for Image RegistrationHoward Schultz F 3D Panoramic ImagingHoward Schultz F 3D Terrain and Site ModelingEdward Riseman F Aided Search and Target CueingGary Whitten F Knowledge-Based and Adaptive Image Processing Allen Hanson