Next Generation 4D Distributed Modeling and Visualization Avideh Zakhor (UCB) Ulrich Neumann (USC) William Ribarsky (Georgia Tech) Pramod Varshney (Syracuse) Suresh Lodha (UCSC)

Accomplishments (1) Developed a mobile 3D data acquisition system capable of working in unconstrained Urban environments. Developed algorithms for building 3D point clouds using 2D laser scanners. Developed new techniques for multi-sensor fusion for pose estimation. Developed techniques for extraction of 3D information from aerial images. Our multi- sensor platform 3D point Cloud of a block 3D information extracted from aerial stereo-pairs

Accomplishments (2) New algorithms for obtaining 6DOF tracking of a moving camera (without requiring any calibrated features in the scene) from multiple 5DOF-motion estimates relative to reference images Developed algorithms for tracking moving vehicles– ground, air, water (s imilar sensors and fusion ideas, e.g. EKF) –System: RTK differential GPS, Ashtech Z- Sensor, 3D gyros/compass, Panoramic and forward-down or side-down cameras –Sparse models (LiDAR laser scanner) provides tracking landmarks (and model for texture painting) Developed image projection techniques onto underlying 3D models to facilitate visualization and comprehension of multiple image streams R1 R2 C Tracked camera image Reference image 5DOF motion estimates Camera path Panoramic tracking Outdoor tracking system Image projection on 3D models

Accomplishments (3) Development of mobile visualization capability using laptop version of global geospatial system Initial work on synchronized databases in client-server or peer-to-peer mode for sharing geospatial data and annotations Work on development of wireless network, differential GPS, and orientation tracking for mobile visualization. Work on hand gesture mobile interface using body-mounted camera Gesture interface with heads-up display.

Accomplishments(4) Conceptualization of uncertainty representation, computation and visualization for situation awareness

Accomplishments(5) Formulation of algorithm for integrating uncertain inputs arriving from different battlefield observers at different times Formulation of an algorithm for estimating uncertainty associated with trajectories computed from multiple observers over time Initiated investigation of computation and visualization of uncertainty for terrains and different types of data within the GIS context. Initiated use of multi-modal means to convey uncertainty in an already cluttered and overloaded visualization using sound.

Transitions (1) Established close relationships with both government agencies and industry. Close interaction with army at AFRL Information Directorate, TRADOC Analysis Center in order to determine the current state-of- the-art tools available to the army for planning research directions. Established close relationships with Vexcel Corporation, for collaboration in aerial photogrammetry, and with Bosch, Planet 9 and Geometrix for collaborations on close-range modeling of urban areas. Technical presentations to industries including Carrier Corp., Andro consulting, Niagara Mohawk Power Corp., General Electric R&D Center, Kitware and Raytheon. HRL (Malibu CA) software evaluation - using tracking methods and software for DOD and commercial applications

Transitions (2) Office of Naval Research (ONR) VR lab software evaluation - using inertial and vision tracking methods and software for outdoor AR project (also funding a major portion of the panoramic tracking work here at USC) HP (Palo Alto CA) software evaluation - using tracking methods and software for commercial applications Institute of Creative Technologies (ICT) large-area simulation and training project - possible application and testbed for wide area tracking Boeing (Long Beach CA) for training - using tracking methods and software for DOD and commercial applications NASA (Houston Space Center) for shuttle/ISS/Mars training - using tracking methods and software for training

Transitions (3) Olympus America (New York) software evaluation - using tracking methods and software for commercial applications Briefings and demonstrations to industry are part of IMSC partnership program at USC. Over calendar year 2000, IMSC hosted over 470 visitors from companies and government agencies. Patent application is being written for the use of autocalibration in AR systems. Dr. Jun Park (PhD, 2000) was hired at Rockwell Science Labs (Thousand Oaks, CA). Dr. Ilmi Yoon (PhD, 2000) was hired at CS Dept. Cal State San Francisco.

Transitions (4) Demonstrated mobile visualization capability to Ward Page, DARPA program manager for Command Post of the Future, as one of foundation ideas for a follow-on program. Through CERTIP (Center for Emergency Response Technology, Instrumentation, and Policy), worked with the Marines, with some funding from ONR, on urban emergency response. This has resulted in a field experiment to investigate technologies that could assist responders in a chemical/biological terrorist incident. We then participated in an on-campus simulated terrorist attack. Georgia Tech campus area in mobile visualization Overview and fly-in to attack point on Georgia Tech campus First responder icons updated over Web every 40 seconds. Ultimate goal:complete mobile visualization Simulated Terrorist Attack

Collaborations UC Berkeley and Georgia Tech on extraction of urban building and terrain models and their visualization. UC Berkeley and Syracuse University on computation and characterization of uncertainty for data fusion and change detection. Georgia Tech and USC on research issues in tracking, orientation, and synchronization of common geospatial databases. Georgia Tech and UC Santa Cruz on insertion and visualization of uncertainty in geospatial environments. Syracuse University and UC Santa Cruz on computation, characterization and representation of uncertainty in various data types and modalities.

Publications (1) Hassan Foroosh, “Is Optical Flow Estimation an Under-determined Problem?”, submitted to International Journal of Computer Vision, October Hassan Foroosh, “Optical Flow Estimation Under Integrability Constraint”, under preparation to be submitted to International Conference on Computer Vision and Pattern Recognition, May Hassan Foroosh, “A Closed-Form Solution For Optical Flow By Imposing Temporal Constraints”, submitted to International Conference on Image Processing, January Christian Frueh, Hassan Foroosh, “Fast 3D Model Generation in Urban Environments”, submitted to IEEE Conf. on Multisensor Fusion and Integration for Intelligent Systems, February N.L. Chang and A. Zakhor, “A Multivalued Representation for View Synthesis”, in Proceedings of the International Conference on Image Processing, Kobe, Japan, October 1999, vol. 2, pp ; Also submitted to International Journal of Computer Vision, January 2000

Publications (2) T.Y. Jiang, William Ribarsky, Tony Wasilewski, Nickolas Faust, Brendan Hannigan, and Mitchell Parry,“Acquisition and Display of Real-Time Atmospheric Data on Terrain,” to be published, Proceedings of Eurographics- IEEE Visualization Symposium Mitchell Parry, Brendan Hannigan, William Ribarsky, T.Y. Jiang, and Nickolas Faust, “Hierarchical Storage and Visualization of Real-Time 3D Data,” to be published, Proceedings of SPIE Aerosense Mitchell Parry, William Ribarsky, Chris Shaw, Tony Wasilewski, Nickolas Faust, T.Y. Jiang, and David Krum, “Acquisition, Management, and Use of Large-Scale, Dynamic Geospatial Data,” submitted to IEEE Computer Graphics & Applications J. W. Lee, S. You, and U. Neumann. “Large Motion Estimation for Omnidirectional Vision,” Proc. IEEE Workshop on Omnidirectional Vision, pp , June J.W. Lee, U. Neumann, "Motion Estimation with Incomplete Information Using Omni-Directional Vision," IEEE International Conference on Image Processing, Vancouver Canada, vol. II, pp , September 2000.

Publications (3) J.W. Lee, S. You, U. Neumann. “Motion-Based Pose Tracking,” submitted to IEEE ISMR/VR2001. S. You, U. Neumann. “Fusion of Vision and Gyro Tracking for Robust Augmented Reality Registration”, Accepted in IEEE VR2001, Yokohama, Japan, March Antonio Hasbun, Pramod K. Varshney, Kishan G. Mehrotra and C.K. Mohan, “Uncertainty Introduced by Quantization in Image Processing Applications”, Under preparation. Kishan G. Mehrotra, C.K. Mohan and Pramod K. Varshney, “Theory and Applications of Random Sets: A Review”, Under preparation. Suresh K. Lodha, Krishna M. Roskin, and Jose Renteria,``Hierarchical Topology Preserving Compression of 2D Terrains'', Submitted for publication to Computer Graphics Forum. Suresh K. Lodha and Arvind Verma, ``Spatio-Temporal Visualization of Urban Crimes on a GIS Grid'', Proceedings of the ACM GIS Conference, November 2000, ACM Press, pages

Publications (4) Suresh K. Lodha, Jose Renteria and Krishna M. Roskin, ``Topology Preserving Compression of 2D Vector Fields'', Proceedings of IEEE Visualization '2000, October 2000, pp Christopher Campbell, Michael Shafae, Suresh K. Lodha and D. Massaro, ``Multimodal Representations for the Exploration of Multidimensional Fuzzy Data", Submitted for publication to Behavior Research Methods, Instruments, and Computers.