Virtual Worlds Lab Testbed for Mobile Augmented Battlefield Visualization September, 2003 Testbed for Mobile Augmented Battlefield Visualization September, 2003 William Ribarsky and Nickolas Faust GVU Center and GIS Center Georgia Institute of Technology

Virtual Worlds Lab Matrix of Proposed Activities and Results

Virtual Worlds Lab Mobile Situational Visualization An extension of situation awareness that exploits and integrates interactive visualization, mobile computing, wireless networking, and multiple sensors: Mobile users with GPS, orientation sensing, cameras, wireless User carries own 3D database Servers that store and disseminate information from/to multiple clients (location, object/event, weather/NBC servers) Location server to manage communications between users and areas of interest for both servers and users Ability to see weather, chem/bio clouds, and positions of other users Accurate overviews of terrain with accurately placed 3D buildings Ability to mark, annotate, and share positions, directions, speed, and uncertainties of moving vehicles or people Ability to access and playback histories of movement Placement of multiresolution models from MURI team members into environment Results of real- time collection of GPS path at night (left); screen shot with annotated path in red (right). New lightweight wearable system GPS and orientation tracker

Virtual Worlds Lab Weather/Atmospheric Server Annotation Server Annotated views with updated user location and orientation Spread of dynamic Sarin gas cloud with positions of first responders Mobile Situational Visualization Accurate Shared Locations

Virtual Worlds Lab Mobile Situational Visualization System Drawing Area Buttons Pen Tool Mobile Team Collaboration Example collaborators Shared observations of vehicle location, direction, speed

Virtual Worlds Lab Collaborative Environment Everybody has a location in space and time in the Virtual World Geographic server lookup approach –Users –Location Servers –Data Servers Weather Server User Location Server Traffic Server Annotation Server GeoData Server

Virtual Worlds Lab Everybody has a location in space and time in the Virtual World Geographic server lookup approach –Users –Location Servers –Data Servers User Location Server Traffic Server Annotation Server GeoData Server Weather Server Collaborative Environment

Virtual Worlds Lab Mobile Situational Visualization Video

Virtual Worlds Lab What is Novel and Compelling About Mobile Situational Visualization? Mobile battlefield visualization was an original proposed (and accepted) task. That’s pretty compelling! But, beyond that Instant placement of environmental activity information within the geospatial environment combined with fast sharing and use are novel and compelling. -Fast, accurate, and specific annotation of activity information (both user-controlled and automated logging) -Immediate updates of databases with this information -Server structure for sharing this with collaborators or commanders in the area of interest -Use in computations and simulations (some launched automatically)

Virtual Worlds Lab Matrix of Proposed Activities and Results

Virtual Worlds Lab Integrated, Comprehensive Modeling To build comprehensive models, we need a range of modeling techniques. We also should combine techniques for richer and more complete models. Model Detail Geo- accuracy LowMidHigh Georgia Tech Thousands to tens of thousands of buildings and trees Hundreds of semi-automatically modeled buildings USC Tens to Hundreds of semi- automatically modeled buildings Berkeley Hundreds of automatically modeled buildings

Virtual Worlds Lab Integrated, Comprehensive Modeling To build comprehensive models, we need a range of modeling techniques. We also should combine techniques for richer and more complete models. Detail Geo- accuracy LowMidHigh Integrated, comprehensive models with combined techniques

Virtual Worlds Lab New Results on Modeling Large Collections Generic models extruded from accurate footprints with accurate locations. (11,000 automatically generated from insurance GIS databases). -Complete models with roofs -Generic façade textures -Databases available for automatically generating whole city (hundreds of thousands) Automatic generation of accurately located tree models (thousands) from high-res imagery. Creation of hundreds of specific buildings using commercial or self- developed (semi-automatic) software. (individual 3D buildings have brown roofs) 3D CAD modeled objects on high resolution terrain

Virtual Worlds Lab Automatic Identification and Placement of Trees, Shrubs, and Foliage This can be used with Ulrich Neuman’s or Avideh Zakhor’s results to automatically identify, remove, and model foliage.

Virtual Worlds Lab Automated identification and modeling of trees Application to Tree Modeling Accurate placement of 3D modeled trees

Virtual Worlds Lab Matrix of Proposed Activities and Results

Virtual Worlds Lab Organizing Large Collections of 3D Models for Interactive Display Merging of different types and formats Automated replacement of structures for overlapping areas Common format and organization for different types Q QQQQ Q QQQQ Q QQQQ QQQQ Linked Global Quadtrees

Virtual Worlds Lab Paging, Culling, and Fast Rendering Quadcell Block QQQQ Linked global quadtree Block Out-of core Storage

Virtual Worlds Lab Integrated, Interactive Visualization of Large Collections of Models Video

Virtual Worlds Lab Matrix of Proposed Activities and Results

Virtual Worlds Lab Handling Complicated Models Bounding box Selected LOD View-Dependent LOD for large collections of complicated models Q QQQQ Q QQQQ Q QQQQ Q N Levels Linked Global Quadtrees Results of view-dependent simplification. The blue box is the viewing window; fully textured models with and without meshes displayed are shown on the left and right, respectively. (Top) Full resolution mesh and textures within the window. (Bottom) Significantly reduced resolution mesh and textures within the window without reduction in visual quality. Viewpoint

Virtual Worlds Lab Quadric Error Approach to Simplification Initial development Garland and Heckbert, SIGGRAPH, 1997 Quadric approach yields “optimal” simplification by permitting generalized contractions between vertices and keeping track of the deviation from the original mesh v1v1 v2v2 contraction Use quadric matrix to find a vertex with error within ε; Δ is the surface at error value ε. v1v1 v2v2 general contraction Non-topological simplification

Virtual Worlds Lab Limitations on Basic Quadric Approach No concept of view-dependence and continuous LOD No structure for large collections of objects Geometry error metric; no appearance-preserving metric (e.g., for textures, shading, lighting). A combined metric is best. full resolution w/o appearance metric with appearance metric Application of appearance-preserving metric to a textured object (Cohen et al., SIGGRAPH 98) [ ]

Virtual Worlds Lab View-Dependent Continuous LOD Tree The vertex front is circled. Green nodes are active-interior, blue nodes are active-boundary, and orange nodes are inactive. Here, vertex V 7 is split and vertices V 10 and V 11 are merged. The pink, purple, and dark gray triangles are subfaces of V 7, V 5, and V 4, respectively. (a) Full mesh. (b) Tree on left. (c) Tree on right. Block Façade 1 … Façade N LOD Hierarchy … … … Object M … … … Object 1 … … … … … … … QQQQ Linked global quadtree

Virtual Worlds Lab View-Dependent Appearance-Preserving Simplification Collapse Distance Deviation possible surface M i current surface M i-1 original surface M 0 deviation vectors VaVa VbVb VcVc POPO PCPC Two-Way Incremental Distance Deviation VaVa VbVb VcVc (A) (B) Quadric Error Deviation VaVa VbVb VcVc (C) VaVa VbVb VcVc PCPC One-Way Incremental Texture Deviation (D) VaVa VbVb VcVc PAPA Two-Way Incremental Texture Deviation PBPB PCPC (E) (F) VaVa VbVb VcVc POPO Total Texture Deviation

Virtual Worlds Lab View-Dependent Appearance-Preserving Simplification Video

Virtual Worlds Lab Matrix of Proposed Activities and Results

Virtual Worlds Lab Implementing and Using the Testbed Merging of tens of thousands (and more) of models from multiple sources. Efficient organization and culling of massive collections of 3D objects. Integration of view-dependent methods for accurate and efficient display of complex models. Deployment and use of mobile situational visualization capability.

Virtual Worlds Lab Technology Transfer The VGIS visualization system with capabilities developed here (including mobile visualization) was a key part of the Georgia Tech Homeland Defense Workshop and will be part of the GT Homeland Defense Initiative with support at the State and National levels. The system is being used as part of the Sarnoff Raptor system, which is deployed to the Army and other military entities. In addition our visualization system is being used as part of the Raptor system at Scott Air Force Base. We are in discussion with the Department of the Interior on use of our mobile situational visualization capability to develop Anytime-Anywhere information system resource accessibility for countering asymmetric threats.

Virtual Worlds Lab Plans for Next Year Full deployment of mobile situational visualization capability with sharing of the system and the results with team members. Further development of automated model building from multisource data. This will be a collaborative effort with other team members. We will move towards a robust system with ability to merge and increment model sets and update models (adding improvements to make generic models more detailed and specific as data are available). Development of fully scalable 3D object organization and interactive visualization capability extending to hundreds of thousands of accurately located buildings and trees (or more). Full integration of view-dependent capability for complex models.