NAKAGAWA Masafumi : INSTITUTE OF TECHNOLOGY Panel on 3D data collection and reconstruction Chair: Masafumi Nakagawa, Shibaura Institute of Technology Thursday, 12th, 13:30-14:30 Sander Oude Elberink, University of Twente (via Adobe) Martin Tamke, CITA, KADK (via Adobe) Alain Lapierre, Bentley Susanne Becker, Stuttgart University The Royal Danish Academy of Fine Arts, Schools of Architecture, Design and Conservation (3D reconstruction) (BIM, Indoor localization) (3D modeling and design) (Laser scanning, 3D reconstruction) (3D modeling, Seamless positioning)

NAKAGAWA Masafumi : INSTITUTE OF TECHNOLOGY Panel on 3D data collection and reconstruction What is the biggest challenge in Indoor 3D data collection and reconstruction ( ) - Question 1 - Question 2 - Question 3 - Questions from audiences - Introduction 10 min 15 min 5 min - Summary10 min

NAKAGAWA Masafumi : INSTITUTE OF TECHNOLOGY Indoor mapping and modeling (IMM) Sensors Devices Data structures Acquisition Modeling Navigation Visualization Applications Legal Issues and Standards Algorithms Solving problems in one field of IMM benefits other fields of IMM

NAKAGAWA Masafumi : INSTITUTE OF TECHNOLOGY 3D data collection and reconstruction Sensing and modeling are also required for indoor mapping CAD data CityGML （City Geography Markup Language） LOD1LOD2LOD3LOD4LOD0 BoxBox, roof, facadeOutsideInside Standard, - Point cloud Data reduction Modeling LOD4～5 ? - Moving object - Changing object - Images Sensing

NAKAGAWA Masafumi : INSTITUTE OF TECHNOLOGY Point based rendering ＋ spatial interpolation Improved “point cloud” to be used as “a panorama image” INPUT （ colored points ） OUTPUT （ colored points ） Side looking Transparent effect “Far” from a viewpoint ＝ Dense points “Near” from a viewpoint ＝ Sparse points “Far” from a viewpoint ＝ Dense points Near-far problem Hidden points are visible among near-side points

NAKAGAWA Masafumi : INSTITUTE OF TECHNOLOGY 3D scanned “Gunkanjima” with Terrestrial LiDAR VZ-400 （ RIEGL ） 800 million pts, 65 view points Data source : - Keisoku Research Consultant. Co.,ltd. - Nagasaki University - Nagasaki City Rendering : - Shibaura Institute of Tech.

NAKAGAWA Masafumi : INSTITUTE OF TECHNOLOGY PROBLEMS IN INDOOR MAPPING AND MODELLING Security and levels of access Privacy Copyright The diversity of indoor environments Unification of outdoor and indoor models Gaming Industrial applications Natural description of indoor environments Augmented systems Indoor modelling for crisis response Optimal routing Navigation queries and multiplicity of targets Travelling imperatives Automated space subdivision Navigation models Real-time decision support Discrete vs continuous navigation models Real-time change visualization Complexity visualization Aural cues PoI and landmarks strategies Web and mobile devices Real-time modelling Dynamic abstraction Discovering the context of space Diversity of Indoor Environments Software tool Integration with GIS/BIM Sensor fusion Mobility Real-time acquisition of dynamic environments Variable occupancy, automated feature removal Variable lighting conditions Learning the composition of space Acquisition and Sensors Existing problems Emerging problems Data Structures and Modelling VisualizationNavigationApplications Legal Issues and Standards

NAKAGAWA Masafumi : INSTITUTE OF TECHNOLOGY Objectives Still camera, Video camera, Stereo, Tablet PC, Terrestrial scanner, Mobile scanner, etc. ①The best device for data acquisition ? Real-time acquisition of dynamic environments, ②Real-time processing is required ? Professionals vs. Cloud sourcing vs. Full-automation ③The best data collector and modeler ? Real-time modelling Dynamic abstraction Discovering the context of space Diversity of Indoor Environments Software tool Integration with GIS/BIM Sensor fusion Mobility Real-time acquisition of dynamic environments Variable occupancy, automated feature removal Variable lighting conditions Learning the composition of space Acquisition and Sensors Data Structures and Modelling Real-time modeling, Dynamic abstraction Accuracy, Cost, etc. Cost, Reliability, Integrity, etc. What is the biggest challenge in Indoor 3D data collection and reconstruction ( ) Discovering the context of space

NAKAGAWA Masafumi : INSTITUTE OF TECHNOLOGY ①The best device for data acquisition 3D data acquisition without GNSS - Scanner (+Registration) - Camera - Mobile scanner - Photogrammetry - Structure from Motion - Terrestrial LiDAR - Kinect, TOF imager - Line scanner + IMU × Manual editing works × Moving objects × Cost × Measurement distance × Cost ○ Cost ○ Cost, Speed ○ Speed What is the best device for indoor data acquisition ?

NAKAGAWA Masafumi : INSTITUTE OF TECHNOLOGY ①The best device for data acquisition Sander Martin Alain Susanne What is the best device for indoor data acquisition ? - Knowledge and indoor traces - Depends on the targeted tasks, required fidelity/accuracy and costs. - 3d scannner, potentially lightweight Lidar - Currently: Terrestrial laser scanner - For maintenance/operation purposes, panoramic photos might be sufficient. - For navigation, panoramic photos and/or Laser Scanner. - For fidelity/measurement, combination of Laser Scanner (today static, future mobile) and Photos. - Future (after 2014?): mobile devices (laser scanner). A device that someone can carry in their hand, on their head etc. - Reason: currently the registration of mobile acquired point/image data is not good enough.

NAKAGAWA Masafumi : INSTITUTE OF TECHNOLOGY ②Real-time processing Real-time processing is required ? - Scene - Applications - Objects - BIM - LBS - Public space - Work space - Passageway - Autonomous robots Station, Shopping mall, Exhibition hall, Airport Factory/Plant, Office, School/University - Private space Your house - Shopping mall - RoomWall, Floor, Ceiling, Window, Table, Chair, Electric appliances, etc. Signboard, Goods, Price tags, etc. Wall, Floor, Ceiling, Window, Walker, etc. Infrastructure management Route planning, Navigation, Dynamic abstraction Navigation, Dynamic environment recognition What is your interested real-time processing application(s) ? - Factory Machine, Parts, Pipeline, etc.

NAKAGAWA Masafumi : INSTITUTE OF TECHNOLOGY ②Real-time processing Sander Martin Alain Susanne What is your interested real-time processing application(s) ? - Real-time acquisition and modeling of dynamic environments - For augmented reality, need to register in real-time the reality with the virtual model - Detection of spaces and 3d objects and their semantics - Route planning, evacuations. For evacuations it is necessary to capture the current situation, and adapt to that. So a fast, post event dataset is required. - During construction, progress monitoring and last minute changes.

NAKAGAWA Masafumi : INSTITUTE OF TECHNOLOGY ③The best data collector and modeler Professionals vs. Cloud sourcing vs. Full-automation - Cloud sourcing - Professionals - Full-automation ○ Reliability × Resource ○ Resource × Reliability ○ Speed → Tools, Software, Freeware ? → Cost ? × Integrity → Software, Algorithm ? Who is the best data collector and modeler ? (e.g. OpenStreetMap) # Indoor field is larger than outdoor # Volunteers are nonprofessionals # 100 % success rate ?

NAKAGAWA Masafumi : INSTITUTE OF TECHNOLOGY ③The best data collector and modeler Sander Martin Alain Susanne Who is the best data collector and modeler ? - Automatic modeling from crowd sourced data - Depends on disciplines and user needs. Being a software provider, we are aiming at providing the best software, but the competition and open source community is doing a great job too. Crowd sourcing could apply for less specialized and rigorous tasks. - today: Professional users - The one that best fits the users' needs. How well are the users' needs defined? Maybe the user can best process the data, if he/she is equipped with a variety of interactive tools. - future: algorithms augmented by Professional users

NAKAGAWA Masafumi : INSTITUTE OF TECHNOLOGY Questions from audiences Still camera, Video camera, Stereo, Tablet PC, Terrestrial scanner, Mobile scanner, etc. ①The best device for data acquisition ? Real-time acquisition of dynamic environments, ②Real-time processing is required ? Professionals vs. Cloud sourcing vs. Full-automation ③The best data collector and modeler ? Real-time modelling Dynamic abstraction Discovering the context of space Diversity of Indoor Environments Software tool Integration with GIS/BIM Sensor fusion Mobility Real-time acquisition of dynamic environments Variable occupancy, automated feature removal Variable lighting conditions Learning the composition of space Acquisition and Sensors Data Structures and Modelling Real-time modeling, Dynamic abstraction Accuracy, Cost, etc. Cost, Reliability, Integrity, etc. Discovering the context of space What is the biggest challenge in Indoor 3D data collection and reconstruction ( )

NAKAGAWA Masafumi : INSTITUTE OF TECHNOLOGY Additional questions Glass effect Technical issues in 3D data acquisition Illumination Indoor-Outdoor Pedestrian (dynamic environments)

NAKAGAWA Masafumi : INSTITUTE OF TECHNOLOGY Summary Sander Martin Alain Susanne What is the biggest challenge in Indoor 3D data collection and reconstruction ( ) - Semantic interpretation - Automated processing of huge volume of data (data transfer, feature extraction, semantic interpretation) - Gaining comprehensive data with semantic information - Comment