SA2014.SIGGRAPH.ORG SPONSORED BY Automatic Semantic Modeling of Indoor Scenes from Low-quality RGB-D Data using Contextual Information Kang Chen 1 Yu-Kun Lai 2 Yu-Xin Wu 1 Ralph Martin 2 Shi-Min Hu 1 1 Tsinghua University ， Beijing 2 Cardiff University

SA2014.SIGGRAPH.ORG SPONSORED BY Overview Input Output Semantic Modeling

SA2014.SIGGRAPH.ORG SPONSORED BY Challenges Depth information: noisy may be distorted have large gaps Interior objects: complex 3D geometry with messy surroundings variation between parts

SA2014.SIGGRAPH.ORG SPONSORED BY Previous Work User interaction High-precision 3D scanners [Shao et al. 2012] [Nan et al. 2012]

SA2014.SIGGRAPH.ORG SPONSORED BY Observations Objects normally have strong contextual relationships Interior objects often have an underlying structure

SA2014.SIGGRAPH.ORG SPONSORED BY Pipeline

SA2014.SIGGRAPH.ORG SPONSORED BY 2D Images ill-posed 3D scenes full 3d relationships easy to acquire (Trimble 3D warehouse) Why virtual scenes?

SA2014.SIGGRAPH.ORG SPONSORED BY Can virtual scenes do the job ？ Real-world scene

SA2014.SIGGRAPH.ORG SPONSORED BY Training Data Generation Objects and their parts in virtual scenes are often parallel or perpendicular to each other. unlikely to occur in real world scenes.

SA2014.SIGGRAPH.ORG SPONSORED BY Training Data Generation

SA2014.SIGGRAPH.ORG SPONSORED BY Training Data Generation moveable objects: chair, mouse and keyboard.

SA2014.SIGGRAPH.ORG SPONSORED BY Context Representation

SA2014.SIGGRAPH.ORG SPONSORED BY Comparisons Without Context With Context

SA2014.SIGGRAPH.ORG SPONSORED BY Context-based Optimization

SA2014.SIGGRAPH.ORG SPONSORED BY How much context ?

SA2014.SIGGRAPH.ORG SPONSORED BY Top-down Matching

SA2014.SIGGRAPH.ORG SPONSORED BY Top-down Matching balancing weight geometric closeness color similarity

SA2014.SIGGRAPH.ORG SPONSORED BY Top-down Matching --- Toy Example

SA2014.SIGGRAPH.ORG SPONSORED BY Top-down Matching --- Round 1

SA2014.SIGGRAPH.ORG SPONSORED BY Top-down Matching --- Round 2

SA2014.SIGGRAPH.ORG SPONSORED BY Top-down Matching --- Round 3

SA2014.SIGGRAPH.ORG SPONSORED BY Results

SA2014.SIGGRAPH.ORG SPONSORED BY Limitations Too much depth information is missing. Scenes not conforming to our context.

SA2014.SIGGRAPH.ORG SPONSORED BY Thanks Anonymous reviewers. Trimble 3D Warehouse. Authors providing datasets and results. People offered me their rooms for scanning.

SA2014.SIGGRAPH.ORG SPONSORED BY Thank you very much. Any Questions?