1. ifp University of Stuttgart Institute for Photogrammetry Combined Grammar for the Modeling of Building Interiors INDOOR3D – Cape Town Susanne Becker, Michael Peter, Dieter Fritsch - (ifp) Damian Philipp, Patrick Baier, Christoph Dibak - (IPVS) Com’N’Sense

2. University of Stuttgart ifp 3D Indoor Models 2 © www.theverge.com © www.nokiarules.wordpress.com © www.sa.ee.ic.ac.uk © www.theverge.com © www.infsoft.de © cjwalsh.ie © www.evacmap.com © www.abc.net.au © www. cg.cis.upenn.edu

3. University of Stuttgart ifp 3D Indoor Models 3 © http://buildipedia.com  BIM world  3D GIS world  high geometric and semantic detail  modeled manually © http://stem.cs.pusan.ac.k/ isa2009/keynote.html © www.monstercommercial.com © Budroni & Böhm, 2009  mostly pure geometry models with limited geometric detail  focus on automatic derivation from observation data

4. University of Stuttgart ifp 3D Indoor Models 4  3D GIS world © http://stem.cs.pusan.ac.k/ isa2009/keynote.html © http://buildipedia.com  BIM world  high geometric and semantic detail  modeled manually © www.monstercommercial.com © Budroni & Böhm, 2009  mostly pure geometry models with limited geometric detail  focus on automatic derivation from observation data  Problems when erroneous or incomplete sensor data  Powerful means: formal grammars  Goal: fully automatic approach for grammar-supported indoor modeling based on observation data from low-cost sensors

5. University of Stuttgart ifp Formal Grammar 5  Powerful means for representing complex geometric structures using a set of formal rules  Compact representation of geometries and construction processes  Generative component: automated reconstruction of geometries  A formal grammar  defines a formal language as a set of sequences of symbols  symbols  alphabet  rules  syntax  Notation: G(V,T,P,F)  non-terminals V  terminals T  production rules P  id: lc rc : cond → succ : prob  axiom F (non-terminal defining the start point)

6. University of Stuttgart ifp Shape Grammar 1972 Stiny & Gips Enriched L-Systems Split Grammar 2003 Wonka et al. Müller, 2001 CGA Shape Grammar 2006 Müller et al. Facade Grammar 2009 Becker Indoor Grammar 2010 Gröger & Plümer Formal Grammars for modeling geometric structures 6 Lindenmayer- Systems Prusinkiewicz & Lindenmayer, 1990

7. University of Stuttgart ifp Formal Grammars Lindenmayer-Systems  Parallel rewriting systems over strings  Rewriting rules produce sequences of symbols  Sequences of symbols are interpreted and transferred to explicit geometry  Simple initial object is successively overwritten by more complicated structures 7 (Prusinkiewicz & Lindenmayer, 1990) 7  : F - - F - - F p : F  F + F - - F + F  Example 2: rule application axiom production rule © http://www.youtube.com/watch?v=T2VsULqfq4M  Example 1:

8. University of Stuttgart ifp Formal Grammars Lindenmayer-Systems  Application of L-Systems in urban modeling 8 (Müller, 2001)  Problems with modeling of buildings:  no growth process but iterative partitioning of space  geometric conditions (parallelism, rectangularity,...) difficult to integrate  long and intricate production rules

9. University of Stuttgart ifp Formal Grammars for modeling geometric structures 9 Lindenmayer- Systems Prusinkiewicz & Lindenmayer, 1990 Shape Grammar 1972 Stiny & Gips Enriched L-Systems Split Grammar 2003 Wonka et al. Müller, 2001 CGA Shape Grammar 2006 Müller et al. Facade Grammar 2009 Becker Indoor Grammar 2010 Gröger & Plümer

10. University of Stuttgart ifp Formal Grammars Shape Grammar 10 (Stiny und Gips, 1972)  Problem: complex rule applications  Example: axiom production rule rule application (1)(2)(3)  works not on symbols but on shapes:  alphabet consists of a set of shapes  rules directly work on these shapes

11. University of Stuttgart ifp Formal Grammars Split-Grammar 11 (Wonka et al., 2003)  Specified for the generation of building structures  Production rules are limited to split rules  G(T,V,R,I)  Shapes are treated as symbolic objects  Example:  regular, grid-like structures production rules rule applications START FFFF KS W W W W WIN

12. University of Stuttgart ifp Formal Grammars CGA Shape Grammar 12 (Müller et al., 2006)  Continued development of split grammar  Highly detailed building models in a pre-defined architectural style  Example: virtual Pompeji  Problem:  highly complex production rules depending on the level of detail  manual definition of rules

13. University of Stuttgart ifp Formal Grammars for modeling geometric structures 13 Lindenmayer- Systems Prusinkiewicz & Lindenmayer, 1990 Shape Grammar 1972 Stiny & Gips Enriched L-Systems Split Grammar 2003 Wonka et al. Müller, 2001 CGA Shape Grammar 2006 Müller et al. Facade Grammar 2009 Becker Indoor Grammar 2010 Gröger & Plümer

14. University of Stuttgart ifp 14 Cell Decomposition  Extraction and modeling of facade geometries from terrestrial LiDAR data Knowledge Inference  Detection of repetitive features and structures  Inference of rules Knowledge Propagation  Top-down prediction for completion  Generation of synthetic facades data driven knowledge based Formal Grammars Facade Grammar (Becker, 2009) Facade Grammar

15. University of Stuttgart ifp Formal Grammars Indoor Grammar 15 (Gröger & Plümer, 2010)  Generation of geometrically and topologically consistent 3D indoor models  G(N,T,S,P)  split rules are applied on 3D boxes and 2D rectangles  topology is explicitly modeled through constraints  Example: (a) (b) (c) (d) (e)(f) (g)

16. University of Stuttgart ifp Formal Grammars for modeling geometric structures 16 Lindenmayer- Systems Prusinkiewicz & Lindenmayer, 1990 Shape Grammar 1972 Stiny & Gips Enriched L-Systems Split Grammar 2003 Wonka et al. Müller, 2001 CGA Shape Grammar 2006 Müller et al. Facade Grammar 2009 Becker Indoor Grammar 2010 Gröger & Plümer now ifp, IPVS Combined Indoor Grammar

17. University of Stuttgart ifp Combined Indoor Grammar Design Decisions  Building characteristics  Public buildings are traversed by a system of hallways.  The system of hallways divides each floor into hallway-spaces and non-hallway-spaces.  Non-hallway-spaces can be further divided into smaller room units mostly arranged in a linear sequence parallel to the adjacent hallway. 17  Grammar concept  Hallway system (linear structures)  L-system  Room configurations (spatial partitioning)  split-grammar floorplan of ifp

18. University of Stuttgart ifp Combined Indoor Grammar L-System for Modeling Hallways 18  G hallways = (V, ω,P)  V : set of attributed symbols (modules),  ω : axiom (initial hallway segment)  P : production rules  related to the enriched L-system for modeling streets (Parish & Müller, 2001)  Idea: organize the setting of attributes, probabilities and the constraints induced by the geometric environment through external functions

19. University of Stuttgart ifp Combined Indoor Grammar Split-Grammar for Modeling Rooms 19  G rooms = (N,T,S,R)  N = {Space}  T = {…, space i, space j, …}  S = Space  R = {Split, Merge, Instantiation}       x y z x y  d

20. University of Stuttgart ifp Combined Indoor Grammar Split-Grammar for Modeling Rooms 20  Rule probabilities  a priori probability: P(R i )  relative frequency of occurrence  context aware probability: P(R j |R i )  conditional probability for modeling neighborhood relationships between rooms RiRi … RkRk RjRj …  Implementation through Markov Chain  nodes: rules  edges: neighborhood relationships or transitions  probability for a transition from R i to R j is given by

21. University of Stuttgart ifp Instantiation of Individual Grammars 21 a aaa abc c a b d  G indoor = (G hallways, G rooms )  G hallways  G rooms specialized rule system ?

22. University of Stuttgart ifp Instantiation of Individual Grammars 22  Grammar-based representation of a real floor plan  hallway system using an individual instance of the L-system G hallways floorplan of ifp

23. University of Stuttgart ifp floorplan of ifp Instantiation of Individual Grammars 23  Grammar-based representation of a real floor plan  hallway system using an individual instance of the L-system G hallways  room configurations in non-hallway spaces using an individual instance of the split Grammar G rooms floorplan of ifp Space 2 a Space 1 b c m j k i h g floorplan of ifp Space 1 Space 2 ;

24. University of Stuttgart ifp specialized rule system ? Instantiation of Individual Grammars 24 a aaa abc c a b d  G indoor = (G hallways, G rooms )  G hallways  G rooms specialized rule system !

25. University of Stuttgart ifp high low Instantiation of Individual Grammars Inverse Procedural Modeling  Set up of rules and attributes requires observation data 25 observation data grammar instance quality completeness, accuracy high low  1,  2,  3,  4,  5, d1, d2, d3, d4, d5  1,  2 d1, d2, d3  1,  2,  3,  4,  5, d1, d2, d3, d4, d5 Inverse procedural modeling © Peter et al., 2011 © Peter, 2013

26. University of Stuttgart ifp Grammar Application Procedural Modeling  An instance of an individual indoor grammar contains knowledge about characteristic indoor geometries and room arrangements  geometric properties: e.g. room size  topological properties: e.g. connectivity of rooms  semantic aspects: e.g. functional grouping of rooms  Use the grammar to generate reliable hypotheses about possible indoor geometries in unknown areas 26  1,  2,  3,  4,  5, d1, d2, d3, d4, d5 observation data observation data grammar instance grammar instance Inverse procedural modeling Procedural modeling hypotheses © Peter et al., 2013

27. University of Stuttgart ifp Grammar Application Results  Application scenario:  Generate hypotheses about indoor geometries for the 2 nd floor of university building using erroneous and incomplete trace data and grammar support 27 © www.informatik.uni-stuttgart.de  Input data:  Traces: 250 odometry traces covering the hallways of the 2 nd floor  Grammar: high-level grammar automatically derived from a floor plan of the 1 st floor  3D building shell: LOD2 building model provided by city surveying office

28. University of Stuttgart ifp Grammar Application Results  Procedural modeling process  derive initial hallway segments (axiom)  apply L-system to the axiom  apply split grammar to non- hallway spaces  Comparison to ground trouth (131 rooms)  purely data-driven: 29 rooms  split grammar applied to the data-driven hallways: 92 rooms  split grammar applied to completed hallways through L-system: 116 rooms  average room width error: ~2m 28

29. University of Stuttgart ifp Conclusions and Outlook 29  Combined indoor grammar to support the reconstruction of building interiors from real sensor data  concepts of L-system and split grammar  different geometric and topological characteristics of hallways and rooms  Individual grammars can be derived automatically from observation data  Grammar can be integrated in continuous update and enhancement loop  Future work  link indoor grammar to facade grammar (step over to real 3D)  transfer the concept of split grammar from spaces to faces (model wall openings) © Peter, 2013© Becker, 2011

30. ifp University of Stuttgart Institute for Photogrammetry Thank you for your attention susanne.becker@ifp.uni-stuttgart.de