Conjoining Gestalt Rules for Abstraction of Architectural Drawings Liangliang(Leon) Nan 1, Andrei Sharf 2, Ke Xie 1, Tien-Tsin Wong 3 Oliver Deussen 4, Daniel Cohen-Or 5, Baoquan Chen 1 1 SIAT, China 2 Ben Gurion Univ., Israel 3 CUHK, China 4 Konstanz Univ., Germany 5 Tel Aviv Univ., Israel 1

Abstraction of Architectural Drawings 2

3

How Our Visual System Simplifies Things 4 How we get the idea?

How Our Visual System Simplifies Things 5

Regularity 6 How Our Visual System Simplifies Things

7

8 Gestalt – grouping → simplification

Proximity 9 How Our Visual System Simplifies Things

Similarity 10 How Our Visual System Simplifies Things

Continuity Gestalt Gestalt Theory Several principles Group objects into forms Create internal representation [Wertheimer 1923] Proximity Gestalt 11

Conjoining Gestalt Independent rules Different interpretations Complex interactions Input Horizontal regularity Vertical regularity Both 12

Computational framework Abstraction of architectural drawings Contributions 13

Related Work Qualitative and empirical studies [Wertheimer 1923] Quantification of Gestalt principles and their interactions [Desolneux et al. 2002], [Cao et al. 2007], [Kubovy and Van den Berg 2008] Perceptual principle based abstraction – No conjoining [DeCarlo and Santella 2002], [Barla et al. 2005; 2006], [Mi et al. 2009] 14

Related Work Stroke density based simplification – structures are broken [Grabli et al. 2004], [Shesh and Chen 2008] Building representation – limited to regular patterns [Loya et al. 2008], [Adabala et al. 2009] Quantification and interaction of Similarity & Proximity [Kubovy and van den Berg 2008] Challenge: interaction of multiple Gestalt principles 15

Our Solution Conjoining Gestalts ↔ Consistent segmentation Subset of Gestalt principles Our method –Energy minimization –Graph-cut solution 16

Initial groups Overview 17 Input Optimal groupsAbstraction

Proximity Graph Structure 18

Proximity Graph Structure Spatial relation graph –Element → node –Connect neighbors by edges 19

Optimization via Graph Cut Initial groups → labels 20

Optimization via Graph Cut Initial groups –Proximity groups 21

Optimization via Graph Cut Initial groups –Similarity groups 22

Optimization via Graph Cut Initial groups –Regularity groups 23

Optimization via Graph Cut Goal : find the optimal groups Grouping → Energy minimization Proximity groupsSimilarity groupsRegularity groups 24

Optimization via Graph Cut Energy minimization – function terms Data cost 25

Optimization via Graph Cut Energy minimization – function terms –Data cost Assigned for different Gestalt labels Penalty of assigning a label to an element 26

Optimization via Graph Cut Energy minimization – function terms –Data cost for Regularity Alignment 27

Optimization via Graph Cut Energy minimization – function terms –Data cost for Similarity Average shape similarity 28

Optimization via Graph Cut Energy minimization – function terms –Data cost for Proximity Closest distance of the elements to the group 29

Optimization via Graph Cut Energy minimization – function terms Smoothness cost 30

Optimization via Graph Cut Energy minimization – function terms –Smoothness cost Spatial correlation of neighboring elements 31 p, q - neighboring elements

Optimization via Graph Cut Energy minimization – function terms Label cost 32

Optimization via Graph Cut Energy minimization – function terms –Label cost Penalty for over-complex groups Concise explanation of the inputs 33 - set of labels - non-negative label cost for different Gestalt

Optimization via Graph Cut Overall energy function –Labeling → minimization Multi-label graph-cut [Delong et al. 2010] 34 Data cost Smoothness cost Label cost

Optimization via Graph Cut Groupings complying with Gestalt principles 35

Computation Results Regularity VS. Proximity 36 Speed x0.5

Computation Results Proximity VS. Similarity 37

Visual Abstraction Abstraction Operators –Embracing –Summarization 38

Level of Detail Progressively simplified results 39

Results 40

Results 41

Results 42

Results 43

Results 44

Results 45

Extension to Mosaics Additional continuity & closure Gestalt 46

Extension to Mosaics 47

Conclusion Computational framework –Conjoining Gestalt principles –Energy minimization, Graph-cut solution Abstraction of architectural drawings –Effective –Preserving structure Extension to mosaics 48

49