Course: Structure-Aware Shape Processing Introduction to Geometric ‘Structure’ Extracting Structures –analysis of Individual Models –analysis of Shape Collections (co-analysis) –Structural Hierarchies Manipulating Structures –Modeling as Structural Variations –Structure-guided Design –Organization + Exploration of Shape Collections Future Directions Course structure

Course: Structure-Aware Shape Processing Modeling as Structural Variation Niloy J. MitraMichael Wand Hao Zhang Daniel Cohen-Or Vladimir KimQi-Xing Huang

Course: Structure-Aware Shape Processing Inspiration  a readily usable digital 3D model Inspiration? Modeling: 3D content creation

Course: Structure-Aware Shape Processing Inspiration = real-world data Realistic reconstruction

Course: Structure-Aware Shape Processing Creation of novel 3D shapes Inspiration = design, sketch, photo, other examples sketch High demand in VFX, games, simulation, VR, … Focus on creative modeling

Course: Structure-Aware Shape Processing 2D-to-3D: an ill-posed problem –Shape from shading, sketch-based modeling, … Creation from scratch is hard: job for skilled artists Main reason why graphics is not as ubiquitous as we wanted it to be  Jim Kajia’s SIG’11 Award Talk 3D content creation is hard

Course: Structure-Aware Shape Processing Models created are meant for subsequent use Desire to create readily usable 3D models Usable 3D content even harder

Course: Structure-Aware Shape Processing Models created are meant for subsequent use Desire to create readily usable 3D models Usability: higher-level information beyond low-level mesh –Part or segmentation information –Structural relations between parts –Correspondence to relevant models, etc. Hard shape analysis problems! Usable 3D content even harder

Course: Structure-Aware Shape Processing Reuse existing 3D models and associated information Data- or model-driven approach: creation is driven by or based on existing (pre-analyzed) models Key: model reuse

Course: Structure-Aware Shape Processing Reuse existing 3D models and associated info Data- or model-driven approach: creation is driven by or based on existing (pre-analyzed) models Key: model reuse

Course: Structure-Aware Shape Processing New model created by varying existing models Paradigm I: much like shape editing Modeling as variation Variation as modification of an existing model, e.g., a warp or a deformation

Course: Structure-Aware Shape Processing New model created by varying existing models Paradigm I: much like shape editing Paradigm II: shape synthesis Modeling as variation Variation as modification of an existing model, e.g., a warp or a deformation Variation by part composition, from multiple models

Course: Structure-Aware Shape Processing Available models are assumed to be structurally valid or proper New model should maintain that validity, but only preserve the “essential structures” Conflicting goals = challenge = “fit & diverse”: –Fit: structure preservation from existing models –Diverse: encourage significant deviation from examples = novelty or creativity Two conflicting goals

Course: Structure-Aware Shape Processing iWires: [Gal et al. 2009] Component-wise controllers: [Zheng et al. 2010] Photo-inspired modeling: [Xu et al. 2011] Structural retargeting: [Lin et al. 2011, Bokeloh et al. 2012, Bao et al. 2012, Yeh et al. 2013, Zhang et al. 2013] Topology-varying blending: [Alhashim et al. 2014] Modeling by deformation

Course: Structure-Aware Shape Processing Wires as control or deformation handles [Singh & Fiume 1999] Editing preserves structural relations, e.g., symmetry, co-planarity, etc. iWire: analyze-and-edit [Gal et al. 2009]

Course: Structure-Aware Shape Processing Cuboids and generalized cylinders as control handles Use the analyze-and-edit paradigm like iWires Preserve structural relations among controllers: symmetry and proximity Component-wise controllers [Zheng et al. 2010]

Course: Structure-Aware Shape Processing Not editing; modeling inspired by a single photograph Warp an existing 3D model to fit object silhouette in the photograph Structure preservation ensures a coherent 3D model Photo-inspired modeling [Xu et al. 2011]

Course: Structure-Aware Shape Processing photo Photo-inspired modeling Use the controllers from [Zheng et al. 2010] [Xu et al. 2011]

Course: Structure-Aware Shape Processing Use the controllers from [Zheng et al. 2010] photo Photo-inspired modeling Retrieved candidate 3D model [Xu et al. 2011]

Course: Structure-Aware Shape Processing Result of deformation to fit silhouette photo Use the controllers from [Zheng et al. 2010] Photo-inspired modeling [Xu et al. 2011] Retrieved candidate 3D model

Course: Structure-Aware Shape Processing Use the controllers from [Zheng et al. 2010] Structure preservation at work symmetry [Xu et al. 2011]

Course: Structure-Aware Shape Processing proximity Use the controllers from [Zheng et al. 2010] Structure preservation at work [Xu et al. 2011] symmetry

Course: Structure-Aware Shape Processing additional optimization Use the controllers from [Zheng et al. 2010] Structure preservation at work proximity symmetry [Xu et al. 2011]

Course: Structure-Aware Shape Processing output Use the controllers from [Zheng et al. 2010] Structure preservation at work [Xu et al. 2011] additional optimization proximity symmetry

Course: Structure-Aware Shape Processing Key analysis: regularity detection and organization Repetition of 1D or 2D regular patterns is relatively easy Retarget amid irregularity is hard Structural retargeting Key words: analyze-and-stretch; pattern repetition rather than geometric stretch!

Course: Structure-Aware Shape Processing [Lin et al. 2011]: –Hierarchical structural organization of façade elements obtained manually –Auto decompose facades into 1D sequences and retarget sequence by sequence Retarget with semi-auto analysis [Lin et al. 2011]

Course: Structure-Aware Shape Processing Automatic hierarchical decomposition of irregular 2D facades Optimal decomposition via SYMAX (next week’s talk by Wallace) Retarget facade by altering the generative hierarchical model Automatic analysis [Zhang et al. 2013]

Course: Structure-Aware Shape Processing Structure-aware blending between 3D shapes with different topologies Topology-varying blending or morphing [Alhashim et al. 2014]

Course: Structure-Aware Shape Processing Modeling by example [Funkhouser et al. 2004] Fit & diverse: [Xu et al. 2012] Structure recovery by part assembly: [Shen et al. 2012] Replacing functional sub-structures: [Zheng et al. 2013] Modeling by part composition

Course: Structure-Aware Shape Processing New models composed by parts retrieved from an existing data repository Key: retrieve relevant parts by geometric similarity of parts Many variants to date [Funkhouser et al. 2004] Modeling by example

Course: Structure-Aware Shape Processing Major difference: instead of generating one model at a time, evolve a set of shapes together Inspired by the biological process of evolution [Xu et al. 2012] Set evolution

Course: Structure-Aware Shape Processing Off-springs by part mutation (warp) and cross-over (reassembly of parts by fuzzy replaceability): leads to diversity Design gallery: user specifies “like” or “dislike” to define fitness function [Xu et al. 2012] Fit & diverse

Course: Structure-Aware Shape Processing Modeling from a single Kinect depth scan + RGB image Unlike [Xu et al. 2011], model is built by part assembly from multiple shapes, which is more versatile than just warping one candidate model [Shen et al. 2012] Structure recovery by assembly

Course: Structure-Aware Shape Processing Detect a specific type of common sub-structures in a set of shapes by sub-graph matching: a step towards functionality analysis Replacement can cross different object categories [Zheng et al. 2013] Replacing sub-structures

Course: Structure-Aware Shape Processing Deformation applied mainly for editing/morphing or to model by fitting to something, e.g., a photo, a sketch, or a Kinect scan Part combination mainly for creating novel shapes There are more structural changes with part combination Deform vs. part combination