SmartBoxes for Interactive Urban Reconstruction Liangliang Nan1, Andrei Sharf1, Hao Zhang2, Daniel Cohen-Or3, Baoquan Chen1 1 Shenzhen Institutes of Advanced Technology (SIAT), China 2 Simon Fraser University, Canada 3University of Tel Aviv, Israel

3D Cities Virtual Philadelphia Virtual Berlin

Acquisition of Urban Environments Remote Sensing Systems Airborne LIDAR Cameras/videos Auto-mounted LIDAR 3D Digital City 3 3

3D LiDAR scanner Street-level 60km/h 180 pitch [100-300m] range 100K points/second 5cm XY accuracy

Outdoor Urban Scanning

Imperfect Scans - Occlusions Point cloud contains holes due to various occlusions (“shadows”)

Imperfect Scans – Angle & Range Oblique scanning angle Laser energy attenuation on range

Urban Building Characteristics Dominant planes Axis-aligned basic primitives Repetitions, intra symmetry and regularity

SmartBoxes Box-up and Smart!

SmartBoxes Box prior shape fitting Smart context awareness Data Both

Live Demo

Related Work Procedural modeling of buildings and facades [Wonka2003;Muller2007] Automatic 3D reconstruction from 2D images [Zisserman2002;Xiao2009;Furukawa2009] Interactive modeling of architectural structures [Debevec1996;Schindler2003; Xiao2008;Sinha2008;Jiang2009] Primitive fitting to data [Gal2007; Schnabel 2009]

Preprocessing Automatic detection of planes and edges assuming dominant orthogonal axes RANSAC planes Line sweep edges

Snapping a Box 2D rubber band ROI Collect planes, edges, corners Find the best fitting box using data fitting force D(B,P)

Data fitting D(B,P) snap Data fitting force Facet Edge Data quality (Confidence + Density) Distance

Grouping Simple SmartBox Compound SmartBox Align to remove gaps and intersections cluster and align close to co-linear edges

Drag-and-drop context C(Bi-1, Bi) The context of Bi Interval Alignment Scale Bi Bi-3 Bi-2 Bi-1 ••• Context

Drag-and-drop context C(Bi-1, Bi) The context of Bi Interval term Bi ••• Bi-3 Bi-2 Bi-1

Drag-and-drop context C(Bi-1, Bi) The context of Bi Alignment term Bi ••• Bi-3 Bi-2 Bi-1

Drag-and-drop context C(Bi-1, Bi) The context of Bi Scale term Bi Bi ••• Bi-3 Bi-2 Bi-1

Discrete objective minimization Find linear transformation T(excluding rotation) to minimize: Data fitting force Contextual force

Discrete objective minimization Find linear transformation T(excluding rotation) to minimize: Data fitting force Contextual force ••• Bi-3 Bi-2 Bi-1

Balance between two forces

Emerging city of Shenzhen, China

Results: textured buildings

The Crooked House (Sopot, Poland) Manchester Civil Justice Centre (Manchester, UK) Habitat 67 (Montreal, Canada)

(Thank You)!