1. A Bayesian Estimation of Building Shape using MCMC
Palladio’s Villa Rotonda, near Venice. Complete symmetry and regularity.
Anthony Dick1 Phil Torr2 Roberto Cipolla1
1Department of Engineering 2Microsoft Research,
University of Cambridge Cambridge

2. Bayesian Estimation of Building Shape using MCMC - ECCV'02
Our goal
Reconstruction and recognition of architecture
Reconstruction and interpretation
Bayesian Estimation of Building Shape using MCMC - ECCV'02

3. Bayesian Estimation of Building Shape using MCMC - ECCV'02
Our approach
Generic method that uses domain-specific prior information to interpret ambiguous or misleading image data
Bayesian framework for incorporating prior knowledge with image data
Given this framework, we have 3 main problems:
How to represent our model
How to formulate our prior knowledge
How to estimate the best model for our data
This is what we’ll be talking about…
Bayesian Estimation of Building Shape using MCMC - ECCV'02

4. Shape representation Model is a collection of “wall” planes
Each wall plane may contain primitives defined by 4 – 8 parameters
Eg:
Window
Door
Pediment
Pedestal
Entablature
Column
Buttress
Drainpipe c
Example shape (window) b
(x,y) d r a a
Front view
Overhead view
Bayesian Estimation of Building Shape using MCMC - ECCV'02

5. Bayesian Estimation of Building Shape using MCMC - ECCV'02
The prior
There is a prior on shape and texture
A texture prior is learnt for each type of primitive
Show examples of the primitive from a suite of approx 100 architectural images
Details in [ICCV01]
A shape prior is more problematic
Shape + scale of individual primitives
Layout of multiple primitives (e.g. alignment, symmetry)
We use MCMC to simulate sampling from this complex distribution
Requires a decent starting point
Bayesian Estimation of Building Shape using MCMC - ECCV'02

6. Sampling the shape prior
Requires a scoring function and jumping distributions
Scoring function is a function of the model parameters
Combination of scale, shape, alignment and symmetry terms
Jumping distribution is a mixture of several types of jump:
Add/Remove/Modify shape
Add/Remove/Modify wall
Add/Remove/Modify row/column of shapes
Regularise row/column of shapes
Symmetrise row/column of shapes
Perturb row/column of shapes
Bayesian Estimation of Building Shape using MCMC - ECCV'02

7. Verifying the shape prior
“Seed” buildings:
Samples on a city grid:
Bayesian Estimation of Building Shape using MCMC - ECCV'02

8. Bayesian Estimation of Building Shape using MCMC - ECCV'02
Model estimation
Initial shape estimate obtained via existing structure and motion algorithms
Extract and match corners and lines
Self-calibrate cameras
Plane fitting post-process to estimate walls
Search for likely primitives on each wall [ICCV01]
This produces seed points for the MCMC process
Likelihood measure is based on sum squared error of reprojected pixels
Assumes Lambertian model
Now run Reversible Jump MCMC on seed models
At least 2000 iterations
Bayesian Estimation of Building Shape using MCMC - ECCV'02

9. Bayesian Estimation of Building Shape using MCMC - ECCV'02
Reconstructed model
Bayesian Estimation of Building Shape using MCMC - ECCV'02

10. Bayesian Estimation of Building Shape using MCMC - ECCV'02
Completed model
Bayesian Estimation of Building Shape using MCMC - ECCV'02

11. Bayesian Estimation of Building Shape using MCMC - ECCV'02
Other likely models
Sills
Included:
Split
windows:
Door
Not included:
Extra
Columns:
Bayesian Estimation of Building Shape using MCMC - ECCV'02

12. Bayesian Estimation of Building Shape using MCMC - ECCV'02
Effect of prior
Wall of Downing College library:
Without alignment prior
With alignment prior
Bayesian Estimation of Building Shape using MCMC - ECCV'02

13. Bayesian Estimation of Building Shape using MCMC - ECCV'02
Effect of prior
This is an exception – usually Classical and Gothic priors give the same result
Classical shape prior
Gothic shape prior
Bayesian Estimation of Building Shape using MCMC - ECCV'02

14. Bayesian Estimation of Building Shape using MCMC - ECCV'02
Gothic model
Bayesian Estimation of Building Shape using MCMC - ECCV'02

15. Bayesian Estimation of Building Shape using MCMC - ECCV'02
Completed model
Bayesian Estimation of Building Shape using MCMC - ECCV'02

16. Bayesian Estimation of Building Shape using MCMC - ECCV'02
Ground truth
Bayesian Estimation of Building Shape using MCMC - ECCV'02

17. Ground truth comparison
Ratio
Ground truth
Model
Lower
Upper
Height / width
1.48
1.52
1.50
1.64
Width / depth
5.67
6.37
5.00
7.40
Wall-col / width
2.22
2.24
2.18
2.28
Col circ / width
2.56
2.77
2.74
3.00
All values within bounds, but the model is less precise than hand measurement
Need for more accurate image measurements e.g. super-resolution
Bayesian Estimation of Building Shape using MCMC - ECCV'02

18. Bayesian Estimation of Building Shape using MCMC - ECCV'02
Conclusion
Combination of structure from motion and recognition
Use of high level prior information is crucial
Bayesian Estimation of Building Shape using MCMC - ECCV'02