1. VideoTrace: Interactive 3d modelling for all
Anton van den Hengel
Director, Australian Centre for Visual technologies
Associate Professor, Adelaide University, South Australia
Director, PunchCard Visual technologies

2. Input

3. Modelling

4. Results

5. Interactive 3D modelling
3D modelling is critical to all sorts of application
Special effects, but also mining, architecture, defence, urban planning, …
People are getting more visually sophisticated
More 3D data is being generated
More cameras, but also scanners etc
The interfaces of modelling programs are usually very hard to fathom

6. Why? Insert your own objects into a game
Put your couch into second life
Model your house for Google Earth
Video editing
Cut and paste between sequences
Remove someone from your home videos

7. Put your truck into a game

8. Put your truck into a game

9. Modelling for animation

10. Video editing requires models

11. Dense surface reconstruction

12. Video editing requires models

13. Structure from motion

14. The process Capture and import the video
Perform structure and motion analysis
Interact with the system to generate and edit the model
Export to your application

15. The approach Pre-compute where possible Then interact
Structure from motion (camera tracking)
Superpixels
Then interact
Interactions allow user to exploit precomputed results

16. Structure from motion Camera tracking Calculates
Reconstructed point cloud
Camera parameters
Location
Orientation
Intrinsics (eg. Focal length)
Informs interaction interpretation process

17. Interactive modelling from video

18. Interactions Straight lines Curves Mirroring Extrusion
Closed sets of lines define planar polygons
Curves
For planar shapes with curved edges
For NURBS surfaces
Mirroring
Duplicates existing geometry
Extrusion

19. Fitting planar faces User specifies boundary
Boundary specifies infinitely many planes
Similar to pre-emptive RANSAC
Generate bounded plane hypotheses from point cloud
Eliminate hypotheses that fail a series of tests
Run simplest / most robust tests first
Generally 3d tests before 2d tests

20. Fitting planar faces
Line of sight
Object points
Image plane

21. Hierarchical RANSAC Generate bounded plane hypotheses Tests
Support from point cloud
Reprojects within new image boundaries
Constraints on relative edge length and face size
Colour histogram matching on faces
Colour matching on edge projections
Reprojection is not self-occluding

22. Curves

23. Mirroring

24. Extrusion

25. Dense surface reconstruction
Needs to be at interactive speed
Calculated as a max-flow graph-cut over a Markov Random Field
Link cost based on photoconsitency

26. Modelling without features

27. Modelling without features

28. Recent model

29. Future work Other interactions Other data sources Occluding contours
Interactive SFM
De / Re-lighting