1. Interactive 3D Modeling Using Only One Image
Sujin Liu and Zhiyong Huang
School of Computing

2. 1. Introduction
Problem: for VR systems, to create the models, usually with irregular shapes
the current CAD modeling software addressed different problems
Ideas: study the use of human interaction and one image
balance the interaction and automation
11/21/2018
VRST 2000

3. 2. Related Work Forward methods CSG/B-Rep Implicit Surfaces
Zeleznik et al.. SKETCH. SIGGRAPH 96
Igarashi et al. Teddy. SIGGRAPH 99
Implicit Surfaces
Shen and Thalmann. Metaball Human, Implicit Surfaces 95
Production systems
Sakaguchi and Ohya. Botanical Tree. VRST 99
11/21/2018
VRST 2000

4. Reverse methods Computational Geometry based Model based
Edelsbrunner and Mucke, 3D Alpha Shapes. TOG 94
Hoppe et al. Curless and Levoy. Zero-set. SIGGRAPH 92, SIGGRAPH 96
Amenta et al. Crust. SIGGRAPH 99
Turner et al., Line Drawing Interpretation, VRST 99
Model based
Thalmann and Thalmann. Human. IEEE CG&A 87
Pighin et al. Face. SIGGRAPH 98
Lee et al. Face. Eurographics 00
11/21/2018
VRST 2000

5. Other one image based method
Hybrid method
Debevec et al. Façade. SIGGRAPH 96
Other one image based method
Beymer and Poggio. Face Recognition. ICCV 95
Horry et al. TIP (Tour Into Picture). SIGGRAPH 97
11/21/2018
VRST 2000

6. 3. Our Work 3D model One 2D image Photogrammetric Modeling
Human Interaction
Texture Mapping
11/21/2018
VRST 2000

7. Photogrammetric Modeling
Purpose: to achieve automation by exploring the use of one image
Major steps:
Contour extraction
2D skeleton computation
2D meshing
3D meshing
Texture mapping
11/21/2018
VRST 2000

8. Contour Extraction Using the color clustering
the algorithm classifies the pixels into different clusters by comparing result of the color threshold of each cluster
Two clusters: foreground and background
the foreground is distinguishable from the background by colors
not necessary a pure color background
11/21/2018
VRST 2000

9. Example
11/21/2018
VRST 2000

10. 2D Skeleton Computation
Purpose: to derive the skeleton of the 2D shape
The algorithm is based on the feature tracking and minimal spanning tree
KLT feature tracking: derive the feature points of the image
Minimal spanning tree: derive the skeleton of the 2D shape
11/21/2018
VRST 2000

11. Example
11/21/2018
VRST 2000

12. 2D Meshing
Purpose: to derive a 2D mesh of the shape using the skeleton and contour
A variation of the constrained Delaunay triangulation, Qhull
11/21/2018
VRST 2000

13. Example
11/21/2018
VRST 2000

14. 3D Meshing
Purpose: to derive the 3D mesh as an initial shape of the model
Intuition: lift the 2D mesh with different heights for every vertices
Height is estimated by the color intensity
Requires human interaction most
11/21/2018
VRST 2000

15. Example
11/21/2018
VRST 2000

16. Two problems Resolution decreases Back meshing similar to Teddy
human interaction
11/21/2018
VRST 2000

17. Texture Mapping Straight forward for the front mesh
each vertex of the 3D mesh has its texture coordinate in photogrammetrc modeling
Problem for the back mesh
human interaction
11/21/2018
VRST 2000

18. Example
11/21/2018
VRST 2000

19. Other Human Interaction
Common to any modeling systems
picking, grouping, adding, deleting, displacing, etc.
11/21/2018
VRST 2000

20. Summary A hybrid modeling framework requires human interaction
has automations from the use of one image
not a stereo vision
not a model-based
11/21/2018
VRST 2000

21. 4. More Experimental Results
Video
11/21/2018
VRST 2000

22. 5. Conclusion and Future Work
We have proposed and implemented a hybrid modeling frame work using only one image
Future work: to address more general shapes
11/21/2018
VRST 2000

23. 6. Acknowledgement Dr. Leow Wee Kheng, Zhang Yong
NUS Academic Research Grant RP
11/21/2018
VRST 2000