Download presentation
Presentation is loading. Please wait.
1
Exchanging Faces in Images SIGGRAPH ’04 Blanz V., Scherbaum K., Vetter T., Seidel HP. Speaker: Alvin Date: 21 July 2004
2
Alivn/GAME Lab./CSIE/NDHU Exchanging Faces in Images 2 Outline Introduction Morphable Models Estimation Exchanging Faces Compositing Application Results Conclusions
3
Alivn/GAME Lab./CSIE/NDHU Exchanging Faces in Images 3 Introduction Pasting somebody’s face into an existing image. A novel type of image manipulation: Always needs pairs of images with the same viewpoint and the same illumination. The system only need one image, and can across large differences in viewpoint and illumination.
4
Alivn/GAME Lab./CSIE/NDHU Exchanging Faces in Images 4 Introduction (cont.) Manual interaction: Click on a set of about 7 feature points.feature points Mark the hairline in the target image. Example Two applications: Virtual try-on for hairstyles Face recognition
5
Alivn/GAME Lab./CSIE/NDHU Exchanging Faces in Images 5 Previous Works
6
Alivn/GAME Lab./CSIE/NDHU Exchanging Faces in Images 6 Previous Works (cont.)
7
Alivn/GAME Lab./CSIE/NDHU Exchanging Faces in Images 7 Previous Works (cont.)
8
Alivn/GAME Lab./CSIE/NDHU Exchanging Faces in Images 8 Previous Works (cont.)
9
Alivn/GAME Lab./CSIE/NDHU Exchanging Faces in Images 9 Previous Works (cont.)
10
Alivn/GAME Lab./CSIE/NDHU Exchanging Faces in Images 10 Previous Works (cont.)
11
Alivn/GAME Lab./CSIE/NDHU Exchanging Faces in Images 11 Outline Introduction Morphable Models Estimation Exchanging Faces Compositing Application Results Conclusions
12
Alivn/GAME Lab./CSIE/NDHU Exchanging Faces in Images 12 Morphable Models A vector space of 3D shapes and textures. Derived from 200 texture Cyberware (TM) laser scans. 100 male and 100 female. In a cylindrical representation with radii r(h, Φ) of surface points 512 equally-spaced angles Φ. 512 equally-spaced vertical steps h.
13
Alivn/GAME Lab./CSIE/NDHU Exchanging Faces in Images 13 Morphable Models (cont.) Dense correspondence is computed automatically with an algorithm derived from optical flow.
14
Alivn/GAME Lab./CSIE/NDHU Exchanging Faces in Images 14 Morphable Models (cont.) After performing a PCA m = 149
15
Alivn/GAME Lab./CSIE/NDHU Exchanging Faces in Images 15 Fitting
16
Alivn/GAME Lab./CSIE/NDHU Exchanging Faces in Images 16 Light Direction And Intensity Estimation
17
Alivn/GAME Lab./CSIE/NDHU Exchanging Faces in Images 17 Outline Introduction Morphable Models Estimation Exchanging Faces Compositing Application Results Conclusions
18
Alivn/GAME Lab./CSIE/NDHU Exchanging Faces in Images 18 Estimation All parameters are estimated simultaneously in an analysis-by- synthesis loop. All scene parameters are recovered automatically, starting from a frontal pose in the center of the image, and at frontal illumination.
19
Alivn/GAME Lab./CSIE/NDHU Exchanging Faces in Images 19 Estimation (cont.) Cost Function
20
Alivn/GAME Lab./CSIE/NDHU Exchanging Faces in Images 20 Estimation (cont.) The optimization is performed with a Stochastic Newton Algorithm. The linear combination of texture T i cannot reproduce all local characteristics of the novel faces. Extract the texture by an illumination-corrected texture extraction method.
21
Alivn/GAME Lab./CSIE/NDHU Exchanging Faces in Images 21 References A morphable model for the synthesis of 3D faces. SIGGRAPH’99, pp. 187–194. Face recognition based on fitting a 3D morphable model. IEEE Trans. on Pattern Analysis and Machine Intell. 25, 9 (2003), 1063– 1074.
22
Alivn/GAME Lab./CSIE/NDHU Exchanging Faces in Images 22 Outline Introduction Morphable Models Estimation Exchanging Faces Compositing Application Results Conclusions
23
Alivn/GAME Lab./CSIE/NDHU Exchanging Faces in Images 23 Exchanging Faces Both 3D shapes are aligned to each other in 3D with 3D Absolute Orientation Algorithm. Both textures have similar illumination. Illumination-corrected Texture Extraction Algorithm. Render the face that was reconstructed from the source image with the rendering parameters that were estimated from the target image.
24
Alivn/GAME Lab./CSIE/NDHU Exchanging Faces in Images 24 Outline Introduction Morphable Models Estimation Exchanging Faces Compositing Application Results Conclusions
25
Alivn/GAME Lab./CSIE/NDHU Exchanging Faces in Images 25 Compositing
26
Alivn/GAME Lab./CSIE/NDHU Exchanging Faces in Images 26 Background Layer The scene of target image, and the original person’s face, hair and body. The novel face may be smaller than the original. Solved by a background continuation method. Based on a reflection of pixels beyond the original contour into the face area.
27
Alivn/GAME Lab./CSIE/NDHU Exchanging Faces in Images 27 Face Layer The silhouette of this region: Occluding contours. Boundaries of hair regions that occlude the skin. Mesh boundaries at the neck and the forehead. Skin may be partly covered by hair. This hair would be mapped on the face as a texture.
28
Alivn/GAME Lab./CSIE/NDHU Exchanging Faces in Images 28 Hair Layer Drawn in front of the face. Can be used for all faces. Automated classification of pixels into skin and hair is a difficult task. Manually define alpha values for opacity.
29
Alivn/GAME Lab./CSIE/NDHU Exchanging Faces in Images 29 Outline Introduction Morphable Models Estimation Exchanging Faces Compositing Application Results Conclusions
30
Alivn/GAME Lab./CSIE/NDHU Exchanging Faces in Images 30 Applications Current systems are restricted to frontal view of faces.
31
Alivn/GAME Lab./CSIE/NDHU Exchanging Faces in Images 31 Applications (cont.)
32
Alivn/GAME Lab./CSIE/NDHU Exchanging Faces in Images 32 Outline Introduction Morphable Models Estimation Exchanging Faces Compositing Application Results Conclusions
33
Alivn/GAME Lab./CSIE/NDHU Exchanging Faces in Images 33 Results
34
Alivn/GAME Lab./CSIE/NDHU Exchanging Faces in Images 34 Results (cont.)
35
Alivn/GAME Lab./CSIE/NDHU Exchanging Faces in Images 35 Results (cont.)
36
Alivn/GAME Lab./CSIE/NDHU Exchanging Faces in Images 36 Outline Introduction Morphable Models Estimation Exchanging Faces Compositing Application Results Conclusions
37
Alivn/GAME Lab./CSIE/NDHU Exchanging Faces in Images 37 Conclusions A novel way of processing images on a high level. Only needs simple manual processing steps. For a wide range of applications. Transferring technology from CG to CV. Combines the benefit of image-based method with the versatility of 3D graphics.
38
Alivn/GAME Lab./CSIE/NDHU Exchanging Faces in Images 38 Future Works Fully automated: Detecting facial features. Hair Segmentation. Exchange faces in video sequences. Tracking head motion.
39
Alivn/GAME Lab./CSIE/NDHU Exchanging Faces in Images 39 Thank you for your patience
40
Alivn/GAME Lab./CSIE/NDHU Exchanging Faces in Images 40 Example
41
Alivn/GAME Lab./CSIE/NDHU Exchanging Faces in Images 41 Feature Points
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.