1. MACHINE VISION GROUP Face image mapping from NIR to VIS Jie Chen Machine Vision Group http://www.ee.oulu.fi/mvg

2. MACHINE VISION GROUP Outline Problem Methods Preliminary results Plans for next period

3. MACHINE VISION GROUP Face image mapping from NIR to VIS Problem –NIR: Near infrared imaging –VIS: Visual light imaging

4. MACHINE VISION GROUP Face image mapping from NIR to VIS Problem –NIR: Near infrared imaging –VIS: Visual light imaging

5. MACHINE VISION GROUP Algorithm: Patches mapping Training Training

6. MACHINE VISION GROUP Algorithm: Patches mapping Training Mapping φi,jφi,j

7. MACHINE VISION GROUP Look up the KNN A patch of an input sample in S 3 k-th nearest patch in S 1 Weight of k-th nearest neighbor A patch of an input sample in S 4 Corresponding patch of in S 2

8. MACHINE VISION GROUP Weight computing

9. MACHINE VISION GROUP Experiments Setup –both S1 and S2 is composed of 300 samples. 50 subjects, each subject has 6 images but in different expression (anger, disgust, fear, happiness, sadness, and surprise). –w f =64, h f =80, w p =16, h p =16, w o =12, h o =12 –Testing:using leave-one-out and K=15.

10. MACHINE VISION GROUP Reconstructed images (a) Input images in NIR 24.8823.7627.5526.2926.4827.4321.5421.18 (b) Reconstructed images in VIS using LBP(8,1) and the PSNR 31.8932.11 34.4132.1131.6832.1831.08 (c) Reconstructed images in VIS using the combined Multi-resolution LBP and their PSNR (d) Ground truth in VIS

11. MACHINE VISION GROUP Multi-resolution LBP (MLBP) (P=4,R=1) (P=8,R=1) (P=12,R=1.5) (P=16,R=2) (P=24,R=3)

12. MACHINE VISION GROUP PNSR Pixel wise LBP

13. MACHINE VISION GROUP Multi-resolution LBP

14. MACHINE VISION GROUP PSNR on MLBP

15. MACHINE VISION GROUP Plans for next period Training data: –Use more samples (192*10 from CASIA, a group in Beijing, China) Methods: –Combine the methods proposed in the paper (A. Hertzmann, SIGGRAPH, 2001) for better performance