1. Su-ting, Chuang 2010/8/2

2. Outline Introduction Related Work System and Method Experiment Conclusion & Future Work 2

3. Outline Introduction Related Work System and Method Experiments Conclusion & Future Work 3

4. Introduction Non-uniform lighting problem Various finger touch response among different position Low computation efficiency No such tool that helps users determine parameters automatically 4

5. Outline Introduction Related Work System and Method Experiments Conclusion & Future Work 5

6. Related Work FTIR (Frustrated Total Internal Reflection) J. Y. Han, “Low-cost multi-touch sensing through frustrated total internal reflection," in Proceedings of the 18th annual ACM symposium on User interface software and technology (UIST '05). New York, NY, USA: ACM Press, 2005, pp. 115-118. 6

7. Related Work DI (Diffused Illumination) J. Rekimoto and N. Matsushita, “Perceptual surfaces: Towards a human and object sensitive interactive display," Workshop on Perceptural User Interfaces (PUI'97), 1997. 7

8. Related Work TouchLib A multi-touch development kit Finger detection processing flow chart 8 Background Subtraction Simple Highpass ScaleThreshold Finger Analysis

9. Related Work DirectShow Filter-based framework GShow GPU-accelerated framework Combination of DirectX and DirectShow 9

10. Outline Introduction Related Work System and Method Experiments Conclusion & Future Work 10

11. Hardware Configuration (2) IR Camera (3) IR Illuminator (1) Peripheral Projector 11

12. Hardware Configuration Order of diffuser layer and touch-glass layer 12 Diffuser layer IR illuminator IR camera spot IR illuminator IR camera Touch-glass layer IR camera spot IR camera 2 1 4 3

13. Hardware Configuration Problem: IR rays reflected by the touch-glass will result in hot spot regions in camera views Solution: Use other cameras to recover the regions which are sheltered by IR spots 13

14. Software Architecture Detection system Image Stitching Finger Detection Finger Tracking Parameter determination 14 Image Stiching Image Stiching Finger Detection Finger Detection Finger Tracking Finger Tracking

15. Software Architecture 15 Image Stiching Image Stiching Finger Detection Finger Detection Finger Tracking Finger Tracking

16. Image Stitching Goal Combine multi-camera view into a virtual camera view 16

17. Image Stitching Advantages Remove IR spot effect Unify finger size among different position of table Reduce matching problem Be compatible with existent finger detection system 17

18. Image Stitching 18 Image Blending IR Camera(L) IR camera(R) Undistortion HomoWarp

19. Image Stitching HomoWarp 19 5 5 2 2 1 1 6 6 2 2 5 5 4 4 3 3 1 1 4 4 3 3 6 6 2 2 5 5

20. Image Stitching Image Blending 20

21. Finger Detection TouchLib Our method 21 Normalization Difference of Gaussian Background Subtraction Binary Finger Analysis Simple Highpass Scale Background Subtraction Binary Finger Analysis

22. Finger Detection Normalization Method Model distribution of IR illumination Use specific material to simulate foreground Construct normalization map Normalize foreground image Result Before normalization: mean = 75, standard variation = 30 After normalization: mean = 255, standard variation = 3 22

23. Finger Detection Difference of Gaussian (DoG) Modified from simple highpass in TouchLib 23

24. Fingertip Tracking Goal Smooth the trajectory of finger Fix lost results Method Kalman filter Smooth the path Predict the new state and its uncertainty Correct the tracker with its new measurement Assume white noise and uniform velocity Original After Kalman filter 24

25. Parameter Determination Requirements of ideal finger detection system Sensitive  miss ↓ Noise-free  false alarm ↓ Goal Find an applicable set of parameters for finger detection system fulfilling the requirements 25

26. Parameter Determination 26 Parameters Determinator Parameter Combination Detection Result Applicable set of Parameters Test Set Touch Data Ground Truth (Trace) Detection System

27. Parameter Determination Evaluation of parameters Data Collection Depict trace Measurement Minimize # of miss and false alarm 27

28. Parameter Determination Ideal finger detection Only one fingertip landing on trace Continuity among frames 28

29. Outline Introduction Related Work System and Method Experiments Conclusion & Future Work 29

30. Experiments Performance evaluation 30

31. Experiments Parameter determination Decide parameters in our system Adopt sampling-based parameter search technique 31 Normalization Difference of Gaussian Background Subtraction Binary Finger Analysis Subtract value Smooth kernel Smooth kernel Threshold Finger Size Finger Size

32. Experiments Parameter determination Exhaustive search Parameter combination 5 (step) *5 (step) *5 (step) *5 (step) = 625 Applicable parameter num 16/625 = 2.56% 32 Subtract value Smooth kernel ThresholdFinger size Low bound 0510 Step55510 High bound 20253050

33. Experiments Parameter determination Particle filtering 33 Sampling Measure

34. Outline Introduction Related Work System and Method Experiments Conclusion & Future Work 34

35. 35