1. Su-ting, Chuang 2010/8/2

2. Outline Introduction Related Works System and Method Experiment Conclusion & Future Work 2

3. Outline Introduction Related Works System and Method Experiments Conclusion & Future Work 3

4. Introduction i-m-Top interactive multi-resolution tableTop interactive: multi-touch multi-resolution: fovea + peripheral projectors 4 Chia, Yi-Wei, i-m-Top, an Interactive Multi-resolution Tabletop Display System (2007)

5. Introduction i-m-Top SDK A software development toolkit for rapid prototyping multi-resolution and multi-touch applications 5

6. Introduction Low computation efficiency Non-uniform lighting problem Various finger touch responses among different positions No tools available for helping users determine parameters automatically 6

7. Outline Introduction Related Works System and Method Experiments Conclusion & Future Work 7

8. Related Works DI (Diffused Illumination) J. Rekimoto and N. Matsushita, “Perceptual surfaces: Towards a human and object sensitive interactive display," Workshop on Perceptural User Interfaces (PUI'97), 1997. 8

9. Related Works FTIR (Frustrated Total Internal Reflection) J. Y. Han, “Low-cost multi-touch sensing through frustrated total internal reflection," in Proceedings of the 18th annual ACM symposium on User interface software and technology (UIST '05). New York, NY, USA: ACM Press, 2005, pp. 115-118. 9

10. Related Work TouchLib A multi-touch development kit Finger detection processing flow chart 10 Background Subtraction Simple Highpass Intensity Scaling Thresholding Finger Analysis Parameters to be adjusted manually Image Enhancement a1a2, a3a4a5a6

11. Related Work DirectShow Filter-based framework GShow GPU-accelerated framework Combination of DirectX and DirectShow 11

12. Outline Introduction Related Work System and Method Experiments Conclusion & Future Work 12

13. Hardware Configuration (2) IR Camera (3) IR Illuminator (1) Peripheral Projector 13

14. Hardware Configuration Order of diffuser layer and touch-glass layer 14 Diffuser layer IR illuminator IR camera spot IR illuminator IR camera Touch-glass layer IR camera spot IR camera 2 1 4 3

15. Hardware Configuration Problem: IR rays reflected by the touch-glass will result in hot spot regions in camera views Solution: Two cameras to cover spot regions for each other’s sheltered IR spot zone 15

16. Software Architecture Detection system Image Stitching Finger Detection Finger Tracking Parameter determination 16 Image Stiching Image Stiching Finger Detection Finger Detection Finger Tracking Finger Tracking

17. Software Architecture 17 Image Stiching Image Stiching Finger Detection Finger Detection Finger Tracking Finger Tracking Enhance in GPU

18. Image Stitching Combine multi-camera view into a virtual camera view 18

19. Image Stitching Remove IR spot effect Unify finger size among different positions of table Easily fit in existing finger detection systems 19

20. Image Stitching 20 Image Blending IR Camera(L) IR camera(R) Undistortion HomoWarp

21. Image Stitching HomoWarp 21 5 5 2 2 1 1 6 6 2 2 5 5 4 4 3 3 1 1 4 4 3 3 6 6 2 2 5 5

22. Image Stitching Image Blending 22

23. Finger Detection TouchLib Our method 23 Normalization Difference of Gaussians Background Subtraction Thresholding Finger Analysis Simple Highpass Scale Background Subtraction Thresholding Finger Analysis

24. Finger Detection Insert normalization 24

25. Finger Detection Normalization Method Model distribution of IR illumination Use specific material to simulate foreground Construct normalization map Normalize foreground image Result Before normalization: mean = 75, standard variation = 30 After normalization: mean = 254.8, standard variation = 1.25 25

26. Finger Detection Approximate Difference of Gaussians (DoG) Modified from simple highpass in TouchLib TouchLib: (Original image – Blurred image ) + median filter 26

27. Fingertip Tracking Goal Smooth the trajectory of finger Method Kalman filter Use Kalman filter to predict the current position of t n 27

28. Parameter Determination Requirements of ideal finger detection system High sensitivity  miss ↓ Noise-free  false alarm ↓ Goal Find an appropriate set of parameters for finger detection system fulfilling the requirements 28

29. Parameter Determination 29 Parameters Determinator Parameter Combination Detection Result Applicable set of Parameters Test Set Touch Data Ground Truth (Trace) Detection System

30. Parameter Determination Evaluation of parameters Data Collection Depict trace Measurement Minimize # of miss and false alarm 30

31. Parameter Determination Ideal finger detection Only one fingertip landing on trace Continuity among frames 31

32. Outline Introduction Related Work System and Method Experiments Conclusion & Future Work 32

33. Experiments Performance evaluation 33 640 x 480

34. Experiments Parameter determination Decide parameters in our system Adopt sampling-based parameter search technique 34 Normalization Difference of Gaussian Background Subtraction Binary Finger Analysis b4 b1: subtraction value b2: kernel size b3: threshold b4: finger size b3 b2 b1

35. Experiments Parameter determination Exhaustive search Parameter combination 5 (Iteration) *5 (Iteration) *5 (Iteration) *5 (Iteration) = 625 Applicable parameter num 16/625 = 2.56% 35 Subtract value Smooth kernel ThresholdFinger size Low bound 0510 Step55510 High bound 20253050

36. Outline Introduction Related Work System and Method Experiments Conclusion & Future Work 36

37. Conclusion & Future Work Multi-touch detection system GPU-accelerated Non-uniform lighting problem solved Automatically parameter determination tool proposed Future Work Optimize parameter determination 37

38. 38