Su-ting, Chuang 2010/8/2

Outline Introduction Related Work System and Method Experiment Conclusion & Future Work 2

Outline Introduction Related Work System and Method Experiments Conclusion & Future Work 3

Introduction Non-uniform lighting problem Various finger touch response among different position Low computation efficiency No such tool that helps users determine parameters automatically 4

Outline Introduction Related Work System and Method Experiments Conclusion & Future Work 5

Related Work FTIR (Frustrated Total Internal Reflection) J. Y. Han, “Low-cost multi-touch sensing through frustrated total internal reflection," in Proceedings of the 18th annual ACM symposium on User interface software and technology (UIST '05). New York, NY, USA: ACM Press, 2005, pp

Related Work DI (Diffused Illumination) J. Rekimoto and N. Matsushita, “Perceptual surfaces: Towards a human and object sensitive interactive display," Workshop on Perceptural User Interfaces (PUI'97),

Related Work TouchLib A multi-touch development kit Finger detection processing flow chart 8 Background Subtraction Simple Highpass ScaleThreshold Finger Analysis

Related Work DirectShow Filter-based framework GShow GPU-accelerated framework Combination of DirectX and DirectShow 9

Outline Introduction Related Work System and Method Experiments Conclusion & Future Work 10

Hardware Configuration (2) IR Camera (3) IR Illuminator (1) Peripheral Projector 11

Hardware Configuration Order of diffuser layer and touch-glass layer 12 Diffuser layer IR illuminator IR camera spot IR illuminator IR camera Touch-glass layer IR camera spot IR camera

Hardware Configuration Problem: IR rays reflected by the touch-glass will result in hot spot regions in camera views Solution: Use other cameras to recover the regions which are sheltered by IR spots 13

Software Architecture Detection system Image Stitching Finger Detection Finger Tracking Parameter determination 14 Image Stiching Image Stiching Finger Detection Finger Detection Finger Tracking Finger Tracking

Software Architecture 15 Image Stiching Image Stiching Finger Detection Finger Detection Finger Tracking Finger Tracking

Image Stitching Goal Combine multi-camera view into a virtual camera view 16

Image Stitching Advantages Remove IR spot effect Unify finger size among different position of table Reduce matching problem Be compatible with existent finger detection system 17

Image Stitching 18 Image Blending IR Camera(L) IR camera(R) Undistortion HomoWarp

Image Stitching HomoWarp

Image Stitching Image Blending 20

Finger Detection TouchLib Our method 21 Normalization Difference of Gaussian Background Subtraction Binary Finger Analysis Simple Highpass Scale Background Subtraction Binary Finger Analysis

Finger Detection Normalization Method Model distribution of IR illumination Use specific material to simulate foreground Construct normalization map Normalize foreground image Result Before normalization: mean = 75, standard variation = 30 After normalization: mean = 255, standard variation = 3 22

Finger Detection Difference of Gaussian (DoG) Modified from simple highpass in TouchLib 23

Fingertip Tracking Goal Smooth the trajectory of finger Fix lost results Method Kalman filter Smooth the path Predict the new state and its uncertainty Correct the tracker with its new measurement Assume white noise and uniform velocity Original After Kalman filter 24

Parameter Determination Requirements of ideal finger detection system Sensitive  miss ↓ Noise-free  false alarm ↓ Goal Find an applicable set of parameters for finger detection system fulfilling the requirements 25

Parameter Determination 26 Parameters Determinator Parameter Combination Detection Result Applicable set of Parameters Test Set Touch Data Ground Truth (Trace) Detection System

Parameter Determination Evaluation of parameters Data Collection Depict trace Measurement Minimize # of miss and false alarm 27

Parameter Determination Ideal finger detection Only one fingertip landing on trace Continuity among frames 28

Outline Introduction Related Work System and Method Experiments Conclusion & Future Work 29

Experiments Performance evaluation 30

Experiments Parameter determination Decide parameters in our system Adopt sampling-based parameter search technique 31 Normalization Difference of Gaussian Background Subtraction Binary Finger Analysis Subtract value Smooth kernel Smooth kernel Threshold Finger Size Finger Size

Experiments Parameter determination Exhaustive search Parameter combination 5 (step) *5 (step) *5 (step) *5 (step) = 625 Applicable parameter num 16/625 = 2.56% 32 Subtract value Smooth kernel ThresholdFinger size Low bound 0510 Step55510 High bound

Experiments Parameter determination Particle filtering 33 Sampling Measure

Outline Introduction Related Work System and Method Experiments Conclusion & Future Work 34

35