1. By shooting 2009/10/1

2. outline imTop overview imTop detection Finger Mobile Finger detection evaluation Mobile detection improvement

3. imTop Architecture Scene Touch/Mobile Detection Touch/Mobile Detection View Application (oForm) Node Other Sensors Projector PTU Speaker Socket Window Message COM Message Parser OpenGL c# Device Control Virtual Director (oDirector) Event queue LED Light CamNode

4. Background Subtraction Opening Down- scale Integral Image Connected Component Palm Tracking Connected Component PCA Analysis Palm-finger Association Finger Tracking Connected Component PCA Analysis Palm Association Finger Fusion - - - fingerpalm

5. Camera Synchronize manager Capture camera (Source filter) Capture camera (Source filter) imTopFilter (Transform Filter) Merge finger results Match current result to previous Process un-matched finger points Smooth finger results imTopFilter (Transform Filter) Show detection results (Render filter) Send current results

6. Peripheral Projector Fovea Projector IR Camera IR Illuminator Mirror Pan-tilt Unit 56” 96 ppi 24 ppi

7. Undistortion Preprocess Build integral image Smooth image Background subtraciton Finger detection Finger result Detect lighter spot Connected component Finger analyze Kernel scan Sufficient energy Appropriate center imTopFilter (Transform Filter)

8. Flow chart Load Image Undistotion Pre-process Finger detection Show result Send Result to imTop Calculate Background image by 10 initial frames (initial 10 frames) Calculate Background image by 10 initial frames (initial 10 frames) Background subtraction (exceed 10 frames) Background subtraction (exceed 10 frames)

9. Finger detection Detect lighter spot Connected component finger analyze Set finger result Image preprocessing

10. Mobile detection Detect contour mobile analyze Set finger result Image preprocessing

11. Result management Match current result to the previous Process for un-matched finger points/mobile contours Process for un-matched finger points/mobile contours Smooth the detection results Send results Copy current results from each camera

12. outline imTop overview imTop detection Finger Mobile Finger detection evaluation Mobile detection improvement

13. Image preprocessing Smooth current image Smooth Image by Gaussian filter (kernel 3x3) Build integral image Calculate the M+2 by N+2 integral image of current M by N background subtracted image

14. Detect lighter spot Searching for region where the average intensity of inner part is higher enough then that of outer part Corners & Finger-size regions will be detected

15. Connected component Using previous finger candidates Using queue Calculate the mean position of each connected candidates Mark each connected component with groupID which is the position of the initial candidate

16. Finger analyze Calculate region center and energy of each connected component Accept as a Finger input if Sufficient Energy The Center is within its region

17. outline imTop overview imTop detection Finger Mobile Finger detection evaluation Mobile detection improvement

18. Mobile detection Image preprocessing Smooth current image Smooth Image by Gaussian filter (kernel 3x3) Detect contour Mobile Analyze Filter contour Not rectangle Inappropriate area

19. Detection result

20. outline imTop overview imTop detection Finger Mobile Finger detection evaluation Mobile detection improvement

21. Single-tapping test Fixed Target Appear sequentially Calculate accuracy Variation between Target center & finger detection point + ++ +

22. Finger-moving test Demo a video of moving a target Tell user try to move this target as demo video Record finger detection point Calculate variation between target center & finger detection point +

23. outline imTop overview imTop detection Finger Mobile Finger detection evaluation Mobile detection improvement

24. Microsoft’s tag