PortableVision-based HCI A Hand Mouse System on Portable Devices 連矩鋒 (Burt C.F. Lien) Computer Science and Information Engineering Department National Taiwan University
Problems A Portable Vision-based HCI –Hand mouse operating on a projected interface –Real-time detection of user hand motion from a user PDA/SmartPhone’s video camera (target platform) A part of “mTeller” project –Natural storytelling support system
Why important Vision-based HCI is a more instinct way to manipulate data Mobility –Steerable interface everywhere
Related Works I A Portable System for Anywhere Interactions –Sukaviriya et al., IBM Research Real-time hand tracking using a set of cooperative classifiers based on Haar-like features –Barczak1 et al., Institute of Information & Mathematical Sciences Massey University
Everywhere Display (IBM) Figure 1: Interactive store application
Related Works II Real-Time Hand-Arm Motion Analysis using a single Video Camera – Hienz et al. MMX-Accelerated Real-Time Hand Tracking System –Liu et al. (2001)
Related Works III Rapid Object Detection Using a Boosted Cascade of Simple Features. –Viola, P., & Jones, M. (2001). Robust real-time object detection. –Viola, P., & Jones, M. Robust real-time face detection –P. Viola and M. Jones. Adaboost-based real-time pedestrian detection –P. Viola, M. Jones, and D. Snow.
Target Devices
System Configuration Hand motion capture and interpretation Wireless projector data transmission Interactive Interface
What is new A portable vision-based HCI on handheld device Design for general data manipulation
System Implementation Platform (prototype) –“Laptop” + “Low Cost Camera (USB)” Software tools –“MS VC++” + “Intel OpenCV library”
Technical Challenges and approaches Real-time hand motion tracking under an intense lighting condition (projector light) –Adaboost Hand position vs. Corresponding programs on host –Comparison between PDA’s screen and the projected screen(??) An efficient algorithm to run detection system on handheld devices (limited computation) –Lower framing rate (2fps?) + fewer classifier Noise filter –A sliding window or a simple filter to filter misbehaving hand motion
Expect Result A generic vision-based HCI software program running on a laptop (1st phase) A user can manipulate the laptop directly on the projected image –Hand mouse (one-click function only)
The End