Members: Nicholas Allendorf - CprE Christopher Daly – CprE Daniel Guilliams – CprE Andrew Joseph – EE Adam Schuster – CprE Faculty Advisor: Dr. Daji Qiao Client: Dr. Stephen Gilbert Virtual Reality Application Center

Create a system capable of accurately tracking fingertips in three dimensions Incorporate the ability to support as many as six users simultaneously Design the system so that it is easily reproducible Slide of 12 2

Project Requirements Provide a 3D position of all tracked fingertips within a 2m x 2m x 2m indoor region with 1 centimeter accuracy Update positions 15 times per second (15 Hz) with low latency The system shall be capable of tracking as many as 60 fingertip positions simultaneously Positions shall be displayed in a graphical interface so the position may be viewed in real time Slide of 12 3

Project Plan Use Optical/Infrared tracking – Most practical solution and cost effective solution Use stereo cameras to track and localize IR LEDs embedded on gloves Process images with a desktop computer and open source computer vision software Slide of 12 4

System Design and Layout Glove – Contains IR LEDs/color markers on fingertips which will be tracked by the cameras Infrastructure – Provides stable and measurable mounting points for the cameras Cameras – Mounted in stereo pairs around periphery of infrastructure – Detect IR LEDs and pass images to server for processing Server/Computer – Performs image processing, calculates position, and runs the GUI Slide of 12 5

System Hardware/Software Cameras : Logitech QuickCam Pro 9000 Computer : Dell XPS LEDs : 950 nm Surface Mount Infrared LEDs Infrastructure : 8020 Aluminum Framing Operating System: Windows 7 Image Processing: OpenCV Slide of 12 6

Preliminary Results Gloves – A seamstress is working on making a glove of our design – Battery Pack consists of 4 AAA batteries to give ~20 hrs of continuous use Stereo Cameras – The first stereo camera is assembled and working well – Another set of cameras arrived just this week – assembly imminent IR Filter – The developed film filters worked ok, but image quality was an issue – Changed to a low cost commercial Longpass IR Filters, which is much more effective than than the old developed film Infrastructure – Rudimentary infrastructure in place, but more parts are needed Slide of 12 7

IR LED with new filter on cameraIR LED with old filter on camera Preliminary Results Slide of 12 8

Assembled stereo camera with filtersInfrastructure around TV Preliminary Results Slide of 12 9

Preliminary Results Camera Calibration – Using a checkerboard pattern and OpenCV to rectify images – Initially, Calibration was inaccurate and unusable – Calibration has improved, and our average error values are now in an acceptable range Image Processing and Finger Identification – Currently able to easily identify locations of IR LEDs in filtered images – Initially we were unsure of how to discriminate between fingers – New solution: One camera with a filter, and one without, and colored fingertips on the glove to discriminate between fingers Localization/Tracking – With an accurate calibration and good LED location we are able to produce a 3D location of a single LED – Nearly able to do so for multiple LEDs – Localization needs to be improved – Good precision, poor accuracy Slide of 12 10

Current Schedule Slide of 12 11

Questions? Slide of 12 12

Thanks for your time!