Cynthia Atherton

 Methodology  Code  Results  Problems  Plans

“Hand Detection and Segmentation in Cluttered Environments” by Christopher Schwarz and Niels da Vitoria Lobo School of Computer Science, University of Central Florida, Orlando, FL

 Hand tracking system  Detect and segment straight fingered hands ▪ Any complex background ▪ All reasonable lighting conditions ▪ Single frame analysis  Existing systems ▪ Work with silhouettes ▪ Work under normal lighting conditions

 Preprocessing  Detecting candidate fingers  Grouping finger detections into candidate hand detections  Post-processing candidate hand groups  Generating output

 Break down image into lines and curves  Search for a pair of nearly parallel lines and a fingertip curve to constitute a candidate finger  Run Kanade-Lucas-Tomasi (KLT) Feature Tracker to find locations of textured regions

Combination of Burns Line Finder and Canny Edge Detection Curvefinder software is separate program written by Jan Prokaj

Used in phase 4 to strengthen the detection confidence in finger and hand detections in regions of low texture and weaken those in regions of more complex texture.

 Build the initial candidate list  Remove duplicates  Correct bases  Reapply Area Tests

 Tip is found correctly, but base does not extend to location where finger meets palm  Method of base correction on each finger:  Sample gradient magnitude in region around original base point.  Move sampling bounding box along median line past the original base in increments of several pixels.  Compare strength of data in region to previous.  Assume new base when magnitude drastically change.

 When a finger is extended through base correction, should the original finger be discarded?  Apply length-to-width ratio to original  If ratio is in range, keep original and add duplicate detection with “corrected” base.

 Form initial groups  Score groups for correctness  Reinstate candidates  Remove false positive hand groups  Merge similar hand groups  Remove fingers from hand groups Repeat once before continuing to phase 4

 To get an estimated palm center: 1. Draw a ray along the medial axis of the finger through the center of its base and beyond 2. Take a point in the ray estimated as the palm center by following it for a distance past the base that is a percentage of the length of the finger (0.6x)

 To group candidate fingers into candidate hands:  2 candidate fingers are connected if they meet several criteria, for example: ▪ Similar median brightness values ▪ Estimated palm centers are close (Euclidean distance)  Results in cliques of fingers

 For a detected hand group  Average all of the estimated palm centers for the fingers to get the palm center  Estimate palm region as a circle around the center, with a radius proportion to the average length of the detected fingers

 2 levels of promotion  Promotion 1 ▪ Stronger method of promotion ▪ Candidate must match every finger in the group  Promotion 2 ▪ Candidate’s tip-base median ray must come within some distance of group’s palm radius ▪ Candidate must point in same direction as other fingers in group

 Correct palm center locations  Reapply Remove fingers from hand groups  Calculate openness rating of each group  Reapply Remove false positive hand groups  Remove false positive groups according to the positions of others  Reapply Correct palm center locations

 Improve location of palm center by looking for internal wedges  Curves found in negative space of hand and open fingers  Points along perimeter of the palm  Wedges used to nudge original palm center in the right direction

 Detected hands  Pink – palm center  Blue – detected fingers  Green – Maybe class

 Curvefinder written in C by Jan Prokaj  KLT written in C by Stan Birchfield of Clemson University  Line sketch/Finger finder written in Java by Chris Schwarz  Curve sketches generated prior to running Fingerfinder

 System is good at detecting palm locations, but not straight fingers.  Curvefinder not designed to run on Windows  Learning how to run Linux image in VMWare Player

 Curved fingers  Finding fingers using silhouettes and AdaBoosting  Get with Dr. Lobo for info on head detection  Learn about AdaBoosting  Apply new method for finger detection