The Free Safety Problem Using Gaze Estimation as a Meaningful Input to a Homing Task Albert Goldfain CSE 668: Animate Vision Principles Final Project Presentation

Defining the problem American Football Terminology A “quarterback” is an offensive player who is responsible for throwing the ball to “receivers”. A “receiver” is an offensive (moving target) player who runs in a predefined pattern (or “route”) known to the quarterback and the rest of his team. The receiver attempts to catch the ball if the quarterback decides to throw to him. The “free safety” is a defensive player who is responsible for impeding the targeted receiver from catching the ball. The free safety is called “free” because he is not assigned to any particular receiver, and must make the decision of who to guard during play. He is labeled “safety” because he is usually the last line of defense and stands 6 or 7 yards behind all of the other defenders. The goal of this project is to explore the visual aspects of the free safety’s decision making process as he selects a receiver to guard.

Existing Literature/Research Gaze / Pose Estimation –Perception of head orientation. [Wilson et al] –Motion Segmentation and Pose Recognition with Motion History Gradients. [Bradski, Davis] –Head pose estimation without manual initialization. [Fitzpatrick] –3D Face pose estimation and tracking from a monocular camera. [Ji, Hu] –Pose determination of human faces by using vanishing points. [Wang, Sung] –Many, many more. Computational Vision in Sports –RoboCup [ –Computers Watching Football. [MIT Media Lab/Vision and Modelling Group]

Initial Observations The problem is too large without domain knowledge/constraints. A robotic (embodied computational) free safety would have to act very quickly…no time for recovery paradigm approach…a good application for active/animate vision. From the free safety’s point of view, the task of guarding a receiver can be done using only two dimensions (width and depth). This is why a coach can draw x’s and o’s on a chalkboard without confusing his players.

Domain Constraints Players wear uniforms Markings on field can be used to determine distances with precision Good lighting can be assumed Sidelines 2D internal representation for free safety QB head pose restrictions

Free Safety Subtasks

Quarterback Centered Coordinate System At each instant t, wide receiver i’s location is given by the parametric coordinates (x i (t),y i (t)) in this space. The initial pre-snap values for the safety position are given by

Safety Centered Polar Coordinate System Coordinates for wide receiver i can be expressed with a distance from the safety r i and rotation angle  s We can convert from safety coordinates to quarterback coordinates using

Finding the Quarterback Safety could use one of two techniques: Use constraints of football formations to determine who is behind the center (a lineman on the line of scrimmage) at time t 0 Search image for a player that has the ball immediately after time t 0 Once the quarterback is found, the head pose image must be segmented out.

Given a new image… …find best correlation to a stored image Image Database: Views of Quarterback Pose at 10 deg. Angle Increments.

Using Image Difference Matching as a Similarity Metric Difference will be minimized when test image and stored image are the most similar. Thus the darkest of these image differences should be closest match. The catch: often times, symmetries in image differences at certain angles will yield false positives. Perhaps image difference is a weak similarity metric! The solution: Look at the angular “nearest neighbors” of a proposed solution.

Some Test Results for Image Difference Classification Acute Angle Obtuse Angle Test Image Stored Image

Labeling the Receivers in Quarterback’s FOV Safety begins turning motion based on θ q If no receiver is found, safety turns back to the quarterback to estimate θ q once again. If multiple receivers are found, a suggested egomotion “action” is output and θ q is estimated again. If only one receiver is found, the safety approaches that receiver.

Further Work / Potential Applications Working on decision making egomotion “action” function for the safety. Such a function needs to balance the early goal of maintaining a large field of view with the later goal of approaching the wide receivers. This function needs to be time based. If the defensive team is doing its job, the quarterback should be running out of time to throw. I am willing to take any suggestions I am willing to take any suggestions I believe further research of this problem could yield important results in robotics, video game AI and computer vision in general.