WP3 - 3D reprojection Goal: reproject 2D ball positions from both cameras into 3D space Inputs: – 2D ball positions estimated by WP2 – 2D table positions selected by user (GUI) – Camera matrix and additional parameters

WP3 - 3D reprojection Objectives: – Use known table dimensions and its projection to estimate the position (+rotation) of the cameras – Use camera positions, camera matrix and 2D ball points to reproject and estimate real ball positions

WP3 - 3D reprojection Two phase process – 1 x scene analyses = 2 x pose estimation – N x reprojection Step 1: scene analyses POSIT (Pose from Orthography and Scaling with ITeration) – Originally proposed in 1992 – Computes the pose (position and rotation) of a known object

WP3 – 3D reprojection step 1: scene analysis POSIT (continued) – Requires: At least four non-coplanar points of the object Image projections of these object points Focal length in pixels f x,y  assumes square pixels f x = F * s x, f y = F * s y s x and s y being the number of pixels/mm on the imager – Estimates: Translation vector T from center of projection towards origin object model Rotation matrix R relative to object model origin

WP3 – 3D reprojection step 1: scene analysis POSIT (continued) – Resctriction: weak-perspective approximation Assumes that the points on the object are all at effectively the same depth  which means internal depth differences within the object are neglectable Still converges properly, probably due to: Regular shape of the table Imager and table being approximately aligned

WP3 – 3D reprojection step 1: scene analysis POSIT (continued) – Obvious choice of points would be:

WP3 – 3D reprojection step 1: scene analysis POSIT (continued) However, – Algorithm does not benifit from additional coplanar points – Experimental results are only descent if coordinates variate enough on each axis. Proposed points vary to less in y-direction  use height of table

WP3 – 3D reprojection step 1: scene analysis POSIT (continued) Different set of points: Advantage: + Converges properly Disadvantages: - Position of table leg not official - Bottom table not on our footage (camera 2)

WP3 – 3D reprojection step 2: reprojection Step 2: reprojection Input, for both cameras : – Rotation matrix R of object model – Tranlation vector T of object model – Focal length F – Pixels/mm on the imager – Cx cy (principle ray does not go through center of imager exactly) Assumes a simple camera pinhole model

WP3 – 3D reprojection step 2: reprojection Step 2: reprojection 3D point is located on the ray r from the center of projection, through the point on the projection plane where a ball was detected For each camera, this ray can be calculated using: – F, sx, sy, and the coordinates of the detected ball Using R and T these rays can be converted to the coordinate system of the table. The 3D point can be approximated by the crossing of the rays

WP3 – 3D reprojection step 2: reprojection However, this step is executed for each frame  has to be computationally efficient  using intersection of planes 1.For each camera, a vertical plane is constructed defined by – Normal n, being cross product of: » Ray r (through center of projection and projected point) » Unity vector (0,1,0) – Point p, being the projected point 2.For each camera, a horizontal plane is constructed defined by – Normal n, being cross product of: » Ray r » Unity vector (1,0,0) – Point p, being the projected point

WP3 – 3D reprojection step 2: reprojection 1.vertical planes construction 2.Horizontal planes construction 3.All planes are converted to the table coordinate system – Using R and T – Including 180 degree turn 4.Intersection between vertical planes is calculated  results in line l 5.Intersections between line l and horizontal planes is calculated  results in points p1 and p2 6.3D point is approximated by the average of p1 and p2

WP3 - 3D reprojection Issues: – Should have exact location of model point (and its projections) which varies in y-direciton solution: can use table leg (unofficial and not in our footage) – Not enough good frames for calibration  estimed focal lengths wrong solution: focal lengths defined experimentally