Passive Object Tracking from Stereo Vision Michael H. Rosenthal May 1, 2000

Outline Purpose Camera Calibration Data Collection Object Tracking Depth from Stereo Results Conclusions

Purpose 3D visualization - blood vessels, organs, teapots Stereo displays require head tracking Cumbersome trackers are undesirable

Purpose Passive tracking offers tetherless option Speed and accuracy is a concern

How To Do It Mount reflectors on tracking target Calibrate pair of cameras Identify reflectors in each image Calculate positions using stereo disparity Update tracking model using new positions

Camera Calibration Select six points on known calibration target Solve for six extrinsic and four intrinsic parameters (ignore distortion) Used derivative of Tsai’s calibration code ( Thanks to Herman Towles and Ruigang Yang of Office of the Future for adapted code

Data Collection Mounted target on optical rail Translated target at 1mm per frame over 20cm Rotated target through 120 degrees

Object Tracking We want to follow an object through a scene Challenge - what targets are best for tracking? Spheres yield spatially ambiguous results Complex shapes limit ambiguity but are hard to track Model of glasses with tracking targets

Object Tracking I chose a square and a rectangle - unambiguous, but surprisingly hard to track Square is easy - calculate centroid as weighted average of position and intensity Rectangle is the problem - need center and orientation, makes problem non-trivial Model of glasses with tracking targets

Object Tracking Calculate centroid of rectangle Search for shortest axis at some angle through centroid Find edges along short and long axes using derivatives Use endpoints of long axes as tracking targets Use prior results for future frames Model of glasses with tracking targets

Depth from Stereo Use point pairings to get depth Find shortest segment between the two pixel rays Use midpoint as position estimate Trucco section 7.4 describes equations

Results Translation Moved stage by 210mm Average measured translation: 178mm 16% error Rotation Rotated target by 60 degrees Averaged measured rotation: 70 degrees 16% error Neither showed high noise, so systematic error is likely (calibration?)

Conclusions Camera calibration is somewhat challenging Good camera calibration is very challenging Robust object tracking will require significant development Simplified targets will reduce complexity