Active Vision Sensor Planning of CardEye Platform Sherif Rashad, Emir Dizdarevic, Ahmed Eid, Chuck Sites and Aly Farag ResearchersSponsor US Army Mounted Maneuver BattleSpace Lab

Objective The main objective of this project is to design and implement an active sensor planning algorithm for the CardEye platform. For this system, generalized camera parameters such as position, orientation, and optical settings have to be determined according to the new position of the robot arm so that its features are within the field of view of the CardEye cameras and are in focus.

System Overview Robot Arm Super Computer Transmit current coordinates to super computer using the serial port Reading robot coordinates from super computer Sensor Planning Sending planned parameters to CardEye to adjust the cameras’ settings according to the new settings CardEye active vision system Sending the captured images of robot arm to be displayed

CardEye Platform This platform uses an agile trinocular vision head contains three CCD cameras (c’, c”, c’’’) with their lenses for the automated zoom and focus. The cameras are placed at equal distances from each other. The cameras can translate (t) along their mounts to change the baseline distance. At the same time, the cameras can rotate towards each other to fixate to a point in space by changing the vergence angle (  ). The target is assumed to be inside a sphere that has a radius (R).

Geometry for Sensor Planning X Y Z O C’ C’’’ C’’ t t t d G R l l l    Object contained in a sphere Optical axis Fixation point Optical Center Camera Parameters: translation ( t ) vergence angle (  ) filed of view angle(  ) (for zoom setting) R l GC’ 

The system's fixation point is the center C of the sphere. The center of the sphere is at distance d from the origin along the z axis. Every time, the sensor planning module will have the radius R and the distance d only (to be calculated from the initial position and the current coordinates of the robot). For suitable planning, we should calculate t for translation,  for adjusting the vergence angle so all cameras can fixate on the same point in 3D space, and  to set the zoom of the cameras. Sensor Planning

System Constraints (a) Overlap Constraint 3t 2 C’ C’’’ G R 3t 2  d’ O’ where By maximizing , overlap area is also maximized. By decreasing t, we increase the overlap.

System Constraints (b) Disparity Constraint 3t 2 C’ C’’’ R 3t 2 P G d’   O’ By increasing t, more adequate depth information can be recovered from the imaged object. Total Angular Disparity

For effective reconstruction, the images must display adequate depth information (increase t) and have a fairly large overlap area (decrease t). Solution Solution: 1. Analyze the effect of object distance on overlap and disparity angles and compute translation t. 2. Normalize the translation values based on the physical range of the system translation. 3. Estimate the system workspace 4. Repeat step 1 and compute t as function of object distance d. Analysis of System Constraints

Five cases of object size are analyzed and their solution for t is estimated for each case: Case 1: Case 1: 0.2m <R < 0.3m, 1.200m < d < 7m t = d d [m] Case 2: Case 2: 0.3m < R < 0.5m, 1.925m < d < 7m t = 0: d2 + 0:04702 d [m] Case 3: Case 3: 0:5m < R < 0.7m, 2.650m < d < 7m t = 0: d2 + 0:05530 d [m] Case 4: Case 4: 0.7m < R <0.9 m, 3.375m < d< 7m t = 0: d2 + 0:06668 d [m] Case 5: Case 5: 0:9m < R <1.0m, 4.100m < d < 7m t = 0: d2 + 0:08380 d [m] 2nd-order polynomial equation for sensor placement t 3rd-order polynomial equation was for the voltage used to control the zoom in lenses

After Sensor Planning Sample of Results (1) Before Sensor Planning At d=2.091m and R=0.399m

At d=1.525m and R=0.200m After Sensor Planning Before Sensor Planning Sample of Results (2)