Intellectual Property Rights are governed by PEGASE Contract Annex II Part C and PEGASE consortium agreements. Before using, reproducing, modifying or disclosing the information contained in that document, it is under the user responsibility to check its rights and if mandatory ask to the Intellectual Property Owner the authorisation to use, reproduce, modify or disclose the information. WP6: IST / INRIA-Sophia Tiago F. Gonçalves, José R. Azinheira, Patrick Rives March 12-13, 2009 Ljubljana

Intellectual Property Rights are governed by PEGASE Contract Annex II Part C and PEGASE consortium agreements. Before using, reproducing, modifying or disclosing the information contained in that document, it is under the user responsibility to check its rights and if mandatory ask to the Intellectual Property Owner the authorisation to use, reproduce, modify or disclose the information. Contents Previous Work Visual Tracking (INRIA-Sophia / IST) –Dynamic management of ROIs –Direct visual tracking Visual Servoing (IST / INRIA-Sophia) –Sensor-based –Position-based visual servoing –Image-based visual servoing Results Conclusions / Future Work

Intellectual Property Rights are governed by PEGASE Contract Annex II Part C and PEGASE consortium agreements. Before using, reproducing, modifying or disclosing the information contained in that document, it is under the user responsibility to check its rights and if mandatory ask to the Intellectual Property Owner the authorisation to use, reproduce, modify or disclose the information. Previous Work (1) Matlab simulation framework: –Visual tracking based on feature points –Camera model with image discretization –Homography computation using SVD ESM visual tracker: –Dense visual tracking –Estimation of lightning parameters –Real time performance

Intellectual Property Rights are governed by PEGASE Contract Annex II Part C and PEGASE consortium agreements. Before using, reproducing, modifying or disclosing the information contained in that document, it is under the user responsibility to check its rights and if mandatory ask to the Intellectual Property Owner the authorisation to use, reproduce, modify or disclose the information. Previous Work (2) Visual Servoing (PBVS) –Optimal control design (LQR) –Linearization of PEGASE aircraft flight model –Direct control of actuators

Intellectual Property Rights are governed by PEGASE Contract Annex II Part C and PEGASE consortium agreements. Before using, reproducing, modifying or disclosing the information contained in that document, it is under the user responsibility to check its rights and if mandatory ask to the Intellectual Property Owner the authorisation to use, reproduce, modify or disclose the information. Visual Tracking (1) Dynamic management of ROIs: –Avoid that the followed ROI goes out of the field-of-view when the aircraft is manoeuvering –Selection of the most textured region of the image

Intellectual Property Rights are governed by PEGASE Contract Annex II Part C and PEGASE consortium agreements. Before using, reproducing, modifying or disclosing the information contained in that document, it is under the user responsibility to check its rights and if mandatory ask to the Intellectual Property Owner the authorisation to use, reproduce, modify or disclose the information. Visual Tracking (2) Direct visual tracking: –Reference path defined by images Each sector is sampled by images and indexed by respective camera pose –Output Homography between current and reference image ROIs –Integrated into the PEGASE simulation framework

Intellectual Property Rights are governed by PEGASE Contract Annex II Part C and PEGASE consortium agreements. Before using, reproducing, modifying or disclosing the information contained in that document, it is under the user responsibility to check its rights and if mandatory ask to the Intellectual Property Owner the authorisation to use, reproduce, modify or disclose the information. Visual Servoing Sensor-based –u = u 0 - K x (x - x 0 ) - K v (v - v 0 ) x/v -> pose/velocities states measured by aircraft sensors Position-based visual servoing (PBVS) –u = u 0 - K y (y - y 0 ) - K v (v - v 0 ) y = f (H = R+tn’/d) pose data extrated from homography by decompositon Image-based visual servoing (IBVS) –u = u 0 - K x y e - K v (v - v 0 ) y e = (L s c W 0 ) + (H(:) – H*(:)), with H * = I(3)

Intellectual Property Rights are governed by PEGASE Contract Annex II Part C and PEGASE consortium agreements. Before using, reproducing, modifying or disclosing the information contained in that document, it is under the user responsibility to check its rights and if mandatory ask to the Intellectual Property Owner the authorisation to use, reproduce, modify or disclose the information. Visual Tracking Results (1) PBVS (closed-loop) Flog density – 0.4 Rain density – 0.8 Rain thickness – 1p Wind: 10m/s N -1m/s E No turbulence

Intellectual Property Rights are governed by PEGASE Contract Annex II Part C and PEGASE consortium agreements. Before using, reproducing, modifying or disclosing the information contained in that document, it is under the user responsibility to check its rights and if mandatory ask to the Intellectual Property Owner the authorisation to use, reproduce, modify or disclose the information. Visual Tracking Results (2)

Intellectual Property Rights are governed by PEGASE Contract Annex II Part C and PEGASE consortium agreements. Before using, reproducing, modifying or disclosing the information contained in that document, it is under the user responsibility to check its rights and if mandatory ask to the Intellectual Property Owner the authorisation to use, reproduce, modify or disclose the information. Visual Tracking Results (2)

Intellectual Property Rights are governed by PEGASE Contract Annex II Part C and PEGASE consortium agreements. Before using, reproducing, modifying or disclosing the information contained in that document, it is under the user responsibility to check its rights and if mandatory ask to the Intellectual Property Owner the authorisation to use, reproduce, modify or disclose the information. Visual Servoing Results (1) IBVS Flog density – 0.4 Rain density – 0.8 Rain thickness – 2p Wind: 10m/s N -1m/s E Turbulence – L1

Intellectual Property Rights are governed by PEGASE Contract Annex II Part C and PEGASE consortium agreements. Before using, reproducing, modifying or disclosing the information contained in that document, it is under the user responsibility to check its rights and if mandatory ask to the Intellectual Property Owner the authorisation to use, reproduce, modify or disclose the information. Visual Servoing Results (2)

Intellectual Property Rights are governed by PEGASE Contract Annex II Part C and PEGASE consortium agreements. Before using, reproducing, modifying or disclosing the information contained in that document, it is under the user responsibility to check its rights and if mandatory ask to the Intellectual Property Owner the authorisation to use, reproduce, modify or disclose the information. Conclusions Visual Tracking: –Visual tracking software validated and integrated into the PEGASE simulation framework –Not valid for low altitudes (<30m) due to the high pixel displacement Homography-based visual servoing –PBVS and IBVS validated and integrated into the PEGASE simulation framework –IBVS does not consider wind disturbances

Intellectual Property Rights are governed by PEGASE Contract Annex II Part C and PEGASE consortium agreements. Before using, reproducing, modifying or disclosing the information contained in that document, it is under the user responsibility to check its rights and if mandatory ask to the Intellectual Property Owner the authorisation to use, reproduce, modify or disclose the information. Future Work Pan-Tilt control –Pan-tilt control to compensate wind disturbance for IBVS Non-Linear control –Homography-based non-linear control law Complete automatic landing –Runway side lines following for flare, roll-out and also take-off (Rives and Azinheira)