« Structure from motion » algorithms for the CYCAB robot Guillaume CERNIER ENSIMAG - 2nd year internship 19th June - 15th September 2006

Tuesday, 16th January 07 Guillaume CERNIER « Structure from motion » algorithms for the CYCAB robot INRIA presentation Well-known french research centre in : –Computer science –Control theory –Applied mathematics 6 research centres (Lorraine, Rennes, Rhône-Alpes, Rocquencourt, Sophia-Antipolis, Futurs) Partner of the new research centre of Microsoft at Orsay INRIA Rhône-Alpes unit : –26 research teams among 8 associated with foreign laboratories –Near 500 people (engineers, researches, PhD students, interns, …) –Local partners : IMAG, UJF, INPG, CNRS, … –Member of 10 european research networks –271 contracts among 53 with industry and 26 european –14 start-ups created since 1998 (more than 560 jobs created)

Tuesday, 16th January 07 Guillaume CERNIER « Structure from motion » algorithms for the CYCAB robot e-Motion team presentation “GEometry and Probability for MOtion and AcTION” Belongs to the GRAVIR lab (INRIA, CNRS, UJF, INPG) Goals : –Developing algorithmic models and methods allowing to build artificial systems –With capacities of perception, decision, and action –Operating in open and dynamic environments Start-ups : Itmi (1982), Getris Images (1985), Aleph Technologies (1989), Aleph Med (1992), Probayes (2003). Industrial collaborations : Robotsoft, Renault, PSA, AW Europe (linked to Toyota), XL-Studio, Aesculap, Teamlog, Kelkoo.

Tuesday, 16th January 07 Guillaume CERNIER « Structure from motion » algorithms for the CYCAB robot CYCAB presentation Large R&D program between INRIA & INRETS Autonomous vehicle intended for urban areas Automatic taxi computer-guided Can be adapted for blind and/or handicapped people Autonomous + urban areas => avoiding obstacles Before my internship : avoidance realized by a laser telemeter Internship goal : use a video camera to estimate the structure of the whole scene in front of the vehicle Called “structure from motion”

Tuesday, 16th January 07 Guillaume CERNIER « Structure from motion » algorithms for the CYCAB robot Internship goals Videos, sequence of pictures3D estimation of the structure Constraints :  Software development : C/C++  Real-time processing  Integration to the whole platform embedded in the Cycab (plug-in format) Helps :  Use of an open-source 3D vision library : “OpenCV”

Tuesday, 16th January 07 Guillaume CERNIER « Structure from motion » algorithms for the CYCAB robot “Structure from motion” (1/3) The problem : only one camera => unknown motion between 2 snapshots So, structure from motion is a 2-step process : – Optical flow (vector field) computation between 2 pictures –Thanks to this optical flow, computing : First, the motion of the camera between the 2 pictures Finally, the 3D structure of the scene

Tuesday, 16th January 07 Guillaume CERNIER « Structure from motion » algorithms for the CYCAB robot Optical flow computation (2/3) SPARSE Vector field –To lighten computation –Vectors tracking real points in the scene How choose the points to track ? –GoodFeaturesToTrack algorithm –Exists in OpenCV library Computation of the optical flow : –Pyramidal algorithms –Establishing correspondence between points of the 2 pictures –Eliminate “outliers” –Let to know the “fundamental matrix”

Tuesday, 16th January 07 Guillaume CERNIER « Structure from motion » algorithms for the CYCAB robot 3D structure computation (3/3) 2 steps : –Find the motion between the 2 pictures : Thanks to fundamental & intrinsic matrixes Find the rotation and translation resulting –Here, we know the position of the camera at the 2 snapshots –Computes the 3D point for each point pair (q 1,q 2 ) in the 2 pictures : Triangulation methods Least-square methods (retained) Depends of the projection matrixes P 1 & P 2 P i = KR i ( I 3x3 | -t i ) (4x3 matrixes)

Tuesday, 16th January 07 Guillaume CERNIER « Structure from motion » algorithms for the CYCAB robot Other development Camera Calibration GUI

Tuesday, 16th January 07 Guillaume CERNIER « Structure from motion » algorithms for the CYCAB robot Difficulties and successes 3D vision library OpenCV contains some bugs … Tutors absence … Very interesting research field ! Reach the internship goals 2 weeks before the planned date Research article writing for a robotics conf (ICRA 07) Learning a GUI API (Gtk+)

Tuesday, 16th January 07 Guillaume CERNIER « Structure from motion » algorithms for the CYCAB robot Team life Different culture people (Italian, Brazilian, Asiatic, …) Summer at INRIA is …