1. AVITRACK Project FP6-502818 INRIA WP1 - Apron Activity Model WP3 - Scene Tracking WP4 - Scene Understanding Brussels, January 17th 2006

2. AVITRACK Project FP6-502818 WP1 : Apron Activity Model

3. WP1 – Apron Activity Model  Task 1.3 : Formalism for Model Representation Design of a language to describe AVITRACK scenarios corresponding to servicing operations (e.g. Unloading operation) Design of the following formalisms : Scene model of the apron (zones of interest, airport walls) Parked vehicles (zones related to parked vehicles) Expected objects in the scene (size of vehicles, aircraft and people) Camera information and algorithms parameters  Task 1.4 : Models creation Creation of the scenario models in collaboration with Silogic

4.  Example of the Unloading Front Operation model CompositeEvent(UnLoading_Front_Operation, PhysicalObjects( (v1 : Vehicle), (v2 : Vehicle), (z1 : Zone), (z2 : Zone), (z3 :Zone)) Components( (c1 : CompositeEvent Loader_Arrival(v1, z1, z2)) (c2 : CompositeEvent Transporter_Arrival(v2, z1, z3)) Constraints( (v1->SubType = LOADER) (v2->SubType = TRANSPORTER) (z1->Name = ERA) (z2->Name = RF_DoorC_Access) (z3->Name = LOADER_BackZone) (c1 before c2))) WP1 – Apron Activity Model

5.  Illustration of Dynamic Context : WP1 – Apron Activity Model

6.  Delivered documents :  Formalism for Scene Model (IN_AVI_03_001)  Ontology (IN_AVI_01_020, with Silogic)  ORION Scenario Description Language (IN_AVI_03_002) WP1 – Apron Activity Model

7. AVITRACK Project FP6-502818 WP3 : Scene Tracking Task 3.6 : Complex Scene Tracking

8. WP3 - Scene Tracking  Task 3.6 : Complex Scene Tracking Objective : Improve Tracking in complex situations (lost of objects, occlusions, lack of contrast, reflections, shadows…) Long-Term Tracking (LTT) uses a temporal window (typically 10 to 20 frames) to explore possible tracks for each mobile object uses specific algorithms for each type of mobile objects (vehicles, people, aircraft) Global Tracking (GT) global spatio-temporal analysis of mobile object trajectories using a rules based architecture

9.  Tracking of a vehicle partially detected Without Global-Tracker - With Global-Tracker WP3 - Scene Tracking

10.  Deliverables of two software : Long-Term Tracking on Dec. 04 (first) and Jan. 06 (final) Global Tracking on June 05 (first) and Jan. 06 (final) WP3 - Scene Tracking

11. AVITRACK Project FP6-502818 WP4 : Scene Understanding

12. WP4 - Scene Understanding  Objectives : Development of a software to recognise any type of operations described by the Scenario Description Language (WP1) Adaptation, recognition and evaluation of specified scenarios (AVITRACK operations)  Task 4.1 : Basic Event Recognition (e.g. stop of vehicle v)  Task 4.2 : Single Vehicle Movement Interpretation (e.g arrival of vehicle v)  Task 4.3 : Interaction Recognition  Task 4.4 : Complex Activity Recognition

13.  61 Modelled and recognised scenarios 21 general video events (Task 4.1)  10 primitive states (e.g. person inside a zone)  5 composite states (e.g. vehicle stopped in a zone)  6 primitive events (e.g. vehicle changes zone) 28 scenarios involving a single vehicle (Task 4.2, Task 4.3)  8 scenarios for the GPU (Aircraft Arrival Preparation on 10 seq. )  8 scenarios for the Tanker (Refuelling operation on 7 seq. )  3 scenarios for the Aircraft (Aircraft Arrival on 2 seq.)  3 scenarios for the Tow Tractor (Tow Tractor Arrival on 2 seq.)  3 scenarios for the Conveyor (Conveyor Arrival on 2 seq.)  3 scenarios for the Catering (Catering vehicle Arrival on 3 seq.) WP4 - Scene Understanding

14.  Modelled and recognised scenarios 12 scenarios involving multiple vehicles (Task 4.3, Task 4.4)  12 scenarios for the Loader and the Transporter  Unloading Front operation (global) recognised on 2 seq.  Unloading Front operation (detailed) recognised on 2 seq.  Other scenarios : detection of the handler, platform elevation… WP4 - Scene Understanding

15.  Example of the Unloading Front Operation (global) CompositeEvent(UnLoading_Front_Global_Operation, PhysicalObjects( (v1 : Vehicle), (v2 : Vehicle), (z1 : Zone), (z2 : Zone), (z3 :Zone)) Components( (c1 : CompositeEvent Loader_Arrival(v1, z1, z2)) (c2 : CompositeEvent Transporter_Arrival(v2, z1, z3)) Constraints( (v1->SubType = LOADER) (v2->SubType = TRANSPORTER) (z1->Name = ERA) (z2->Name = RF_DoorC_Access) (z3->Name = LOADER_BackZone) (c1 before c2))) WP4 - Scene Understanding

16.  Results on the Sequence S28 : “Unloading Global Operation”

17.  Example of the Unloading Front Operation (detailed) CompositeEvent(UnLoading_Front_Detailed_Operation, PhysicalObjects( (p1 : Person), (v1 : Vehicle), (v2 : Vehicle), (v3 : Vehicle), (z1 : Zone), (z2 : Zone), (z3 :Zone), (z4 : Zone)) Components( (c1 : CompositeEvent Loader_Arrival(v1, z1, z2)) (c2 : CompositeEvent Transporter_Arrival(v2, z1, z3)) (c3 : CompositeState Worker_Manipulating_Container(p1, v3, v2, z3, z4))) Constraints( (v1->SubType = LOADER) (v2->SubType = TRANSPORTER) (z1->Name = ERA) (z2->Name = RF_DoorC_Access) (z3->Name = LOADER_BackZone) (z4->Name = Behind_RF_DoorC_Access) (c1 before c2) (c2 before c3))) WP4 - Scene Understanding

18.  Results on the Sequence S28 : “Unloading Detailed Operation”

19.  Deliverables Delivery of 2 software : Basic Event Interpretation delivered on Dec. 04 (first) and Jan. 06 (final) Understanding Prototype delivered on Dec. 04 (first) and Jan. 06 (final) Delivery of 2 documents : D4.1-4.2 Scene Understanding First Results on October 04 D4.3-4.4 Scene Understanding for Interaction and Complex Activity Recognition on June 05 WP4 - Scene Understanding

20.  Aircraft Arrival Preparation (involving the GPU) recognised on 10 Sequences

21. WP4 - Scene Understanding  Tow Tractor Arrival recognised on 2 Sequences

22. WP4 - Scene Understanding  Tanker Arrival recognised on 7 Sequences