Scene Understanding Francois BREMOND perception, multi-sensor fusion, spatio-temporal reasoning and activity recognition. Francois BREMOND Orion project-team, INRIA Sophia Antipolis, FRANCE Francois.Bremond@sophia.inria.fr http://www-sop.inria.fr/orion/orion-eng.html Key words: Artificial intelligence, knowledge-based systems, cognitive vision, human behavior representation, scenario recognition
A scene understanding platform: real-time interpretation of videos from pixels to events Segmentation Classification Tracking Scenario Recognition Alarms access to forbidden area 3D scene model Scenario models A priori Knowledge
Video Understanding Applications metro station monitoring airport monitoring train monitoring building control access
ETISEO: performance evaluation depends on sensors (position, type) and scenes conditions strong perspective shadows tiny objects lighting conditions close view clutter
ETISEO: selection of 6 sites for ETISEO video acquisition.
Classification into more than 8 classes (e. g Classification into more than 8 classes (e.g. Person, Groupe, Train) based on 2D and 3D descriptors (position, 3D ratio height/width, …)
Vandalism in metro station
Fighting in metro station
Posture Recognition : silhouette comparison Real world Virtual world Generated silhouettes
Multi sensors information fusion: Lateral Shape Recognition
Bank agency monitoring
Parked aircraft monitoring in Toulouse: Toulouse-Blagnac Airport France Apron Video Sensors
Parked aircraft monitoring in Toulouse: “Unloading Front Operation”
Scenario recognition: “Disturbing people in a train”
HealthCare Monitoring : Détecteur de présence Contacts de portes Caméra Capteur de chute Capteur de pouls Microphones Pèse personne Surveillance respiratoire
HealthCare Monitoring
HealthCare Monitoring: Object detection and tracking (a) Segmentation (b) Classification (c) Tracking
HealthCare Monitoring: Object detection and tracking (a) Segmentation (b) Classification (c) Tracking
HealthCare Monitoring: Object detection and tracking (a) Segmentation (b) Classification (c) Tracking
HealthCare Monitoring: Object detection and tracking (a) Segmentation (b) Classification (c) Tracking
HealthCare Monitoring: Object detection and tracking (a) Segmentation (b) Classification (c) Tracking
HealthCare Monitoring: Object detection and tracking (a) Segmentation (b) Classification (c) Tracking
Trajectory clustering in Torino subway (45min), 2052 trajectories
People Classification (based on 3D parallelepiped) into 3 people classes : 1Person, 2Persons, 3Persons, Unknown
Posture Recognition : Set of Specific Postures Standing Sitting Bending Lying Hierarchical representation of postures