1. REAL-TIME DETECTOR FOR UNUSUAL BEHAVIOR
Showcase

2. Highlights Events Motion and shape based
usual  non-usual
Motion and shape based
Statistically relevant  irrelevant
Alert generation on unusual event
Storing events in database

3. Platform Visualisation: Web browser Software platform:
SZTAKI will provide a communication module that will call the module functions provided by the partners.
Software platform:
C++, OpenCv, IPP
Web technologi
Hardware platform:
Pc, laptop (x86 like)

4. Partners ACV Tracking, pedestrian detection BILKENT
UPC
SZTAKI
Tracking, pedestrian detection
Multimodal human actions, HMM
2D Body actions, motion fields
Unusual event detection, annotation process, statistical analysis, shadow removing
(formerly ARC)
BILKENT

5. Distribution of work Sztaki UPC ACV BILKENT Moving Cam. Static Cam.
Unusual
event
mosaicing
ACV
Foreground
Detect.
Region alert
Motion
features
Pedestrian
detection
Body model
shilouettes
sound
Tracking
Periodicity
HMM class.
classification
BILKENT

6. Contribution of ACV
Non-parametric clustering of moving objects in difference images
Occlusion handling for interacting targets
Kernel-based tracking using motion features for multiple targets
Video data set and evaluation of the motion detection and tracking performance (Benchmark competition)

7. Details on the algorithmic modules
Human detection by clustering and model-based verification
Kernel-based human tracking using motion information
Occlusion handling
Tracking evaluation (comparison to manual ground truth)
VIDEO
VIDEO
VIDEO

8. Evaluation video datasets (street scenarios)
Sequence Street_01.avi: 720x576 pixels, 8628 frames (tracking ground truth available for 1040 frames)
Sequence Street_02.avi: 720x576 pixels, 763 frames (tracking ground truth available for 763 frames)

9. Contribution of Bilkent
Motion and silhoutte based person detector
detect motion and moving blocks and observe periodicity in bounding boxes of moving blocks in video.
use silhouttes to classify moving objects in video
combine the results of periodicity and silhoutte based detectorIn this way,
Determine the number of people in the scene.
HMM classification (fight, fall or simply walk)
Record the sounds and classify the sounds to (car sounds, walking person, and loud screems)
Combine the results of 3 and 4 to reach a final decision.

10. Human Recognition in Video
BILKENT
Utilizes objects’ silhouettes for different poses
Silhouettes are extracted using contour tracing
Compare silhouette signature functions using wavelet energy signatures

11. Falling Person Detection using Motion Clues (visual)
BILKENT
Falling Person Detection using Motion Clues (visual)
Observation: Walking and falling person T1 T2
Quasi-periodic behaviour in time for both the aspect ratio and the corresponding wavelet signal is observed for walking case. No such periodicity for the falling case.

12. Contribution of UPC
Foreground detection and automatic features extraction
motion history descriptors
simple body model
Apply the integrated system to different environments
crowded scenes in automatic stairs

13. Motion Analysis
Motion History and Motion Energy descriptors introduced by Bobick et al. in 2D and Canton et al. in 3D allows robust motion analysis
MEV
MHV

14. Model Based Analysis
Analyzing input data by means of a Human Body Model, allows retrieving information about limbs positions

15. Silhouette analysis for detection of body extremities
Scene capture
User segmentation
CoG computation
Creation of the geodesic distance map
Contour tracking
Creation of the distance/silhouette border position function
H-maxima operation on the function
Local maxima extraction
Morphological skeleton computation and crucial point labeling
Pixel position
Geodesic Distance

16. Contribution of SZTAKI
Foreground detection
View region surveillance
Alert event generation
Event History
Search & display

17. Contribution of SZTAKI
Foreground detection in moving camera

18. Contribution of SZTAKI
Mosaicing

19. Contribution of SZTAKI
Usual – non usual motion
Actual motion masked with usual motion
Pixel-wise motion estimation
black: right, white: left
Input
Motion statistics

20. Contribution of SZTAKI
SG based unusuality detector on
motion fields
motion tracks
Software Environment
Interface module to user dll/lib/module
Separates and bridge modules
Server
Serves image/video streams
Transcodes images
Forward requests to modules
DB server
Metadata store & search
Webserver
Generate html pages with links to Server (later)
Client dynamic web
Javascript/flash based graphics display
Mozilla native mjpeg stream + SVG

21. Contribution of SZTAKI - architecture
Internet
User modules DB
metadata
Web
server
SERVER
Matlab
tcp/ip
Comm.
Interface
Html
Module Register
Web Page
tcp/ip
DLL/LIB
Comm.
Interface
Controller
json
tcp/ip
mjpg
C++
Comm.
Interface
json
Streams
Data Source
Comm.
Interface