Abandoned Object Detection for Indoor Public Surveillance Video Dept. of Computer Science National Tsing Hua University

Outline  Introduction  Common Surveillance Scenarios and Schemes Scenario of Few Pedestrians Scenario of Normal Case Scenario of Rush Hours  Proposed Abandoned Object Detection Scheme  Experimental Results  Conclusions and Future Works

Applications of Video Surveillance Systems  Security Surveillance of housing, public area Detecting or tracking suspicious objects  Behavior analysis Segmentation of the human body Classify the behavior of the human

Scenario Types Few Pedestrians (Lib) Normal Case (DingXi Station) Rush Hours (Taipei Main Station) Object Presence Frequently Object Presence Occasionally Object Presence Rarely

Scenarios of Environments Scenarios Types Object PresenceObject Detection Few Pedestrians Frequent Background Subtraction Normal Case Somewhat Frequently Simple Motion Filter Rush HoursRare Advanced Motion Filter

Scenario of Few Pedestrians – Background Subtraction The reference backgroundCurrent frame

Scenarios of Environments Scenarios Types Object PresenceObject Detection Few Pedestrians Frequent Background Subtraction Normal Case Somewhat Frequently Simple Motion Filter Rush HoursRare Advanced Motion Filter

Scenario of Normal Case- Background Subtraction The reference backgroundCurrent frame

Scenario of Normal Case- Most Frequent Intensity X Frame Counter Pixel Intensity Background or Stationary Objects Most Frequent Intensity !!

Scenario of Normal Case- Most Frequent Intensity The reference background Most Frequent Intensity The Most Frequent Intensity Picture

Scenarios of Environments Scenarios Types Object PresenceObject Detection Few Pedestrians Frequent Background Subtraction Normal Case Somewhat Frequently Simple Motion Filter Rush Hours RareAdvanced Motion Filter

Proposed Abandoned Object Detection Scheme for Scenario of Rush Hours  Pixel-based MoG  Advanced Motion Filter for Scenario of Rush Hours Using Vertical Scan Line Eliminate the Sparse Background Clutter Extracting the Complete Shape of an Abandoned Object Tracing Through Vertical Scan Lines Controllable System Alarm Response Time Grouping Abandoned Pixels to Objects

A Multi-model Background Modeling Algorithm － Mixture of Gaussian (MoG) 1 frame # weight 0 x Background distribution

Observations from vertical scan line h1h1 h2h2

Observations from Vertical Scan Line h1h1 h2h2

Proposed Motion Filter using Vertical Scan Line

Proposed Motion Filter -Eliminate the Sparse Background Clutter The referenced backgroundCurrent frame

Proposed Motion Filter -Eliminate the Sparse Background Clutter

Proposed Motion Filter -Extracting the Complete Shape of the Abandoned Object The referenced backgroundCurrent frame

Proposed Motion Filter - Extracting the Complete Shape of the Abandoned Object First foreground pointComplete Shape

Proposed Motion Filter -Tracing Through Vertical Scan Lines x Stop at first foreground section Tracing through the next foreground section Current frame

Proposed Motion Filter -Controllable System Alarm Response Time  Different reasonable response time for different applications  Avoid to issue the alarm for temporally still pedestrians

Proposed Motion Filter - Grouping Abandoned Pixels to Objects Background Pixel Abandoned Object Pixel Constraint: Object size ≥ 4 pixels One Alarm

Experimental Results -Test Sequences and Parameters Sequence Name Total Frames The Amount of Pedestrians Abandoned Object is Shot First Taipei Station Metro 1200Rush HoursIn the 99 th Frame DingXi Metro1000Normal CaseIn the 1 st Frame NTHU Library1000FewIn the 1 st Frame

Experimental Results -Evaluating Parameters  Application-depended Thresholds Eliminate the Sparse Background Clutter  Te Size of an Abandoned Object  Ts Controllable System Alarm Responding Time  Tr  Performance Evaluation Response Time (<25s) Alarms for Abandoned Objects / Total Alarms ↑

Eliminate the Sparse Background Clutter (Taipei Station) Response TimeAlarms Count <25s 2/( )=1/15 5/(5+3+7)=1/3

Size of an Abandoned Object (Taipei Station) Response Time Alarms Count 7/(7+8+15)=7/30 5/(5+3+7)=1/3 <25s

Controllable System Alarm Responding Time (Lib) Response TimeAlarms Count <25s

Experimental Results-Comparisons with Related Works [11] [12] Demo

Experimental Results-Time Complexity Analysis 47.7

Conclusions & Future Works  An Abandoned Object Detection Scheme to Deal with all of the Scenarios of Few Pedestrians, Normal case and Rush hours  Define different method for new scenarios  Object Detection Scheme for adaptive environment (Light changes, outdoor)  Define new interested events