Object Based Processing for Privacy Protected Surveillance Karl Martin Kostas N. Plataniotis University of Toronto Dept. of Electrical and Computer Engineering February 6, 2008

2 Confidential - Not to be reproduced without permissionFebruary 6, 2008 Overview Introduction Object Based Image/Video Processing Object Based Coding/Compression Privacy Protection via Object Processing Smart Surveillance Technologies

3 Digital Signal Processing Image and Video Processing Digital Camera Processing cDNA Micro-array Imaging Visual Object Compression Universal Multimedia Access Multimedia Security Authentication & Fingerprinting Image and Video Encryption Visual Secret Sharing Secure Streaming Watermarking Biometrics & Surveillance Face Recognition Gait Recognition Visual Surveillance Professor: Kostas Plataniotis

4 Confidential - Not to be reproduced without permissionFebruary 6, 2008 Object Based Image/Video Processing Semantic processing of frames/video sequences Objects: –Face, body, suitcase, vehicle, etc. –Defined by: Shape + Texture Enables: –Privacy Protection –Event Detection –Etc.

5 Confidential - Not to be reproduced without permissionFebruary 6, 2008 Object Based Image/Video Processing Requires detection and segmentation –Rough box or accurate contour –Computer vision and pattern recognition techniques

6 Confidential - Not to be reproduced without permissionFebruary 6, 2008 Object Based Coding/Compression Traditional frame/video sequence coding –MPEG-1/2/4, H.263, Motion JPEG, Motion JPEG2000 –Content unaware Object based coding –MPEG-4, ST-SPIHT Wavelet (UofT, Martin and Plataniotis 2006) –Content aware – enabling technology for object based processing Video Compression Engine Compressed Data Stream Frame Object Compression Engine Compressed Shape and Texture Stream (Object 1) Compressed Shape and Texture Stream (Background) Compressed Shape and Texture Stream (Object 2)

7 Confidential - Not to be reproduced without permissionFebruary 6, 2008 Privacy Protection via Object Processing Object obscuration – irreversible Object Encryption - reversible Texture Removal (blackout) Texture Blur Decryption Key

8 Confidential - Not to be reproduced without permissionFebruary 6, 2008 Privacy Protection via Object Processing SecST-SPIHT Secure Object Coding (UofT, Martin and Plataniotis, 2007) –Secure texture OR texture+shape Encryption can be reversed with correct key –Real-time or offline release of key based on authority –Allows investigation of original content Original FrameSecured TextureSecured Texture+Shape

9 Confidential - Not to be reproduced without permissionFebruary 6, 2008 Privacy Protection via Object Processing Example System Topologies Object Segmentation SecST-SPIHT (secure coding) Recorder/ Storage Server Authorized Key Release Local Real-time Monitor Central Real-time Monitor Law Enforcement Physical Transfer Key Management

10 Confidential - Not to be reproduced without permissionFebruary 6, 2008 Privacy Protection via Object Processing Example System Topologies Object Segmentation Recorder/ Storage Server Authorized Key Release Local Real-time Monitor Central Real-time Monitor Law Enforcement Physical Transfer Key Management SecST-SPIHT (secure coding)

11 Confidential - Not to be reproduced without permissionFebruary 6, 2008 Smart Surveillance Technologies Digital vs. Analog DigitalAnalog Operating Central operationIsolated, site-by-site Managing Central managementDifficult to manage large number of sites Recording Flexible framerate/qualityFixed framerate/quality Storing Flexible based on storage, compression parameters – highly scalable Tapes requiring substantial storage space – potential video loss during tape change Searching Single query search across multiple storage sites – near instantaneous Manual, physical retrieval of tapes

12 Confidential - Not to be reproduced without permissionFebruary 6, 2008 Smart Surveillance Technologies General processing stages for smart processing Detected Behaviours –Violence –Suspected objects –Robbery –Vandalism –Overcrowding –Trespassing Event Detection Object Recognition Tracking Behavoural Analysis

13 Confidential - Not to be reproduced without permissionFebruary 6, 2008 Smart Surveillance Technologies Integrated Approach –Technical infrastructure –Organizational human-led procedures –Ethical framework (privacy vs security) –Behavioural studies Requirements for Implementation –Robust –Adaptable –Easy installation –Off-the-shelf hardware –Real-time operation

14 Confidential - Not to be reproduced without permissionFebruary 6, 2008 Smart Surveillance Technologies Example Commercial Solutions CompressionOther Features MC Electronics MPEG-4Audio, PTZ control, alarm inputs Visiowave Visiobox 3D WaveletPTZ control, motion detection 3 Baxall Analog and digital interfaces Teleeye III+ NF WaveletMotion detection, wireless Sony SNCV10P WaveletMultiple interfaces, embedded LINUX, built in security

15 Confidential - Not to be reproduced without permissionFebruary 6, 2008 END