1. Respectful Cameras Jeremy Schiff EECS Department University of California, Berkeley Ken Goldberg, Marci Meingast, Deirdre Mulligan, Pam Samuelson IEOR, EECS, Law University of California, Berkeley http://www.cs.berkeley.edu/~jschiff/RespectfulCameras NSF Science and Technology Center, Team for Research in Ubiquitous Secure Technologies, NSF CCF-0424422, with additional support from Cisco, HP, IBM, Intel, Microsoft, Symmantec, Telecom Italia and United Technologies.

2. Background New class of Robotic Cameras since 9/11/2001 New class of Robotic Cameras since 9/11/2001 $20,000 -> Under $1,000 $20,000 -> Under $1,000 Static -> Pan, tilt, zoom (21x) Static -> Pan, tilt, zoom (21x) UK - 3 Million Outdoor Cameras UK - 3 Million Outdoor Cameras Now Deploying in Large US Cities Now Deploying in Large US Cities Zoom Example

3. Invasiveness

4. Objective

5. Static Marker Detection Adaboost Adaboost Training Phase Training Phase Input is data and label Input is data and label Classifying Phase Classifying Phase Data -> label Data -> label Linear function of weak classifiers Linear function of weak classifiers Example Example Construction Hat Color Construction Hat Color

6. Features Input from images Input from images Each pixel Each pixel red, green, blue (RGB) red, green, blue (RGB) Values 0 to 255 Values 0 to 255 Project into higher dimension Project into higher dimension Convert to 9 dimensions Convert to 9 dimensions RGB RGB HSV HSV Stable over changing lighting Stable over changing lighting LAB LAB Good for detecting specularities Good for detecting specularities 10 243 13 9 241 16 12 252 8 60 201 73 69 225 74 42 17 38 65 209 78 74 220 171 45 112 16

7. Classifiers Operates on each dimension Operates on each dimension Threshold value Threshold value Above good and below bad Above good and below bad Above bad and below good Above bad and below good Example Example

8. Connected Component Groups adjacent pixels Groups adjacent pixels Threshold Threshold Minimum Area Minimum Area Bounding Box Bounding Box Acceptable Ratio Between Dimensions Acceptable Ratio Between Dimensions

9. Marker Tracking Particle Filtering Particle Filtering Probabilistic Method for Tracking Probabilistic Method for Tracking Motivates Probabilistic AdaBoost Motivates Probabilistic AdaBoost

10. Particle filters Non-Parametric Non-Parametric Sample Based Method (Particles) Sample Based Method (Particles) Particle Density ~ Likelihood Particle Density ~ Likelihood Tracking requires three distributions Tracking requires three distributions Initialization Distribution Initialization Distribution Transition Model (Intruder Model) Transition Model (Intruder Model) Observation Model Observation Model Determines Determines

11. Observation Model p p 1-p 0.80.6 0.70.9 0.1 0.0 0.4 0.10.2 0.10.2 0.4 0.2 0.30.2 0.80.6 0.70.9 1.0 0.6 0.90.8 0.90.8 0.6 0.8 0.70.8 0.79375

12. Transition Model State State Position Position Bounding-box Width Bounding-box Width Bounding-box Height Bounding-box Height Orientation Orientation Speed Speed Add Gaussian Noise to width, height, orientation and speed Add Gaussian Noise to width, height, orientation and speed Euler Integration to determine new position Euler Integration to determine new position

13. Multiple Filters Single Filter Per Marker Single Filter Per Marker Define overlap Define overlap Add Filter when overlap of Static Image Cluster and all filters is below threshold Add Filter when overlap of Static Image Cluster and all filters is below threshold Delete Filter when prob. of best particle < 0.5 Delete Filter when prob. of best particle < 0.5 Delete Filter when 2 filters overlap > threshold Delete Filter when 2 filters overlap > threshold

14. Video – Nearby Hats

16. Video – Lighting

18. Video – Crossing

20. Video – Shirt

22. Future Work Other Features Other Features Edge Detection Edge Detection Feature Structure Feature Structure Generalize to Other Domains Generalize to Other Domains Other Obstruction Mechanisms Other Obstruction Mechanisms Encryption Encryption Full Body Full Body Multiple Cameras Multiple Cameras

23. Thank You Jeremy Schiff: jschiff@cs.berkeley.edu Jeremy Schiff: jschiff@cs.berkeley.edujschiff@cs.berkeley.edu URL: www.cs.berkeley.edu/~jschiff/RespectfulCameras URL: www.cs.berkeley.edu/~jschiff/RespectfulCameras www.cs.berkeley.edu/~jschiff/RespectfulCameras