University of Zagreb, Faculty of Electrical Engineering and Computing Automated Segmentation of Humans in Videos for Soft- and Non-Biometric De-identification Tomislav Hrkać University of Zagreb, Faculty of Electrical Engineering and Computing Unska 3, 10 000 Zagreb, Croatia

Context: de-identification Security cameras Privacy concerns De-identification Biometric Soft biometric Non-biometric Important first step: Person detection Input: image or video sequence Persons detection Identifier segmentation Output: de- identified content De-identifying transformation

Pedestrian segmentation Output of human detector – bounding boxes around humans More precise localisation of persons needed

GrabCut algorithm Semi-authomatic algorithm Minimal user input Easily combined with background subtraction to acheive full authomatic character

GrabCut algorithm In theory – very good results In reality, however... Three types of problems: “Cutting through” at the high contrast boundary Snapping to high contrast in the background Enclosing concavities

GrabCut– how does it work? Preliminary BG and FG specs -> GMM models Segmentation as energy minimization: E = U + V Minimization by graph cuts Iterative procedure: refining BG and FG models in each step “data term” “smoothnes term”

GrabCut – details Data term: Smoothnes term:

Our modifications to GrabCut Preliminary step: find out the rough object boundary based on background subtraction followed by morphological post-processing Modified GrabCut: Weighted modeling of FG and BG Modification to the data term U to take into account preliminary classification Modification to the smoothness term V to discourage boundaries far from the preliminary boundary

Modifications – details (1/3) Weighted modeling of BG and FG Originally each pixel initially classified as BG or FG contributes equally to BG/FG GMM models Instead, we propose to build GMMs by weighting the contribution of each pixel based on its distance from the preliminary boundary

Modifications – details (2/3) Modification of the data term Originally: Modified:

Modifications – details (3/3) Modification of the smoothness term Originally: Modified:

Results (1/3) Qualitative evaluation (Left –input image; middle – GrabCut; right- our method)

Results (2/3) Quantitative evaluation (CDnet2014 Pedestrian dataset)

Results (3/3) Quantitative evaluation (CDnet2014 Pedestrian dataset)

Conclusion We proposed fully authomatic video foreground segmentation, based on GrabCut, initialized by background subtraction mask the more relevant GMM models of foreground and background are built by weighing pixel contribution depending on their distance from the boundary of the object prior the data and smoothness term of the energy equation are modified in a way that discourages placing boundaries far from the object prior boundary. Evaluation on the CDNet 2014 Pedestrian Dataset: significant improvement of recall slight drop in precision much better F1 measure Further experimentation and investigation of possible improvements needed

Acknowledgments This work has been supported by the Croatian Science Foundation, within the project “De-identification Methods for Soft and Non-Biometric Identifiers” (DeMSI, UIP-11-2013-1544), and by the COST Action IC1206 “De-Identification for Privacy Protection in Multimedia Content”. This support is gratefully acknowledged