Algirdas Beinaravičius Gediminas Mazrimas Salman Mosslem

 Introduction  Background subtraction techniques  Image segmentation ◦ Color spaces ◦ Clustering  Blobs  Body part recognition  Problems and conclusion

 Background subtraction/Foreground extraction  Color spaces and K-Means clustering  Blob-level introduction  Body part recognition

 What is background subtraction?  Background subtraction models: ◦ Gaussian model ◦ “Codebook” model

 Learning the model  Gaussian parameters estimation  Thresholds - Foreground/Background determination

Original image Background subtraction using Gaussian model Background subtraction using Codebook model

 Color spaces ◦ RGB ◦ HSI ◦ I3 (Ohta) ◦ YCC (Luma Chroma)  Clustering ◦ K-Means ◦ Markov Random Field

What does image segmentation? Why we needed different color spaces? What was clustering for?

 RGB (Red Green Blue) ◦ Classical color space ◦ 3 color channels (0-255)  In this project: ◦ Used in background subtraction

 HSI (Hue Saturation Intensity/Lightness) ◦ Similar to HSV (Hue Saturation Value) ◦ 3 color channels:  Hue – color itself  Saturation – color pureness  Intensity – color brightness ◦ Converted from normalized RGB values ◦ Intensity significance minimized  In this project: ◦ Used in clustering ◦ Blob formation ◦ Body part recognition

 Image data (pixels) classification to distinct partitions (labeling problem)  Color space importance in clustering

 Clustering without any prior knowledge  Working only with foreground image  Totally K clusters  Classification based on cluster centroid and pixel value comparison ◦ Euclidean distance: ◦ Mahalanobis distance:

Euclidean distanceMahalanobis distance

RGBHSI

 Probabilistic graphical model using prior knowledge  Usage: ◦ Pixel-level ◦ Blob level  Concepts from MRF: ◦ Neighborhood system ◦ Cliques

Neighborhood system Cliques

 Blob parameters  Blob formation  Blob fusion conditions  Blob fusion

 Higher level of abstraction ◦ Ability to identify body parts ◦ Faster processing

 Label.  Set of area pixels.  Centroid.  Mean color value.  Set of pixels, forming convex hull.  Set of neighboring blobs.  Skin flag.

 Input: K-means image/matrix.  Output: Set of blobs

 Particularly important in human body part recognition.  Can not be fused.  Technique to identify skin blobs: ◦ Euclidean distance

 Conditions: ◦ Blobs have to be neighbors ◦ Blobs have to share a large border ratio ◦ Blobs have to be of similar color ◦ Small blobs are fused to their largest neighbor  Neither of these conditions apply to skin blobs

 Associate blobs to body parts

 Skin blobs play the key role: ◦ Head and Upper body:  Torso identification  Face and hands identification ◦ Lower body:  Legs and feet identification

 Computational time  Background subtraction quality  Subject clothing  Subject position  Number of clusters in K-Means algorithm  Skin blobs

 Main tasks completed  Improvements are required for better results  Possible future work: ◦ Multiple people tracking ◦ Detailed body part recognition

?