Loris Bazzani Marco Cristani "Person re-identification: a recent issue for the videosurveillance community and a technique for approaching it Loris Bazzani Marco Cristani ? Modena, 17 maggio 2011

Before we start… [opt.] Check out our CVPR 2010 paper: Download code and datasets for the exercises (iLIDS, VIPeR, CAVIAR): http://profs.scienze.univr.it/~bazzani/TMP/S4_SDALF_reid.zip [opt.] Check out our CVPR 2010 paper: http://www.lorisbazzani.info/papers/proceedings/FarenzenaetalCVPR10.pdf [opt.] Check out the website: http://www.lorisbazzani.info/code-datasets/sdalf-descriptor/

Outline of the lesson Person Re-identification (few minutes…) A possible solution: SDALF, Symmetry-Driven Accumulation of Local Features (20 minutes…) Matlab exercises (~1 hour)

Person Re-identification Goal: Recognizing an individual in different timings Same camera T = 145 T = 222 T = 1 T = 23 Different overlapping cameras Different non overlapping cameras

Person Re-identification Issues: Many, you will see them in the exercises…

STEP 0-1: Axes of Symmetry and Asymmetry STEP 2: Chromatic Feature A possible solution: SDALF, Symmetry-Driven Accumulation of Local Features Overview of the proposed descriptor: STEP 3: Per-region Feature For each body part STEP 0-1: Axes of Symmetry and Asymmetry Descriptor Accumulation t STEP 2: Chromatic Feature STEP 4: Texture Feature

Step 0 – Isolating the silhouette We need to focus on the body of the person We perform background subtraction or We apply a statistical model of the human appearance [Jojic et al. 2009]

Step 1 – Axes of (A)simmetry We draw axes of symmetry and asymmetry Features near the axes of symmetry are more reliable

Step 1 – Axes of (A)simmetry Chromatic operator Spatial covering operator BG subtraction using STEL generative model

Step 2 - Chromatic feature For each part (no head), we compute a weighted color histograms HSV color space “Gaussian Kernel” for each body part: Low-weight to the background clutter Robust to illumination changes, partial occlusions

Step 3 - Per-region feature Maximally Stable Color Region (MSCR) detector Detect “stable blobs” Look at successive steps of an agglomerative clustering of image pixels Covariant to affine transformations Clustering of the detected blobs to reduce the computational cost of the matching

Step 4 - Texture feature Recurrent High-Structured Patches (RHSP) detector

Accumulation of features Descriptor: Single-shot: SDALF with only one image (no accumulation) Multi-shot: SDALF with multiple images

Testing the person re-identification methods A (probe) B (gallery) Pick a selection Rank We have two sets, the probe set and the gallery set. We have a set of individuals, each of them appears one or more time in both the sets. From the probe set we randomly pick a subset of test instances of individuals. For each individual we have to sort the gallery set, trying to put in the first positions the images that portray the the test instance.

Matching algorithm Distance between two signatures WHERE , Bhattacharyya distance between HSV histograms Distances between blob descriptors

How to evaluate Cumulative Matching Characteristic (CMC) curve, the expectation of finding the correct match in the top n matches

For this, you can use the MATLAB file: Ex. 1: The Datasets Exercise 1: take a look at the datasets and try to find out the challenges of the re-id problem For this, you can use the MATLAB file: DEMO0_dataset.m The first exercise is kind of simple. When we are dealing with a problem, the first thing that a researcher should usually do is to retrieve the available datasets, Take a look closely at the data, find out the challenging problems and then start to think of a solution. So, the first thing I ask you to do is to have a look at the datasets. Let’s open the first demo script DEMO0_dataset.m [EXPLAIN briefly the code] [RUN the code and discuss about the challenges] [TIME = 7 min.]

Ex. 2: SDALF Exercise 2: qualitative analysis of the SDALF descriptor: display the weighted HSV hist., MSCR, RHSP For this, you can use the MATLAB file: DEMO1_SDALFextraction.m Now that you know the re-identification problems, let’s have a look at the SDALF descriptor. In particular, we are going to plot the symmetry and asymmetry axes and each of the 3 feature we engineered for the task. For doing this, we have to open the script, DEMO1_SDALFextraction.m . First of all, run the code and have a look at the out. [DISCUSS each deature] Now, we analyze in detail the code. [EXPLAIN the code] [TIME = 20 min.]

Ex. 3: Cross-validation Exercise 3: try the cross-validation code evaluating CMC, SRR and nAUC Compare SvsS and MvsM case Vary the number of images for the MvsM case For this, you can use the MATLAB file: DEMO2_crossvalid.m [set MAXCLUSTER=1 (SvsS) or >1 (MvsM)] After you found out a solution of a certain problem, you need to test if the solution is a good one. So, what you need to do is to perform cross-validation and compute statistics so that you can compare your solution with the other state-of-the-art methods. In the re-id problem, 4 statistics are very useful. [EXPLAIN briefly CMC, CMC(1), nAUC, SRR] Cumulative Matching Characteristic (CMC) curve: represents the expectation of finding the correct match in the top $n$ matches. Synthetic Recognition Rate (SRR) curve: represents the probability that any of the $m$ best matches is correct. Normalized Area Under the Curve (nAUC): gives an overall score of your method [EXPLAIN the matching and cross-validation function] Our method is able to deal with single images and multiple images (available when tracking). Set MAXCLUSTER=1 (SvsS) or >1 (MvsM). [RUN the code varying the number of images and with different dataset] [TIME = 15 min.]

For this, you can use the MATLAB file: Ex 4: Matching Exercise 4: evaluate qualitatively the output of the matching procedure For this, you can use the MATLAB file: DEMO2_crossvalid.m [set plotMatch=1] And DEMO3_crossvalid.m The last thing I ask you to do is to have a look at the matching results sorted by the distance. In this way, you can see the probe individual and the gallery individuals that are more similar to the probe. The similarity is defined by our matching procedure. [RUN the code with plotMatch = 1, MAXCLUSTER = 1, dataset = VIPeR] [TIME = 13 min.]

Take-home Message Why this lesson? Questions? To be able to use our system on new datasets Compare your personal methods with SDALF Questions?