1. Loris Bazzani Marco Cristani
"Person re-identification: a recent issue for the videosurveillance community and a technique for approaching it
Loris Bazzani
Marco Cristani ?
Modena, 17 maggio 2011

2. Before we start… [opt.] Check out our CVPR 2010 paper:
Download code and datasets for the exercises (iLIDS, VIPeR, CAVIAR):
[opt.] Check out our CVPR 2010 paper:
[opt.] Check out the website:

3. Outline of the lesson Person Re-identification (few minutes…)
A possible solution:
SDALF, Symmetry-Driven Accumulation of Local Features (20 minutes…)
Matlab exercises (~1 hour)

4. Person Re-identification
Goal: Recognizing an individual in different timings
Same camera
T = 145
T = 222
T = 1
T = 23
Different
overlapping
cameras
Different
non overlapping
cameras

5. Person Re-identification
Issues:
Many, you will see them in the exercises…

6. STEP 0-1: Axes of Symmetry and Asymmetry STEP 2: Chromatic Feature
A possible solution: SDALF, Symmetry-Driven Accumulation of Local Features
Overview of the proposed descriptor:
STEP 3:
Per-region Feature
For each body part
STEP 0-1: Axes of Symmetry and Asymmetry
Descriptor
Accumulation t
STEP 2: Chromatic Feature
STEP 4: Texture Feature

7. Step 0 – Isolating the silhouette
We need to focus on the body of the person
We perform background subtraction or
We apply a statistical model of the human appearance [Jojic et al. 2009]

8. Step 1 – Axes of (A)simmetry
We draw axes of symmetry and asymmetry
Features near the axes of symmetry are more reliable

9. Step 1 – Axes of (A)simmetry
Chromatic operator
Spatial covering operator
BG subtraction using STEL generative model

10. Step 2 - Chromatic feature
For each part (no head), we compute a weighted color histograms
HSV color space
“Gaussian Kernel” for each body part:
Low-weight to the background clutter
Robust to illumination changes, partial occlusions

11. Step 3 - Per-region feature
Maximally Stable Color Region (MSCR) detector
Detect “stable blobs”
Look at successive steps of an agglomerative clustering of image pixels
Covariant to affine transformations
Clustering of the detected blobs to reduce the computational cost of the matching

12. Step 4 - Texture feature
Recurrent High-Structured Patches (RHSP) detector

13. Accumulation of features
Descriptor:
Single-shot: SDALF with only one image (no accumulation)
Multi-shot: SDALF with multiple images

14. Testing the person re-identification methods
A (probe)
B (gallery)
Pick a selection
Rank
We have two sets, the probe set and the gallery set. We have a set of individuals, each of them appears one or more time in both the sets. From the probe set we randomly pick a subset of test instances of individuals. For each individual we have to sort the gallery set, trying to put in the first positions the images that portray the the test instance.

15. Matching algorithm Distance between two signatures WHERE ,
Bhattacharyya distance between HSV histograms
Distances between
blob descriptors

16. How to evaluate
Cumulative Matching Characteristic (CMC) curve, the expectation of finding the correct match in the top n matches

17. For this, you can use the MATLAB file:
Ex. 1: The Datasets
Exercise 1: take a look at the datasets and try to find out the challenges of the re-id problem
For this, you can use the MATLAB file:
DEMO0_dataset.m
The first exercise is kind of simple. When we are dealing with a problem, the first thing that a researcher should usually do is to retrieve the available datasets,
Take a look closely at the data, find out the challenging problems and then start to think of a solution.
So, the first thing I ask you to do is to have a look at the datasets. Let’s open the first demo script DEMO0_dataset.m
[EXPLAIN briefly the code]
[RUN the code and discuss about the challenges]
[TIME = 7 min.]

18. Ex. 2: SDALF
Exercise 2: qualitative analysis of the SDALF descriptor: display the weighted HSV hist., MSCR, RHSP
For this, you can use the MATLAB file:
DEMO1_SDALFextraction.m
Now that you know the re-identification problems, let’s have a look at the SDALF descriptor. In particular, we are going to plot
the symmetry and asymmetry axes and each of the 3 feature we engineered for the task.
For doing this, we have to open the script, DEMO1_SDALFextraction.m .
First of all, run the code and have a look at the out.
[DISCUSS each deature]
Now, we analyze in detail the code.
[EXPLAIN the code]
[TIME = 20 min.]

19. Ex. 3: Cross-validation
Exercise 3: try the cross-validation code evaluating CMC, SRR and nAUC
Compare SvsS and MvsM case
Vary the number of images for the MvsM case
For this, you can use the MATLAB file:
DEMO2_crossvalid.m
[set MAXCLUSTER=1 (SvsS) or >1 (MvsM)]
After you found out a solution of a certain problem, you need to test if the solution is a good one.
So, what you need to do is to perform cross-validation and compute statistics so that you can compare your solution with the other state-of-the-art methods.
In the re-id problem, 4 statistics are very useful.
[EXPLAIN briefly CMC, CMC(1), nAUC, SRR]
Cumulative Matching Characteristic (CMC) curve: represents the expectation of finding the correct match in the top $n$ matches.
Synthetic Recognition Rate (SRR) curve: represents the probability that any of the $m$ best matches is correct.
Normalized Area Under the Curve (nAUC): gives an overall score of your method
[EXPLAIN the matching and cross-validation function]
Our method is able to deal with single images and multiple images (available when tracking).
Set MAXCLUSTER=1 (SvsS) or >1 (MvsM).
[RUN the code varying the number of images and with different dataset]
[TIME = 15 min.]

20. For this, you can use the MATLAB file:
Ex 4: Matching
Exercise 4: evaluate qualitatively the output of the matching procedure
For this, you can use the MATLAB file:
DEMO2_crossvalid.m
[set plotMatch=1]
And DEMO3_crossvalid.m
The last thing I ask you to do is to have a look at the matching results sorted by the distance.
In this way, you can see the probe individual and the gallery individuals that are more similar to the probe. The similarity is defined by our matching procedure.
[RUN the code with plotMatch = 1, MAXCLUSTER = 1, dataset = VIPeR]
[TIME = 13 min.]

21. Take-home Message Why this lesson? Questions?
To be able to use our system on new datasets
Compare your personal methods with SDALF
Questions?