1. D31 Entity Recognition Results with Auto- associative Memories Nicolas Gourier INRIA PRIMA Team GRAVIR Laboratory CAVIAR Project

2. Entity recognition Can be performed either by local or global approaches Local approaches Use information contained in the neighboorhood of pixels Global approaches Use the entire appearance of the image

3. Why a global approach ? No landmarks have to be detected No model has to be constructed Can handle Low resolution Partial occlusions => Only the object has to be detected

4. Existing Global Approaches PCA, KDA,… [Pentland91] Sensitive to alignment Number of dimensions ? Neural networks Number of cells in the hidden layer ? Recovery of prototypes of image classes ? => Auto-associative Memories [Abdi94]

5. Plan of the talk 1) Our approach 2) Results 3) Comparison with other techniques

6. 1. Normalized object imagette Grey scale face imagette normalized in size and slant: 25x25 pixels => Computation time reduction => Size and slant robustness

7. 1. Auto-associative memories Linear auto- associative memory Input patterns associated with themselves Connection between input units Portion of an input =>Complete pattern X’ = W.X X : Source image X’ : Output image W : Weights

8. 1. Hebbian learning rule W = X k.X k T Faces not well discriminated [Valentin94]

9. 1. Widrow-Hoff learning rule (1) Learned images are reconstructed Other images are degraded [Valentin94]

10. 1. Widrow-Hoff learning rule (2) Creation of prototypes Eigenvalues egalization [Abdi & Valentin94] We adapt Widrow-Hoff learning to entity recognition

11. 1. Entity Recognition Compare the input image to all responses => Score between 0 and 1 Winner-takes-all process ->½ videos for training, ½ videos for test

12. 1. Training and Test Training -> Test V

13. 2. Experiments 3 Experiments : 1) Classes 0 / 1 person Without training a 0 person class 2) Classes 0 / 1 person 3) Classes 0 / 1+ person

14. 2. Result of the first experiment (1)

15. 2. Result of the first experiment (2) Not sufficient for reliable classification 0 person class imagettes have non- uniform variations in appearance => Learn a 0 person class from random images of the background

16. 2. Result of the second experiment

17. 2. Result of the third experiment

18. 2. Recall and precision Experiment Classes 1 1/0 2 1/0 3 1+/0 1st class recall -99 % 2 nd class recall -68 %70 % 1st class precision -95 %93 % 2 nd class precision -93 %90 %

19. 3. Discussion Training the 0 person class improves discrimination Some 0 person class images are misclassified :

20. 3. Advantages Varying the size of the imagette do not have much influence -> 25x25 pixels Normalization + Classification is done at video-rate Prototypes can be saved and reused Can be adapted to entity recognition

21. Conclusion + Invariance to scale, slant and alignment + Not disrupted by local changes - Needs to train a non-person class Adapted to the project Low resolution Changes of viewpoint Fast