1. Olshausen’s Demo

2. 1.The Training set ?  Natural Images (Olhausen’s database)  How much do we learn ?  face database and car database 2.The Sparseness term ?  Prior steepness  Sparseness function 3.Natural encoding or hacking?  Whitening the data  Non-stationary hypothesis How Important Is:

3. Training with Natural Images  Training: 10 images (512x512)  10,000 presentations  Batch size: 100  Basis Function: 16x16

4. Face Database  Training: 100 images (100x100)  10,000 presentations  Batch size: 100  Basis Function: 16x16

5. Encoding Properties Original 50 basis 10 basis 30 basis40 basis 20 basis

6. Car Database  Training: 200 images (128x128)  10,000 presentations  Batch size: 100  Basis Function: 16x16

7. Comments 1.The algorithm seems to capture the structure of the images (cf car):  Learning is experience-dependent 2.Basis functions found in good agreement with properties of neurons in visual cortex:  Receptive fields are localized, oriented, bandpass

8. 1.The Training set ?  Background, face and car databases 2.The Sparseness term ?  Prior steepness  Sparseness function 3.Natural encoding or hacking?  Whitening the data  Non-stationary hypothesis How Important Is:

9. Prior Steepness Steepness 2.2 Steepness 10 Steepness 5 Steepness 100

10. Prior Steepness Steepness 2.2Steepness 1.5 Steepness 0.2

11. Sparseness Function

13. S(x)=|x| S(x)=log(1+x^2)

14. Sparseness Function batch of 100 samples: Mean Error: abs=.471 / log =.504

15. 1.The Training set ?  Background, face and car databases 2.The Sparseness term ?  Prior steepness  Sparseness function 3.Natural encoding or hacking?  Whitening the data  Non-stationary hypothesis How Important Is:

16. Whitening the Data Data are filtered with whitening/low-pass filter: How important is it for the convergence of the algorithm? The question is to know whether it is just a speed-up or is it required for convergence?

17. Non-preprocessed Car Images  Training: 100 images (100x100)  30,000 presentations  Batch size: 100  Basis Function: 16x16

18. Non-stationary Hypothesis: Encoding the Full Face After few iterations…

19. Code + images available: http://web.mit.edu/serre/www/ http://web.mit.edu/serre/www/