Olshausen’s Demo

1.The Training set ?  Natural Images (Olhausen’s database)  How much do we learn ?  face database and car database 2.The Sparseness term ?  Prior steepness  Sparseness function 3.Natural encoding or hacking?  Whitening the data  Non-stationary hypothesis How Important Is:

Training with Natural Images  Training: 10 images (512x512)  10,000 presentations  Batch size: 100  Basis Function: 16x16

Face Database  Training: 100 images (100x100)  10,000 presentations  Batch size: 100  Basis Function: 16x16

Encoding Properties Original 50 basis 10 basis 30 basis40 basis 20 basis

Car Database  Training: 200 images (128x128)  10,000 presentations  Batch size: 100  Basis Function: 16x16

Comments 1.The algorithm seems to capture the structure of the images (cf car):  Learning is experience-dependent 2.Basis functions found in good agreement with properties of neurons in visual cortex:  Receptive fields are localized, oriented, bandpass

1.The Training set ?  Background, face and car databases 2.The Sparseness term ?  Prior steepness  Sparseness function 3.Natural encoding or hacking?  Whitening the data  Non-stationary hypothesis How Important Is:

Prior Steepness Steepness 2.2 Steepness 10 Steepness 5 Steepness 100

Prior Steepness Steepness 2.2Steepness 1.5 Steepness 0.2

Sparseness Function

S(x)=|x| S(x)=log(1+x^2)

Sparseness Function batch of 100 samples: Mean Error: abs=.471 / log =.504

1.The Training set ?  Background, face and car databases 2.The Sparseness term ?  Prior steepness  Sparseness function 3.Natural encoding or hacking?  Whitening the data  Non-stationary hypothesis How Important Is:

Whitening the Data Data are filtered with whitening/low-pass filter: How important is it for the convergence of the algorithm? The question is to know whether it is just a speed-up or is it required for convergence?

Non-preprocessed Car Images  Training: 100 images (100x100)  30,000 presentations  Batch size: 100  Basis Function: 16x16

Non-stationary Hypothesis: Encoding the Full Face After few iterations…

Code + images available: