1. Activity Recognition Journal Club “Neural Mechanisms for the Recognition of Biological Movements” Martin Giese, Tomaso Poggio (Nature Neuroscience Review, 2003)

2. Objective Recognition of complex movements and actions using a neurophysiologically plausible and quantitative model Biology has already generated a system that is robust and has high selectivity - let’s mimic it.

3. Biological Intuitions Separate dorsal and ventral streams http://upload.wikimedia.org/wikipedia/commons/thumb/c/c0/Gray722.png/300px-Gray722.png http://hebb.mit.edu/courses/8.515/lecture3/sld007.htm

4. Biological Intuitions/Assumptions Hierarchical architecture with increasing complexity along the hierarchy Mainly feedforward Activities and views are learned Interaction between two streams not necessary for some recognition

5. Biological Intuitions

6. Artificial Neural Networks Feedforward  r dv/dt = -v + F(W·u) Recurrent  r dv/dt = -v + F(W·u + M·u)

7. Form Pathway Object Recognition Riesenhuber, Poggio 2002

8. Form Pathway Simple Cells –Modeled by Gabor Filters Output via Linear threshold function Complex Cells –‘MAX’ function –Output via linear threshold function –Invariant bar detectors Giese, Poggio, AIM-2002-012, 2002.

9. Form Pathway View/Object Tuned Units –Radial Basis Function –u is the vector of the responses of the significant invariant bar detector –u 0 signifies the preferred input pattern –C is a diagonal matrix with the elements C u that are inversely proportional to the variance of the l -th component of u in the training set Giese, Poggio, AIM-2002-012, 2002.

10. Form Pathway Motion pattern neurons –Leaky integrator –  y dy/dt + y(t) =  f(u n (t)) –Most active when input follows synaptically encoded sequence

11. Motion Pathway Represents dorsal stream Similar hierarchy –Increasing complexity and invariance

12. Motion Pathway Lowest level –Optical flow, computed directly Second Level –First Class Translational flow (four tuned directions) –Second Class Motion edges (horizontal and vertical) MAX operator Third Level –“Snapshot Neurons” –Modelled by RBF’s Fourth Level –Motion Pattern Neurons

13. Motion Pathway

14. Model Parameters Giese, Poggio, AIM-2002-012, 2002. Simple Cells –8 orientations, 2 scales –  1 =10,  2 =7,  =0.35 Motion Pattern & Motion Snapshot Neurons –  s =150ms

15. Results

18. Limitations Does not address ‘attentional’ effects No model for computing optical flow Does not address interaction between form and motion streams Form stream does not recognize point- light motion as per experimental data

20. Recap

22. Example Pattern (s n ) a b u(:,t+1) = sn(:,t) + w * tanh(u(:,t)); v(:,t+1) = sum( tanh(u(1:3,t)) );

23. Example