1. Medical University of Lübeck Institute for Signal Processing Nonlinear visual coding from an intrinsic-geometry perspective E. Barth* & A. B. Watson NASA Ames Research Center http://vision.arc.nasa.gov Supported by DFG grant Ba 1176/4-1 to EB and NASA grant 199-06-12-39 to ABW

2. Medical University of Lübeck Institute for Signal Processing Intrinsic dimensionality in 2D i0D: constant in all directions i1D: constant in one direction i2D: no constant direction

3. Medical University of Lübeck Institute for Signal Processing i0D FT

4. Medical University of Lübeck Institute for Signal Processing i1D e.g. straight lines and edges, gratings FT

5. Medical University of Lübeck Institute for Signal Processing i2D e.g. corners, line ends, curved edges and lines FT

6. Medical University of Lübeck Institute for Signal Processing i2D i1D i0D

8. i1D i2D

9. Medical University of Lübeck Institute for Signal Processing Intrinsic dimensionality in 3D i0D: constant in all (space-time) directions i1D: constant in 2 directions i2D: constant in one direction i3D: no constant direction

10. Medical University of Lübeck Institute for Signal Processing i0D FT

11. Medical University of Lübeck Institute for Signal Processing i1D e.g. drifting spatial grating FT

12. Medical University of Lübeck Institute for Signal Processing i2D e.g. drifting corner, flashed grating FT

13. Medical University of Lübeck Institute for Signal Processing i3D e.g. flow discontinuities, flashed corners FT

14. Medical University of Lübeck Institute for Signal Processing Intrinsic dimensionality and motion FT of (rigid) motion signal is in a plane

15. The visual input as a hypersurface luminance hypersurface Visualization of surfaces is easier:

16. Medical University of Lübeck Institute for Signal Processing Geometric view on intrinsic dimensionality

17. Medical University of Lübeck Institute for Signal Processing Curvature and motion (“plane” = “more than line but no volume”)

18. Medical University of Lübeck Institute for Signal Processing The Riemann tensor R most important property of (hyper)surfaces measures the curvature of the (hyper)surface has 6 independent components in 3D vanishes in 1D.

19. Medical University of Lübeck Institute for Signal Processing The Riemann tensor components are nonlinear combinations of derivatives, i.e., of linear filters with various spatio-temporal orientations.

20. Medical University of Lübeck Institute for Signal Processing R components and speed v Multiple representation of speed.

21. Medical University of Lübeck Institute for Signal Processing R and direction of motion q Multiple, distributed representation of direction.

22. Medical University of Lübeck Institute for Signal Processing Sectional curvatures

23. Medical University of Lübeck Institute for Signal Processing Direction tunings of R components vertical motion horizontal motion

24. Wallach, 1935 Barber pole

25. Kooi, 1993 “abolished illusion”

26. Rodman & Albright, 1989 Analytical predictions based on R components Typical Type II MT neuron, macaque monkey Direction tuning Orientation tuning Orthogonal orientation and direction tunings

27. Recanzone, Wurtz, & Schwarz, 1997 Analytical predictions based on R components Typical MT neuron, macaque monkey Multiple motions

28. Medical University of Lübeck Institute for Signal Processing (Reference to 3D world of moving objects is not needed.) Conclusion Hypothesis that a basic (geometric) signal property (the intrinsic dimensionality) is encoded in early- and mid-level vision explains –orientation selectivity (derivatives, and R2, R3) –endstopping (all R components are endstopped for translations) –velocity selectivity –direction selectivity –some global-motion percepts (by integration) –some properties reported for MT neurons.