Medical University of Lübeck Institute for Signal Processing Nonlinear visual coding from an intrinsic-geometry perspective E. Barth* & A. B. Watson NASA Ames Research Center Supported by DFG grant Ba 1176/4-1 to EB and NASA grant to ABW

Medical University of Lübeck Institute for Signal Processing Intrinsic dimensionality in 2D i0D: constant in all directions i1D: constant in one direction i2D: no constant direction

Medical University of Lübeck Institute for Signal Processing i0D FT

Medical University of Lübeck Institute for Signal Processing i1D e.g. straight lines and edges, gratings FT

Medical University of Lübeck Institute for Signal Processing i2D e.g. corners, line ends, curved edges and lines FT

Medical University of Lübeck Institute for Signal Processing i2D i1D i0D

i1D i2D

Medical University of Lübeck Institute for Signal Processing Intrinsic dimensionality in 3D i0D: constant in all (space-time) directions i1D: constant in 2 directions i2D: constant in one direction i3D: no constant direction

Medical University of Lübeck Institute for Signal Processing i0D FT

Medical University of Lübeck Institute for Signal Processing i1D e.g. drifting spatial grating FT

Medical University of Lübeck Institute for Signal Processing i2D e.g. drifting corner, flashed grating FT

Medical University of Lübeck Institute for Signal Processing i3D e.g. flow discontinuities, flashed corners FT

Medical University of Lübeck Institute for Signal Processing Intrinsic dimensionality and motion FT of (rigid) motion signal is in a plane

The visual input as a hypersurface luminance hypersurface Visualization of surfaces is easier:

Medical University of Lübeck Institute for Signal Processing Geometric view on intrinsic dimensionality

Medical University of Lübeck Institute for Signal Processing Curvature and motion (“plane” = “more than line but no volume”)

Medical University of Lübeck Institute for Signal Processing The Riemann tensor R most important property of (hyper)surfaces measures the curvature of the (hyper)surface has 6 independent components in 3D vanishes in 1D.

Medical University of Lübeck Institute for Signal Processing The Riemann tensor components are nonlinear combinations of derivatives, i.e., of linear filters with various spatio-temporal orientations.

Medical University of Lübeck Institute for Signal Processing R components and speed v Multiple representation of speed.

Medical University of Lübeck Institute for Signal Processing R and direction of motion q Multiple, distributed representation of direction.

Medical University of Lübeck Institute for Signal Processing Sectional curvatures

Medical University of Lübeck Institute for Signal Processing Direction tunings of R components vertical motion horizontal motion

Wallach, 1935 Barber pole

Kooi, 1993 “abolished illusion”

Rodman & Albright, 1989 Analytical predictions based on R components Typical Type II MT neuron, macaque monkey Direction tuning Orientation tuning Orthogonal orientation and direction tunings

Recanzone, Wurtz, & Schwarz, 1997 Analytical predictions based on R components Typical MT neuron, macaque monkey Multiple motions

Medical University of Lübeck Institute for Signal Processing (Reference to 3D world of moving objects is not needed.) Conclusion Hypothesis that a basic (geometric) signal property (the intrinsic dimensionality) is encoded in early- and mid-level vision explains –orientation selectivity (derivatives, and R2, R3) –endstopping (all R components are endstopped for translations) –velocity selectivity –direction selectivity –some global-motion percepts (by integration) –some properties reported for MT neurons.