1. Optical Illusions KG-VISA Kyongil Yoon 3/31/2004

2. Introduction http://www.cfar.umd.edu/~fer/optical/index.html http://www.cfar.umd.edu/~fer/optical/index.html http://www.cfar.umd.edu/~fer/optical/index.html A new theory of visual illusions A new theory of visual illusions A computational nature. A computational nature. The theory predicts many of the well known geometric optical illusions The theory predicts many of the well known geometric optical illusions Illusions of movement in line drawings Illusions of movement in line drawings Illusions of three-dimensional shape Illusions of three-dimensional shape Nearly every illusion has a different cause Nearly every illusion has a different cause Robinson in introduction to geometrical optical illusions "There is no better indicator of the forlornness of this hope [the hope of some to find a general theory] than a thorough review of the illusions themselves " Robinson in introduction to geometrical optical illusions "There is no better indicator of the forlornness of this hope [the hope of some to find a general theory] than a thorough review of the illusions themselves " The scientific study of illusions The scientific study of illusions Beginning of the nineteenth century when scientists got interested in perception Beginning of the nineteenth century when scientists got interested in perception Illusions have been used as tools in the study of perception Illusions have been used as tools in the study of perception An important strategy in finding out how perception operates is to observe situations in which misperceptions occur. By carefully altering the stimuli and testing the changes in visual perception psychologists tried to gain insight into the principles of perception. An important strategy in finding out how perception operates is to observe situations in which misperceptions occur. By carefully altering the stimuli and testing the changes in visual perception psychologists tried to gain insight into the principles of perception. Theories about illusions Theories about illusions On geometric optical illusions: accounting for a number of illusions On geometric optical illusions: accounting for a number of illusions Referring to image blurring Referring to image blurring The new theory The new theory

3. Introduction The Proposed Theory Image interpretation - number of estimation processes Image interpretation - number of estimation processes Noise  best estimate Noise  best estimate However, the best estimate does not correspond to the true value However, the best estimate does not correspond to the true value The estimates are biased The estimates are biased The principle of uncertainty of visual processes The principle of uncertainty of visual processes In certain patterns, where the error is repeated, it becomes noticeable. In certain patterns, where the error is repeated, it becomes noticeable. The principle of uncertainty is the main cause for many optical illusions The principle of uncertainty is the main cause for many optical illusions Geometric Optical Illusions Geometric Optical Illusions Early computational processes: The extraction of features, such as lines and points, or intersections of lines Early computational processes: The extraction of features, such as lines and points, or intersections of lines An erroneous estimation  erroneous perception An erroneous estimation  erroneous perception Illusions of Movement Illusions of Movement For cleverly arranged patterns with spatially separated areas having different biases For cleverly arranged patterns with spatially separated areas having different biases Shape Illusions Shape Illusions Extracting the shape of the scene in view from image features, called shape from X computations Extracting the shape of the scene in view from image features, called shape from X computations The bias can account for many findings in psychophysical experiments on the erroneous estimation of shape The bias can account for many findings in psychophysical experiments on the erroneous estimation of shape An understanding the bias allows to create illusory displays. An understanding the bias allows to create illusory displays. The bias is a computational problem, and it applies to any vision system The bias is a computational problem, and it applies to any vision system These illusion is experienced by humans, also should be experienced by machines. These illusion is experienced by humans, also should be experienced by machines.

4. Introduction: The Proposed Theory Bias in Linear Estimation The constraints underlying visual processes The constraints underlying visual processes Formulated as an over-determined linear equations Formulated as an over-determined linear equations A x = b where A an n × k matrix, and b an n-dimensional vector denoting measurements, that is the observations, and x a k-dimensional vector denoting the unknowns. The observations are noisy, that is, they are corrupted by errors. We can say that the observations are composed of the true values (A', b') plus the errors (δA, δb), i.e. A = A' + δA and b = b' + δb. In addition the constraints are not completely true, they are only approximations; in other words there is system error, ε. The constraints for the true value, x', amount to A x = b where A an n × k matrix, and b an n-dimensional vector denoting measurements, that is the observations, and x a k-dimensional vector denoting the unknowns. The observations are noisy, that is, they are corrupted by errors. We can say that the observations are composed of the true values (A', b') plus the errors (δA, δb), i.e. A = A' + δA and b = b' + δb. In addition the constraints are not completely true, they are only approximations; in other words there is system error, ε. The constraints for the true value, x', amount to A' x' = b' + ε. A' x' = b' + ε. We are dealing with what is called the errors-in-variable model in statistics. We have to use an estimator, that is a procedure, to solve the equation system. The most common choice is by means of least squares (LS) estimation. However, it is well known, that LS estimation is biased. We are dealing with what is called the errors-in-variable model in statistics. We have to use an estimator, that is a procedure, to solve the equation system. The most common choice is by means of least squares (LS) estimation. However, it is well known, that LS estimation is biased. Under some simplifying assumptions (identical and independent random variables δA and δb with zero mean and variance σ2 ) the LS estimate converges to Under some simplifying assumptions (identical and independent random variables δA and δb with zero mean and variance σ2 ) the LS estimate converges to Large variance in δA, an ill-conditioned A', or an x' which is oriented close to the eigenvector of the smallest singular value of A' all could increase the bias and push the LS solution away from the real solution. Generally it leads to an underestimation of the parameters. Large variance in δA, an ill-conditioned A', or an x' which is oriented close to the eigenvector of the smallest singular value of A' all could increase the bias and push the LS solution away from the real solution. Generally it leads to an underestimation of the parameters. There are other, more elaborate estimators that could be used. None, however will perform better if the errors cannot be obtained with high accuracy. There are other, more elaborate estimators that could be used. None, however will perform better if the errors cannot be obtained with high accuracy. Examples of visual computations which amount to linear equation systems are the estimation of image motion or optical flow, the estimation of the intersections of lines, and the estimation of shape from various cues, such as motion, stereo, texture, or patterns. Examples of visual computations which amount to linear equation systems are the estimation of image motion or optical flow, the estimation of the intersections of lines, and the estimation of shape from various cues, such as motion, stereo, texture, or patterns.

5. Errors in Image Intensity: How images change when smoothed As a noisy version of the ideal image signal As a noisy version of the ideal image signal We create the most likely image the vision system works with by smoothing the image We create the most likely image the vision system works with by smoothing the image Many illusions can be understood from the behavior of straight lines and edges Many illusions can be understood from the behavior of straight lines and edges Three cases Three cases An edge at the border between regions of different intensity, such as black and white An edge at the border between regions of different intensity, such as black and white No change No change A line on a background of different intensity A line on a background of different intensity Drift apart each other Drift apart each other A gray line between a bright and a dark region A gray line between a bright and a dark region Move toward each other Move toward each other

7. Errors in Image Intensity: Café Wall Illusion The horizontal mortar lines being tilted The horizontal mortar lines being tilted Effects of smoothing Effects of smoothing

8. Errors in Image Intensity: Café Wall Illusion Local edge detection  linked to longer lines Local edge detection  linked to longer lines

9. Errors in Image Intensity: Café Wall Illusion Counteract the effect Counteract the effect

10. Errors in Image Intensity: Spring Pattern Square grid with black squares superimposed Square grid with black squares superimposed

11. Errors in Image Intensity: Spring Pattern Combination of type-1 (single) and type-2 (drift apart) edges Combination of type-1 (single) and type-2 (drift apart) edges Flash Anim Flash Anim

12. Errors in Image Intensity: Waves Pattern Black and white checkerboard with small squares Black and white checkerboard with small squares Flash Anim Flash Anim

13. Errors in Line Estimation: The Theory Two intersecting lines Two intersecting lines Local edge detection Local edge detection Noisy Noisy Intersection point Intersection point The point closest to all the lines using least squares estimation The point closest to all the lines using least squares estimation The estimation of the intersection point is biased The estimation of the intersection point is biased For an acute angle For an acute angle The estimated intersection point is between the lines. The estimated intersection point is between the lines. The bias increases as the angle decreases. The bias increases as the angle decreases. The component of the bias in the direction perpendicular to a line decreases as the number of line segments along the line increases The component of the bias in the direction perpendicular to a line decreases as the number of line segments along the line increases

14. The two ends of the straight diagonal line passing behind the rectangle appear to be offset The two ends of the straight diagonal line passing behind the rectangle appear to be offset Can be predicted by the bias Can be predicted by the bias The diagonal line segments The diagonal line segments The lines at the border of the rectangle The lines at the border of the rectangle The illusory effect increases with a decrease in the acute angle The illusory effect increases with a decrease in the acute angle Errors in Line Estimation: Poggendorff Illusion Java Anim Java Anim

15. Errors in Line Estimation: Zöllner Illusion Tilted segments are estimated Tilted segments are estimated Input to the higher computational processes which fits long line to the segments Input to the higher computational processes which fits long line to the segments Parametric studies Parametric studies A stronger illusory perception for more tilted obliques A stronger illusory perception for more tilted obliques A stronger illusory effect when the pattern is rotated by 45 degrees A stronger illusory effect when the pattern is rotated by 45 degrees In neurophysiological studies, our cortex responds more to lines in horizontal and vertical than oblique orientations In neurophysiological studies, our cortex responds more to lines in horizontal and vertical than oblique orientations Less response from the main lines, more bias Less response from the main lines, more bias Java Anim Java Anim

16. Errors in Line Estimation: Luckiesh Pattern Distorted circle Distorted circle The bias depends on the direction of the intersecting lines The bias depends on the direction of the intersecting lines Changing the direction of the background lines causes a change in the bias and thus a change in the estimated curve, with the circle bumping at different locations Changing the direction of the background lines causes a change in the bias and thus a change in the estimated curve, with the circle bumping at different locations Java Anim Java Anim

17. Errors in Movement: How image movement is estimated Optical flow Optical flow Representation of image motion by comparing sequential images and estimating how patterns move between images Representation of image motion by comparing sequential images and estimating how patterns move between images The movement of the point from the first image to the second image. It can only be computed where there is detail, or edges, in the image. And it requires two computational stages to estimate optical flow. The movement of the point from the first image to the second image. It can only be computed where there is detail, or edges, in the image. And it requires two computational stages to estimate optical flow. Normal flow (First stage) Normal flow (First stage) Through a small aperture, we can only compute the component of the motion vector perpendicular the edge Through a small aperture, we can only compute the component of the motion vector perpendicular the edge Local information only provides information about the line Local information only provides information about the line constraint line: on which the optical flow vector lies constraint line: on which the optical flow vector lies

18. Errors in Movement: How image movement is estimated Second stage Second stage Combination of the motion components from differently oriented edges within a small patch Combination of the motion components from differently oriented edges within a small patch Estimate the optical flow vector closest to all the constraint lines Estimate the optical flow vector closest to all the constraint lines The minimum squared distance from the lines The minimum squared distance from the lines Over-determined system Over-determined system Solution is biased Solution is biased Does not correspond to the actual flow Does not correspond to the actual flow Depends on the features in the patch, texture Depends on the features in the patch, texture

19. Errors in Movement: Ouchi Illusion Estimated flows of surrounding area and inset area are different Estimated flows of surrounding area and inset area are different Smaller in length than the actual flow and it is closer in direction to the majority of normal flow vectors in a region Smaller in length than the actual flow and it is closer in direction to the majority of normal flow vectors in a region

20. Errors in Movement: Ouchi Illusion Flash Anim Flash Anim

21. Errors in Movement: Wheels Illusion Every point on the image moves on a straight line through the image center Every point on the image moves on a straight line through the image center Actual flow vectors are moving radially from the image center outwards, otherwise they are moving inwards Actual flow vectors are moving radially from the image center outwards, otherwise they are moving inwards

22. Errors in Movement: Wheels Illusion

25. Errors in Movement: Spiral Illusion Spiral rotation around its center Spiral rotation around its center Not circular Not circular Contract or expand Contract or expand Counter-clockwise Counter-clockwise Red: actualmotion vector Red: actualmotion vector Blue: normal flow vectors Blue: normal flow vectors

26. Errors in Movement: Moving sinusoids Smooth curves may be perceived to deform non- rigidly when translated in the image plane Smooth curves may be perceived to deform non- rigidly when translated in the image plane Low amplitude: appears to deform non-rigidly Low amplitude: appears to deform non-rigidly High amplitude: perceived as the true translation High amplitude: perceived as the true translation Flash Anim Flash Anim

27. Shape from Motion: The Constraint A biased estimate for the surface normal A biased estimate for the surface normal Motion parameters Motion parameters Orientation of the image lines (that is the texture of the plane) Orientation of the image lines (that is the texture of the plane) As a parameterization for the surface normal As a parameterization for the surface normal Slant (σ) Slant (σ) The angle between N and the negative optical axis The angle between N and the negative optical axis Tilt (τ) Tilt (τ) The angle between the parallel projection of N on the image plane and the image x-axis. The angle between the parallel projection of N on the image plane and the image x-axis.

28. Shape from Motion: Segmentation of a Plane due to Erroneous Slant Estimation The plane is perceived to be segmented into two differently slanted planes The plane is perceived to be segmented into two differently slanted planes Upper texture: smaller the slant in the than in the lower one Upper texture: smaller the slant in the than in the lower one Appears to be closer in orientation Appears to be closer in orientation Much more bias  a large underestimation of slant Much more bias  a large underestimation of slant