2. Visual Intelligence Chapters 6 & 7 When the World Stopped Moving

3. Tiffany S. Williams 4 th Year Drama and Psych major. 4 th Year Drama and Psych major. Believes she may be quadrichromatic. Believes she may be quadrichromatic.

4. Borna Rangchi 4 th Year Psych Major 4 th Year Psych Major Believes Tiffany may be quadrichromatic. Believes Tiffany may be quadrichromatic.

5. How could you possibly see an object but not its motion?

6. This Question would have puzzled John Locke! Asserted that body and motion were inseparable and that body and its color are separable. Asserted that body and motion were inseparable and that body and its color are separable. Motion is primary and inseparable. Motion is primary and inseparable.

7. The Case of L.M. Had “AKINETOPSIA” – brain damage resulting in the inability to perceive moving objects, despite stationary objects remaining more or less visible. Had “AKINETOPSIA” – brain damage resulting in the inability to perceive moving objects, despite stationary objects remaining more or less visible. Loss of motion is permanent. Loss of motion is permanent. Akinetopsia would have been a hard concept for Locke to comprehend. Akinetopsia would have been a hard concept for Locke to comprehend.

8. Motion is Constructed! Our visual intelligence constructs motion. Our visual intelligence constructs motion. Logic = “If you construct objects, then you must construct their motions. If what moves is your construction then how it moves must be as well. The what and how of motion are, of necessity, intimately linked. Logic = “If you construct objects, then you must construct their motions. If what moves is your construction then how it moves must be as well. The what and how of motion are, of necessity, intimately linked. How can we experience the loss of motion without brain damage? How can we experience the loss of motion without brain damage? TMS (TRANSCRANIAL MAGNETIC STIMULATION) – process where anyone can experience a brief loss of motion through the use of magnetic fields that impair normal electric function in V5. TMS (TRANSCRANIAL MAGNETIC STIMULATION) – process where anyone can experience a brief loss of motion through the use of magnetic fields that impair normal electric function in V5.

9. Motion is Constructed

10. Need Some Proof Do You? Phi Motion (Sigmund Exner; 1875) Phi Motion (Sigmund Exner; 1875) Two dots are flashed, one after the other. Two dots are flashed, one after the other. With the correct interstimulus interval (time and space between dots), two objects are transformed into one moving object. With the correct interstimulus interval (time and space between dots), two objects are transformed into one moving object. If ISI is between a half of a second and a tenth of a second, the illusion of motion is created by your visual intelligence. The precise range of ISIs depends on the distance between dots. If ISI is between a half of a second and a tenth of a second, the illusion of motion is created by your visual intelligence. The precise range of ISIs depends on the distance between dots. Flashed: 200ms Blank: 400ms Flash: 200ms Blank: 50ms Flash: 10ms Blank: 0ms

11. Implications of “Phi” Motion Objects and motions are constructed interdependently. Objects and motions are constructed interdependently. The construction of motion depends on the construction of objects and vice versa. The construction of motion depends on the construction of objects and vice versa. ex. If your visual intelligence opts to construct one object, it will move across the screen rather than two separate blinking objects.

12. Color Phi Motion Color Phi Motion (Max Wertheimer; 1912) Color Phi Motion (Max Wertheimer; 1912) Same as Exner’s experiment but with different color dots. Same as Exner’s experiment but with different color dots. If ISI is appropriate, you should see the first half of the motion represent the color of the first dot, and the second half of the motion represent the color of the second dot. If ISI is appropriate, you should see the first half of the motion represent the color of the first dot, and the second half of the motion represent the color of the second dot. What about the sequence of events? What about the sequence of events?

13. Motion Over Curved Paths With the right ISI the dot should move over the arc. With the right ISI the dot should move over the arc.

14. Curved Motions Continued Curved motions can be constructed when no curve is flashed. Curved motions can be constructed when no curve is flashed. A single dot is constructed moving left to right in a curved path above of below the box. No fancy loops or twists From this we can infer that motions are constructed on minimal (simple) paths and motions. Rule 29: Create the simplest possible motions.

15. Two Dots at a Time  Long ISI = Group Motion  Group Motion: Two dots move together rigidly from left to right.  Long Range Motion  Short ISI = Element Motion  Dot in the middle stays put, and the dot on the left jumps around it from left to right.  Short Range Motion Group Motion Element Motion

16. More Group/Element Motion Multilayered construction Multilayered construction Group MotionElement Motion

17. Two Dots at Once Two different motions are constructed. Two different motions are constructed. Dots move up & down or side to side. Dots move up & down or side to side. Dots DO NOT merge. Either they BOTH move up & down or they BOTH move side to side. Dots DO NOT merge. Either they BOTH move up & down or they BOTH move side to side.

18. Rule 30 Rule 30: When making motion, construct as few objects as possible, and conserve them as much as possible. Rule 30: When making motion, construct as few objects as possible, and conserve them as much as possible. “These motions make two dots merge into one, and leave one dot unmatched. You prefer to conserve objects as much as possible when you construct motion. You don’t want to make two objects magically turn into one, and have another object magically appear, when instead you could just make two objects total, and conserve them over the two frames.” -(Hoffman pg. 148)

19. Coherence of Motion Do you notice that motions of the dots are the same? Do you notice that motions of the dots are the same? Even when one switches from horizontal to vertical and vice versa, the rest switch in concordance. Even when one switches from horizontal to vertical and vice versa, the rest switch in concordance.

20. …..which leads us to……

21. RULE 31!!!!

22. Rule 31 Rule 31: Construct motion to be as uniform over space as possible. Rule 31: Construct motion to be as uniform over space as possible.

23. Rotations Clockwise or Counterclockwise?

24. Rotation with “Inertia” Clockwise or Counterclockwise?

25. Circular Square Given the correct ISI, motion from four points can be interpreted in a circular fashion. Given the correct ISI, motion from four points can be interpreted in a circular fashion. Look off to the side for the full experience.

26. Constructing motion depends on the Context Karl Dunker (1929) – illustrated a “Cycloid.” Karl Dunker (1929) – illustrated a “Cycloid.” See light as bouncing like a ball and not rotating on a wheel. See light as bouncing like a ball and not rotating on a wheel. Dunker was ambitious and decided to add yet another light, this time with one at the hub and one to the wheel…

27. 2 or more Dots We will create two types of motion: We will create two types of motion: 1. Global Motion – all the dots together 2. Relative Motion– the motion of each dot relative to the global motion. How do we divide motion into global and/or relative? How do we divide motion into global and/or relative? James Cutting & Dennis Proffitt (1982) – suggested it is to minimize both common and relative motions. James Cutting & Dennis Proffitt (1982) – suggested it is to minimize both common and relative motions. Doesn’t predict what you see in Dunker, Rubin, and Richards displays. “Because you minimize common motion by making it uniform translation, but you don’t minimize both common and relative motions together. If you did, you’d see both dots rotate about a point halfway between them.” Doesn’t predict what you see in Dunker, Rubin, and Richards displays. “Because you minimize common motion by making it uniform translation, but you don’t minimize both common and relative motions together. If you did, you’d see both dots rotate about a point halfway between them.”

28. What’s up with those barbershop poles? J.P. Gilford (1929) – noticed that stripes on barber shop poles appear to move vertically although it is turning on a horizontal axis. J.P. Gilford (1929) – noticed that stripes on barber shop poles appear to move vertically although it is turning on a horizontal axis. This paradoxical motion is fundamental in understanding the APERTURE PROBLEM. This paradoxical motion is fundamental in understanding the APERTURE PROBLEM.

29. The Aperture Problem Rule 32: Construct the smoothest Velocity Field Rule 32: Construct the smoothest Velocity Field “Velocity Field” – the path of motion “Velocity Field” – the path of motion The smoothest path is where the velocity of the motion changes the least. The smoothest path is where the velocity of the motion changes the least. So in the Case of the Barber shop pole… So in the Case of the Barber shop pole… The smoothest velocity field has the stripes moving straight up. The smoothest velocity field has the stripes moving straight up. The motion you construct at one point depends on the motion you construct at nearby points. The motion you construct at one point depends on the motion you construct at nearby points. Smoothness is a relation between motions at different points. Smoothness is a relation between motions at different points.

30. Exploring the Interaction between the Construction of Objects and Motion Ted Adelson & Tony Movshon – A “grating.” Ted Adelson & Tony Movshon – A “grating.” “motion orthogonal to its lines” – the motion direction you always make for that particular grating. Do u see two independent gratings (combo of 1 st and 2 nd grating) or one unit? contrast You Coordinate your creations of motions and objects by intelligently using size, contrast, depth, and color.

31. What will happen if the gratings differ? Differ in size=no integration Differ in size=no integration The greater the difference the more likely to see 2 independent gratings The greater the difference the more likely to see 2 independent gratings Differ in contrast = no integration The same is true with differences of color and depth.

32. The POWERFUL Principle of Rigid Motion!!! Points in space move rigidly if all distances between them remain constant during the motion.

33. 3D Cylinder 3D Cylinder Dots appearing at slightly different positions from one frame to the next over a span of 12 frames. They appear to take 3D form…different from just a 2D movie. Dots appearing at slightly different positions from one frame to the next over a span of 12 frames. They appear to take 3D form…different from just a 2D movie. Rule 33: If possible, and if other rules permit, interpret image motions as projections of rigid motions in three dimensions. Rule 33: If possible, and if other rules permit, interpret image motions as projections of rigid motions in three dimensions.

34. Um…ok, so how can you do this in practice? Ullman’s Rigidity Theorem: “Suppose you are given three frames, each containing at least four points. If the points are placed at random in each frame, then the probability is zero that they have a rigid interpretation in three dimensions. If the points do have a rigid interpretation, then they almost surely have exactly two interpretations (which are mirror- symmetric).” Ullman’s Rigidity Theorem: “Suppose you are given three frames, each containing at least four points. If the points are placed at random in each frame, then the probability is zero that they have a rigid interpretation in three dimensions. If the points do have a rigid interpretation, then they almost surely have exactly two interpretations (which are mirror- symmetric).” It means you don’t need that much info to decide whether or not to create a rigid 3D object and its motion. All you need is as little as 3 frames each having 4 points! Uh…so what does that mean? This is advancing the field of computer vision systems. At the moment they are good at constructing 3D objects, but not recognizing them.

35. Limitations of Rigidity Principle Biological Motion – motion of the human body Biological Motion – motion of the human body Eg. Black Light Theater of Prague. Eg. Black Light Theater of Prague. Coming to our own Irvine Barclay Theatre March 19 th and 20 th Coming to our own Irvine Barclay Theatre March 19 th and 20 th UCI’s Beall Center for Art and Technology explores the idea of biological motion. UCI’s Beall Center for Art and Technology explores the idea of biological motion.

37. Johansson’s movies Johansson’s movies Placed lights on joints of actors. Placed lights on joints of actors. Issue for Rigidity Principle: You can’t have groups of 4 points that move together rigidly. Issue for Rigidity Principle: You can’t have groups of 4 points that move together rigidly. Knee and ankle points move rigidly, but hip and ankle do not. Knee and ankle points move rigidly, but hip and ankle do not. You can still construct 3D objects w/o having to create groups of 4 rigid points You can still construct 3D objects w/o having to create groups of 4 rigid points

38. So, you mean there’s more than just the Rigidity Principle? Yup! Rule 34: If possible, and if other rules permit, interpret image motions as projections of 3D motions that are rigid and planar. Rule 34: If possible, and if other rules permit, interpret image motions as projections of 3D motions that are rigid and planar. This allows us to construct biologicical motion where rigidity alone, does not. This allows us to construct biologicical motion where rigidity alone, does not. Because of the biological make up of our joints, our bones swing on planes. Because of the biological make up of our joints, our bones swing on planes. Rigidity and Planarity together require only 2 points to construct a 3D object versus the 4 needed with Ullman’s theorem. Rigidity and Planarity together require only 2 points to construct a 3D object versus the 4 needed with Ullman’s theorem. Rigidity and Planarity will usually work with other rules too such as rules about smoothness or dynamical constraints Motion Picture and Gaming: need compelling graphics or people won’t want it.

39. Ok…enough dots already. Let’s Move on… Let’s Move on…

40. Dan Kersten and Collaborators. When you’re clever: When you’re clever: Interpreting a smooth diagonal movement along the checkerboard along with the shadow. Interpreting a smooth diagonal movement along the checkerboard along with the shadow. When you’re not so bright: Interpreting a complicated movement of varying elevation and direction just because of a different position of the shadow even though the solid sphere is moving along the same exact path! Studied how we construct shapes, motions, shadows, and light sources. Studied how we construct shapes, motions, shadows, and light sources.

41. But…motion in 3D depends not only on the sphere, but also on how you interpret the relation between it and its shadow. But…motion in 3D depends not only on the sphere, but also on how you interpret the relation between it and its shadow. The sphere’s move along the same path, but the shadow moves along in arcs. The sphere’s move along the same path, but the shadow moves along in arcs. Why? Why? Because of Rule 35: Light Sources Move Slowly. Because of Rule 35: Light Sources Move Slowly. We don’t like to create light sources that move fast, we’d rather form an interpretation about the objects. We don’t like to create light sources that move fast, we’d rather form an interpretation about the objects. Eg. After class if you walked outside and it was completely pitch black…you may think the world was coming to an end. Eg. After class if you walked outside and it was completely pitch black…you may think the world was coming to an end.

42. Ooh all the Pretty Colors! Tom Albright and Karen Dobkins claim a strong interaction between the magnocellular (motion, luminance, depth, and coarse form) and parvocellular (color, fine form). P. 166 Tom Albright and Karen Dobkins claim a strong interaction between the magnocellular (motion, luminance, depth, and coarse form) and parvocellular (color, fine form). P. 166 Shows that we prefer objects of the same color to move together. Shows that we prefer objects of the same color to move together. “Birds of a feather flock together.”

43. Can motion affect how you create color? “Indeed it can.” “Indeed it can.” Benedict Prevosts “Flicker Colors.” Benedict Prevosts “Flicker Colors.” Rediscovered by C.E. Benham Rediscovered by C.E. Benham Benham’s Top. Benham’s Top. Spin counterclockwise and see an array of colors, spin it clockwise and you see another array of colors but in reverse order. (red to green to blue to violet and vice versa). Spin counterclockwise and see an array of colors, spin it clockwise and you see another array of colors but in reverse order. (red to green to blue to violet and vice versa). Phenomena of flicker colors shows how your creation of motion affects your creation of color. Phenomena of flicker colors shows how your creation of motion affects your creation of color.

44. But Hoffman too was a clever one!

45. Dynamic Color Spreading Creating motion through color with stationary dots. Creating motion through color with stationary dots. Dr. Hoffman and Dr. Carol Cicerone

46. More examples…

47. Computer Vision Systems It is still very limited, but it’s growing and may not be far beyond our reach! It is still very limited, but it’s growing and may not be far beyond our reach! Eg. Richman’s car tracking system Eg. Richman’s car tracking system

49. Chapter 7 The Feel of a Phantom

50. Phantom Limbs The feel that an actual limb is there with real feelings and sensations when in fact it is not there. The feel that an actual limb is there with real feelings and sensations when in fact it is not there. Case of F.A. Case of F.A. Found 2 Systematic maps of phantom hand. Found 2 Systematic maps of phantom hand. First: About 6cm above stump First: About 6cm above stump Second: About 13cm above stump. Second: About 13cm above stump. He could feel the sensations in his hand when those specific areas were touched He could feel the sensations in his hand when those specific areas were touched Maps found on faces in some cases as with patient V.Q. p.175 Maps found on faces in some cases as with patient V.Q. p.175

51. The Somatosensory Cortex The portion of your brain devoted primarily to touch and sensations The portion of your brain devoted primarily to touch and sensations Penfield map: Penfield map: Notice how the hand and face are nearby? Notice how the hand and face are nearby? The face and arm regions of cortex “invade” the cortex that normally processes the hand. The face and arm regions of cortex “invade” the cortex that normally processes the hand. Proven with magnetic source imaging – a non-surgical process used to measure brain activity. Tony T. Yang (1994) Proven with magnetic source imaging – a non-surgical process used to measure brain activity. Tony T. Yang (1994)

52. Are we just making it up? Do we construct everything we see and experience (hear, smell, taste, and feel)? Do we construct everything we see and experience (hear, smell, taste, and feel)? Hoffman argues that all your sensations and perceptions are constructions. Hoffman argues that all your sensations and perceptions are constructions. You don’t just construct simple feelings. You don’t just construct simple feelings. Exp. Diane Ackerman “Our palette of feelings is elaborate.” Exp. Diane Ackerman “Our palette of feelings is elaborate.” You construct complex ones as well. You construct complex ones as well.

53. What if you have all your limbs, is it possible to have trouble with constructiion? Yes. If you’re constructive processes are not intact, you will have problems with touch. Yes. If you’re constructive processes are not intact, you will have problems with touch. Case of E.C. Case of E.C. Stroke in areas 39 and 40 of left hemisphere. Numbness in the right hand. Stroke in areas 39 and 40 of left hemisphere. Numbness in the right hand. Another stroke in temporal and occipital cortex. Another stroke in temporal and occipital cortex. Outcome: Trouble remembering visual objects (RH) and her right hand had tactile agnosia, (LH). Hand still had sensations, it just couldn’t recognize objects. Outcome: Trouble remembering visual objects (RH) and her right hand had tactile agnosia, (LH). Hand still had sensations, it just couldn’t recognize objects.

54. So what about if we don’t have any physical problems like Phantoms or Strokes? Frank Geldard and Carl Sherrick (1970’s) Frank Geldard and Carl Sherrick (1970’s) 3 vibrators on arm that accidentally delivered taps and created the feeling of extra taps between the vibrators. Feeling of a rabbit hopping up your arm. 3 vibrators on arm that accidentally delivered taps and created the feeling of extra taps between the vibrators. Feeling of a rabbit hopping up your arm. Can create this effect anywhere on body and with as little as 2 vibrators. Can create this effect anywhere on body and with as little as 2 vibrators. Can create more than one rabbit. P. 181 Can create more than one rabbit. P. 181 The second tap tells you the direction of where to send the rabbit, so you must create the rabbit afterward. The second tap tells you the direction of where to send the rabbit, so you must create the rabbit afterward.

55. So what does all this mean? It means that we create not just the “what” and the “where” of feeling, but the “when” as well. It means that we create not just the “what” and the “where” of feeling, but the “when” as well. It confirms that you create what you feel!!! It confirms that you create what you feel!!!

56. Now… How does all of this relate to the Mind/Body problem? How does all of this relate to the Mind/Body problem? Does this bring us any closer to solving the problem? Does this bring us any closer to solving the problem? How have your thoughts changed from the first day of class until now? How have your thoughts changed from the first day of class until now?

57. Thank You!