1. Ecological Information
What It Is, Where It Is, and How To Use It In Your Explanations
A Workshop
Andrew D Wilson
Sabrina Golonka

2. Outline
I’ll talk for a little bit about the basic rules for information (what it is, how it’s made, why it is informative)
We’ll play with an example (progressive occlusion) to work up some intuitions about the ecological analysis
I’ll talk a little more about using information to control action
We’ll work together to figure out the information used in coordinated rhythmic movement
I’ll tell you a little about how to scale up the use of information to get into language, etc
We’ll have a robust discussion trying to break (5)

3. Part 1 Ecological Information is a Real Thing
No Really, It’s a Thing. You Can Formally Identify It. It’s Not Just a Convenient Word.

4. Ecological Information
Information is a technical term in Gibson’s ecological approach It does not mean what most people mean when they talk about information (sorry) It is, however, that is a fundamental component of our behaviour and none of you, no matter what kind of behaviour you are studying, can ignore it (sorry)

5. The Evil Demon Gap

6. Task Dynamics
To be functional, our behaviour has to complement some subset of our local environment
Biologically relevant properties of the environment are most completely described at the level of dynamics (Bingham, 1995; Wilson & Bingham, 1999)
Dynamics is the level at which things can be effectively categorised into types
We need access to those dynamics, but these are mostly ‘over there’ (not in mechanical contact)

7. Kinematic Information
That gap is filled with energy media (light, an atmosphere, etc) Task dynamics interact with those media and are projected into those media as real kinematic patterns Ecological hypothesis: these kinematic patterns, while not identical to the dynamics, can specify (map 1:1) the dynamics Specifying patterns can be used as perceptual information for dynamics

8. Time
Position

9. From Dynamics to Kinematics
creates
Dynamics (in the world)
Kinematics (information)

10. From Dynamics to Kinematics
Dynamics (ball 1)
Dynamics (ball 2)
Only the ‘same’ thing at the level of dynamics
Same dynamics, different parameters, similar motion = two instances of the same type of event

11. From Dynamics to Kinematics
Dynamics (ball)
Dynamics (walking)
Different dynamics create different patterns of motion, enabling us to perceive the difference

12. Sensations vs Specification
Specification means that there is a 1:1 mapping from dynamics to kinematics Even though the world and information about the world are not the same thing, using the latter to stand on for the former works How is this 1:1 possible, though? I thought sensations were ambiguous…

13. “Consider the humble marsh periwinkle…”
Is this stem climb-upable?
Is this stem collide-withable?
“Why specification?”
Segue is “No one thinks that there isn’t stuff being detected by visual systems; but that
Turvey, M. T., Shaw, R. E., Reed, E. S., & Mace, W. M. (1981). Ecological laws of perceiving and acting: In reply to Fodor and Pylyshyn (1981).Cognition, 9(3),

14. Collide-withable stems
Climb-upable stems
Collide-withable stems

15. Optical property specifying climb-upable
Has the property Climb-upable
Has the property Collide-withable
Optical property specifying collide-withable

16. Kinematics Can Specify Dynamics
Time-to-contact and Tau (Lee & Reddish, 1981)
Dynamical Event: the time before an approaching surface intercepts your point of observation
Kinematic Information: tau (inverse of the rate of expansion (change in angular size) of the projection of the approaching object in the optic array.
Perception: time!
Behavioural evidence: organisms behave as if perceiving time, e.g. gannets closing wings at critical tau not distance (Lee & Reddish, 1981)
Psychophysical evidence: thresholds for relation ‘tau’ lower than either ratio component in isolation (impossible if computed later using components; Regan & Hamstra, 1993)
NB tau is a pain in the ass

17. Kinematics Can Specify Dynamics
Point light displays of dynamical events (Johannsson, 1950, 1973)
Dynamical Event: pretty much anything, but typically used to study biological motion
Kinematic Information: display is literally the relative motion of the degrees of freedom in the dynamical event
Perception: once in motion, the underlying dynamic is immediately perceived
Perception of size from motion (Jokisch & Troje, 2003)
Weights of lifted objects (Runeson & Frykholm, 1981)
Intent to deceive about the weight of lifted objects (Runeson & Frykholm, 1981)
and so on….

20. Kinematics Can’t Specify No Dynamics
Upcoming changes in friction (e.g. Joh et al, 2006, 2007)
Dynamical Event: a change in locomotion surface and resulting alteration in the frictional forces developed between it and your foot
Kinematic Information: NONE. Friction does not exist before the surfaces come in contact and therefore can produce no information
Perception: no perceptual access to the upcoming change at all, with often disastrous consequences (e.g. falls, broken limbs)
‘Cues’ like shine etc used but deeply unreliable, making actions unreliable
No dynamical property, no information, no perception

21. No Kinematics, No Dynamics
Driving a car under white out conditions
Dynamical Event: trying to control heading so as to keep the car on the road
Kinematic Information: NONE. While there are dynamical objects and events out there, the information they are creating is being wiped out by the white- out conditions.
Perception: little if any perceptual access to upcoming changes at all, with often disastrous consequences (e.g. crashes, losing your bearings)
Might be able to see just a little ahead, but only enough to control one step at a time
Plenty of dynamical properties, information from those erased by the white-out, no perception

22. Ecological Information Is Real and Is Not Just a Reification of a Word

23. Push space, get animation of square moving in a circle

24. Kinematics Can Specify Dynamics
Progressive Occlusion (Gibson et al, 1969; Kaplan, 1969)
Dynamical Event: One surface going out of view behind another (becoming progressively occluded but continuing to exist)
Kinematic Information: Deletion of optical texture at the leading edge, accretion of optical texture at the trailing edge (optical elements going being destroyed and created)
Perception: the dynamical event ‘persisting surface becoming occluded and de-occluded’
Judgments: people never report the accretion or deletion, but the occlusion
Multiple object tracking: can still be tracked through an occlusion but not an actual disappearance/reappearance (Scholl & Pylyshyn, 1999)
Scholl & Pylyshyn (19990

25. Multiple Object Tracking Survives Occlusion
You will see an array of dots; four will flash and then all the dots will move.
Your job is to
a) Fixate on the red circle in the centre, AND
b) Keep track of the 4 dots that flashed

26. MOT: Occlusion Baseline
Video
Scholl, B. J., & Pylyshyn, Z. W. (1999). Tracking multiple items through occlusion: Clues to visual objecthood. Cognitive Psychology, 38,
MOT: Occlusion Baseline

27. MOT: Occlusion Baseline

28. MOT: Virtual Occlusion
MOT-Occ-virtOcc.mov
MOT: Virtual Occlusion

29. MOT: Virtual Occlusion

30. MOT: Implosion http://www.yale.edu/perception/Brian/demos/MOT.html
MOT-Occ-virtOcc.mov
MOT: Implosion

31. ??? MOT: Implosion http://www.yale.edu/perception/Brian/demos/MOT.html
I have no idea what the right answer is 
MOT: Implosion

32. Object-ness Survives Occlusion Only
Occlusion specifies object moving in and out from behind something Implosion/explosion specifies objects coming in and out of existence Object-based attention is driven by this
Scholl, B. J., & Pylyshyn, Z. W. (1999). Tracking multiple items through occlusion: Clues to visual objecthood. Cognitive Psychology, 38,

33. The Curious Case of Coordinated Rhythmic Movement
Part 2 Ecological Information is a Real Thing and We Use It to Control Actions
The Curious Case of Coordinated Rhythmic Movement

34.  V = -a cosφ –b cos 2φ
0° & 180° the only stable states 0° more stable than 180° Other coordinations (e.g. 90°) can but must be learned
Demos

35. Information Experiments
HKB effects persist between people
Schmidt et al, 1990
HKB effects persist between person and a display
e.g. Wilson et al, 2005
HKB effects persist in perceptual judgments (no action!)
e.g. vision, Bingham et al, 2000; proprioception, Wilson et al, (2003)
Altering the feedback changes the HKB effects
e.g. all goes away with Lissajous feedback, Kovacs et al 2009
e.g. 90° becomes stable if feedback altered to show 0°; Wilson et al, 2005
HKB effects = the basic phenomena (stability at 0>180 > everything else, 90 worst; all gets differentially less stable with increasing frequency)
Point here is that the coupling alluded to earlier cannot be internal (either neural or mechanical) – it must be perceptual, and therefore informational.

36. Constraints on the Information
The oscillators must be coupled by a variable that
specifies relative phase
is composed only of state variables (to preserve autonomy)
is most readily detected at 0°, then at 180°, not at all at 90° (basic phenomena)
detection varies with frequency
We’ve set up the coupling has to be perceptual; these are the constraints on what the information is, given all the experiments

37. Constraints on the Information
The information should
be detectable by both vision (e.g. Bingham et al 1999, 2001; Zaal et al, 2001) and proprioception (Wilson et al, 2003)
only be defined for parallel movements (Bogaerts et al, 2003; Wilson et al, 2005; Wimmers et al, 1992)
be available throughout the trajectory but vary with relative speed (Bingham, 2004)

38. ?

39. Perturbation Experiments
Wilson & Bingham (2008)
Selectively rendered 3 kinematic variables uninformative about relative phase
Relative Position
Relative Frequency
Relative Speed
Relative direction impossible to
perturb!
We tried two perturbations of rel direction but you can’t do it AND keep the movement a coordinated rhythmic one
Demos

40. Perturbation Experiments
Nothing* affected visual perception of relative phase at 0 or 180; only thing left is relative direction, therefore that’s the information
Trained perception of relative phase at 90 completely disrupted by perturbing position; learning 90 entails switching variables to relative position
*well, 3 people got hit at 180 by relative position perturbation

41. The Information is Relative Direction
Perceived relative phase (rho)
Relative Speed (noise)
Ρ = sgn(sin(Φ1) sin(Φ2) + α(Vi – Vj)3 Nt )
Relative direction fits the previously described bill
Relative Direction
(information)
Relative direction (kinematic information) specifies relative phase (dynamical property)
0°: relative direction always the same
180°: relative direction always different
90°: relative direction maximally variable

42. Relative Direction in the Optic Array
Ρ = sgn(sin(Φ1) sin(Φ2) + α(Vi – Vj)3 Nt )
Visually, relative direction is a relation between the flow vectors in the local flow field generated by the motion of the oscillators
0°: all common motion
180°: all relative motion
90°: common motion half the time, relative motion half the time
Other relative phases: common & relative motion in various proportions
A note: relative direction specifies relative phase between 0 and 180. After that, it repeats (so 270 and 90 behave the same at the level of relative direction). This is fine, though, because that matches the behaviour; learning 90 transfers only to 270, for example, for just this reason.
So far so good; mechanism components figured out ahead of time using experiments, not model fitting

43. Real Components Action components:
Two autonomous damped mass-spring coupled via perceived relative phase
Information components:
Relative phase specified by relative direction and the detection of this conditioned by relative speed
*detection conditioned by relative speed means rel speed is a noise term to account for the various frequency effects.

44. The perception components
The action components
The perception components
Bingham, 2001, 2004a, b; Snapp-Childs, Wilson & Bingham, 2011

45. Part 3 Ecological Information is a Real Thing and We Use It to Control Actions and It Scales Up to ‘Higher’ Cognition
Golonka & Wilson (2016). Ecological Representations

46. Scaling Up – The Pieces of the Puzzle
Information is a(n intensional) representation
Information can be used in a law-based manner or a conventional manner
Law based information use can support action control and action selection
Conventional information use can support action selection
Some of the neural activity created by interacting with information (in either law based or convention based manner) is a representation
Some neural representations require continuous information to work; these can support action control and action selection
Some neural representations can be just triggered by information (if simple enough); this can support action selection
Analysis takes the first person perspective
Golonka & Wilson (2016). Ecological Representations

47. 1. Informational representations
“An entity X designates [represents] an entity Y relative to a process P, if, when P takes X as input, its behavior depends on Y. (Newell, 1980, p. 156) An ecological information variable designates a dynamical property to a process (e.g. behaviourally coordinating wrt that property) see Bechtel, 1998 for related argument re: Watts Governor Ecological information is a representation
Information can have content because of the intensional analysis so suck it Hutto
Golonka & Wilson (2016). Ecological Representations

48. 2. Neural representations
Neural activity (X) precipitated by ecological information (Y) is a representation if X designates Y to a process P (supporting the coordination of behavior with respect to a task relevant property) and allows P to exhibit action with respect to Y at a distance (the distance between perceptual systems and the rest of the body). Neural activity is only a representation to the extent that it preserves or systematically transforms the spatiotemporal structure of information variables
Golonka & Wilson (2016). Ecological Representations

49. Evidence for neural representations
Spatio temporal structure of neural activity correlates with time to contact information in optic array and strength of this relationship predicts performance and tracks development (van der Meer et al, 2012; van der Weel & van der Meer, 2009)
Neural activity in Broca’s area correlates with the speech envelope of text during silent reading (Margrassi et al, 2015)
Considerable behavioral evidence that variability in action tracks variability in information structure (Bingham, 2001, 2004a, b; Golonka & Wilson, 2012; Snapp-Childs, Wilson & Bingham, 2015; Wilson, Collins & Bingham, 2005a, b; Wilson, Snapp-Childs & Bingham, 2010)
Golonka & Wilson (2016). Ecological Representations

50. Using Information for Action Control
Action control: coupling to one or more information variables so that an action complements the current task dynamics Information used for action control must vary in task-relevant time with the task-relevant property
Golonka, 2015

51. Neural Representations of Information for Action Control
No convincing evidence that we can instantiate a neural representation of information sufficient to support action control unless the relevant information is present in the current environment (or was present recently enough to calibrate activity).
e.g., miming steering (Wallis, Chatziastros, Tresilian & Tomasevic, 2007)

52. Using Information for Action Selection
Action selection: select one action rather than another, parameterize how an action unfolds, begin a new task Information used for action selection must only be reliably detectable Information that is conventionally related to a task-relevant property can influence action selection (Golonka, 2015): Color red for stop light Consequences for action best explained by invoking a culturally, evolutionarily, experimentally, etc. constrained convention
Golonka, 2015

53. Neural Representations of Information for Action Selection
Can often instantiate a neural representation of information used in action selection
if we have an appropriate precipitating event
and if the structure of the information is simple, short, and/or well-practiced and stereotypical enough to have had a reliable functional effect on corresponding neural activity during learning.
Neural representations can impact action selection

54. Neural Representations and Inner Speech
For an experienced language user, word structure is simple, short, well- practiced and relatively stereotypical. The right precipitating event (e.g., reading a sign saying “Danger: Bear Inside”) can reliably instantiate a neural representation of the acoustic information caused by pronouncing these words. The result is that we “hear” the words in our heads (e.g. Breen & Clifton, 2013) and can select appropriate action.

55. The Analysis
Identify the information being used to coordinate and control behaviour
What kinematic structures are being created by the local task dynamics?
Is the information being used to coordinate behaviour with respect to that event?
Yes: action control or action selection, law-based information use
No: action selection, law-based or convention-based information use
What about the behaviour remains to be explained?
What neural activity is being created by information created by the local task dynamics?

56. Responding to Verbal Instructions
Behaviour to be explained: when told to ‘pick up the glass’, you reach and grasp and lift the glass
Information present:
Auditory information carrying the words ‘pick up the glass’ created by dynamics of speech system
Visual information about the prehension affordances of the glass created by that dynamical property
Informational Explanation of Behaviour
Selecting action to grasp that glass explained by conventional meaning of auditory information
Execution of the prehension action controlled by law based meaning of the visual information

57. Creating Those Verbal Instructions
Behaviour to be explained: producing the phrase ‘pick up the glass’
Information present:
Auditory and proprioceptive information about the behaviour of the dynamics of the speech system
Informational Explanation of Behaviour
Production of the sentence controlled by law-based meaning of the auditory and proprioceptive information

58. Selecting Those Verbal Instructions
Behaviour to be explained: selecting the phrase ‘pick up the glass’ to produce
Information present:
Auditory information and associated neural representations about the unfolding sentence
Informational Explanation of Behaviour
Selection of the sentence driven by the conventional meaning of the information

59. These create informational differences which can be detected in PLDs
Social Deception
Perception of the pencil is appropriate, given how the local dynamics have structured light
Deception can have dynamical consequences (e.g. pretending to lift a heavy weight vs actually lifting that weight)
These create informational differences which can be detected in PLDs

60. Talking About Imaginary Things
Typical framing: Language as reference
Talking about imaginary things is complicated
If I say ‘unicorn’ what am I referring to?
Ecological reframing: Language as a tool (e.g. Bickhard, 2009; Everett, 2012) to select actions & influence action selection of others
Talking about imaginary things is simply a behaviour
Spend time in a culture (set of conventions) that uses ‘unicorn’ in particular contexts (constrains actions selected)
Asked to describe a unicorn, select action ‘horse with horn’

61. Other examples
Discussion time to apply the analysis to other problems

62. Ecological information is real and we can’t ignore it anymore
Take Home Message
Ecological information is real and we can’t ignore it anymore
It’s our only point of contact with the dynamical world and behaviour is only functional with respect to that world
BUT
Fortunately information is rich and meaningful
Tools: task dynamics, kinematic information created by those task dynamics, perturbation studies, law-based vs convention-based information use, action control vs action selection, ecological neural representations

63. References
Why Dynamics? Bingham, G.P. (1995). Dynamics and the problem of visual event recognition. In Port, R. & T. van Gelder (eds.), Mind as Motion: Dynamics, Behavior and Cognition, (pp ). Cambridge, MA: MIT Press. Wilson, A. D., & Bingham, G. P. (2001). Dynamics, not kinematics, is an adequate basis for perception – Commentary on Shepard (2001). Behavioral and Brain Sciences, 24(4), From Dynamics to Kinematics Bingham, G.P. (1988). Task specific devices and the perceptual bottleneck. Human Movement Science, 7, Turvey, M. T., Shaw, R. E., Reed, E. S., & Mace, W. M. (1981). Ecological laws of perceiving and acting: In reply to Fodor and Pylyshyn (1981).Cognition, 9(3),

64. References
Coordination Model Bingham, G. (2004). A Perceptually Driven Dynamical Model of Bimanual Rhythmic Movement (and Phase Perception). Ecological Psychology, 16 (1), Snapp-Childs, W., Wilson, A. D., & Bingham, G. P. (2011). The stability of rhythmic movement coordination depends on relative speed: The Bingham model supported. Experimental Brain Research, 215, Scaling Up via Conventional Information Use Golonka, S. (2015). Laws and conventions in language related-behaviours. Ecological Psychology, 27(3), Golonka & Wilson (2016). Ecological Representations (

65. Notes from Two Scientific Psychologists http://psychsciencenotes
Notes from Two Scientific Psychologists Cognition In Action (CIA) Lab

66. Neural representations function as components in conceptual systems
Developing a concept requires experience with multiple individuals of a type
Neural representations reflect repeated exposure to information specifying properties of a given type
Neural activity may achieve sufficient stability to be re-instantiated absent this information
Such stable neural activity is a representation of a type of property

67. Neural representations support componential systems
Neural representations are components –bits that can participate in a number of events made up of other bits
short, stereotyped
represent particular properties, not holistic
Neural representations can impact action selection
Some of these actions can be the instantiation of further neural representations
Particulars of action selection will reflect both the learning history and current task context – combination of neural representations is flexible and adaptive