1. Eye Movements & Change Blindness: What do they tell us?
Ben J. Schlorholtz
Psyc 736 Eye Movements, Theory & Applications

2. What is Change Blindness?
The inability to notice changes to visual scenes, sometimes even very large changes (Rensink et al. 1997)
Flicker paradigm demo
An original image and a slightly altered image are presented consecutively with each image presentation separated by a brief, uniformly colored blank screen
Blank screen serves as a mask so that subjects cannot directly see the changing object between images
Believed to simulate the visual disruption which takes place during a naturally occurring saccade or blink
1 explanation for change blindness is that the mechanisms that generate an eye movement actively suppress perception during the eye movement (saccadic suppression). The rapid movement of the eye causes a blur on the retina and it might be that this blurring is what masks the change signal.
If it is this motion blur that masks the change, then other disruptions should mask them as well. Hence the Flicker paradigm
Detection is improved for objects of central interest compared to those of only marginal interest.

3. Change Blindness examples
Failure to detect changes to actors of central interest in …
short films (Levin & Simons, 1997)
real-world conversations (Simons & Levin, 1998)
Perhaps the richness of the visual world is not as “detailed” as once thought

4. The missing link…
How rich are our mental representation across fixations?
Early studies simulated visual disruptions caused by blinks and saccades, BUT
There was no way to tell where the eyes were actually fixated
Perhaps changes were missed because people weren’t looking at the immediate area of change?
Remedied this by recording eye movements
Eye movements tell us not only where people look, but also potentially to what characteristics of the scene are considered when looking for changes

5. Grimes (1996)
Can results from EM and reading studies be extended to more naturalistic scenes
McConkie & Zola (1979) – subjects were practically oblivious to changes made during saccades, but reading accuracy was maintained
Implemented a procedure similar to the flicker paradigm, used a Dual Purkinje Image Eye Tracker, and a saccade contingent paradigm
Subjects viewed 2-D magazine photographs for 10 secs and were instructed to…
Inspect each photo for a later memory test and report any changes detected while viewing
During a single saccade, a change occurred and remained for the duration of the viewing period
McConkie & Zola (1979) – while subjects read sentences containing alternating upper and lower case letters, the case of the letter changed each time a saccade occurred, subjects were oblivious but reading accuracy was maintained

6. Grimes (1996)
Initially thought part of procedure was faulty because practically no one reported any changes, but after checking all of the apparati, it became evident that subjects simply were unaware that changes were occurring
Examples of changes included two men changing hats of a different style, a large parrot changing from bright red to bright green, 2 cowboys exchanging heads, a child enlarged by 30% in a playground scene, etc…

7. Grimes (1996), Results
80% of photos changed however subjects only reported that 10-30% changed
Because EM’s were recorded, Grimes was able to separate out those who never fixated the object and those who did
For those that did fixate the object before and after a change, most were still oblivious to any changes (note: didn’t report any statistics)
Results were consistent with previous EM & reading literature
suggested that the visual detail retained across saccades was minimal
Grimes suggests that it’s simply more economical to mentally represent the overall object information rather than intricate details.
If more details are needed, one can simply re-fixate the object for deeper analysis
Only reported percentages
Able to show this phenomenon at conferences on projectors and it worked!

8. McConkie & Currie (1996)
Common criticism of the visual perception literature is the overuse of ecologically valid stimuli
Could results from research using simple dot patterns be extended to more naturalistic, color photographs
Ultimate goal was to determine if detection of changes made during saccades was the result of specific, local info acquired from the saccade destination; or the result of more global info regarding the mere position of the entire image

9. McConkie & Currie (1996)
Conducted a series of experiments using a fifth generation Dual Purkinje eye tracker (saccade detection within 10ms of initiation)
Instructed subjects to inspect scenes for a later recognition test and report any changes believed to be occurring

10. McConkie & Currie (1996)
In Exp. 1, the entire image was displaced either horizontally or vertically during predetermined saccades
Only finding replicated from earlier studies using simple stimuli was a decrease in detection as a function of increasing saccade distance
Results suggest that perhaps our perceptual system is working differently than we were led to believe by early research which used very simple dot and line stimuli
f(s) is the likelihood of detection,
A is a parameter indicating the intercept of the function,
K is a parameter controlling the rate of change, and
s is the saccade length
Also found that when the image shifted in the same direction as the current saccade, detection probability increased

11. McConkie & Currie (1996)
In Exp 2, images now increased and decreased in overall size by 10 or 20%
Results suggest that the larger the change in image size, the increased likelihood of change detection
Overall, the authors conclude that findings where the result of inconsistent local information expectations.
Go on to suggest the saccade target theory which states that the visual system assumes consistency in the world during saccades. Changes are detected when there are any discrepancies between this assumption and the saccade target.
But with increasing saccade distance, the variability of the target area also increases
Important to the CB lit in the sense that they provide an explanation for visual stability across saccades using more complex stimuli than traditional visual perception research
When a saccade is launched to a new position and the image changes size during this ballistic motion; the more discrepant the landing eye position is from the initial target which was selected, the more likely change detection is to occur
Changes are less likely to be detected because there is simply a larger, more variable target area within which a given saccade will land
These results are important to the CB lit in the sense that they provide an explanation for visual stability across saccades using more complex stimuli than traditional visual perception research

12. Hayhoe, Bensinger, & Ballard (1997)
How do task demands influence fixation durations during performance of a sensorimotor task?
Subjects have 3 distinct areas to monitor and choose from
Model Area
Resource Area
Workspace Area
Task is to inspect the model and recreate it in the workspace using a computer mouse to drop-and-drag blocks from the resource area
Model area – displays a block configuration that the subjects must recreate
Resource Area – contains a variety of colored blocks from which the subject must choose
Workspace area – is the space subjects use to recreate the model

13. Hayhoe, Bensinger, & Ballard (1997)
Capitalized on stereotypical EM’s
Model – Resource – Model - Workspace
A single blocks color was changed during 2 saccade conditions
Before Pickup
After pickup
Before pickup – changed the color of a block in the model area before the pickup of the next block. This color change occurred as the subject saccaded from the workspace area to the model area directly following the previous blocks placement in the workspace. If color information in the model is maintained through peripheral observation or memory from previous fixations, these changes should be noticed and evidenced by increased fixation durations
In the After pickup condition, a block in the model area changed color as the subject saccaded from the resource area back to the model area prior to its drop off in the workspace. In other words, the color of the block just selected by the subjects becomes inconsistent with the model thus allowing the authors to observe any interference that might occur.

14. Hayhoe, Bensinger, & Ballard (1997)
In the Before pickup condition,
If color information in the model is maintained through peripheral observation or memory from previous fixations, these changes should be noticed and evidenced by increased fixation durations
In the After pickup condition,
The color of the block just selected by the subjects becomes inconsistent with the model thus allowing the authors to observe any interference that might occur.
Because the manipulations occurring in each condition involve the exact same block changing color, the authors were…
Able to observe how the different task demands of each condition affect performance for identical fixation positions

15. Hayhoe, Bensinger, & Ballard (1997)
For the Before pickup condition, critical color information was not encoded prior to fixation
But for the After pickup condition, the color of the block being held was retained in memory to confirm its final location
Changing the color disrupted performance
Results show that changing the color of the block before pickup did not significantly increase fixation duration compared to a control group (127 msec)
However in the After Pickup condition, changing the color of the block increased fixation durations by 275 msec
Fixation duration on the y-axis

16. Hayhoe, Bensinger, & Ballard (1997)
The ability to detect a change at identical fixation positions depends upon the immediate demands of the task at hand
The fact that we can detect a change in one circumstance, but not another lends evidence to the claim that our visual system does not represent everything mentally, but rather only those items which are immediately necessary

17. What is lacking thus far?
Up to this point in the CB literature, the verification of fixation position relative to the changing targets had not been adequately controlled for
Henderson & Hollingworth (1999) attempted to control for this by introducing changes which were entirely dependent upon the current fixation position

18. Henderson & Hollingworth (1999)
Using a dual Purkinje image eye tracker (1000Hz), changes were made to scene target areas in the following three conditions
Toward Condition
Object change occurred during the first saccade in which the eyes landed on the target
Away Condition
Object change occurred during the first saccade away from the target after it had been directly fixated
Control Condition
Object change occurred during the first saccade to a non-target area of the scene
2 types of target changes were also included
Rotation Condition
90 degrees on vertical axis
Deletion condition
Objects disappeared completely
Computer rendered color images of naturalistic scenes were changed contingent on a saccade toward or away from a prespecified target region
When the eyes crossed an invisible boundary during a saccade, the target object changed

19. Henderson & Hollingworth (1999)
Subjects were instructed to inspect each picture for a later memory test and respond immediately if any changes were detected
Hypothesized…
If fixation position is not involved in change detection as implied by the non-EM related CB lit, change detection rates should be equivalent among the 3 saccade conditions
BUT, if information is retained across previous fixations, detection performance should be better in the Away condition compared to the Towards and Control conditions, but this increase in performance should also decline as a function of saccade distance from the target
CB is CB and it will occur regardless of whether or not you are saccading towards or away from the object

20. Henderson & Hollingworth (1999)
Poor change detection for Control condition
Suggests that the visual system does not create a global image representation by combining info from previous fixations
Many changes were not detected until re-fixation of the target region
Suggests that some info was maintained across saccades, but was inaccessible until eyes re-fixated the target
Full bars represent all detections
Lower sections represent immediate detection (<1500msec)
Detection was best when a target object was immediately present prior to a saccade to that object, goes against their prediction of Away condition being optimal
The fact that deleted objects were detected more often than rotated objects suggests that our perceptual system is wired for specific features
Some trials were eliminated due to tracking loss 11%

21. Henderson & Hollingworth (1999)
Graph combines the object type changes with the saccade conditions and looks at how eccentricity effected probability of detection
Suggests that some info with regard to an object being present in an upcoming saccade target was being encoded, people could detect this with high accuracy
For object Deletions, as subjects saccaded away from target a steady decrease was apparent with increasing saccade length
Strikingly, this was not the case for saccades made towards the target area

22. Henderson & Hollingworth (1999)
For saccades toward the target, the fact that detection rates were equivalent for 1-8o suggests that the visual system is encoding certain information from upcoming saccade destinations.
This is also supported by low detection rates for both the saccades towards and away in the rotation condition
E.g. moderate changes in rotation do not appear to be encoded
Our visual system does not appear to recreate a global representation of scenes across saccades, but certain info is encoded and retained
Bullet 2 – data suggest that rotational info was not a critical feature that the visual system was detecting
However noticing whether or not something is or isn’t present does appear to be encoding
This could have evolutionary implications – monitoring for predators, need to know if they are there not if they are rotated slightly or not

23. O’Regan et al. (2000)
Can the changes people miss be replicated during blinks?
Blinks range from ms while saccades range from 20-70ms
Also, to what extent does attention to certain scene aspects affect the rate of change detection?
Operationalized attention by using 5 judges which rated objects in scenes
Central Interest object if chosen by 3 judges
Marginal Interest object if rated by none of the judges
Twas believed that by making changes to scene components in each interest category, they could manipulate the level of attention to certain objects
Objects of both interest groups were of comparable size

24. O’Regan et al. (2000)
Changes occurred during each blink, unbeknownst to the subject
Subjects were told to respond as soon as a change was detected
Used a dual Purkinje image eye tracker to determine how distant the eyes had to be from a change before it was missed
48 digitized color photographs depicting a variety of indoor and outdoor scenes

25. O’Regan et al. (2000), Results
These graphs show how similar EM’s were for both Central and Marginal Interest objects, durations were the same and frequency relative to saccadic amplitude were practically equal
Results were consistent with previous research, objects of Central Interest were detected more often than those of Marginal interest
Also, probability of change detection decreases as a function of eye position relative to the changing object

26. O’Regan et al. (2000), Results
White dots = CI Black Dots = MI
Distance of the eye’s landing positions from the change location, at the moment of the change cumulated over all subjects
It plots the distance of the eye from the change location at the moment of occurrence of each blink
eye’s location tended to be closer to the change location when in it was a CI location that when it was a MI location
The same as A, but only for those blinks that were immediately followed by detection of the change
How far away were the eyes located when a change was detected
Use the info from these 2 graphs to form
Authors used the info from these 2 graphs to form a ratio
# of changes detected after a blink at given eccentricity
total # of changes/blinks at a given eccentricity

27. O’Regan et al. (2000), Results
Prob of detection as a function of eye position relative to the object
For eccentricity >2o, detection declines to only 10% and remains consistent
“seeing without looking”
BUT, prob of detection for direct fixation of changes was only 60%
“you don’t always see where you look”
Authors suggest that there must be other aspects of this fixated region to which attention is being allocated
Ratio of effective to all blinks, i.e. proportion of changes that were detected as a function of the distance of the eye from the change at the moment of the change
Probability of seeing without looking is not very good
We miss 40% of the changes we were directly looking at!!!
It appears that often times we will miss very large changes to scene object, even when we are looking directly at them

28. Hollingworth, Schrock, & Henderson (2001)
Wanted to further examine the original CB studies by measuring EM’s
Conducted a series of experiments using a Flicker paradigm and a Dual Purkinje eye tracker to ask,
“How does fixation position affect change detection performance?”

29. Hollingworth, Schrock, & Henderson (2001), Exp 1
Subjects viewed naturalistic scenes and target objects were either
Deleted, then added
Rotated 90o
If detection is influenced by current fixation position, performance should improve the closer the eyes get to the target
Separated the scene into 4 visual regions
Target object region
Smallest circle around the object
Foveal region
1o ring around the target object
Parafoveal region
2.5o ring around the target object
Peripheral region
All other parts of the scene
Subjects were told that on some trials a change would occur to a single object
Their task was to press a response button immediately upon detecting a change in the scene

30. Hollingworth, Schrock, & Henderson (2001), Exp 1 - Results
When the object was detected, the eyes were much more likely to be directly fixating it (74.5% across change conditions) compared to all other regions
At the moment of change detection, % of fixations directly on the target for Rotated objects (85%) was sig. higher than for Deleted/added objects (63%)
This suggests that Rotated objects were more difficult to identify and therefore required an increased # of direct fixations
Once again, rotating objects does not appear to be a basic level feature that we encode quickly

31. Hollingworth, Schrock, & Henderson (2001), Exp 2
Previous research has shown that visual attention is allocated to upcoming saccade target locations and thus is not entirely attributed to current fixation
Authors wanted to separate visual attention from current fixation position. How?
Used 2 conditions
No-movement
Maintain central fixation while searching for changes
Movement
Free movement of eyes to search for changes
If active fixation of a target is the single factor influencing detection performance, the no-movement condition should be practically impossible

32. Hollingworth, Schrock, & Henderson (2001), Exp 2 - Results
Subjects performed better in the Movement compared to the No-movement condition
More accurate (99% vs. 91%)
Quicker response latency (3.7sec vs. 6.3sec)
Fewer false alarms (0% vs. 18.1%)
Suggests that fixation position does play a significant role in change detection
BUT, the fact that performance was still well above chance suggests that fixation position is not the only critical factor utilized in flicker paradigm studies
For future studies, it’s imperative that fixation position be sufficiently controlled for
How else will you know if they have fixated the object if you don’t measure EM’s???

33. Hollingworth & Henderson (2002)
What types of info are preserved from objects that have previously been fixated?
There are some discrepancies between the CB lit and the Picture Memory lit
This area states that viewers are capable of encoding a substantial amount of visual info into a LTM store

34. Hollingworth & Henderson (2002)
Used a dual Purkinje eye tracker
Instructed subjects to inspect each scene for an upcoming memory test and press a button as soon as they detected a change
In this study a LTM test was actually conducted
3 change conditions
Change after fixation
Change did not occur until the subject fixated the object, then saccaded away to a certain region
Change before fixation
Target object changed immediately prior to fixation
Control
No change occurred
LTM test – 5min delay between study and the test
Memory was tested for the 12 scenes appearing in the control condition
Subjects saw 2 scenes sequentially (the actual and a distracter)
Viewed each scene for 8sec and press a button indicating which scene was the actual one they inspected
Distractors included both Type and Token objects

35. Hollingworth & Henderson (2002)
Participants began by fixating the center of the screen
In the change after fixation condition, the computer waited until the eyes had dwelled in the target object region A for at least 90msec
Then the change triggering region B was activated, and as the eye crossed the boundary to this region B, the change was initiated
In the change before fixation condition, the computer waited until the eyes left the central region C before activating the change triggering region B and the change was initiated as the eyes crossed the change triggering boundary
These region boundaries were not visible to participants

36. Hollingworth & Henderson (2002)
To examine the type of information retained across saccades, objects changed via…
Type
Notepad changed to a floppy disk occupying same space
Token
Notepad changed to a spiral notebook
If the visual system maintains object representation after attention is withdrawn from that object, subjects should be able to detect both Type & Token changes

37. Hollingworth & Henderson (2002) Exp 1 - Results
Higher detection in the “Change After” compared to the “Change Before” fixation conditions
Type changes were detected sig. more than Token changes
Change detection appears to be highly dependent upon previous fixations
This might explain why detection performance was so poor in the original CB studies
The fact that the changes were not deleted/added might explain why performance was low in the change before fixation condition. The object remained, but only changed in its immediate properties

38. Hollingworth & Henderson (2002) Exp 1 - Results
For the fixation duration data, sig. Positive correlation for the Token change detection
Influenced not only by previous fixation, but also by how long the target was fixated
For the Change After condition, 93% of fixations occurred upon re-fixation of the target
Authors suggest that re-fixating a target might serve as a trigger for retrieval of previously attended info
Poor performance in original CB studies might have been due to lack of re-fixation
LTM tests
93% correct in the Type condition
80% correct in the Token condition
Results of LTM test suggests that in certain contexts, our visual system might be retaining more info across fixations than once thought.
A finding that is consistent with the picture memory lit

39. Hollingworth & Henderson (2002) Exp 2
Used only the Change After Fixation and Control Condition
Objects changed with respect to…
Token qualities
Rotational qualities
Only affects visual appearance and not object meaning
The ability of subjects to decipher between different rotational positions on LTM test would offer support for the maintenance of purely visual qualities in memory across fixations

40. Hollingworth & Henderson (2002) Exp 2
Results were consistent with Exp 1
Rotated objects were detected with just as much likelihood as Token changing objects
Purely visual information was retained after attention was distributed else where in the scene
Is it really?
Re-fixation of the target object sig. improved detection performance
The fact that subjects were able to detect both token and rotational qualities lends evidence that on some level, purely visual information was retained across fixations after attention had been allocated elsewhere
The fact that performance was only around 30% does not provide a very strong argument that the representations formed during online scene percpetion contains visual information from previously attended objects
This is the problem with a lot of the CB lit
Modest change detection performance is typically interpreted as evidence for the absence of representation
LTM performance was around 80% for both groups

41. Hollingworth & Henderson (2002) Exp 3 – Forced Choice Procedure
Wanted to test the representation of previously attended objects during online perceptual processing of a scene.
Evidence from Exp 1 & 2 that online detection performance might not be a true indicator of scene representation based on the fact that detection was only moderate while LTM performance was well above chance
This method should eliminate the factors that could have caused the change detection tasks of Exp 1&2 to underestimate the detail of scene representation
Subjects weren’t told to study for a LTM test, all they had to do was perform the discrimination task
After subjects fixated the target region they were cued to saccade towards an arbitrary non-target region.
Panel 1: Initial scene image (regions depicted were not visible to subjects). Participants began by fixating the center of the screen. The computer waited until the eyes had dwelled within the target region A for at least 90 msec.
Subjects were then cued to Region B. As the eyes crossed this boundary, the target object was occluded by a salient mask (2). The mask remained visible until the participant pressed a button to begin the forced choice task.
After a delay of 500msec, the first object alternative was displayed for 4 sec (3), followed by a target object mask (4), the second target alternative for 4 sec (5), and the target object mask (6) which remained until response

42. Hollingworth & Henderson (2002) Exp 3 – Results
Change detection performance was highly accuarate
Performance did NOT decrease as the # of fixations between target object and mask presentation

43. Hollingworth & Henderson (2002) Conclusions
Overall, data suggests that there is some buildup of visual information in some memory store that is retained across multiple fixations
Previous CB studies which used poor detection as evidence for a lack of representations across visual interruptions might not have been accurate
Visual representations do exist in some LTM store and it appears that re-fixating an object plays a significant role in detecting changes across scenes
Re-fixation may serve as a trigger for the system to reactivate certain memory stores specific to where attention had been distributed in previous fixations

44. Static vs. Dynamic CB
To this point, the stimuli used to investigate CB have been primarily static in nature (computer renderings, photographs, etc…)
Researchers have to wonder if the same types of findings would be found in more dynamic situations
Once such area could involve driving
“Looked, but didn’t see” accidents
Research to this point suggests that our ability to detect changes is not as good as we might think

45. Velichkovsky et al. (2002) 3 types of occlusion modes
Blinks, blanks, and saccades
2 viewing conditions
Static (still pictures of driving scenes)
Dynamic (driving simulator)
2 types of changes
Relevant or Irrelevant
(all were insertions or deletions)
Do the different occlusion methods influence detection performance under static and dynamic conditions
Used a SR-R Research Eyelink recorder
No hypotheses were provided

46. Velichkovsky et al. (2002) Results – Static condition
Detection performance was …
sig. higher for relevant changes (80%) compared to irrelevant changes (34%)
150msec faster for relevant changes
No differences were found among the occlusion methods
Insertions were detected more often and quicker than deleted objects
Potentially hazardous objects are usually those that suddenly appear, although this could be argued
RT’s were 225msec longer during an occlusion compared to no occlusion
This suggests that critical events which take place during a visual disruption may require 1/5 sec more time to react
Kid disappearing behind a parked vehicle
Because there were no differences among occlusion methods, it suggests that using screen blanks is a useful substitute for simulating blinks and saccades

47. Velichkovsky et al. (2002) Results – Dynamic condition
Due to equipment delays, authors were only able to observe changes occurring during fixations & blanks (i.e. simulated blanks and saccades
Not an issue based on results of static condition

48. Velichkovsky et al. (2002) Results – Dynamic condition
Results were completely opposite what would be expected given the CB lit
Detection performance was…
More accurate for blanks than fixations (95% vs. 90%)
Faster for blanks than fixations (666ms vs. 702ms)
The fact that false alarm rates were also low served as evidence to the authors that the blank screen stimulus was not serving as a cue “when” to look for a change
Further research is needed to verify this finding
All effects were sig but there were lots of trials so power and effect size could be an issue

49. So what do EM’s tell us about CB?
Observed CB using very simple stimuli and 2-D naturalistic images and photos
Task Demands appear to play a sig. role
Allocation of attention changes quickly
Saccading toward and away from a target object can impact our ability to detect a change
Tells us more about the relationship between impending saccade targets and the allocation of attention
Continue to find our ability to detect a change decreases as a function of increasing fixation distance from a target
There appears to be an attentional split between current fixation and the location of the next saccade target

50. In my own opinion,
CB occurs not because people simply don’t look at changing objects
In O’Regan et al. (2000) subjects missed 40% of changes which were directly fixated
It’s not so much about visually inspecting a scene, but rather where in the scene attention is deployed
Perhaps our perceptual system uses attentional thresholds to regulate what is and is not explicitly detected
for changes not explicitly detected, subjects looked at some changing objects of a given quality more than others (Hollingworth et al., 2001)
The next logical step
CB applications in more dynamic environments
Attention Thresholds
Perhaps on an implicit level, our visual system noticed these changes or something peculiar happening, but because attention did not reach a certain threshold explicit detection was not possible
Future research investigating the change blindness phenomenon might yield important results about how attention functions.
This could have considerable implications for the domain of driving where often times accidents are described as being of the “looked, but didn’t see” nature.
I think the next logical step is to investigate change blindness in more dynamic environments, more similar to those we actually experience in everyday situations.
Learning more about why certain object changes are or are not detected might help us understand to what environmental characteristics we are actually paying attention.