Visuospatial Representation Spatial Knowledge, Imagery, Visual Memory

Representation What is a representation? Four aspects of representation The represented world The representing world Set of informational relations on how the two correspond Set of processes that extract and use information from the representation

Meaning Mental representations are carriers of meaning In order to interact appropriately with the environment we represent info from it and manipulate those representations Correspondence Meaning derived from how representation stands in consistent relation to the represented world Conceptual Meaning determined by relations to other representations

Spatial Knowledge How we represent and use spatial information Separate from strictly verbal knowledge Semantic propositions Dependent on the linear dimension of space.

Spatial Cognition How is the representing world like the represented world? The represented world is a space The representing world is a space What kinds of processes might be involved?

Space as a representation Spatial representation Representing world is a space. What is a space? Geometric entity in which locations are specified relative to a set of axes Dimensionality defined by the number of axes that can point in independent directions Of interest is the distance between items, which can be measured in different ways Euclidian Straight line Non-independent dimensions Saturation and brightness City-block Distinct dimensions Color and size

Space as a representation Physical world experienced (at least perceptually) has three dimensions (+ time) However, the representing world is not confined to any number of dimensions Represented world does not need to be spatial Conceptual info can be represented spatially More on that later

Spatial Representation Analog representation Representation mimics the structure of the represented world Multidimensional scaling Propositional Abstract assertions regarding the state of the represented world Not tied to a particular sensory modality

Multidimensional Scaling (MDS) Mathematical technique for taking a set of distances and finding the best-fitting spatial configuration that corresponds to those distances Input: a distance or proximity matrix that describes how close every object in a set is to every other object N objects are represented by N(N-1)/2 numbers (distances) Output: a geometric representation where every object is represented as a point in D-dimensional space Each object is represented as a point in space N objects are represented by ND numbers (coordinates) Purposes of MDS Give psychological interpretations to the dimensions Reveal the dimensionality of a data set

MDS Difficult to get a sense of relative distance by means of this information

MDS MDS recovers absolute original locations for the objects from the distances Flipping on horizontal axis would give us a rough approximation of NSEW Analog representation

Propositional Representation (A,B) 10 miles east (E,C) 20 miles south, 10 miles east (F,D) 10 miles south, 10 miles west

Analog vs. Propositional Good for configural info Easy incorporation of new info Propositional Time-consuming Lots of info must be represented E.g. one point added may require many propositions Allows for communication of spatial knowledge and incorporation of additional information not related to distance Going south on I35, one must pass through Denton to get to either Fort Worth or Dallas

Cognitive Maps Where is Seattle? Where is Terrill Hall? Large vs. small-scale space Hierarchical representation

Small vs. Large-scale space Maps of small-scale (navigable space) Cognitive geography Maps of large-scale space What is our sense of the locations of items in the world?

Small scale space Survey knowledge Route knowledge Bird’s eye view (map knowledge) Good for global spatial relations Easy acquisition Not so great for orientation Route knowledge Gained from navigating through the environment Locate landmarks and routes within a general frame of reference Landmark knowledge Salient points of reference in the environment More difficult to acquire but better for navigation in irregular environments May lead to survey knowledge Perhaps a different type Cognitive collage vs. orientation free

Large scale space Which is farther north: Denton, TX or Chicago, IL? Portland, OR or Portland, ME? Hierarchical representation of locations

Hierarchical representations Relative locations of smaller regions are determined with respect to larger regions. States are superodinate to cities, countries superordinate to states USA is south of Canada Maine is just south of Canada Oregon is well south of Canada Oregon must be south of Maine Cities in Oregon must be south of cities in Maine In this case such cognitive economy works against us Portland OR is north of Portland ME

Hierarchical representations Judge relative position of cities (Stevens and Coupe) When superordinate info congruent with question, performance better Is x north of y when one of right side maps presented

Hierarchical coding Huttenlocher & Hedges Category-adjustment model Combine info across hierarchical levels If info at subordinate is known with near certainty, there is no appeal to categorical info If info at subordinate (fine-grained) levels is at all uncertain, people use categorical info in estimation Bias toward center of category Bayesian approach utilizing prior knowledge Gist: errors in estimation are due to categorization rather than nonmetric spatial relations

How are maps learned From descriptions From navigation Taylor & Tversky: People learned maps from survey and route descriptions From navigation People can assess distance and direction traveled Integration of information Visual information Vestibular information Maps formed from video games are less accurate than maps in which people really move Rotation is particularly important

Using spatial cognition Adaptive context Locating and way finding Tool Use Mental rotation vs. mental movement Symbolic representations of space Drawings, maps, models Language Thinking

Adaptive context Locating and way finding Consider Hatchling sea turtles finding the sea Salmon finding way back home However these are more behavioral instinct and imprinting than pure navigation Desert ant finding direct route home after meandering paths in featureless environment Marsh tit stores seeds in holes in a hundred various places for later retrieval

Locating and way finding Ego-centered systems Environment-centered systems Hierarchical coding

Locating and way finding Ego-centered system Location of objects coded relative to self Updated as we move through the world Nonconscious Rieser, Guth, Hill (1986) Participants asked to point out previously learned locations in unfamiliar room after blindfolded and led along path Did not matter whether previously told which location they’d be asked about, suggesting attentional focus did not assist in the process Problem: may not always be accurate over larger distances without detailed environmental information

Locating and way finding Environment-centered system Object location coded in relation to stable features of the environment Requires feature-rich environment providing info to dominate sense used by organism If conditions met, then superior to ego-centered Allows for rechecking of position (no drift from accumulation of small errors) Works better for retaining info over long periods of time

Cognitive maps Both humans and animals display errors in judgment that cast doubt in positing a true ‘cognitive map’ Animal studies suggest approximation of distance from a single landmark Humans make many errors in spatial judgments that suggest no real metric representation Distance from A to B judged different from B to A Though again this sort of distortion may be related to categorization (hierarchy)

Tool use Making, using and designing tools for interaction with the environment involves cognitive processes such as mental rotation and imagery for success

Shephard & Metzler (1971)

Mental rotation vs. Mental Movement Logically equivalent However evidence suggests that mental rotation and perspective-taking/mental movement are psychologically distinct Selection task Which these arrays/models would be the correct view from over there? Item question What object would be nearest to you if you were over there? Specify frame of reference of relative to the observer

Mental rotation vs. Mental Movement Selection task Piaget Kids (< 10) not so hot at such a task Usually pick egocentric view Huttenlocher & Presson They do much better when asked to do mentally rotate Can also physically move to new location that matches a particular array Suggests conflict between current physically present perspective and the new (imagined) one they are trying to obtain MR allows them to stay put in the physically present room Physical movement physically transforms that perspective

Mental rotation vs. Mental Movement Item questions If kids do not move item questions help (even as young as 3) Again, this helps them maintain that egocentric perspective If asked to mentally rotate, item questions can actually hurt performance compared to selection tasks It may be that in item questions, whole array must be rotated to determine object relations vs a simple ‘rotation’ of the person or single object in selection task Gist: mental rotation and mental movement can be differentially affected depending on the nature of the question asked, suggesting there may be different underlying processes involved

Drawings maps and models Spatial learning from maps differs from learning by means of navigation Map learning may aid configural knowledge and allow for better estimates of distance between points while navigational learning allows for better route distance estimation and location of unseen points Recall survey vs. route knowledge Orientation-specific vs. orientation-free learning Studies show evidence that navigational learning is more a collection of multiple views than orientation-free, though may lead to a sort of orientation-free type of knowledge Sholl & Friedman

Spatial Language Contrasting experience with communication Experience spatial relations continuously, but language is usually discrete (e.g. near vs. far) Spatial terms function much like other categories (e.g. fuzzy boundaries, prototypes) Experience multiple spatial relations simultaneously, but speak of one relation at a time A frame of reference must be agreed upon in order to communicate spatial relations

Spatial Language Despite the difficulties in communicating spatial knowledge, ambiguities are generally overcome and information encoded (survey, route knowledge) However it does seem that spatial language may bias or constrain spatial representation, and may even affect the development of spatial concepts and categories Even so, the actual link between spatial language and spatial representation is not entirely clear Impaired sight individuals may have difficulties with a variety of spatial tasks but have intact spatial language

Thinking Spatial cognition also contributes to logical reasoning, metaphor, and creativity Transitive reasoning A > B, B > C A ? C Metaphor The future stretched in front of them My heart is a flame turned upside down Structural alignment of spatial and temporal concepts Diagrams as aids to understanding Show conceptual similarity of items, connections amongst various concepts etc. Creativity E.g. visualization for problem solving Taking someone else’s point of view?

Imagery Some information in memory is purely verbal Who wrote the Gettysburg address? Other memories seem to involve mental images Trying to recall a procedure Making novel comparisons of visual items What is a mental image? How are mental images represented and processed? Are mental images like visual images?

Evidence for use of visual imagery Selective interference Segal & Fusella Imagery interferes with detection of stimuli (sensitivity decreased) Auditory imagery interfered with auditory detection, visual imagery with visual stimuli Manipulation of images Mental rotation studies

Evidence for use of visual imagery Kosslyn Learn a map Mentally travel from one point to another Measure time to make this mental trip Results Time to make trip increases with distance Times increase with imagined size of the map.

Evidence for use of visual imagery Moyer 1973 Subjects were given the names of two common animals and asked to judge which was larger Which is larger, a moose or a roach? Wolf or Lion? The time delays as a function of size difference were similar to those usually found for perceptual judgments.

Kosslyn RT True False Kosslyn 1975 Scenario I: Imagine an elephant standing next to a rabbit. Does a rabbit have a beak? Scenario II: Imagine a fly standing next to a rabbit. Does a rabbit have an eyebrow? People made faster judgments when relying on a larger mental image (such as the rabbit next to the fly) than when using a smaller mental image (such as the rabbit next to an elephant) Kosslyn suggested that the size of an image is an important factor in determining how fast we can make judgments about it. RT Elephant Fly True False Inconsistent Consistent

Paivio's Dual-Coding Theory Information is mentally represented either in a verbal system (propositional) or a nonverbal (analogical) system (or both). Each system contains different kinds of information. Each concept is connected to other related concepts in the same system and the other system. Activating any one concept also leads to activation of closely related concepts.

Paivio The hypothesis of multiple codes (verbal and spatial) is based on the demonstration of independence of effects. Pictures of objects Words of objects

Paivio (1975) compared reaction times for consistent and inconsistent visual stimuli If the stimuli are processed semantically, there should be no difference between consistent and inconsistent presentations. If stimuli are processed spatially, inconsistent stimuli should require a mental conversion to appropriate size. Which takes time

Consistent

Inconsistent

Results “Which is larger?” RT Inconsistent Consistent

Paivio Consistent RABBIT FLY

Paivio Inconsistent RABBIT ELEPHANT

Paivio RT Congruity Effect only for Pictures (not words) Picture Words Inconsistent Consistent Congruity Effect only for Pictures (not words) Imagery relies on perceptual detail and semantic does not Such findings as this and picture superiority effect (pictures are better recognized than words), and that verbal + imagery encoding leads to best recall, suggest a Dual Code Theory

Santa 1977 More evidence of dual coding Ss presented array of objects or words On test presentation asked whether the elements were same as studied E.g. In geometric condition first two would be yes responses

Santa 1977 Results of positive responses Spatial configuration is preserved in geometric encoding Compared to verbal presentation, which was encoding in typical English reading style and benefited from the linear configuration

Representation of images What is the relationship between imagery and perception? Can imagining interfere or facilitate detection of stimuli? Similar processes involved?

Spatial Knowledge Symbolic Distance Effect (Moyer 1973) Process of imagery = process of perception As perceptual distance increases so does psychological distance (RT). Items “farther apart” are more quickly distinguished Which is larger? Rabbit-Elephant Rabbit-Dog

Representation of images Contrary evidence Chambers and Reisberg Images are (committed to) a particular interpretation E.g. The rabbit comes once drawn but was only a duck as imaged Contrast with perception which requires interpretation, images are already interpretations

Are visual images visual? Plenty of evidence to suggest a spatial component to visual imagery, but perhaps the visual part is represented propositionally Kerr Congenitally blind also take longer to imagine longer map routes like the one in Kosslyn

Are visual images visual? Images are also not as sharp as real pictures Form a mental image of a tiger Does it have stripes? How many? It is hard to examine details of mental images that would require eye movements

Making new pictures Finke, Pinker, Farah Example Imagine a capital letter H and a triangle Rotate the H 90 degrees Place the triangle on top of it What is it? Suggests images can take on new interpretations

Are visual images visual? Facilitation and interference (Farah) Have people imagine a letter (H or T) Present one of the letters to the screen briefly (20 ms), or present nothing, followed by mask Asked if they saw a letter People are more likely to detect the stimulus if it was the same as what they were imaging, suggesting that visual and imaginal representations joined or fed into the same process

Are visual images visual? Evidence from neuroscience Patients with lesions of visual cortex that lead to perceptual problems also have problems with mental imagery ERP evidence PET evidence: Visual imagery leads to activation of visual cortex. Auditory imagery does not In general, results of studies from mental rotation to brain imaging support the idea of both visual and spatial representation of images

Translating Words to images Franklin and Tversky Create a mental image based on the description Asked to identify location of items in that imagined environment Results are what one might expect given an imagined spatial environment Up-down, front-back more relevant in navigating real world Left-right confusion in real world and imagined world

Visual memory Although our visual memory seems to be excellent, it turns out not to be that great in many respects In general, our memory for details is lost, much like with other types of memory

Visual memory Memory for pictures is quite good generally Again, don’t get too detailed Standing Presented 10000 photos over several days Old-New memory over 80% Picture superiority effect Better memory for pictures than words