“Perceptual Symbol Systems” L. W. Barsalou (1999), Brain and Behavior Science, 22, 577-660. An overview of Barsalou’s Perceptual Symbol System theory James Sulzen June 6, 2001 Psychology 264 Gordon Bower Spring 2001

Terminology P-state - “Perceptual state” A combination of multimodal percepts constituting an experience or some aspect of perception of the real world. P-sym (or p-symbol) - “Perceptual symbol” Recalled or constructed subset of p-states which symbolically stands for a referent of some sort PSS - Perceptual Symbol System System of symbols and processes which operate on them to produce cognitive processes. Frame Simulation

Amodal Symbol Systems Amodal systems transduce p-states into amodal equivalents.

Amodal vs. Modal Issues Evidence: Little/no direct evidence for amodal systems Neuroscience & Psychology: Much evidence for modal processing Awkwardness: Certain computations are very amodally awkward (i.e. spatio-temporal) Transduction and symbol grounding: Amodal systems are essentially arbitrary. (Grounding/associating amodal representations back to perceptual ones just ultimately rates to make the amodal systems redundant.) Too much power: Amodal systems ultimately are too powerful: They can explain anything. Modal systems have a priori limitations (modal ones do not). Modal systems are open to falsifiability and provocative hypothesis

PSS Core Properties P-syms - Stand in as referents Frames - These combine p-syms & frames with relationships Simulators - dynamically bind other elements Language - Can stand for or drive other elements

PSS Core Properties P-syms Frames Have a neural representation / substrate Are schematic representations of p-states (see Fig. 1) Arise from repeated exposure [neuronal recruitment / exclusion] Are inherently multimodal (including proprioception and introspection) Selective attention operates to extract p-sym attributes Frames Organize p-syms (i.e., define primitive relationships - up/down, in/out) Combine together: Predicates Attribute-value bindings Constraints Recursion

Multimodal Organization of Knowledge Sensory registers V- vision G - Gustatory H - Haptic K - Kinesthetic & proprioceptive O - Olfactory A - Auditory L - Language S - Spatial E - Emotional Other systems certainly exist - Each modality can be thought of as a representational system - Each provides certain affordances

Frames Establishing an initial frame for car after processing a first instance. Evolution after processing a second instance. Creating a simulation of second instance from frame in B

Simulators Simulators Simulation Derived properties of simulators [This is an area requiring more development in the theory] Frames a set of p-syms, frames, & simulators to create higher-level structures Temporally dynamic integration of other elements Can be dynamically constructed, modified, componentized, and executed Comparable to mental models, schema, concepts, etc. Simulation This is the execution of a simulator Are always sketchy and incomplete - are never veridical Idealization occurs (i.e., Gestaltist principles apply) Example: Categorization - “if a category simulator can produce a satisfactory simulation of a perceived entity, then the entity belongs in the category” (p. 587) Derived properties of simulators Productivity: Can be combinatorially and recursively combined Propositions: Can be bound to individual entities [framing?] Variability: Implement variable embodiment Abstraction: Can combine physical and introspective events to represent abstractions

Simulator Productivity Object categories Spatial relationships Combinatorial productivity Recursive productivity

Prop- ositions Proposition representation (“the balloon is above the cloud”) Complex hierarchical proposition (balloon above cloud) Alternative proposition (balloon below cloud) simulators simulations Perceived situations

Language Can be linked any other element (p-sym, frame, simulator) Can be used to construct and control simulators [Language is probably the amodal basis of cognition and memory?]

PSS Symbol Manipulation Barsalou essentially shows that p-syms, frames, simulators, etc can: Be composed, associated, combined, subtracted, and so on They constitute a symbol manipulation system He does not demonstrate a formal correspondence between his PSS and an amodal symbol system (such as a semantic knowledge network) Such a demonstration would constitute a form of completeness proof Language as a representational system [and can serve the role of an amodal symbol system]

Abstractions Metaphor (anger => liquid exploding out of a container) Not adequate in & of itself to represent all abstractions Abstraction are constructed from three main elements Framing: Are framed against the background of a simulation Selectivity: Selective attention highlights the core content of the concept in the context of the simulation Introspective symbols (i-syms): These are central to representing abstract concepts (i-syms are the internal sense of an experience or internal perception) Methodology for identifying abstractions Find a frame that characterizes the abstraction Identify seeming p-syms and i-syms Identify the focal element that characterizes the abstraction

Truth & Falsity Mapping succeeds: “truth” Simulated event seq. frames the concept The abstraction is only a focal part of the simulation (i.e., the outcome that construes the concept) Introspective symbols are central to the construing of the meaning of the concept After many mappings, a simulator develops for truth Mapping fails: “falsity” After many mappings abstraction simulator becomes established

Anger Anger involves Core component: Blocked goal Similar to falsity Appraisal of an initiating event, and that the agent’s goal is blocked by the event Intense affective states Behavioral responses (i.e., disapproval, revenge, and redirecting goals) Core component: Blocked goal A goal is a simulated state that the agent desires to achieve A blocked goal is a failed mapping when it is expected to successfully map Similar to falsity A simulated situation fails to map to a perceived situation Except that affective states and behavioral responses are also associated with anger Lie A statement induces a simulation purported to be true that is actually not (i.e., simulation is negative in liar’s simulation, but false in the deceived’s) (P. 602)

Attempted reconstruction Disjunction Attempted reconstruction Original event Partial recall

Transformations

Ad hoc Categories