Selecting, Referring & Predicating: Basic ingredients of the mind-world connection

The functions of focal attention A central notion in the present analysis is the notion of “picking out” or selecting. The usual mechanism that is appealed to in explaining perceptual selection is attention (sometimes called focal attention or selective attention). Why must we select anyway? This is a rarely asked question to which there are several answers: We need to select because we can’t process all the information available. This is the resource-limitation reason. We need to select because certain patterns cannot be computed without first marking certain special elements of a scene We need to select because of the way relevant information in the world is packaged (Strawson’s Collecting Principles). It is a response to the Binding Problem We need to select because selection is a consequence of the first line of causal contact between mind and world: it precedes all conceptualizing and encoding

What does visual attention select? If attention is selection, what does visual attention select? One obvious answer is places. We can select places by moving our eyes so our gaze lands on different places. Must we always move our eyes to change what we attend to? Studies of Covert Attention-Movement: Posner (1980). How does attention switch from one place to another? When a place is selected, is selection automatic (exogenous) or voluntary (endogenous)?

Exogenous movements of attention Example of an experiment using a cue-validity paradigm for showing that the locus of attention moves without eye movements and for estimating its speed. Posner, M. I. (1980). Orienting of Attention. Quarterly Journal of Experimental Psychology, 32, 3-25.

Endogenous movements of attention

Exogenous & endogenous control of attention Attention shifted in exogenous and endogenous ways differs in a number of ways: Only exogenous attention shift leads to Inhibition of Return Automatic attention shifts are faster and the attention effects are stronger. Voluntary attention shifts can be interrupted by exogenous cues, so it is considered secondary to automatic control With voluntary attention control the person only knows which direction to move attention, so it may occupy intermediate locations

Exogenous & endogenous control of attention Attention shifted by exogenous and endogenous ways differs in other ways as well: With automatic shift, the apparent attention increase at intermediate locations can be explained by decreasing attention at the source and increasing attention at the target (Sperling & Weichselgarter, 1995). It is doubtful that there is attentional selection of empty regions – empty space does not have the causal power to attract exogenous attention and voluntary control is special (also some doubt that voluntary movements are continuous – Pylyshyn & Cohen, 1999) If attentional selection is to play the role of initial nonconceptual contact between mental representations and the world, it must be exogenously driven attention – the world must impose itself on the perceptual system.

Evidence that attention is object-based Although the earliest evidence showed that attention moves through space (covert movement) there is now evidence that attention attaches to “objects” as a whole The main source of evidence initially was based on same object superiority

Single object superiority even when the shapes are controlled Pay attention to the blue object. Which vertex is higher, the left or the right? Pay attention to the red object. Which vertex is higher, the left or the right?

Attention spreads over perceived objects Spreads to B and not C Spreads to C and not B Spreads to B and not C Spreads to C and not B Using a priming method (Egly, Driver & Rafal, 1994) showed that the effect of a prime spreads to other parts of the same visual object compared to equally distant parts of different objects.

Inhibition of return Inhibition-of-return is the phenomenon whereby an object that has been attended is less likely to attract attention again in a period of 300 ms to 900 ms after it is first attended. The attended item is said to be inhibited. This is thought to help in visual search since it prevents previously visited objects from being revisited IOR is Object-Based (the only counter-evidence involves easily-marked locations – like between two objects)

But IOR appears to be object-based (so it travels with the object that was attended)

Visual neglect syndrome is object-based When a right neglect patient is shown a dumbbell that rotates, the patient continues to neglect the object that had been on the right, even though It is now on the left (Behrmann & Tipper, 1999).

Simultanagnosic (Balint Syndrome) patients only attend to one object at a time Simultanagnosic patients cannot judge the relative length of two lines, but they can tell that a figure made by connecting the ends of the lines is not a rectangle but a trapezoid (Holmes & Horax, 1919).

Balint patients can only attend to one object at a time even if they are overlapping Luria, 1959

What does attention select preconceptually? Although there is now considerable evidence that attention attaches itself to objects, conventional wisdom insists that to detect properties is to detect properties-at-locations To reconcile this intuitive view with the object-based attention evidence, one might say that what is attended is spatiotemporal regions or “worms” – and many people do believe that* But the problem with this argument – and the problem with most ways of trying to reconcile the location view with empirical data – is that a spatiotemporal “worm” is simply the region that is traced out by a moving object! Without the independent notion of object there would be no worm! This may even be a terminological variant of the object view since objects and worms are mathematical duals – you can always translate one into the other.

The Binding Problem Our perceptual system can distinguish scenes that differ by conjunctions of properties, so early vision must not fuse together or lose the co-occurrence or conjunctiveness of properties it detects. In reporting properties early vision must bind them together. How it binds them together is a central question in vision. The most common answer is that it binds them according to co-location.

The role of attention to location in Treisman’s Feature Integration Theory

The more elaborate version of Treisman’s Feature Integration Theory

An alternative view of how we solve the binding problem Since only properties of indexed objects are encoded properties that belong to the same object are stored in the same Object File This raises problems to which I will return at the end The problems have to do with what a nonconceptual representation is supposed to be First we consider Austen Clark’s views of what he calls representations at the “level of sentience” or what others call sensations

Austen Clark (& P. Strawson) and feature placing languages What kind of representations are provided by (preconceptual) sensations? Strawson’s answer: Just those permitted by feature-placing “languages” – primitive artificial languages without predicates or concepts “The hypothesis ...is that sensation is feature-placing: a pre-linguistic system of mental representation. Mechanisms of spatio-temporal discrimination … serve to pick out or identify the subject-matter of sensory representation. That subject-matter turns out invariably to be some place-time in or around the body of the sentient organism. …the various reasons cited for thinking that sensation is intentional can also be explained on this hypothesis. The ‘aboutness’ of sensation reduces to its spatial character. (Clark, 2000, p 165)” “…there is a sensory level of identification of place-times that is more primitive than the identification of three-dimensional material objects. Below our conceptual scheme – underneath the streets, so to speak – we find evidence of this more primitive system. The sensory identification of place- times is independent of the identification of objects; one can place features even though one lacks the latter conceptual scheme.”

Why Objects are a better target than Locations It would have to be regions rather than locations anyway: points are irrelevant to the binding problem The only regions that are relevant are occupied regions – i.e., “objects”. The boundaries of regions must coincide with the boundaries of things, otherwise it does not help with the binding problem Properties (e.g. features) are properties of things, not of space. If it is to be a primitive nonconceptual contact (a “first responder”), then what is selected must capture attention and therefore must have causal powers. So it can’t be empty regions of space.

Solving the binding problem requires not just picking out places or regions. It requires that the regions coincide with things (objects) in the word that have the relevant properties

Experimental reasons why objects are a better target for attentional selection than location There is experimental evidence that attention attaches to things rather than places The Posner evidence of analog movement of focal attention, when attention is exogenously summoned, can be explained by a punctate object-based theory of attention-allocation – Sperling & Weichselgartner (1995)

Recall Posner’s demonstration of exogenous attention switch Does the improved detection in intermediate locations entail that the “spotlight of attention” moves continuously through empty space?

Sperling & Weichselgartner (1995) “Episodic” or Quantal Theory of Attention switching Assumes a quantal “shift” in attention in which the spotlight pointed at location -2 is extinguished and, simultaneously, the spotlight at location +2 is turned on. Because extinction and onset take a measurable amount of time, there is a brief period when the spotlights partially illuminate both locations simultaneously.

Some philosophical issues that arise from FINST theory Distinguishing causes and codes What causes Object Files to be created vs what is entered into them Conceptual and nonconceptual contents Representing and carrying information The case of clusters, figure-ground, and correspondence Can information-carrying properties (e.g., location on the proximal pattern) create clusters without representing locations of features that are clustered?

An alternative view of how to solve the Binding Problem According to the current version of FINST theory, only properties of indexed objects are encoded (conceptualized) The binding problem never arises because properties are always encoded as properties of an indexed object, and no other properties are encoded at all. This is in conflict with strong intuitions – namely that we see much more than we conceptualize. So what do we do about the things we “see” but do not conceptualize? Some Philosophers say they are represented nonconceptually? But what is such a representation like? And what makes it a representation, as opposed to just a biological reaction? My provisional answer is that such biological reactions (e.g., retinal activity) are not representations at all – they have no truth values. I think I am alone in this view so I must be right!

Another puzzle: Punctate inhibition of moving objects? We have recently obtained evidence that nontargets are inhibited (as measured by detection of small faint probe dots). There appears to be no inhibition of the empty region through which they move The inhibition is spatially local How can a punctate moving object be inhibited unless the object is being tracked? And how can it be tracked if there are many (n > 5) of them? But there is some sense in which moving objects must be being tracked: E.g., Dynamic random-dot stereograms Maybe Indexing is a two-stage process? Individuate Reference (for accessing)

Illustrative sketch of a FINST network

FINSTs and nonconceptual representation (a reprise) What does the early vision system deliver to the mind in a nonconceptual manner? What classes and properties can be recognized without the apparatus of concepts? Causality? Cardinality (of small sets)? 3D object shapes? Shape-from motion? Shape from shading? Shape from contours? What can be selected in a nonconceptual manner, and how does this help with the problem of connecting vision with the world?

Going beyond nonconceptual representations Work with Infants’s numerosity judgment frequently appeals to Index theory (Leslie, Carey, Wynne,…) Some of these findings appear to implicate indexing of nonconceptual properties, but some may not – the distinction is not easy to draw in practice: e,g. Infants can use certain properties to decide to create an new object file but not to recognize if an object is the same one that caused the object file to be created earlier Determination of cardinality in infants appears to be sensitive to such properties of individuals as whether they can be taken apart, whether they were poured, whether their parts moved together, etc. Such effects may indicate either that infants are deploying concepts or it may indicate that the mode of arrival of individuals affects whether they are indexed and tracked. This appears to be true of adult tracking as well!