Symposium on Language and Perception How does language affect what are normally considered to be lower-level perceptual and motor processes? Is language best conceptualized as a symbolic system in which terms derive all of their meaning from their relations to one another? How does the language we acquire augment our ability to represent, reason about, and computationally process our world? How do our perceptual abilities shape language processes normally considered to be arbitrary or ungrounded? How much of our language is perceptually grounded? How else can elements of our language get their meaning? How are the various components of language (word meanings, sounds, and grammatical constructions) influenced by perceptual and action processes? How do these same components influence perceptual and conceptual processes? Indiana University Monday, April 4, 2005, 12:00-2:00 PM Dogwood Room, Indiana Memorial Union Panelists Lera Boroditsky, Assistant Professor of Psychology, Stanford University Stevan Harnad, Canada Research Chair and Professor, Université du Québec à Montréal Linda Smith, Chancellor’s Professor of Psychology, Indiana University

Language --> Perception or Perception --> Language Does what we see determine what we say? Or does what we say determine what we see? (Let’s C…) Stevan Harnad U. du Québec à Montreal

1. Commensurability the fundamental question of whether it can be said that the objective shape of things in the world "resembles" the shape of their subjective appearance in our minds at all, rather than just being behaviorally correlated with it: e.g., how/why does bigger sound-wave amplitude resemble sounding louder, rather than softer, or for that matter, greener? how/why does looking curved resemble being curved?

2. Constructivism/Coherence/Consistency the notion that what things look like -- the shape of their subjective appearance -- is something we "construct", by social convention, culturally, or arbitrarily, with perhaps the only constraint being that the construction should be consistent or coherent with itself

3. Congruity/Conformity the notion that what things look like -- the shape of their appearance -- is congruent with the shape of what they really are like: that the subjective "shape" of appearances, not being constructed by culture, simply conforms to the objective shape of things in reality

4. Correlation the empirical fact -- objectively measurable through psychophysics and adaptive behavior -- that there is a correlation (which is not the same as a congruity) between the real shape of things and the shape of our behavior in response to them: we reliably call more "more," and act accordingly

5. Conformity/Constraint the notion that the shape of appearances is merely constrained by the shape of reality [and logic], at least inasmuch as I/O correlation and adaptive behavior are concerned: i.e., apart from whatever "more" may look/feel like, subjectively, we should feel as if more feels like more, rather than like less, and (more important) we should reliably act in conformity with that constraint (e.g., running more from a bigger threat, etc.)

6. Communication language, when we describe the shape of things to one another, can sometimes convey their objective shape faithfully, to an approximation [e.g., "a circle is a set of points equidistant from its center" or "she is the one with the hat"], and it can also sometimes bias its subjective shape, to a degree [e.g., "there is cynicism behind that smile", ”all El Greco subjects are distorted”]

7. Computation the fact that [just about] all objective shapes are informally describable in language and are even formally computable does not mean that cognition -- i.e., whatever goes on in our heads in between input and output -- is just computation: computation is symbol manipulation; symbol shapes are arbitrary (in relation to the shapes of the things the symbols are about), so the meanings of the symbols have to be grounded in something more than just further symbols and symbol-manipulations (this is the “symbol-grounding problem”)symbol-grounding problem

8. Corporality the fact that the subjective shape of things is determined in part by the objective shape of our bodies (e.g., one property of chairs is that they afford sittability-upon) means that some cognition is sensorimotor [and other] dynamics

9. Categorization to a great extent, cognition amounts to whatever is going on in our heads that gives us the capacity to know or learn how to do the kinds of things we do with the kinds of things we do it with: everything from sitting on them to manipulating, naming and describing them: to cognize is to categorize

10. Correction/Consequences to have and know a category, there has to be a right or wrong of the matter, there have to be consequences of miscategorizing that can serve as a reliable corrective to guide future correct categorization; there are behavioral continua too, where more/less is right, rather than all/none; but categorization concerns all/none -- i.e., categorical -- behavior; and where there is no possibility of error, no objective consequences of miscategorization, there are no categories, only arbitrary and incontrovertible subjectivity (Wittgenstein’s “private language”), hermeneutics, or free- floating metaphor (because everything resembles everything else, to an infinite degree, if freed of objective constraints and consequences)

11. Concreteness all categorization is abstraction: selectively hewing to some properties of the kind of thing categorized, ignoring others; all categories are hence abstract; types are always more abstract than their tokens; concreteness hence really refers only to how much sensorimotor experience is directly involved in the categorization: "red" is more concrete than "color" and "color" is more concrete than "property", but all categories, even those based purely on verbal hearsay, must be recursively grounded, bottom-up, in sensorimotor categories

12. Context/Confusability both categories and their invariant properties are context-dependent (and often also provisional and approximate): the properties that distinguish members from non-members depend on what proves to be reliably invariant in the range of variation the categorizer has sampled and from which he must, based on the consequences of miscategorization, successfully generalize to categorize future instances: "What is an X?" "Compared to what?”: category invariants are whatever features are sufficient to reliably resolve the confusability between members and nonmembers (i.e., there isn’t and never was anything wrong with the “classical theory” of categorization! whether the invariants prove to be conjunctive, disjunctive, monadic, polyadic, conditional, constructive, computational, deformational or statistical -- they must exist and be detectable if categorization is to succeed, and hence they must be “classical”)context-dependent

13. Compression in selectively detecting/attending to some properties and ignoring others, categorization results in a compression ["warping", "Whorfing"] of appearances in similarity space, making the subjective shapes of members of the same category resemble one another more, and resemble the shapes of members of other categories less; this is sometimes detectable as a psychophysical compression/separation of similarities, as in color and phoneme categorical perception [mostly innate], and even (more subtly) as a result of sensorimotor training or even verbal suggestion, in learned categorical perceptioncategorical perception

14. Cohabitation/Complementarity inside our heads there co-habit effects in both directions: the sensorimotor interactions between the objective shapes of our bodies and the objective shapes of things and kinds in the world constrain the shape of the subjective appearance of things and kinds, and verbal communication about shapes and kinds also influences the shape of their subjective appearance and the kinds of things we do with and to them

15. Collaboration/Cumulativity/Collectivity/Correctivity it is clear that not all of this can transpire within one head alone: both categories and their subjective appearance are shaped by interactions not only with the things in the world, but with the sayings and doings of other categorizers

References Harnad, Stevan (1982) Metaphor and Mental Duality, in Simon, T. and Scholes, R., Eds. Language, mind and brain, pages pp Hillsdale NJ: Erlbaum. Harnad, Stevan (1987) Category Induction and Representation, Chapter 18 of: Harnad, Stevan (ed.) (1987) Categorical Perception: The Groundwork of Cognition. New York: Cambridge University Press. Harnad, Stevan (2001) Explaining the Mind: Problems, Problems. The Sciences 41:pp Harnad, Stevan (2003) Symbol-Grounding Problem, in Encyclopedia of Cognitive Science. MacMillan: Nature Publishing Group. Harnad, Stevan (2003) Categorical Perception, in Encyclopedia of Cognitive Science. MacMillan: Nature Publishing Group. Harnad, S. (2005) To Cognize is to Categorize: Cognition is Categorization. To appear in Lefebvre C., & H. Cohen (Eds.) (2005) Handbook on Categorization. Elsevier