1. Cognitive Maps: How the Brain Organizes Knowledge Ling 411 – 18

2. The Cognitive Map Hypothesis  Hypothesis: Knowledge is organized in the cortex as maps  Established (hence not hypothetical): The cognitive map of the body  Primary motor and somatosensory areas The map of pitch frequency  In primary auditory area  Hypothesized: Conceptual Phonological

3. Properties of cognitive maps  Established for somatic and frequency maps Local specificity  Every cortical location has a specific function Adjacency  Adjacent locations for adjacent functions  Nearby locations for related functions  Comes in degrees  Hypothesis: these properties apply to all homotypical cortical areas all types of knowledge represented in the cortex

4. First step in exploring the hypothesis: Categories  Understanding phonology Phonological structure is organized around phonological categories  E.g., vowels and consonants, voiceless stops  Understanding semantics Semantic structure is largely a matter of conceptual categories Understanding how categories work is the key to unlock the mysteries of semantics To understand how categories work we need to understand how the brain manages categorial information

5. What is a concept? Concepts vs. percepts  Percept: one sensory modality Locations are known Auditory: temporal lobe Visual: occipital lobe Somatosensory: parietal lobe  Concept: more than one sensory modality Higher level (more abstract) Locations, for nominal concepts:  Angular gyrus  (?)MTG

6. Types of Conceptual Categories  Discrete Even integers Counties in Texas  Radial Birds Vehicles  Family resemblance Games Furniture  Ill-defined Thought Mind

7. Phenomena associated with conceptual categories 1. No small set of defining features (with rare exceptions) 2. Fuzzy boundaries 3. Prototypical members and peripheral members 4. Categories are in the mind, not in the real world 5. Categories and their memberships vary from one language/culture system to another 6. Categories influence thinking, in both appropriate and inappropriate ways 7. Subcategories, and sub-subcategories, in hierarchical chains

8. Phenomena associated with categories: 1 1. No small set of defining features (with rare exceptions) The feature-attribute model fails  Works for some mathematical objects, but doesn’t apply to the way people’s cognitive systems apprehend most things  Example: CUP

9. Phenomena associated with categories: 2 1. No small set of defining features (with rare exceptions) 2. Fuzzy boundaries Example: VEHICLE  Car, truck, bus  Airplane?  Boat?  Toy car, model airplane?  Raft?  Roller skate?  Snowboard?

10. Fuzzy Categories  No fixed boundaries  Membership comes in degrees Prototypical Less prototypical Peripheral Metaphorical  The property of fuzziness relates closely to the phenomenon of prototypicality

11. Phenomena associated with categories: 3 1. No small set of defining features (with rare exceptions) 2. Fuzzy boundaries 3. Prototypical members and peripheral members Prototypical  CAR, TRUCK, BUS Peripheral:  AIRPLANE, TOY CAR, RAFT, ROLLER SKATE, etc. Varying degrees of peripherality

12. Prototypicality phenomena  The category BIRD Some members are prototypical  ROBIN, SPARROW Others are peripheral  EMU, PENGUIN  The category VEHICLE Prototypical : CAR, TRUCK, BUS Peripheral: ROLLER SKATE, HANG GLIDER

13. Phenomena associated with categories: 4 1. No small set of defining features (with rare exceptions) 2. Fuzzy boundaries 3. Prototypical members and peripheral members 4. Categories are in the mind, not in the real world In the world, everything  is unique  lacks clear boundaries  changes from day to day (even moment to moment) Whorf: “kaleidoscopic flux”

14. Phenomena associated with categories: 5 1. No small set of defining features (with rare exceptions) 2. Fuzzy boundaries 3. Prototypical members and peripheral members 4. Categories are in the mind, not in the real world 5. Categories and their memberships vary from one language/culture system to another cloche(of a church) clochette(on a cow) sonnette(of a door) grelot(of a sleigh) timbre(on a desk) glas(to announce a death) English: French: bell

15. Phenomena associated with categories - 6 1. No small set of defining features (with rare exceptions) 2. Fuzzy boundaries 3. Prototypical members and peripheral members 4. Categories are in the mind, not in the real world 5. Categories and their memberships vary from one language/culture system to another 6. Categories influence thinking, in both appropriate and inappropriate ways B.L. Whorf Example: Racial profiling

16. Phenomena associated with categories - 7 1. No small set of defining features (with rare exceptions) 2. Fuzzy boundaries 3. Prototypical members and peripheral members 4. Categories are in the mind, not in the real world 5. Categories and their memberships vary from one language/culture system to another 6. Categories influence thinking, in both appropriate and inappropriate ways 7. Subcategories, and sub-subcategories, in hierarchical chains ANIMAL – MAMMAL – CARNIVORE – CANINE – DOG –TERRIER – JACK RUSSELL TERRIER – EDDIE Each subcategory has the properties of the category plus additional properties Smallest subcategory has the most properties

17. Beyond description to explanation  How can we explain these phenomena?  The answer this question depends on how our information about categories is represented in the brain  The brain is where our linguistic and cultural knowledge is represented

18. Facts and a hypothesis that we can build on  Fact: The brain is a network Composed, ultimately, of neurons Cortical neurons are clustered in columns  Columns come in different sizes  Each minicolumn acts as a unit  Therefore a person’s linguistic and conceptual system is a network  Hypothesis: Every word and every concept is represented as a sub-network Term: functional web (Pulvermüller 2002)

19. Properties of functional webs  I: Functional Webs A concept is represented as a functional web  II: Columnar Nodes Nodes are implemented as cortical columns  III: Nodal Specificity Every node in a functional web has a specific function  III(a): Adjacency Nodes of related function are in adjacent locations  More closely related function, more closely adjacent

20. Property III(a): Adjacency  Nodes of related function are in adjacent locations More closely related function, more closely adjacent  Examples: Adjacent locations on cat’s paw represented by adjacent cortical locations Similar line orientations represented by adjacent cortical locations

21. Hypotheses concerning conceptual webs  Hypothesis I: Extrapolation to Humans The findings about cortical structure and function from experiments on cats, monkeys, and rats can be extrapolated to humans Hypothesis I(a): The extrapolation can be extended to linguistic and conceptual structures and functions  Hypothesis II: Hierarchy A functional web is hierarchically organized  Hypothesis III: Cardinal nodes Every functional web has a cardinal node Hypothesis III(a):  Each subweb likewise has a cardinal node

22. (Part of) the functional web for CAT V P A M C The cardinal node for the entire functional web T Cardinal nodes of the subwebs

23. Phenomena associated with categories 1. No small set of defining features (with rare exceptions) 2. Fuzzy boundaries 3. Prototypical members and peripheral members 4. Subcategories, and sub-subcategories, in hierarchical chains 5. Categories are in the mind, not in the real world 6. Categories and their memberships vary from one language/culture system to another 7. Categories influence thinking, in both appropriate and inappropriate ways REVIEW

24. How to explain?  Description is fine, but its only a start  Next step: Explanation  How to explain? By answering the question of how categories are represented in the brain REVIEW

25. Phenomena associated with categories: 1-3 1. No small set of defining features (with rare exceptions) Example: CUP More realistic alternative: radial categories 2. Fuzzy boundaries Example: VEHICLE 3. Prototypical members and peripheral members VEHICLE  Prototypical: CAR, TRUCK, BUS  Peripheral: AIRPLANE, TOY CAR, RAFT, ROLLER SKATE, etc. Varying degrees of peripherality  These three phenomena are interdependent

26. How do radial categories work?  Different connections have different strengths (weights)  More important properties have greater strengths  For CUP, Important (but not necessary!) properties:  Short (as compared with a glass)  Ceramic  Having a handle  Cups with these properties are more prototypical

27. The properties of a category have different weights T CUP MADE OF GLASS CERAMIC SHORT HAS HANDLE The properties are represented by nodes which are connected to lower-level nodes The cardinal node The threshold More important properties have greater weights, represented by greater thicknesses of lines

28. Activation of a category node  The node will be activated by any of many different combinations of properties  The key word is enough – it takes enough activation from enough properties to satisfy the threshold  The node will be activated to different degrees by different combinations of properties When strongly activated, it transmits stronger activation to its downstream nodes.

29. Prototypical exemplars provide stronger and more rapid activation T CUP MADE OF GLASS CERAMIC SHORT HAS HANDLE Stronger connections carry more activation Activation threshold (can be satisfied to varying degrees ) Inhibitory connection

30. Explaining Prototypicality  Cardinal category nodes get more activation from the prototypical exemplars More heavily weighted property nodes  E.g., FLYING is strongly connected to BIRD Property nodes more strongly activated  Peripheral items (e.g. EMU) provide only weak activation, weakly satisfying the threshold (emus can’t fly)  Borderline items may or may not produce enough activation to satisfy threshold

31. Activation of different sets of properties produces greater or lesser satisfaction of the activation threshold of the cardinal node CUP MADE OF GLASS CERAMIC SHORT HAS HANDLE

32. Explaining prototypicality: Summary  Variation in strength of connections  Many connecting properties of varying strength  Varying degrees of activation  Prototypical members receive stronger activation from more associated properties  BIRD is strongly connected to the property FLYING Emus and ostriches don’t fly But they have some properties connected with BIRD Sparrows and robins do fly  And as commonly occurring birds they have been experienced often, leading to entrenchment – stronger connections

33. Phenomena associated with categories: 4 1. No small set of defining features (with rare exceptions) 2. Fuzzy boundaries 3. Prototypical members and peripheral members 4. Categories are in the mind, not in the real world In the world, everything  is unique  lacks clear boundaries  changes from day to day (even moment to moment) Whorf: “kaleidoscopic flux”

34. Phenomena associated with categories: 5 1. No small set of defining features (with rare exceptions) 2. Fuzzy boundaries 3. Prototypical members and peripheral members 4. Categories are in the mind, not in the real world 5. Categories and their memberships vary from one language/culture system to another cloche(of a church) clochette(on a cow) sonnette(of a door) grelot(of a sleigh) timbre(on a desk) glas(to announce a death) English: French: bell REVIEW

35. Phenomena associated with categories - 6 1. No small set of defining features (with rare exceptions) 2. Fuzzy boundaries 3. Prototypical members and peripheral members 4. Categories are in the mind, not in the real world 5. Categories and their memberships vary from one language/culture system to another 6. Categories influence thinking, in both appropriate and inappropriate ways B.L. Whorf Example: Racial profiling

36. These phenomena (4-6) are interrelated 4. Categories are in the mind, not in the real world 5. Categories and their memberships vary from one language/culture system to another 6. Categories influence thinking, in both appropriate and inappropriate ways B.L. Whorf Example: Racial profiling

37. Bidirectional processing and inference T CUP MADE OF GLASS CERAMIC SHORT HANDLE These connections are bidirectional Separate fibers for the two directions; shown as one line in the notation

38. Bidirectional processing and inference T CUP SHORT HANDLE Thought process: 1. The cardinal concept node is activated by a subset of its property nodes 2. Feed-backward processing activates other property nodes Consequence: We “apprehend” properties that are not actually perceived

39. Another hypothesis of Whorf  Grammatical categories of a language influence the thinking of people who speak the language  Can we explain this too in terms of brain structure?

40. Example: Grammatical gender  Does talking about inanimate objects as if they were masculine or feminine actually lead people to think of inanimate objects as having a gender?  Could the grammatical genders assigned to objects by a language influence people’s mental representation of objects? Boroditsky (2003)

41. Experiment: Gender and Associations (Boroditsky et al. 2002)  Subjects: speakers of Spanish or German All were fluent also in English English used as language of experiment  Task: Write down the 1 st 3 adjectives that come to mind to describe each object All the (24) objects have opposite gender in German and Spanish  Raters of adjectives: Native English speakers

42. Examples:  Key (masc in German, fem in Spanish) Adjectives used by German speakers:  Hard, heavy, jagged, metal, serrated, useful Adjectives used by Spanish speakers:  Golden, intricate, little, lovely, shiny, tiny  Bridge (fem in German, masc in spanish) Adjectives used by German speakers:  Beautiful, elegant, fragile, peaceful, pretty Adjectives used by Spanish speakers:  Big, dangerous, long, strong, sturdy, towering

43. Results of the Experiment (Boroditsky et al. 2002)  Raters of adjectives were native English speakers  Result: Adjectives were rated as masculine or feminine in agreement with the gender in subject’s native language

44. Categories and the brain All of these phenomena associated with categories can be explained as inevitable consequences of the structure and function of the human brain

45. Phenomena associated with categories: 7 7. Subcategories, and sub-subcategories, in hierarchical chains ANIMAL – MAMMAL – CARNIVORE – CANINE – DOG –TERRIER – JACK RUSSELL TERRIER – EDDIE Each subcategory has the properties of the category plus additional properties Smallest subcategory has the most properties

46. How to explain? Perceptual Neuroscience  Hypothesis I: Extrapolation The findings described by Mountcastle can be extrapolated to humans  Hypothesis I(a): Extrapolation can be extended to linguistic and conceptual structures  Why? Cortical structure, viewed locally, is Uniform across mammalian species Uniform across different cortical regions  Cortical structure and function, locally, are essentially the same in humans as in cats and monkeys and rats Moreover, in humans, the regions that support language have the same structure locally as other cortical regions

47. Support for the extrapolation hypothesis  Conceptual systems in humans evidently use the same structures as perceptual systems  Therefore it is not too great a stretch to suppose that experimental findings on the structure of perceptual systems in monkeys can be applied to an understanding of the structure of conceptual systems of human beings  In particular to the structures of conceptual categories REVIEW

48. Columns of different sizes  Minicolumn Basic anatomically described unit 70-110 neurons (avg 75-80) Diameter barely more than that of pyramidal cell body (30-50 μ)  Maxicolumn (term used by Mountcastle) Diameter 300-500 μ Bundle of about 100 continuous minicolumns  Hypercolumn – up to 1 mm diameter Can be long and narrow rather than cylindrical  Functional column Intermediate between minicolumn and maxicolumn A contiguous group of minicolumns

49. Functional Columns  Intermediate in size between minicolumn and maxicolumn  Hypothesized functional unit whose size is determined by experience/learning  A maxicolumn consists of multiple functional columns  A functional column consists of multiple minicolumns  Functional column may be further subdivided with learning of finer distinctions

50. Columns of different sizes In order according to size  Minicolumn The smallest unit 70-110 neurons  Functional column Variable size – depends on experience Intermediate between minicolumn and maxicolumn  Maxicolumn (a.k.a. column) 100 to a few hundred minicolumns  Hypercolumn Several contiguous maxicolumns

51. Hypercolums: Modules of maxicolumns A visual area in temporal lobe of a macaque monkey

52. Perceptual subcategories and columnar subdivisions of larger columns  Nodal specificity applies for maxicolumns as well as for minicolumns  The adjacency hypothesis likewise applies to larger categories and columns Adjacency applies for adjacent maxicolumns  Subcategories of a category have similar function Therefore their cardinal nodes should be in adjacent locations

53. Functional columns  The minicolumns within a maxicolumn respond to a common set of features  Functional columns are intermediate in size between minicolumns and maxicolumns  Different functional columns within a maxicolumn are distinct because of non-shared additional features Shared within the functional column Not shared with the rest of the maxicolumn Mountcastle: “The neurons of a [maxi]column have certain sets of static and dynamic properties in common, upon which others that may differ are superimposed.”

54. Similarly..  Neurons of a hypercolumn may have similar response features, upon which others that differ may be superimposed  Result is maxicolumns in the hypercolumn sharing certain basic features while differing with respect to others  Such maxicolumns may be further subdivided into functional columns on the basis of additional features  That is, columnar structure directly maps categories and subcategories

55. Hypercolums: Modules of maxicolumns A visual area in the temporal lobe of a macaque monkey Category (hypercolumn) Subcategory (can be further subdivided)

56. Category representations in the cortex  Hypercolumn  Maxicolumn  Functional column  Sub-functional column  Supercategory  Category  Subcategory  Sub-subcategory

57. Hypothesis applied to conceptual categories  A whole maxicolumn gets activated for a category Example: BEAR  Different functional columns within the maxicolumn for subcategories BROWN BEAR, GRIZZLY, POLAR BEAR, etc.  Adjacent maxicolumns for categories related to BEAR (sharing various features) I.e., other carnivores  Similarly, CUP has a column surrounded by columns for other drinking vessels

58. Perceptual subcategories and columnar subdivisions of larger columns  Nodal specificity applies for maxicolumns as well as for minicolumns  The adjacency hypothesis likewise applies to larger categories and columns Adjacency applies for adjacent maxicolumns  Subcategories of a category have similar function Therefore their cardinal nodes should be in adjacent locations

59. Support from patients with brain damage (from Rapp & Caramazza 1995)  J.B.R. and S.B.Y. (905b-906a)  Herpes simplex encephalitis  Both temporal lobes affected  Could not define animate objects ostrich, snail, wasp, duck, holly  Much better at defining inanimate objects  tent, briefcase, compass, wheelbarrow, submarine, umbrella  Conclusion: cortical areas for conceptual categories

60. Additional support from cases of brain damage  J.J. and P.S. (Hillis & Caramazza 1991) (906-7) J.J. – left temporal, basal ganglia (CVA)  Selective preservation of animal concepts P.S. – mostly left temporal (injury)  Selective impairment of animate category P.S J.J.

61. Two different patients with anomia Deficit in retrieval of animal names (Damage from stroke) Inability to retrieve words for unique entities (Left temporal lobectomy)

62. Two more patients with anomia Deficit in retrieval of words for man-made manipulable objects (Damage from stroke) Severe deficit in retrieval of words for concrete entities (Herpes simplex encephalitis)

63. What is it that determines location?  Logical categories like ANIMALS vs. TOOLS/UTENSILS ? If so, why?  Abstract categories based on cognitively salient properties?

64. Animals vs. Tools/Utensils?  These two categories have been studied most extensively in the literature  What is it that determines location?  Observations: Most animals are known mostly in the visual modality Many tools and utensils are known largely in the somatosensory and motor modalities

65. Proximity principle and nominal concepts  Supramarginal gyrus, angular gyrus, and middle temporal gyrus are all close to Wernicke’s area  Angular gyrus occupies intermediate location between the major perceptual modalities  Supramarginal gyrus especially close to somatosensory perception  Middle temporal gyrus especially close to visual perception

66. Functional columns in phonological recognition A hypothesis  Demisyllable (e.g. /de-/) activates a maxicolumn  Different functional columns within the maxicolumn for syllables with this demisyllable /ded/, /deb/, /det/, /dek/, /den/, /del/

67. Demisyllables [di, de, da, du] F1 and F2 For [de] It is unlikely that [d] is represented as a unit in perception

68. Functional columns in phonological recognition A hypothesis [de-] A maxicolumn (ca. 100 minicolumns ) Divided into functional columns (Note that all respond to /de-/) deb ded den de- det del dek

69. Phonological hypercolumns (a hypothesis)  Maybe we have Hypercolumn of contiguous maxicolumns for /e/ With maxicolumns for /de-/, /be-/, etc. Each such maxicolumn subdivided into functional columns for different finals  /det/, /ded/, /den/, /deb/, /dem/. /dek/  N.B.: This is a hypothesis, not proven But there is indirect evidence Maybe someday soon we’ll be able to test with sensitive brain imaging

70. Adjacent maxicolumns in phonological cortex? ge- ke- be- pe- te- de- A module of six contiguous maxicolumns Each of these maxicolumns is divided into functional columns Note that the entire module responds to [-e-] Hypercolum

71. Adjacent maxicolumns in phonological cortex? ge- ke- be- pe- te- de- A module of six contiguous maxicolumns The entire module responds to [-e-] deb ded den de- det del dek The entire maxicolumn responds to [de-]

72. Learning phonological distinctions: A hypothesis ge- ke- be- pe- te- de- 1. In learning, this hypercolumn gets established first, responding to [-e-] 2. It gets subdivided into maxicolumns for demisyllables deb ded den de- det del dek 3. The maxicolumn gets divided into functional columns

73. Indirect evidence for the hypothesis  Fits the structural organization demonstrated in monkey vision  Cortical structure and function have a high degree of uniformity  MEG is able to pick up different locations in Wernicke’s area for different vowels MEG can only detect activity of at least 10,000 contiguous apical dendrites (Papanicolaou)  Requires perhaps at least 250 adjacent minicolumns  The size of a maxicolumn or hypercolumn

74. Remaining question: The process of learning distinctions  When a hypercolumn is first recruited, no lateral inhibition among its internal subdivisions  Later, when finer distinctions are learned, they get reinforced by lateral inhibition  Question: How does this work?

75. Inhibitory connections Based on Mountcastle (1998)  Columnar specificity is maintained by pericolumnar inhibition (190) Activity in one column can suppress that in its immediate neighbors (191)  Inhibitory cells can also inhibit other inhibitory cells (193)  Inhibitory cells can connect to axons of other cells (“axoaxonal connections”)  Large basket cells send myelinated projections as far as 1-2 mm horizontally (193)

76. Neural processes for learning  Basic principle: when a connection is successfully used, it becomes stronger Successfully used if another connection to same node is simultaneously active  Mechanisms of strengthening Biochemical changes at synapses Growth of dendritic spines Formation of new synapses  Weakening: when neurons fire independently of each other their mutual connections (if any) weaken

77. Neural processes for learning A B C If connections AC and BC are active at the same time, and if their joint activation is strong enough to activate C, they both get strengthened (adapted from Hebb) Synapses here get strengthened

78. end