1. I - Cortical Column Functions II - Functional Webs Ling 411 – 12

2. Uniformity of cortical function  If cortical function is uniform across mammals and across different cortical areas, then the findings presented by Mountcastle can be extended to language  Claims: Locally, all cortical processing is the same The apparent differences of function are consequences of differences in larger-scale connectivity  Conclusion (if the claim is supported): Understanding language, even at higher levels, is basically a perceptual process

3. Testing the claim  Claim: The apparent differences of function are consequences of differences in larger-scale connectivity  To test, we need to understand cortical function  That means we have to understand the function of the cortical column

4. Quote from Mountcastle “[T]he effective unit of operation…is not the single neuron and its axon, but bundles or groups of cells and their axons with similar functional properties and anatomical connections.” Vernon Mountcastle, Perceptual Neuroscience (1998), p. 192

5. Columns do not store symbols!  They only Receive activation Maintain activation Inhibit competitors Transmit activation  Important consequence: We have linguistic information represented in the cortex without the use of symbols It’s all in the connectivity  Challenge: How?

6. Why the usual approach won’t work  Let us suppose that words are stored in some kind of symbolic form  What form?  If written, there has to be.. something in there that can read them something in there that can write them something in there that can move them around, from one place to another something in there to compare them with forms entering the brain as it hears someone speaking – otherwise, how can an incoming word be recognized?

7. Why the usual approach won’t work (cont’d)  If not written, then represented in some other medium  Doesn’t solve the problem  You still need whatever kind of sensory detectors can sense the symbols in whatever medium you choose  Plus means of performing all those other operations

8. Compare imagery  Visual images Little pictures? If so, what is in there to see them?  Auditory images Little sounds vibrating in the brain? If so, what is in there to hear them?  There has to be another way!

9. There must be another way  Visual imagery (e.g. of your grandmother) Reactivation of some of the same nodes and connections that operate when actually seeing her  Auditory imagery (e.g. of a tune) Reactivation of some of the same nodes and connections that operate in actually hearing it

10. Another way, for language  A syllable Activation of the nodes and connections needed to recognize or produce it  A word Activation of the nodes and connections needed to recognize it  A syntactic construction Activation of the nodes and connections needed to recognize or produce it

11. The postulation of objects as something different from the terms of relationships is a superfluous axiom and consequently a metaphysical hypothesis from which linguistic science will have to be freed. Louis Hjelmslev Prolegomena to a Theory of Language (1943: 61) Quotation

12. Columns do not store symbols!  They only Receive activation Maintain activation Inhibit competitors Transmit activation  Important consequence: We have linguistic information represented in the cortex without the use of symbols It’s all in the connectivity  Challenge: How?

13. Columnar Functions: Integration and Broadcasting  Integration: A column is activated if it receives enough activation from Other columns Thalamus  Can be activated to varying degrees  Can keep activation alive for a period of time  Broadcasting: An activated column transmits activation to other columns Exitatory Inhibitory  Learning : adjustment of connection strengths and thresholds

14. Integration and Broadcasting  Broadcasting To multiple locations In parallel  Integration

15. Integration and Broadcasting Integration Broadcasting Wow, I got activated! Now I’ll tell my friends!

16. What matters is not ‘what’ but ‘where’  What distinguishes one kind of information from another is what it is connected to  Lines and nodes are approximately the same all over  Hence, uniformity of cortical structure Same kinds of columnar structure Same kinds of neurons Same kinds of connections  Different areas have different functions because of what they are connected to

17. Operations in relational networks  Activation moves along lines and through nodes Integration Broadcasting  Connection strengths are variable A connection becomes stronger with repeated successful use A stronger connection can carry greater activation

18. What about the rest of language?  Words and their meanings  Syntax and morphology  Conceptual relationships

19. Sequence  In language, sequence is very important Word order Order of phonological elements in syllables Etc.  Also important in many non-linguistic areas Dancing Eating a meal  Can cortical columns handle sequences?

20. Lasting activation in minicolumn Subcortical locations Connections to neighboring columns not shown Cell Types Pyramidal Spiny Stellate Inhibitory Recurrent axon branches keep activation alive in the column – Until is is turned off by inhibitory cell

21. Notation for lasting activation > Thick border for a node that stays active for a relatively long time > Thin border for a node that stays active for a relatively short time

22. Recognizing items in sequence This link stays active a b Node c is satisfied by activation from both a and b If satisfied it sends activation to output connections Node a keeps itself active for a while Suppose that node b is activated after node a Then c will recognize the sequence ab c This node recognizes the sequence ab

23. Demisyllables in recognizing stops  Consider stop consonants, e.g. t, d  At the time of closure For voiceless stops there is no sound to hear For voiced stops, very little sound  The stops are identified by transitions To following vowel From preceding vowel

24. Demisyllables [di, de, da, du] F1 and F2 For [a] It is unlikely that [d] is represented as a unit in perception

25. Recognizing a syllable and its demisyllables dim di- -im Cardinal node for dim Functional subweb for dim Auditory features of [di-] Auditory features of [-im] Just labels

26. Another syllable and its demisyllables bil bi- -il Cardinal node for bill Subweb for bill

27. Multiple connections of -il bil hil kil bi- -il Bill hill mill kill etc. One and the same /-il/ in all of them

28. Multiple connections of -il bil hil kil bi- -il Bill hill mill kill etc. Similarly for multiple connections of bi- bit, bib, bid, etc.

29. Multiple connections of -il bil hil kil bi- -il Bill hill mill kill etc. To lower level nodes in the subwebs, for phonological features

30. Syntactic Recognition – same principle This link stays active a b Let node a represent Noun Phrases (Subject) and let b represent Predicates (Verb Phrases etc.) Then c represents Clauses: the sequence ab c This node recognizes the sequence ab

31. Syntactic Recognition: higher-level perception This link stays active a b The whole process is one of recognition, just as at lower levels (e.g., phonological recognition) Same structures, different connections c This node recognizes the sequence ab

32. Conclusion: All of linguistic structure is relational  The whole of linguistic structure is a connectionist system  Good thing, since that is exactly the kind of system that the cortex is built to represent and to operate with

33. Findings relating to columns ( Mountcastle, Perceptual Neuroscience, 1998)  The column is the fundamental module of perceptual systems probably also of motor systems  Perceptual functions are very highly localized Each column has a very specific local function  This columnar structure is found in all mammals that have been investigated  The theory is confirmed by detailed studies of visual, auditory, and somatosensory perception in living cat and monkey brains

34. Operation of the Network  The linguistic system operates as distributed processing of multiple individual components – cortical columns  Columnar Functions Integration: A column is activated if it receives enough activation from other columns  Can be activated to varying degrees  Can keep activation alive for a period of time An activated column transmits activation to other columns  Exitatory – contribution to higher level  Inhibitory – dampens competition at same level  Columns do not store symbols! Review

35. Neuronal Structure and Function (Pulverműller 2002, Chapter 2)

36. Neuronal Structure and Function: The Cortex as a Network  Pulvermüller (2002) : The brain is not like a computer “…any hardware computer configuration can realize almost any computer program or piece of software.” “… it may be that the neuronal structures themselves teach us about aspects of the computational processes that are laid down in these structures.”  Connectivity as key property

37. The cortex operates by means of connections  Grey matter Cortical columns Horizontal connections among neighboring columns  White matter Connections between distant columns

38. Computers and Brains: Different Structures, Different Skills  Computers Exact, literal Rapid calculation Rapid sorting Rapid searching Faultless memory Do what they are told Predictable  Brains Flexible, fault tolerant Slow processing Association Intuition Adaptability, plasticity Self-driven activity Unpredictable Self-driven learning

39. What brains but not computers can do  Acquire information to varying degrees “Entrenchment” How does it work?  Variable connection strength  Connections get stronger with repeated use  Perform at varying skill levels Degrees of alertness, attentiveness Variation in reaction time Mechanisms:  Global neurotransmitters  Variation in blood flow  Variation in available nutrients  Presence or absence of fatigue  Presence or absence of intoxication

40. Neuronal Structure and Function: Connectivity  White matter: it’s all connections Far more voluminous than gray matter Cortico-cortical connections  The fibers are axons of pyramidal neurons  They are all excitatory White since the fibers are coated with myelin  Myelin: glial cells  There are also grey matter connections Unmyelinated Local Horizontal, through gray matter Excitatory and inhibitory

41. Pyramidal neurons and their connections  Connecting fibers Dendrites (input): length 2mm or less Axons (output): length up to 10 cm  Synapses Afferent synapses: up to 50,000  From distant and nearby sources Distant – to apical dendrite Local – to basal dendrites or cell body Efferent synapses: up to 50,000  On distant and nearby destinations Distant – main axon, through white matter Local – collateral axons, through gray matter

42. Proportion of pyramidal cells in the cortex  Abeles (1991: 52) says 70%  Mountcastle says 70% - 80% (1998: 54) Based on information from Feldman (1984)  Pulvermüller (2002: 13) says 85% Based on information from Braitenburg & Schüz (1998)  Some difference comes from how spiny stellate cells are counted Pyramidal or not?  No discrete boundary between these categories

43. Connecting fibers of pyramidal neurons Apical dendrite Basal dendrites Axon

44. Interconnections of pyramidal neurons Input from distant cells Input from neighboring columns Output to distant cells

45. Neuronal Structure and Function: Connectivity  Synapses of a typical pyramidal neuron: Incoming (afferent) – 50,000 (5 x 10 4 ) Outgoing (efferent) – 50,000  Number of synapses in cortex: 28 billion neurons (Mountcastle’s estimate)  i.e., 28 x 10 9  Synapses in the cortex (do the math) 5 x 10 4 x 28 x 10 9 = 140 x 10 13 = 1.4 x 10 15 Approximately 1,400,000,000,000,000 i.e., over 1 quadrillion

46. Cortical connectivity properties  Probability of adjacent areas being connected: >70% (Pulvermüller p. 17) But if we count by minicolumns instead of cells the figure is probably higher, maybe close to 100%  Probability of distant areas being connected: 15-30% (p. 17) Distant areas: at least one intervening area In Macaque monkey, most areas have links to 10 or more other areas within same hemisphere

47. More cortical connectivity properties  Most areas are connected to homotopic area of opposite hemisphere  Most connections between areas are reciprocal  Primary areas not directly connected to one another, except for motor- somatosensory Connections under central sulcus

48. Degrees of separation between cortical neurons or columns  For neurons of neighboring columns: 1  For distant neurons in same hemisphere Range: 1 to about 5 or 6 (estimate) Mostly 1, 2, or 3, especially if functionally closely related Average about 3 (estimate)  For opposite hemisphere Add 1 to figures for same hemisphere  Probably, for any two columns anywhere in the cortex, whether functionally related or not, fewer than 6 degrees of separation

49. Neural processes for learning  Basic principle: when a connection is successfully used, it becomes stronger Successfully used if another connection to same node is simultaneously active  Mechanisms of strengthening Biochemical changes at synapses Growth of dendritic spines Formation of new synapses  Weakening: when neurons fire independently of each other their mutual connections (if any) weaken

50. Neural processes for learning A B C If connections AC and BC are active at the same time, and if their joint activation is strong enough to activate C, they both get strengthened (adapted from Hebb) Synapses here get strengthened

51. Pulvermüller’s functional webs  For example, a web for the concept CAT  Pulvermüller: A significant portion of the web’s neurons are active whenever the cat concept is being processed The function of the web depends on the intactness of its member neurons If neurons in the functional web are strongly linked, they should show similar response properties in neurophysiological experiments (2002:26)

52. The neural basis of cognition  Earlier proposals (p. 23) Individual neurons (Barlow 1972)  Individual neurons too noisy and unreliable  Would require more information processing capacity than one neuron has Mass activity and interference patterns in the entire cortex (Lashley 1950)  Better alternative: Functional webs of neurons (Pulvermüller)  Even better Functional webs of cortical columns (not mentioned by Pulvermüller)

53. Pulvermüller’s functional webs  A large set of neurons that Are strongly connected to each other Are distributed over a set of cortical areas Work together as a functional unit Are functionally interdependent so that each is necessary for the optimal functioning of the web (p.24)

54. Hypothesis I: Functional Webs  A word is represented as a functional web  Spread over a wide area of cortex Includes perceptual information  Relating to the meaning As well as specifically conceptual information For nominal concepts, mainly in Angular gyrus (?) For some, middle temporal gyrus (?) For some, supramarginal gyrus As well as phonological information  Temporal, parietal, frontal

55. Example: The meaning of dog  We know what a dog looks like Visual information, in occipital lobe  We know what its bark sounds like Auditory information, in temporal lobe  We know what its fur feels like Somatosensory information, in parietal lobe  All of the above.. constitute perceptual information are subwebs with many nodes each have to be interconnected into a larger web along with further web structure for conceptual information

56. The Wernicke-Lichtheim concept node (1885) Where?

57. The “C” Node  Not just in one place Conceptual information for a single word is widely distributed Conceptual information is in different areas for different kinds of concepts  The second of these points and probably also the first were already recognized by Wernicke  But.. There may be a single “C” node anyway as cardinal node of a distributed network

58. “C” node as cardinal node of a web V M C For example, FORK Labels for Properties: C – Conceptual M – Motor T – Tactile V - Visual Each node in this diagram represents the cardinal node of a subweb of properties T

59. Some connections of the “C” node for FORK V C Each node in this diagram represents the cardinal node of a subweb of properties For example, Let’s zoom in on this one M T

60. Zooming in on the “V” Node.. FORK Etc. etc. (many layers) A network of visual features V

61. Add phonological recognition node V M C For example, FORK Labels for Properties: C – Conceptual M – Motor P – Phonological image T – Tactile V – Visual T P The phonological image of the spoken form [fork] (in Wernicke’s area)

62. Add node in primary auditory area V M C T P PA Primary Auditory: the cortical structures in the primary auditory cortex that are activated when the ears receive the vibrations of the spoken form [fork] For example, FORK Labels for Properties: C – Conceptual M – Motor P – Phonological image PA – Primary Auditory T – Tactile V – Visual

63. Add node for phonological production V M C T P PA PP For example, FORK Labels for Properties: C – Conceptual M – Motor P – Phonological image PA – Primary Auditory PP – Phonological Production T – Tactile V – Visual Arcuate fasciculus Articulatory structures (in Broca’s area) that control articulation of the spoken form [fork]

64. Some of the cortical structure relating to fork V MC T P PA PP

65. Functional web of a simple lexeme: fork V M C T P PA PP Phonological form Meaning Link betw form and meaning

66. Part of the functional web for FORK (showing cardinal nodes only) V M C T P PA PP Each node shown here is the cardinal node of a subweb For example, the cardinal node of the visual subweb

67. An activated functional web (with two subwebs partly shown) V PR PA M C PP T Visual features C – Cardinal concept node M – Memories PA – Primary auditory PP – Phonological production PR – Phonological recognition T – Tactile V – Visual

68. Ignition of a functional web from visual input V PR PA M C Art T

69. V PR PA M C Art T Ignition of a functional web from visual input

70. V PR PA M C Art T

71. Ignition of a functional web from visual input V PR PA M C Art T

72. Ignition of a functional web from visual input V PR PA M C Art T

73. Ignition of a functional web from visual input V PR PA M C Art T

74. Ignition of a functional web from visual input V PR PA M C Art T

75. Ignition of a functional web from visual input V PR PA M C Art T

76. Ignition of a functional web from visual input V PR PA M C Art T

77. Ignition of a functional web from visual input V PR PA M C Art T

78. Ignition of a functional web from visual input V PR PA M C Art T

79. Ignition of a functional web from visual input V PR PA M C Art T

80. Ignition of a functional web from visual input V PR PA M C Art T

81. Ignition of a functional web from visual input V PR PA M C Art T

82. Speaking as a response to ignition of a web V PR PA M C Art T

83. Speaking as a response to ignition of a web V PR PA M C Art T

84. Speaking as a response to ignition of a web V PR PA M C Art T From here (via subcortical structures) to the muscles that control the organs of articulation

85. An MEG study from Max Planck Institute Levelt, Praamstra, Meyer, Helenius & Salmelin, J.Cog.Neuroscience 1998

86. Pulvermüller’s line of reasoning 1. “If neurons in the functional web are strongly linked, they should show similar response properties in neurophysiological experiments. 2. “If the neurons of the functional web are necessary for the optimal processing of the represented entity, lesion of a significant portion of the network neurons must impair the processing of this entity. This should be largely independent of where in the network the lesion occurs. 3. “Therefore, if the functional web is distributed over distant cortical areas, for instance, certain frontal and temporal areas, neurons in both areas should (i) share specific response features and (ii) show these response features only if the respective other area is intact.” (2002: 26, see also 27)

87. end