Short-Term Reorganization of Auditory Analysis Induced by Phonetic Experience Liebenthal et al. (2003). JoCN. Audrey Kittredge 593: Neuroimaging of Language Audrey Kittredge 593: Neuroimaging of Language

MRI: physics  Hydrogen nuclei act as magnets (spinning, charged particle)

MRI: physics  In strong magnetic field: spin-axes form vector parallel to field

MRI: procedure  Radio Frequency pulse  Changes direction and strength of vector  Eventually, nuclei relax and vector returns to original position  As nuclei relax, give out pulse  Pulse type depends on water/fat ratio of tissue --> MRI images!  Radio Frequency pulse  Changes direction and strength of vector  Eventually, nuclei relax and vector returns to original position  As nuclei relax, give out pulse  Pulse type depends on water/fat ratio of tissue --> MRI images!

Functional MRI  Hemoglobin shows up better than deoxyhemoglobin on MRI SO  Brain areas with more oxygenated blood will show up better (BOLD)  Hemoglobin shows up better than deoxyhemoglobin on MRI SO  Brain areas with more oxygenated blood will show up better (BOLD)

Connection to neural activity?  Increase in net neural activity --> increase in oxygenated blood supply (slow)  Quick succession of images: BOLD signal at various times  Increase in net neural activity --> increase in oxygenated blood supply (slow)  Quick succession of images: BOLD signal at various times

Pros  Good spatial resolution  Less risky, faster acquisition than PET  Event-related design  Good spatial resolution  Less risky, faster acquisition than PET  Event-related design

Cons  Poor temporal resolution  BOLD signal degraded near air/bone boundary  Movement artifacts  High speed data acquisition = noisy!  Poor temporal resolution  BOLD signal degraded near air/bone boundary  Movement artifacts  High speed data acquisition = noisy!

Phonetic perception  How does this occur?  Automatic phonetic analysis module (Liberman & Mattingly, 1989)  Stimulus-independent auditory analysis (Kluender & Greenberg, 1989)  How does this occur?  Automatic phonetic analysis module (Liberman & Mattingly, 1989)  Stimulus-independent auditory analysis (Kluender & Greenberg, 1989)

Past Research  PET, fMRI studies  Speech vs nonspeech: superior temporal cortex  PET, fMRI studies  Speech vs nonspeech: superior temporal cortex

Problem!  Confound: perception or stimuli?  Goal: study perception mode independent of stimulus properties  How do we do this?…  Confound: perception or stimuli?  Goal: study perception mode independent of stimulus properties  How do we do this?…

…Sinewave speech!  Sinewave example

Original sentence  “The steady drip is worse than a drenching rain”

Sinewave speech: properties  Sinusoid fit to center frequency and amplitude (over time) of F1-F3 or F4  Result: rapidly changing pure tones  Lack fine-grained acoustic properties of speech  Sinusoid fit to center frequency and amplitude (over time) of F1-F3 or F4  Result: rapidly changing pure tones  Lack fine-grained acoustic properties of speech

Past studies on sinewave speech  Remez et al. (1981):  “Describe”: most say non-speech  “Transcribe”: most write all/some of sentence correctly  Remez et al. (1981):  “Describe”: most say non-speech  “Transcribe”: most write all/some of sentence correctly

Tone-matching Task (Remez et al., 2001)  Stimuli  Sinewave word e.g. juice  Isolated T2 from T123/4 complex  Task: is tone constituent of complex?  Listeners can do this…  When uninformed (not speech)  While matching tone complex to printed word  Difficult task!  Stimuli  Sinewave word e.g. juice  Isolated T2 from T123/4 complex  Task: is tone constituent of complex?  Listeners can do this…  When uninformed (not speech)  While matching tone complex to printed word  Difficult task!

Creation of stimuli  Phonetic stimulus (sinewave word)  3 lowest formants = 1 sinewave each  Tone probe  “True”: from word  “False”: from other sinewave word  Nonphonetic stimulus  T1 and T3 temporally reversed  Phonetic stimulus (sinewave word)  3 lowest formants = 1 sinewave each  Tone probe  “True”: from word  “False”: from other sinewave word  Nonphonetic stimulus  T1 and T3 temporally reversed

Spectrogram of Stimuli

Pilot studies  Phonetic transcribed 52.1% accuracy, multiple choice 89.5% accuracy  Rated as “Clearly identifiable word”:  61% phonetic  22% nonphonetic  “Nonspeech”:  58% nonphonetic  20% phonetic  Phonetic transcribed 52.1% accuracy, multiple choice 89.5% accuracy  Rated as “Clearly identifiable word”:  61% phonetic  22% nonphonetic  “Nonspeech”:  58% nonphonetic  20% phonetic

Stimuli: summary  288 stimuli total  108 pairs of phonetic, nonphonetic stimuli  1/3 repeated  1/2 trials = false  288 stimuli total  108 pairs of phonetic, nonphonetic stimuli  1/3 repeated  1/2 trials = false

Experimental Design Naïve 1Naïve 2Practice Phonetic Practice Informed 1 Informed 2

Procedure  Practice  Stimuli: arbitrarily composed sinusoids  Sinewaves: same/diff pitch contour?  Tone-matching task (T2-T1234)  Naïve condition  “single tone”, “tone complex”  2 blocks  Practice  Stimuli: arbitrarily composed sinusoids  Sinewaves: same/diff pitch contour?  Tone-matching task (T2-T1234)  Naïve condition  “single tone”, “tone complex”  2 blocks

Procedure  Phonetic practice  Sinewave stimuli: 8 sentences, 18 words  Chose from 4 transcriptions  Feedback given for every 5th sentence  Accuracy data collected  Informed condition  “words”  2 blocks  Phonetic practice  Sinewave stimuli: 8 sentences, 18 words  Chose from 4 transcriptions  Feedback given for every 5th sentence  Accuracy data collected  Informed condition  “words”  2 blocks

Results: RT  Phonetic:  Test Block p <.o4 (N1-N2 p <.02, N2-I1 p <.03, I1-I2 p <.05)  Nonphonetic  Test Block p <.001 (N1-N2 p <.01)  In naïve condition, effect of stimulus type p <.04  Phonetic:  Test Block p <.o4 (N1-N2 p <.02, N2-I1 p <.03, I1-I2 p <.05)  Nonphonetic  Test Block p <.001 (N1-N2 p <.01)  In naïve condition, effect of stimulus type p <.04

Results: Accuracy  Phonetic:  No significant effect of Test Block p <.11  Nonphonetic  No significant effect of Test Block p <.53  In naïve condition, no effect of stimulus type p <.07  Phonetic:  No significant effect of Test Block p <.11  Nonphonetic  No significant effect of Test Block p <.53  In naïve condition, no effect of stimulus type p <.07

Results: Phonetic Form Practice  Sentence task: 84 +/- 21% accuracy  Words: 60 +/- 16% accuracy  Chance = 25% in both tasks  Sentence task: 84 +/- 21% accuracy  Words: 60 +/- 16% accuracy  Chance = 25% in both tasks

Results: Subjective Reports  29/31 unaware of phonetic quality during naïve blocks  13/31 recognized words during informed blocks  29/31 unaware of phonetic quality during naïve blocks  13/31 recognized words during informed blocks

Conclusions: Behavior  Phonetic awareness interferes with task  Naïve: subjects perceived only auditory form  Informed: subjects perceived both, focused on auditory  NO explanation for stimulus RT difference in Naïve  Phonetic awareness interferes with task  Naïve: subjects perceived only auditory form  Informed: subjects perceived both, focused on auditory  NO explanation for stimulus RT difference in Naïve

Within each block… 2 phonetic trials 2 nonphonetic trials Baseline (silence) Clustered image acquisition 9s 9s 9s 9s 9s9s 9s 9s

Image acquisition  18 images per trial type per block  36 images per condition/trial type  E.g. Naïve, phonetic  18 images per trial type per block  36 images per condition/trial type  E.g. Naïve, phonetic

fMRI Images  16 slices:  Axially oriented (horizontal)  Contiguous  3x3x4mm voxels  Slice coverage:  Most of temporal lobes  Part of frontal and parietal lobes  Occipital lobe  Anatomical (MRI) images (1x1x1mm)  16 slices:  Axially oriented (horizontal)  Contiguous  3x3x4mm voxels  Slice coverage:  Most of temporal lobes  Part of frontal and parietal lobes  Occipital lobe  Anatomical (MRI) images (1x1x1mm)

fMRI analysis: individuals  AFNI software package  Trial - Baseline-->BOLD difference maps  Difference maps:  averaged (BOLD vs baseline)  Voxel-wise ANOVA (sorted by trial type and condition)  AFNI software package  Trial - Baseline-->BOLD difference maps  Difference maps:  averaged (BOLD vs baseline)  Voxel-wise ANOVA (sorted by trial type and condition)

fMRI analysis: averaging  Individual statistical maps transformed into standard space  Talairach brain  Complicated statistics, smoothing…  t values at each voxel averaged across subjects  Individual statistical maps transformed into standard space  Talairach brain  Complicated statistics, smoothing…  t values at each voxel averaged across subjects

fMRI analysis: significance testing  Randomization testing:  t values >/=.37 significant  uncorrected voxel-wise p <.001  Activation foci < 300 microL removed  Randomization testing:  t values >/=.37 significant  uncorrected voxel-wise p <.001  Activation foci < 300 microL removed

fMRI Result Summary

fMRI Images  

Phonetic: Informed-Naive  Left Heschl’s gyrus (HG/BA42)  Left posterior superior temporal gyrus (STG/BA 42/22)  Right HG/BA42  Left Heschl’s gyrus (HG/BA42)  Left posterior superior temporal gyrus (STG/BA 42/22)  Right HG/BA42

Phonetic Experience  Decreased activation = decreased task execution  Underlies reduced performance  Interference masks information like noise  STG  Primate HG/post STG analogues involved in complex sound analysis, auditory STM  Left-lateralized  Specialization for speech  Decreased activation = decreased task execution  Underlies reduced performance  Interference masks information like noise  STG  Primate HG/post STG analogues involved in complex sound analysis, auditory STM  Left-lateralized  Specialization for speech

Phonetic Experience cont’d  No shift to other areas  No conscious phonetic perception  Phonetic experience induces “short-term functional reorganization of auditory analysis” and is contingent on “dynamic structure”  No shift to other areas  No conscious phonetic perception  Phonetic experience induces “short-term functional reorganization of auditory analysis” and is contingent on “dynamic structure”

Phonetic: Informed-Naive  Dorsomedial thalamic nucleus  Superior frontal gyrus (BA8)  Left middle frontal gyrus (MFG/BA10)  Dorsomedial thalamic nucleus  Superior frontal gyrus (BA8)  Left middle frontal gyrus (MFG/BA10)

Unexplained Results  Dorsomedial thalamic nucleus, medial prefrontal cortex:  Areas with reciprocal connections to each other and ST area  Connected neural system…  Engaged in task  Sensitive to interference  Dorsomedial thalamic nucleus, medial prefrontal cortex:  Areas with reciprocal connections to each other and ST area  Connected neural system…  Engaged in task  Sensitive to interference

Nonphonetic: Informed- Naive  Left posterior STG (BA 42/22)

Phonetic: Blocks2-Blocks1  Left middle frontal gyrus (BA9)

Nonphonetic: Blocks1- Blocks2  Left inferior frontal gyrus (IFG/BA44)

Proficiency Effects  Left IFG, MFG:  Initial difficulty in verbal production task (Raichle et al., 1994)  Not cause of Informed-Naïve difference (no anatomical overlap)  Left IFG, MFG:  Initial difficulty in verbal production task (Raichle et al., 1994)  Not cause of Informed-Naïve difference (no anatomical overlap)

What do YOU think?

Conclusions…?  “Centrality” of this function  Naïve: Phonetic vs nonphonetic RT  Reorganization contingent on speech?  Decreased activation: underlies reduced performance?  Proficiency/Informed: frontal overlap?  “Centrality” of this function  Naïve: Phonetic vs nonphonetic RT  Reorganization contingent on speech?  Decreased activation: underlies reduced performance?  Proficiency/Informed: frontal overlap?

Methodology…?  Response/accuracy inclusion criteria?  RT/accuracy data not parallel  RT: correct, incorrect, true, false trials  Word length?  Age variation (18-57)?  Naïve: phonetic vs nonphonetic? (fMRI)  Response/accuracy inclusion criteria?  RT/accuracy data not parallel  RT: correct, incorrect, true, false trials  Word length?  Age variation (18-57)?  Naïve: phonetic vs nonphonetic? (fMRI)

Some questions…  Role of thalamus/medial frontal areas?  Task difficulty --/--> activation increase  Role of thalamus/medial frontal areas?  Task difficulty --/--> activation increase

Some more questions…  Given phonetic practice, is reorganization entirely stimulus- driven?  How generalizable to normal speech-nonspeech analysis?  Original question: automatic phonetic module or auditory analysis?  Given phonetic practice, is reorganization entirely stimulus- driven?  How generalizable to normal speech-nonspeech analysis?  Original question: automatic phonetic module or auditory analysis?

Acknowledgements  You--thanks for listening!  Steve Higgins  explanation of fMRI procedure, analysis  Gary Oppenheim  practice presenting  You--thanks for listening!  Steve Higgins  explanation of fMRI procedure, analysis  Gary Oppenheim  practice presenting

References  Sinewave speech information and samples obtained from:  MISC/SWS/SWS.html MISC/SWS/SWS.html  fMRI physics information obtained from publicly accessible websites and “Handbook of Functional Neuroimaging of Cognition”, Cabeza & Kingstone (Ed),  Sinewave speech information and samples obtained from:  MISC/SWS/SWS.html MISC/SWS/SWS.html  fMRI physics information obtained from publicly accessible websites and “Handbook of Functional Neuroimaging of Cognition”, Cabeza & Kingstone (Ed), 2001.

References: cont’d  Images of atoms courtesy of Duke-UNC Brain Imaging and Analysis Center website  Schematic and anatomical brain images (as well as Uncle Sam image) obtained from various publicly accessible websites and MNI brain images  Images of atoms courtesy of Duke-UNC Brain Imaging and Analysis Center website  Schematic and anatomical brain images (as well as Uncle Sam image) obtained from various publicly accessible websites and MNI brain images