Synesthesia Synesthesia Romke Rouw University of Amsterdam

Synesthesia automatic (fast and seemingly effortless) consistent since early youth subjective (individual) highly specific familial trait

Grapheme – Color Synesthesia “The letter A is pale blue” “100 is black and white” “w is light-brownish-yellow”

Perceptual reality

visual search Palmeri et al. 2002

visual search Palmeri et al. 2002

Subtypes Grapheme – Color Synesthesia A 

projector versus associator A 

Explanation?

Cross – activation grapheme and color areas adjacently located in the fusiform gyrus Hubbard et al.

Disinhibited feedback Feedback can also flow back from a multisensory nexus such as the superior temporal sulcus, posterior parietal lobe, intraparietal cortex, or temporo-parietal- occipital junction

Re-entrant processing Feedback from processing graphemic meaning in anterior fusiform travels back to posterior inferior temporal regions and V4/V8

Synesthesia & DTI with Steven Scholte

Subjects 18 females Age range 18 – 43 (mean age 28.7) 90 % consistency No history of psych. disease 18 matched controls 

Synesthetic colors? Colors of digits/letters/days of the week, retest after three weeks Eight digit/letters that give a strong, weak, or no synesthetic color experience

How to measure... Behavioral, fMRI and DTI measurements

Synesthesia & DTI Two questions 1. Are there structural differences between synesthetes and non-synesthetes ? 2. Different neurological mechanisms for projectors and associators?

Synesthesia & DTI Question 1 Are there structural differences between synesthetes and non-synesthetes?

DTI FA: Diffusion Anisotropy IndiceFA: Diffusion Anisotropy Indice Degree to which the three tensor eigenvalues differ from one anotherDegree to which the three tensor eigenvalues differ from one another

DTI FA: Diffusion Anisotropy IndiceFA: Diffusion Anisotropy Indice Degree to which the three tensor eigenvalues differ from one anotherDegree to which the three tensor eigenvalues differ from one another White matter properties White matter properties

Synesthetes > non-synesthetes bilateral cluster in frontal cortex left superior parietal cortex right inferior temporal cortex Increased structural connectivity in synesthesia

Synesthetes > non-synesthetes

Tractography results

Synesthetes > non-synesthetes

Tractography results

DTI & fMRI

Synesthesia & DTI Question 2 Projectors versus Associators?  A  A

How can this be measured... Geef aan in hoeverre deze stellingen overeenkomen met uw synesthetische ervaringen (1 =sterk mee oneens, 5 = sterk mee eens), mits anders aangegeven. 1 Wanneer ik naar een bepaalde letter of cijfer kijk, dan zie ik een specifieke kleur Wanneer ik naar een bepaalde letter/cijfer kijk, verschijnt de bijbehorende kleur alleen in mijn gedachten en niet ergens buiten mijn hoofd (zoals op het papier)

Questionnaire “projector” vs “associator” questions: – when I look at a certain letter or digit, I really see a particular color –I see the color of the letter/digit only in my head five-point Likert Scale questions

Projector versus Associator? PA score projector associator

Projector versus Associator? Max. range is from -4 to 4 Mean = 0.08 St.Dev = N = 18

A   A

FA values subtypes of synesthesia rs(17) =.548, p =.019 rs(17) = -.098, p =.699

Current study…

FA values subtypes of synesthesia rs(10) = rs(6) =.679

Synesthesia & DTI Two questions 1. Structural differences between synesthetes and non-synesthetes ? 2. Neurological mechanisms different for projectors and associators?

Conclusions (1) Increased connectivity in synesthetes:Increased connectivity in synesthetes: –Frontal, parietal, inferior temporal cortex Increased connectivity and correlation between FA and BOLD provides support for cross- activation theory However, both fMRI and DTI show a role of higher brain areas in synesthesia.

Conclusions (2) Of particular interest is a cluster of increased connectivity in inferior temporal cortex, near fusiform gyrus The strength of connectivity in this area correlates with the nature of synesthetic experienceThe strength of connectivity in this area correlates with the nature of synesthetic experience Rouw, R. & H.S. Scholte (2007). Nature Neuroscience, (10), Rouw, R. & H.S. Scholte (2007). Nature Neuroscience, (10),

Thank you

Increased FA in sup frontal cortex

Increased FA in left parietal cortex

Hubbard et al.

Synesthetes > non-synesthetes Cluster size (mm 3 ) Talairach coordinates (mm) x y z Max effect size Sup. frontal l Sup. frontal r Parietal l Temporal r

grapheme task normal italic

grapheme task normal

Results Synesthetic color (Strong + Weak) versus no color: synesthetes > non-synesthetes – Left middle frontal cortex 2944 mm3; CoG 49, -62, -18 (z); t max 3.98 –Right cerebellum, 984 mm3; 42, -68, -22; t max 3.47 –Right middle temporal gyrus 864 mm3; 54, -42, -12; t max 3.8 –Right fusiform gyrus (in posterior right temporal cortex) 1136 mm3; 23, 29, 29; t max 3.5 No activation non-synesthetes > synesthetes

Inferior temporal cortex

Results Synesthetic color (Strong + Weak) versus no color: synesthetes > non-synesthetes – Left middle frontal cortex 2944 mm3; CoG 49, -62, -18 (z); t max 3.98 –Right cerebellum, 984 mm3; 42, -68, -22; t max 3.47 –Right middle temporal gyrus 864 mm3; 54, -42, -12; t max 3.8 –Right fusiform gyrus (in posterior right temporal cortex) 1136 mm3; 23, 29, 29; t max 3.5 No activation controls > synesthetes

FA value – BOLD near/in fusiform gyrus Strong color Weak color No color Controls Spearman's rho Sig. (1-tailed) Synesth. Spearman's rho Sig. (1-tailed)

Questionnaire “projector” vs “associator” questions: – when I look at a certain letter or digit, I really see a particular color –I see the color of the letter/digit only in my head “perceptual” vs “conceptual” questions –I have to actually see a letter/digit to have a synesthetic experience – the synesthetic experience is strongest when I think of a certain letter/digit five-point Likert Scale questions

fMRI studies Inconsistent results: activation in Early visual areas ? Color area/V4 ? Parietal lobe Frontal regions

Visual processes Visual experience

Projectors - Associators Different nature of synesthetic experience. 

FA values synesthetes versus non-synesthetes

FA values in subtypes of synesthesia

Fiber tracking

Diffusion Tensor Imaging Free diffusion Restricted diffusion

Anisotropic diffusion Free diffusion Restricted diffusion Isotropic diffusion Anisotropic diffusion

V1,V2, V3 1 st,2 nd,3 rd eigenvectors L1,L2, L3 1 st,2 nd,3 rd eigenvalues FA Fractional Anisotropy Diffusion Anisotropy indice

Fiber tracking

Inferior temporal cortex Recognition of visual categories Including the visual shape of a word

Sean A. Day's Colored Musical Instruments: Sean A. Day's Colored Musical Instruments: bass saxophone dark neon purple; blobs cornet very bright yellow-orange; metallic, shiny trumpet medium yellow-orange; metallic French horn school-bus yellow; construction paper trombone orange-red; metallic bass trombone very dark orange-red; shiny, glossy harp golden; like elongated raindrops piano light to medium blue; misty clouds, sometimes plasticine harmonica light lime green; like a lime electric guitar pinks, oranges, and reds electric bass dark purple; blobs and walls triangle silver-white; extremely shiny light

fMRI Graphemes task: –Graphemes that elicit strong synesthetic color –Graphemes that elicit weak synesthetic color –Graphemes that elicit no synesthetic color Is this letter/digit/symbol in italic?

Subjects with better performance on the behavioral experiments showed larger fMRI responses in early retinotopic visual areas Individual differences?

FA values synesthetes versus controls

Perceptual tasks

Number forms

Hubbard et al. Neuron, 2005 Activation of both number/letter region and color region of the fusiform gyrus in grapheme-color synesthetes

Controls Synesthetes

Increased FA r inferior temporal cortex

directions

Increased FA in sup frontal cortex

Increased FA in left parietal cortex

Synesthetic Experiences