2. Computational Linguistics Analysis of Charismatic Speech: Cross-Cultural and Political Perspectives Andrew Rosenberg NLP & Psychology 11/12/2015

3. Collaborators original slides by Julia Hirschberg Erica Cooper, Svetlana Stenchicova Stoyanchev, Wisam Dakka, Aron Wahl, Judd Sheinholtz, Fadi Biadsy, Rolf Carlson, Eva Strangert, Ian Kaplan, William (Yang) Wang

4. 12345 Which of these speakers sound charismatic? Listening Experiment

5. 1. Vladimir Lenin 3. Mao Zedong2. Franklin D. Roosevelt 4. Warren G. Harding 5. Adolf Hitler

6. What is Charisma?

7. The ability to attract, and retain followers by virtue of personal characteristics -- not traditional or political office (Weber ‘ 47) –E.g. Gandhi, Hitler, Castro, Martin Luther King Jr.,.. –Personalismo What makes an individual charismatic? (Bird ’ 93, Boss ’ 76, Dowis ’ 00, Marcus ’ 67, Touati ’ 93, Tuppen ’ 74, Weber ‘ 47) –Their message? –Their personality? –Their speaking style?

8. What is Charismatic Speech? Circularly… –Speech that leads listeners to perceive the speaker as charismatic What aspects of speech might contribute to the perception of a speaker as charismatic? –Content of the message? –Lexico-syntactic features? –Acoustic-prosodic features?

10. Our Goals Understand which aspects of speech are correlated with observer perceptions of charisma To identify charismatic in political leaders – current and future To create more charismatic text-to-speech synthesizers

11. Our Approach Collect tokens of charismatic and non-charismatic speech from a small set of speakers on a small set of topics Ask listeners to rate the ‘ The speaker is charismatic ’ plus statements about a number of other attributes (e.g. The speaker is …boring, charming, persuasive,…) Correlate listener ratings with lexico-syntactic and acoustic-prosodic features of the tokens to identify potential cues to perceptions of charisma

12. Outline Charisma in American English political speech Comparison with text: what is said vs. how it is said Charisma in Arabic speech Cross-cultural perceptions of charisma Charisma in recent American politics: –Democrats in 2008 –Republican today

13. Outline Charisma in American English political speech Comparison with text: what is said vs. how it is said Charisma in Arabic speech Cross-cultural perceptions of charisma Charisma in recent American politics: –Democrats in 2008 –Republican today

14. American English Perception Study Data: 45 2-30s speech segments, 5 each from 9 candidates for Democratic nomination for U.S. president in 2004 –Liberman, Kucinich, Clark, Gephardt, Dean, Moseley Braun, Sharpton, Kerry, Edwards –2 ‘ charismatic ’, 2 ‘ not charismatic ’ –Topics: greeting, reasons for running, tax cuts, postwar Iraq, healthcare –4 genres: stump speeches, debates, interviews, ads 8 subjects rated each segment on a Likert scale (1- 5) 26 questions in a web survey26 questions Duration: avg. 1.5 hrs, min 45m, max ~3hrs

15. Results: How Much Do Subjects Agree with Each Other? Over all statements? –Using Cohen’s) kappa statistic with quadratic weighting, mean  = 0.207 On the charismatic statement?  = 0.232 (8 th most agreed upon statement) By token? –No significant differences across all tokens although Edwards and Liberman tokens show significantly more agreement across all statements By statement? –Individual statements demonstrate significantly different agreements (most agreement: The speaker is accusatory, angry, passionate, intense; least agreement: The speaker is trustworthy, believable, reasonable, trustworthy)

16. Results: What Do Subjects Mean by Charismatic? Which other statements are most closely correlated with the charismatic statement? (determined by kappa): a functional definition The speaker is enthusiastic0.620 The speaker is persuasive0.577 The speaker is charming0.575 The speaker is passionate0.543 The speaker is boring-0.513 The speaker is convincing0.499

17. Results: Does Whether a Subject Agrees with the Speaker or Finds the Speaker ‘ Clear ’ Affect Charisma Judgments Whether a subject agrees with a token does not correlate highly with charisma judgments (  = 0.30) Whether a subject finds the token clear does not correlate highly with charisma judgments (  = 0.26)

18. Results: Does the Identity of the Speaker Affect Judgments of Charisma? There is a significant difference between speakers (p=2.20e-2) Most charismatic –Rep. John Edwards (mean 3.86) –Rev. Al Sharpton (3.56) –Gov. Howard Dean (3.40) Least charismatic –Sen. Joseph Lieberman (2.42) –Rep. Dennis Kucinich (2.65) –Rep. Richard Gephardt (2.93)

19. Results: Does Recognizing a Speaker Affect Judgments of Charisma? Subjects asked to identify which, if any, speakers they recognized at the end of the study. Mean number of speakers believed to have been recognized, 5.8 Subjects rated ‘ recognized ’ speakers as significantly more charismatic than those they did not believe they had recognized (mean 3.39 vs. mean 3.30).

20. Results: Does Genre or Topic Affect Judgments of Charisma? Recall that tokens were taken from debates, interviews, stump speeches, and campaign ads –Genre does influence charisma ratings (p=.0004) Stump speeches were the most charismatic (3.38) Interviews were the least (2.96) Topic does affect ratings of charisma significantly (p=.0517) –Healthcare > post-war Iraq > reasons for running neutral > taxes

21. What makes Speech Charismatic? Features Examined Duration (secs, words, syls) Charismatic speech is personal: Pronoun density Charismatic speech is contentful: Function/content word ratio Charismatic speech is simple: Complexity: mean syllables/word (Dowis) Disfluencies Repeated words Min, max, mean, stdev F0 (Boss, Tuppen) –Raw and normalized by speaker Min, max, mean, stdev intensity Speaking rate (syls/sec) Intonational features: –Pitch accents –Phrasal tones –Contours

22. Results: Lexico-Syntactic Correlates of Charisma Length: Greater number of words positively correlates with charisma (r=.13; p=.002) Personal pronouns: –Density of first person plural and third person singular pronouns positively correlates with charisma (r=.16, p=0; r=.16, p=0) –Third person plural pronoun density correlates negatively with charisma (r=-.19,p=0) Content: Ratio of adjectives/all words negatively correlates with charisma (r=-.12,p=.008) Complexity: Higher mean syllables/word positively correlates with charisma (p=.034)

23. Disfluency: greater % negatively correlates with charisma (r=-.18, p=0) Repetition: Proportion of repeated words positively correlates with charisma (r=.12, p=.004)

24. Results: Acoustic-Prosodic Correlates of Charisma Pitch: –Higher F0 (mean, min, mean HiF0, over male speakers) positively correlates with charisma (r=.24,p=0;r=.14 p=0;r=.20,p=0) Loudness: Mean rms and sdev of mean rms positively correlates with charisma (r=.21,p=0;r=.21,p=0) Speaking Rate: –Faster overall rate (voice/unvoiced frames) positively correlates with charisma (r=.16,p=0)

25. Duration: Longer duration correlates positively with charisma (r=.09,p=.037) Length of pause: sdev negatively correlates with charisma (r=-.09,p=.004)

26. Results: Intonational Correlates of Charisma (Hand-Annotated Features) Pitch Accent Type: –Positive correlation with !H* and L+H* accents (r=.09,p=0;r=.09,p=.034) –Negative correlation with L*, H* and L*+H accents (r=-.13,p=.002;r=-.11,p=.014;r=-.08,p=.052) Phrasal Types –Negative correlation with !H-L% and !H- endings (r=-.11,p=.015;r=-.10,p=.026)

27. Summary for American English In Standard American English, charismatic speakers tend to be those also highly rated for enthusiasm, charm, persuasiveness, passionateness and convincingness – they are not thought to be boring Charismatic utterances tend to be longer than others, to contain a lower ratio of adjectives to all words, a higher density of first person plural and third person singular pronouns and fewer third person plurals, fewer disfluencies, a larger percentage of repeated words, and more complex words than non-charismatic utterances

28. Charismatic utterances are higher in pitch (mean, min) with more regularity in pause length, faster, louder with more variation in intensity

29. Outline Charisma in American English political speech Comparison with text: what is said vs. how it is said Charisma in Arabic speech Cross-cultural perceptions of charisma Charisma in recent American politics: –Democrats in 2008 –Republican today

30. Replication of Perception Study from Text Alone Lower statement agreement, much less on charismatic statement, different speakers most/least charismatic (Clark most, Sharpton least) `Agreement with speaker ’, genre and topic had stronger correlations with charisma Lexico-syntactic features show weaker correlations –1 st person pronoun density negatively correlated and complexity not at all –Similar to speech experiment for duration, function/content, disfluencies, repeated words

31. Outline Charisma in American English political speech Comparison with text: what is said vs. how it is said Charisma in Arabic speech Cross-cultural perceptions of charisma Charisma in recent American politics: –Democrats in 2008 –Republican today

32. Hypothesis: Perception of Charisma Differs by Culture People of different languages and cultures perceive charisma differently In particular, they perceive charismatic speech differently –Do Arabic listeners respond to American politicians the same way Americans do? –Do Americans respond to Arabic politicians the way Arabs do?

33. Charismatic Speech in Palestinian Arabic Are these tokens charismatic?: Are these?:

34. Palestinian Arabic Perception Study Same paradigm as for SAE Materials: –44 speech tokens from 22 male native-Palestinian Arabic speakers taken from Al-Jazeera TV talk shows –Two speech segments extracted for each speaker from the same topic (one we thought charismatic and one not) Web form with statements to be rated translated into Arabic Subjects: 12 native speakers of Palestinian Arabic

35. Data Total corpus duration: 10.3 minutes Token with min duration: 3 seconds Token with max duration: 28 seconds Average token duration: 14 seconds Total number of words: 1322 words Token with min words: 9 words Token with max words: 65 words Average number of words in token: 30 words

36. How Does Charisma Differ in Arabic? Subjects agree on judgments a bit more (κ=.225) than for English (κ=.207) but still low –Agree most on clarity of msg, enthusiasm, charisma, intensity – all differing from Americans –Agree least on desperation (as SAE), friendliness, ordinariness, spontaneity of speaker –Charisma statement correlates (positively) most strongly with speaker toughness, powerfulness, persuasiveness, charm, and enthusiasm and negatively with boringness

37. Role of speaker identity important in judgments of charisma in Arabic as in English –Most charismatic speakers: Ibrahim Hamami (4.75), Azmi Bishara (4.42), Mustafa Barghouti (4.33) –Least: Shafiq Al-Hoot (3.10), Mohammed Al-Tamini (3.42), Azzam Al-Ahmad (3.33) –Raters claimed to recognize only.55 (of 22) speakers on average, perhaps because the speakers were less well known than the Americans Topic important in charisma ratings (r=0,p=.043) Israeli separation wall > assassination of Hamas leader > debates among Palestinian groups > the Palestinian Authority and calls for reform > the Intifada and resistance

38. Lexical Cues to Charisma Length in words positively correlates with charisma, as for Americans Disfluency rate negatively correlates, as for Americans Repeated words positively correlates with charisma, as for Americans Presence of Arabic ‘ dialect markers ’ (words, pronunciations) negatively correlates with charisma Density of third person plural pronouns positively correlates w/ charisma – differing from Americans

39. Acoustic/Prosodic Cues to Charisma Duration positively correlated with charisma, as for Americans Speaking rate approaches negative correlation – opposite from American –But rate of the fastest intonational phrase in the token positively correlated for both languages –Sdev of rate across intonational phrases positively correlated for charisma in Arabic Pauses –#pauses/words ratio positively correlated with charisma but not for Americans

40. –Sdev of length of pause positively correlated in Arabic but negatively for Americans Pitch: –Mean pitch positively correlates (as for Americans) but also F0 max and sdev –Min pitch negatively correlates (opposite from Americans) Intensity: Sdev positively correlates w/ charisma

41. Outline Charisma in American English political speech Comparison with text: what is said vs. how it is said Charisma in Arabic speech Cross-cultural perceptions of charisma Charisma in recent American politics: –Democrats in 2008 –Republican today

42. How Are Perceptions of Charisma Similar Across Palestinian and Arabic Cultures? Level of subject agreement on statements Role of speaker ID, topic in charisma judgments Positive correlations with charisma –Mean pitch and range –Duration, repeated words –Speaking rate of fastest IP Negative correlations with charisma –Disfluencies

43. How Do Charisma Judgments Differ Across Cultures? Statements most and least agreed upon For Arabic vs. English: –Positive correlations with charisma Sdev of speaking rate, pause/word ratio, sdev of pause length, F0 max and sdev, sdev intensity –Negative correlations with charisma Dialect, density of third person plural pronouns Speaking rate, min F0

44. Ratings Within and Across Cultures Cross-cultural comparison of subjects ratings: –American raters/Arabic speech –Palestinian raters/English speech, –Swedish raters/English speech Do native and non-native raters differ on mean scores per token? Yes, Eng/Pal rating English, Swe/Pal rating English Do mean scores correlate per token? Yes, for all

45. Amer and Swe rating English: –paired t-test betw means per token: p-value = 0.03064 –cor between means of rater-normalized ratings: r = 0.60, p-value = 1.170e-05 Amer and Pal rating English: –paired t-test betw means: p-value = 0.1048 –cor between means of rater-normalized ratings: r = 0.47, p-value = 0.0009849 Amer and Pal rating Arabic: –paired t-test betw means: p-value = 0.00164 –cor between means of rater-normalized ratings: r = 0.72, p-value = 3.049e-08 Swe and Pal rating English: –paired t-test betw means: p-value = 0.8479 –cor between means of rater-normalized ratings: r = 0.55, p-value = 9.467e-05

46. Charisma and Politics Can we predict the Winners? or The Losers?

47. Goals and Research Questions Investigate emotion and charisma in political discourse –Collect a corpus of political speech: debates –Determine lexical and acoustic correlates of political success: correlate with subsequent polls –Characterize similarities and differences between Democrats and Republicans

48. Corpus Collection Democratic primaries from 2008: –Biden, Clinton, Dodd, Edwards, Gravel, Kucinich, Obama, Richardson –New Hampshire MSNBC Debate, September 26, 2007 Fall 2011 Republican primaries: –Bachmann, Cain, Gingrich, Huntsman, Johnson, Paul, Perry, Romney, Santorum –Fox News / Google Debate, September 22, 2011 Interviews for all candidates

49. Gallup Poll Results DemocratsPoll ResultsRepublicalsPoll Results Clinton47%Romney20% Obama26%Cain18% Edwards11%Perry15% Richardson4%Paul8% Biden2%Gingrich7% Dodd1%Bachmann5% Kucinich1%Santorum3% Gravel0.5%Huntsman2% Johnson0%

50. Lexical Features For Debate speech only –Word Count –Laughter, applause, and interruptions –Number of speaker turns –Average number of words per turn –Syllables per word –Disfluencies per word –Linguistic Inquiry and Word Count (LIWC) features

51. Acoustic Features Debates and Interviews: mean, min, max, and standard deviation of f0 Difference for these values between debates and interviews

52. Results: Corrlates with Post-Debate Poll Standing FeaturesDemocractsRepublicans Word Count Laughs Applause Turns Interruptions Avg Words Per Turn Avg Syllables Per Word Disfluencies Per Word Mean-f0 Difference Min-f0 Difference 0.804 0.084 0.194 0.846 0.052 0.199 0.015 -0.805 0.059 0.002 0.582 0.780 0.350 0.381 0.247 0.406 -0.768 -0.130 0.479 0.105

53. LIWC Positive Correlates for Democrats FeatureExamplesCorrelates InsightThink, know consider0.74 InclusiveAnd, with, include0.680 CognitiveCause, know, ought0.679 Words per sentece0.661 Common verbsWalk, went, see0.657 Positive emotionLove, nice, sweet0.632 PronounsI, them, itself0.626 NonfluenciesEr, hmm, umm0.611

54. LIWC Features: Negative Correlates for Democrats FeaturesExamplesCorrelation ArticlesA, an, the-0.684 All Punctuation-0.653 Periods.-0.639 Dashes--0.595 Numerals12, 38, 156-0.581 Questions marks?-0.578 IngestionDish, eat, pizza-0.575-0.575 MoneyAudit, cash, owe-0.548 2 nd Person PronounsYou, your-0.506

55. LIWC Features: Correlates for Republicans FeaturesExamplesCorrelation LeisureCook, chat, movie0.869 Past tenseWent, ran, had0.732 “Other” punctuation!”:%$0.665 Dictionary words0.541 Personal pronounsI, them, her0.525 6+ lettersAbandon, abrupt, absolute -0.741 HomeApartment, kitchen, family -0.513

56. General Party Differences Democrat audiences laughed more Republican audience applauded more Democrats got interrupted more Republicans had more words per turn Both had about the same number of syllables per word Democrats' debate speech was more different from their interviews, but Republicans responded slightly more positively to these differences.

57. Conclusions Democrats: Talk more, use fewer disfluencies, and use more Insight words Republicans: Make the audience laugh, use Leisure-related words, and avoid longer words

58. Historical Political Debates Televised presidential debates first held in 1960. Then 1976-present. Polling information is available from CBS polling from 1976-present

59. Predicting debate winners Try to predict winners of the current debate given all prior debates. –“fighting the last war” –Only those debates with one democrat and one republican –Both VP and Presidential Debates

60. Debate features Word Usage: 1) personal pronouns, 2) number of questions, 3) pleasantries, 4) contractions, 5) numbers, 6) references to the opponent, 7) references to running mate. Topic Words: seed WordNet search with “god”, “health”, “religion, “tax”, and “war” –WordNet-defined attribute, synonym, antonym, derivationally-related form, synset entry, troponym, meronym, member meronym, domain category, of the seed term and first-degree words added from the seed term

61. Debate features Affective Content: average and sum of Dictionary of Affect in Language activation, imagery & pleasantness Named Entities: number of times and rate of mentions of people, locations and organizations Term Vector: tf*idf of the top 10k words (stemmed, omitting stop words) Length-features: syllable, word, sentence length of turns Complexity: Flesch-Kincaid grade readability.

62. Debate winner prediction results Train on all previous Just 2008 Unsplit (full debate) Split (predict each 20 turns) Debate (full debate) 0.64 (0.68) p=0.75 0.61 (0.65) p=1.0 Debater (only one turn) 0.57 (0.5) p=0.230.51 (0.5) p=0.47 Unsplit (full debate) Split (predict each 20 turns) Debate (full debate) 1.0 (0.68) p=0.21 0.72 (0.65) p=0.35 Debater (only one turn) 0.75 (0.5) p=0.140.61 (0.5) p=0.12

63. Debate Results: Important Features First person plural possessives (“our”) used by Democrats “sat”, “estimates”, “plain”, “advocate”, “roughly” “atomic” Grade level readability of Democrats Second person singular object pronouns used by Republicans “disagreements”, “unilateral”, “soviets”

64. Debate Results: Feature analysis First person plural possessives (“our”) used by Democrats –Negatively correlates with Democratic wins Grade level readability of Democrats –Lower complexity in Democratic wins Second person singular object pronouns (“you”) used by Republicans –Negatively correlates with Republican wins “atomic”, “unilateral”, “soviets” –important political topics “achieve”, “disagreements” –political action

65. Future Work Investigate additional acoustic features (intensity, speaking rate, voice quality) Higher level prosodic features (intonational contours, phrasing) Lexical features from interviews and debates More complicated / interesting lexical features –Cross-cultural similarities and differences –Entrainment Interpersonal relationships

66. Thank you!

67. The Prediction of Charisma Arabic English

68. Arabic Prosodic Phenomena MSA vs. Dialect A word is considered dialectal if: –It does not exist in the standard Arabic lexicon –It does not satisfy the MSA morphotactic constraints –Phonetically different (e.g., ya?kul vs. ywkil) In corpus of tokens –8% of the words are dialect. –80% of the dialect words are accented.

69. Arabic Prosody: Accentuation 70% of words are accented 60% of the de-accented words are function words or disfluent items –Based on automatic POS analysis (MADA) –12% of content words are deaccented Distribution of accent types: –H* or !H* pitch accent, 73% –L+H* or L+!H*, 20% –L*, 5% –H+!H*, 2%

70. Arabic Prosody: Phrasing Mean of 1.6 intermediate phrases per intonational phrase Intermediate phrases contain 2.4 words on average Distribution of phrase accent/boundary tone combinations –L-L%59% –H-L%26% –L-H%8% –H-L%6% –H-H%1%

71. Arabic Prosody – most common contours H* L-21.9 H* H-13.4 L+H* L-9.7 H* H* L-7.6 H* !H* L-4.1 L* L-4.1 L+H* !H* L-3 H* H* H-3 H* !H* !H* L-2.3 L+H* H-2.1

72. Arabic Prosody – Disfluency In addition to standard disfluency: –Hesitations –filled pauses –self-repairs In Arabic, speakers could produce a sequence of all of the above. (see praat: file: 1036 and 2016) Disfluency may disconnect prepositions and conjunctions from the content word: – ولتأتي => و... لـ... يعني... تأتي –w- l- uh- yEny uh- t?ty instead of wlt?ty

73. Functional Definition of Charisma According to our 12 native American English (AE) speakers who rated AE speech, “ charisma is ”

74. Analyzing Charismatic Speech

75. Functional Definition of Charisma According to our 12 native Palestinian Arabic (PA) speakers who rated PA speech, “ charisma is ”

76. Examples of features that correlate with Charisma Variation in length of pauses: negative correlate in English but positive in Arabic Speaking Rate: positive correlate in English and approaching negative correlate in Arabic Mean Pitch: positive correlate in both studies

77. Examples of features that correlate with Charisma High variation in loudness: positive correlate in Arabic Disfluency: rate of disfluencies (repetitions, repairs, and filled pauses) negatively correlated with charisma in both First person plural pronoun: positive correlate in English only Ratio of adverbs and adjectives to number of words in utterance: negative correlate in English