Deceptive Speech Frank Enos April 25, 2005

Defining Deception Deliberate choice to mislead a target without notification (Ekman‘’01) Often to gain some advantage Excludes:  Self-deception  Theater, etc.  Falsehoods due to ignorance/error  Pathological behaviors

Why study deception? Law enforcement / Jurisprudence Intelligence / Military / Security Business Politics Mental health practitioners Social situations  Is it ever good to lie?

Why study deception? What makes speech “believable”? Recognizing deception means recognizing intention. How do people spot a liar? How does this relate to other subjective phenomena in speech? E.g. emotion, charisma

Problems in studying deception? Most people are terrible at detecting deception — ~50% accuracy (Ekman & O’sullivan 1991, etc.) People use subjective judgments — emotion, etc. Recognizing emotion is hard

Problems in studying deception? Hard to get good data  Real world  Laboratory Ethical issues  Privacy  Subject rights  Claims of success But also ethical imperatives:  Need for reliable methods  Debunking faulty methods  False confessions

Frank Tells Some Lies Maria: I’m buying tickets to Handel’s Messiah for me and my friends — would you like to join us? Frank: When is it? Maria: December 19th. Frank: Uh… the 19th… Maria: My two friends from school are coming, and Robin… Frank: I’d love to!

How to Lie (Ekman‘’01) Concealment Falsification Misdirecting Telling the truth falsely Half-concealment Incorrect inference dodge.

Frank Tells Some Lies Maria: I’m buying tickets to Handel’s Messiah for me and my friends — would you like to join us? Frank: When is it? Maria: December 19th. Frank: Uh… the 19th… Maria: My two friends from school are coming, and Robin… Frank: I’d love to! Concealment Falsification Misdirecting Telling the truth falsely Half-concealment Incorrect inference dodge.

Frank Tells Some Lies Maria: I’m buying tickets to Handel’s Messiah for me and my friends — would you like to join us? Frank: When is it? Maria: December 19th. Frank: Uh… the 19th… Maria: My two friends from school are coming. Concealment Falsification Misdirecting Telling the truth falsely Half-concealment Incorrect inference dodge. Frank: Oh gee, I’m having an appendectomy that night.

Reasons To Lie (Frank‘’92 ) Self-preservation Self-presentation *Gain Altruistic (social) lies

How Not To Lie (Ekman‘’01) Leakage  Part of the truth comes out  Liar shows inconsistent emotion  Liar says something inconsistent with the lie Deception clues  Indications that the speaker is deceiving  Again, can be emotion  Inconsistent story

How Not To Lie (Ekman‘’01) Bad lines  Lying well is hard  Fabrication means keeping story straight  Concealment means remembering what is omitted  All this creates cognitive load  harder to hide emotion Detection apprehension (fear)  Target is hard to fool  Target is suspicious  Stakes are high  Serious rewards and/or punishments are at stake  Punishment for being caught is great

How Not To Lie (Ekman‘’01) Deception guilt (vs. shame)  Stakes for the target are high  Deceit is unauthorized  Liar is not practiced at lying  Liar and target are acquainted  Target can’t be faulted as mean or gullible  Deception is unexpected by target Duping delight  Target poses particular challenge  Lie is a particular challenge  Others can appreciate liar’s performance

Features of Deception Cognitive  Coherence, fluency Interpersonal  Discourse features: DA, turn-taking, etc. (Some addressed by Statement Analysis) Emotion

Describing Emotion Primary emotions  Acceptance, anger, anticipation, disgust, joy, fear, sadness, surprise One approach: continuous dim. model (Cowie/Lang) Activation – evaluation space Add control/agency Primary E’s differ on at least 2 dimensions of this scale (Pereira)

Problems With Emotion and Deception Relevant emotions may not differ much on these scales Othello error  People are afraid of the police  People are angry when wrongly accused  People think pizza is funny Brokow hazard  Failure to account for individual differences

20th Century Lie Detection Polygraph   The Polygraph and Lie Detection (N.A.P. 2003) Voice Stress Analysis  Microtremors 8-12Hz  Universal Lie response   Reid  Behavioral Analysis Interview  Interrogation

Deception Experiments (Frank‘’92) Addresses lying as dependent variable. Type and form of lie  Concealment  Falsification  Misdirecting  Telling the truth falsely  Half-concealment  Incorrect inference dodge. Motive for Lying  Self-preservation  Self-presentation  Gain  Altruistic (social) lies

Deception Experiments (Frank‘’92) Addresses lying as dependent variable. Scenario  *Topic of the lie: opinion; state; event.  Stakes for lying / stakes for telling the truth.  Interval between event and subject’s account. Interpersonal structure  Characteristics of the liar  Characteristics of the target  Presence or absence of a “coach”  Presence or absence of others

The Good Old Days Mehrabian 1971: Nonverbal Betrayal of Feeling

Bulk of extant deception research… Not focused on verifying 20th century techniques Done by psychologists Considers primarily facial and physical cues “Speech is hard” Little focus on automatic detection of deception

Modeling Deception in Speech Lexical Prosodic/Acoustic Discourse

Deception in Speech (Depaulo ’03) Positive Correlates  Interrupted/repeated words  References to “external” events  Verbal/vocal uncertainty  Vocal tension  F0

Deception in Speech (Depaulo ’03) Negative Correlates  Subject stays on topic  Admitted uncertainties  Verbal/vocal immediacy  Admitted lack of memory  Spontaneous corrections

Problems, revisited Differences due to:  Gender  Social Status  Language  Culture

Columbia/SRI/Colorado Corpus With Julia Hirschberg, Stefan Benus, Sarah Friedman, Sarah Gilman, and colleagues from SRI/ICSI and U. C. Boulder Goals  Examine feasibility of automatic deception detection using speech  Discover or verify acoustic/prosodic, lexical, and discourse correlates of deception  Model a “non-guilt” scenario  Create a “clean” corpus

Columbia/SRI/Colorado Corpus Inflated-performance scenario Motivation: financial gain and self-presentation 32 Subjects: 16 women, 16 men Native speakers of Standard American English Subjects told study seeks to identify people who match profile based on “25 Top Entrepreneurs”

Columbia/SRI/Colorado Corpus Subjects take test in six categories:  Interactive, music, survival, food, NYC geography, civics Questions manipulated   2 too high; 2 too low; 2 match Subjects told study also seeks people who can convince interviewer they match profile  Self-presentation + reward Subjects undergo recorded interview in booth  Indicate veracity of factual content of each utterance using pedals

CSC Corpus: Data 15.2 hrs. of interviews; 7 hrs subject speech Lexically transcribed & automatically aligned  lexical/discourse features Lie conditions: Big Lie / Little Lie Segmentations (LT/LL): slash units (5709/3782), phrases (11,612/7108), turns (2230/1573) Acoustic features (± recognizer output)

CSC Corpus: Results Classification (Ripper rule induction, randomized 5-fold cv)  Slash Units / Little Lies — Baseline 39.8% err Lexical & acoustic: 37.2 %; + subject dependent: 33.6%  Phrases / Little Lies — Baseline 38.2% err Lexical & acoustic 34.0%; + subject dependent: 27.9% Other findings  Positive emotion words  deception (LIWC)  Pleasantness  deception (DAL)  Filled pauses  truth  Some pitch correlation — varies with subject

Our Future Work Individual differences  Wizards of deception Mark Frank Mock Theft Paradigm New paradigm  Shorter  Addition of personality test  Higher stakes?