Deceptive Speech Frank Enos April 19, 2006

Defining Deception Deliberate choice to mislead a target without prior notification (Ekman‘’01) Often to gain some advantage Excludes:  Self-deception  Theater, etc.  Falsehoods due to ignorance/error  Pathological behaviors

Why study deception? Law enforcement / Jurisprudence Intelligence / Military / Security Business Politics Mental health practitioners Social situations  Is it ever good to lie?

Why study deception? What makes speech “believable”? Recognizing deception means recognizing intention. How do people spot a liar? How does this relate to other subjective phenomena in speech? E.g. emotion, charisma

Problems in studying deception? Most people are terrible at detecting deception — ~50% accuracy (Ekman & O’sullivan 1991, Aamodt 2006, etc.) People use subjective judgments — emotion, etc. Recognizing emotion is hard

People Are Terrible At This Group#Studies#SubjectsAccuracy % Criminals Secret service Psychologists Judges Cops Federal officers Students1228, Detectives Parole officers

Problems in studying deception? Hard to get good data  Real world (example)  Laboratory Ethical issues  Privacy  Subject rights  Claims of success But also ethical imperatives:  Need for reliable methods  Debunking faulty methods  False confessions

20th Century Lie Detection Polygraph   The Polygraph and Lie Detection (N.A.P. 2003) Voice Stress Analysis  Microtremors 8-12Hz  Universal Lie response   Reid  Behavioral Analysis Interview  Interrogation

Frank Tells Some Lies An Example…

Frank Tells Some Lies Maria: I’m buying tickets to Händel’s Messiah for me and my friends — would you like to join us? Frank: When is it? Maria: December 19th. Frank: Uh… the 19th… Maria: My two friends from school are coming, and Robin… Frank: I’d love to!

How to Lie (Ekman‘’01) Concealment Falsification Misdirecting Telling the truth falsely Half-concealment Incorrect inference dodge.

Frank Tells Some Lies Maria: I’m buying tickets to Handel’s Messiah for me and my friends — would you like to join us? Frank: When is it? Maria: December 19th. Frank: Uh… the 19th… Maria: My two friends from school are coming, and Robin… Frank: I’d love to! Concealment Falsification Misdirecting Telling the truth falsely Half-concealment Incorrect inference dodge.

Reasons To Lie (Frank‘’92 ) Self-preservation Self-presentation *Gain Altruistic (social) lies

How Not To Lie (Ekman‘’01) Leakage  Part of the truth comes out  Liar shows inconsistent emotion  Liar says something inconsistent with the lie Deception clues  Indications that the speaker is deceiving  Again, can be emotion  Inconsistent story

How Not To Lie (Ekman‘’01) Bad lines  Lying well is hard  Fabrication means keeping story straight  Concealment means remembering what is omitted  All this creates cognitive load  harder to hide emotion Detection apprehension (fear)  Target is hard to fool  Target is suspicious  Stakes are high  Serious rewards and/or punishments are at stake  Punishment for being caught is great

How Not To Lie (Ekman‘’01) Deception guilt  Stakes for the target are high  Deceit is unauthorized  Liar is not practiced at lying  Liar and target are acquainted  Target can’t be faulted as mean or gullible  Deception is unexpected by target Duping delight  Target poses particular challenge  Lie is a particular challenge  Others can appreciate liar’s performance

Features of Deception Cognitive  Coherence, fluency Interpersonal  Discourse features: DA, turn-taking, etc. Emotion

Describing Emotion Primary emotions  Acceptance, anger, anticipation, disgust, joy, fear, sadness, surprise One approach: continuous dim. model (Cowie/Lang) Activation – evaluation space Add control/agency Primary E’s differ on at least 2 dimensions of this scale (Pereira)

Problems With Emotion and Deception Relevant emotions may not differ much on these scales Othello error  People are afraid of the police  People are angry when wrongly accused  People think pizza is funny Brokow hazard  Failure to account for individual differences

Bulk of extant deception research… Not focused on verifying 20th century techniques Done by psychologists Considers primarily facial and physical cues “Speech is hard” Little focus on automatic detection of deception

Modeling Deception in Speech Lexical Prosodic/Acoustic Discourse

Deception in Speech (Depaulo ’03) Positive Correlates  Interrupted/repeated words  References to “external” events  Verbal/vocal uncertainty  Vocal tension  F0

Deception in Speech (Depaulo ’03) Negative Correlates  Subject stays on topic  Admitted uncertainties  Verbal/vocal immediacy  Admitted lack of memory  Spontaneous corrections

Problems, revisited Differences due to:  Gender  Social Status  Language  Culture  Personality

Columbia/SRI/Colorado Corpus With Julia Hirschberg, Stefan Benus, and colleagues from SRI/ICSI and U. C. Boulder Goals  Examine feasibility of automatic deception detection using speech  Discover or verify acoustic/prosodic, lexical, and discourse correlates of deception  Model a “non-guilt” scenario  Create a “clean” corpus

Columbia/SRI/Colorado Corpus Inflated-performance scenario Motivation: financial gain and self-presentation 32 Subjects: 16 women, 16 men Native speakers of Standard American English Subjects told study seeks to identify people who match profile based on “25 Top Entrepreneurs”

Columbia/SRI/Colorado Corpus Subjects take test in six categories:  Interactive, music, survival, food, NYC geography, civics Questions manipulated   2 too high; 2 too low; 2 match Subjects told study also seeks people who can convince interviewer they match profile  Self-presentation + reward Subjects undergo recorded interview in booth  Indicate veracity of factual content of each utterance using pedals

CSC Corpus: Data 15.2 hrs. of interviews; 7 hrs subject speech Lexically transcribed & automatically aligned  lexical/discourse features Lie conditions: Global Lie / Local Lie Segmentations (LT/LL): slash units (5709/3782), phrases (11,612/7108), turns (2230/1573) Acoustic features (± recognizer output)

um i was visiting a friend in venezuela and we went camping Columbia University– SRI/ICSI – University of Colorado Deception Corpus: An Example Segment Breath Group SEGMENT TYPE LABEL ACOUSTIC FEATURES LEXICAL FEATURES LIE max_corrected_pitch 5.7 mean_corrected_pitch 5.3 pitch_change_1st_word -6.7 pitch_change_last_word normalized_mean_energy 0.2 unintelligible_words 0.0 Obtained from subject pedal presses. has_filled_pause YES positive_emotion_word YES uses_past_tense NO negative_emotion_word NO contains_pronoun_i YES verbs_in_gerund YES Produced using ASR output and other acoustic analyses Produced automatically using lexical transcription. LIE PREDICTION

CSC Corpus: Results Classification (Ripper rule induction, randomized 5-fold cv)  Slash Units / Local Lies — Baseline 60.2% Lexical & acoustic: 62.8 %; + subject dependent: 66.4%  Phrases / Local Lies — Baseline 59.9% Lexical & acoustic 61.1%; + subject dependent: 67.1% Other findings  Positive emotion words  deception (LIWC)  Pleasantness  deception (DAL)  Filled pauses  truth  Some pitch correlation — varies with subject

Example JRIP rules: (cueLieToCueTruths >= 2) and (TOPIC = topic_newyork) and (numSUwithFPtoNumSU PEDAL=L (231.0/61.0) (cueLieToCueTruths >= 2) and (numSUwithFPtoNumSU = ) and (wu_F0_SLOPES_NOHD__LAST >= ) => PEDAL=L (284.0/117.0) (cueLieToCueTruths >= 2) and (wu_F0_RAW_MAX >= ) and (wu_DUR_PHONE_SPNN_AV PEDAL=L (262.0/115.0)

CSC Corpus: A Perception Study With Julia Hirschberg, Stefan Benus, Robin Cautin and colleagues from SRI/ICSI 32 Judges Each judge rated 2 interviews Judge Labels:  Local Lie using Praat  Global Lie on paper Takes pre- and post-test questionnaires Personality Inventory Judge receives ‘training’ on one subject.

By Judge 58.2% Acc. By Interviewee

Personality Measure: NEO-FFI Costa & McCrae (1992) Five-factor model  Openness to Experience  Conscientiousness  Extraversion  Agreeability  Neuroticism Widely used in psychology literature

Neuroticism, Openness & Agreeableness correlate with judge performance WRT Global lies.

These factors also provide strongly predictive models for accuracy at global lies.

Other Perception Findings No effect for training Judges’ post-test confidence did not correlate with pre-test confidence Judges who claimed experience had significantly higher pre-test confidence  But not higher accuracy! Many subjects used disfluencies as cues to D.  In this corpus, disfluencies correlate with TRUTH! (Benus et al. ‘06)

Our Future Work Individual differences  Wizards of deception Predicting Global Lies  Local lies as ‘hotspots’ New paradigm  Shorter  Addition of personality test for speakers  Addition of cognitive load