1. Acoustic-Prosodic and Lexical Entrainment in Deceptive Dialogue
Sarah Ita Levitan
With: Jessica Xiang, Julia Hirschberg
Advanced Topics in Spoken Language Processing
March 15, 2019

2. Research Questions
Do interlocutors entrain in acoustic-prosodic and lexical dimensions in deceptive dialogues?
Is entrainment related to deception outcomes?
Is entrainment correlated with ability to deceive or detect deception?
Is there a difference in entrainment behavior between truthful and deceptive speech?

3. Columbia X-Cultural Deception (CXD) Corpus
Corpus of within-subject deceptive and non-deceptive speech
Fake resume paradigm - interview format using 24-item biographical questionnaire
Native speakers of SAE and MC, all speaking in English
170 dialogues between 340 subjects, >122 hours of speech
Global and local deception annotation
Global labels of perception of deception

4. Features 8 acoustic-prosodic features 4 lexical features
Intensity {mean, max}
Pitch {mean, max}
Speaking rate
Jitter, shimmer, NHR
4 lexical features
100, 25 top frequency words
Hedge words, phrases
Cue phrases

5. Entrainment measures Local Global Proximity
Local
Global
Proximity
Local partner_diff < other_diff
Global partner_diff < other diff
Convergence
Correlation between partner_diff and time
partner_diff in last 5 min < partner_diff in first 5 min
Synchrony
Correlation between partner IPU features NA

6. Local Entrainment Feature Proximity Convergence Synchrony t p rt p r
Max Pitch
-3.12 ** 3 NS
0.02
***
Mean Pitch
4.87
-0.006
0.03
Max Intensity
12.82
0.15
Mean Intensity
10.67
0.04
0.16
Speaking Rate
6.04
-0.01
0.08
Jitter
3.95 *
0.05
Shimmer
2.48
0.0005
NHR
2.75
0.012

7. Global Entrainment Feature Proximity Convergence t p High Freq. 100t p
High Freq. 100
0.33 NS
1.99 *
High Freq. 25
2.56
2.05
Hedge
2.82 **
1.29
Cue
0.18
1.18
Max Pitch
2.10
-0.56
Mean Pitch
0.89
0.14
Max Intensity
3.94
***
0.02
Mean Intensity
4.26
-0.49
Speaking Rate
3.98
1.04
Jitter
3.20
0.37
Shimmer
3.44
1.58
NHR
2.31
0.92

8. Deception Analysis
Is there a difference in entrainment behavior between truthful and deceptive speech? ✔ Greater local proximity in max intensity in deceptive speech than truthful speech (t(7244)=3.08; p=0.02)) ✔ Greater local proximity in jitter in truthful speech than deceptive speech (t(7226)=2.66; p=0.008))

9. Deception Analysis
Is there a difference in entrainment behavior between speech that is trusted or mistrusted? ✔ Greater local proximity in mean intensity in speech judged to be deceptive than speech judged to be truthful (t(7222)=2.45; p=0.014)

10. Deception Analysis
Is there a difference in entrainment behavior between successful and unsuccessful lies? ✘ No significant difference for any feature. It seems that interviewers were not able to perceive entrainment differences in truthful and deceptive speech.

11. Deception Analysis
Is entrainment related to ability to deceive or to judge deception? ✔ Global proximity in high frequency (25) words is correlated with interviewer ability to judge deception (r=0.13; p=0.015) ✘ No significant correlation between entrainment and ability to deceive.

12. Conclusions
Evidence of entrainment in deceptive speech in acoustic-prosodic and lexical dimensions, at both global and local levels
Identified differences in entrainment behavior in truthful vs. deceptive speech, and trusted vs. mistrusted speech
In future work, we will explore leveraging entrainment features for deception and trust classification tasks