2. Schizophrenia and Depression – Evidence in Speech Prosody Student: Yonatan Vaizman Advisor: Prof. Daphna Weinshall Joint work with Roie Kliper and Dr. Shirley Portuguese

3. Agenda Presenting the approach – Speech prosody – Mental states, mental pathologies, Schizophrenia Our work – Data – Methods – Results Future directions

4. Information in speech prosody Syntactic disambiguation from intonation Prosodic content as signal for mental state

5. Large variability wave formspectrogram

6. Estimating mental states

7. Previous works Voice analysis to detect cancer of the larynx (Murry, T. and E. Doherty 1980) Differences in speech between Schizophrenians and normals (e.g. Stassen, H., et al 1995)

8. Our work - Data Strict expressionFree expression Reading a list of words Reading a passage Free interview Dr. (M.D.) Shirley Portuguese Time (sec)

9. Our work - Methods Auditory signal processing Collecting acoustic features – Duration features – Variability features – Speech density features Time (sec)

10. Acoustic feature extraction Duration features: utterancegap Mean utterance duration Mean log utterance duration Mean gap duration Mean log gap duration

11. Acoustic feature extraction Variability features: Meso scale Pitch (f0) / period Variability Power variability Shimmer Jitter Micro scale

12. Results – acoustic features List of wordsFluent text During task mean utterance duration (sec) mean jitter score

13. Results - classification S vs. DC vs. DC vs. S 0.71430.700.7381 List of words 0.52000.690.8095 Passage 0.76670.87100.75 Interview DepressionSchizophreniaControl 202220List of words 151011Passage 293133Interview Sample sizes: Correct classification rates (LOU): Linear classification with SVM

14. Results – classification separated by gender S vs. DC vs. DC vs. S 0.76670.87100.75 Both genders 0.45160.73080.6857 Male only 0.92310.93330.6923 Female only S vs. DC vs. DC vs. S 0.71430.700.7381 Both genders 0.47830.63160.5417 Male only 0.63160.75000.8824 Female only List of words: Interview:

15. Results – Correlation SANS total (cc, p-val)Acoustic feature (-0.5953, 0.0004)Utterance duration (0.4223, 0.0180)Gap duration (-0.5134, 0.0031)Spoken ratio (-0.4831, 0.0059)Fragmented speech (-0.4782, 0.0065)Emphasis (-0.3475, 0.0554)Inflection HAM-DAcoustic feature (0.4661, 0.0164)Utterance duration (-0.5007, 0.0092)Inflection (-0.4555, 0.0194)Jitter Schizophrenia: Depression:

16. Results - summary Possible to find signal to mental state in prosodic features Local and micro-scale features are discriminative Type of speech test influences discrimination Possibly signal is more prominent in females

17. Future directions Spectral features Using linguistic elements: phonemes, words Forced alignment and functional data analysis Language and speaker identification methods Personal directions: – Other signal domains: general sounds, music…

18. Thank you!