Schizophrenia and Depression – Evidence in Speech Prosody Student: Yonatan Vaizman Advisor: Prof. Daphna Weinshall Joint work with Roie Kliper and Dr. Shirley Portuguese

Agenda Presenting the approach – Speech prosody – Mental states, mental pathologies, Schizophrenia Our work – Data – Methods – Results Future directions

Information in speech prosody Syntactic disambiguation from intonation Prosodic content as signal for mental state

Large variability wave formspectrogram

Estimating mental states

Previous works Voice analysis to detect cancer of the larynx (Murry, T. and E. Doherty 1980) Differences in speech between Schizophrenians and normals (e.g. Stassen, H., et al 1995)

Our work - Data Strict expressionFree expression Reading a list of words Reading a passage Free interview Dr. (M.D.) Shirley Portuguese Time (sec)

Our work - Methods Auditory signal processing Collecting acoustic features – Duration features – Variability features – Speech density features Time (sec)

Acoustic feature extraction Duration features: utterancegap Mean utterance duration Mean log utterance duration Mean gap duration Mean log gap duration

Acoustic feature extraction Variability features: Meso scale Pitch (f0) / period Variability Power variability Shimmer Jitter Micro scale

Results – acoustic features List of wordsFluent text During task mean utterance duration (sec) mean jitter score

Results - classification S vs. DC vs. DC vs. S List of words Passage Interview DepressionSchizophreniaControl List of words Passage Interview Sample sizes: Correct classification rates (LOU): Linear classification with SVM

Results – classification separated by gender S vs. DC vs. DC vs. S Both genders Male only Female only S vs. DC vs. DC vs. S Both genders Male only Female only List of words: Interview:

Results – Correlation SANS total (cc, p-val)Acoustic feature ( , )Utterance duration (0.4223, )Gap duration ( , )Spoken ratio ( , )Fragmented speech ( , )Emphasis ( , )Inflection HAM-DAcoustic feature (0.4661, )Utterance duration ( , )Inflection ( , )Jitter Schizophrenia: Depression:

Results - summary Possible to find signal to mental state in prosodic features Local and micro-scale features are discriminative Type of speech test influences discrimination Possibly signal is more prominent in females

Future directions Spectral features Using linguistic elements: phonemes, words Forced alignment and functional data analysis Language and speaker identification methods Personal directions: – Other signal domains: general sounds, music…

Thank you!