1. Analysis of Spontaneous Speech in Dementia of Alzheimer Type: Experiments with Morphological and Lexical Analysis
Nick Cercone
Vlado Keselj
Calvin Thomas
Computer Science
Dalhousie University
Kenneth Rockwood
Medicine, Dalhousie University
Elissa Asp
English Deparment
Saint Mary’s University
PUL Workshop, Dalhousie University, Halifax, 23 Apr 2004

2. Overview Introduction
Related work: Bucks et al, authorship attribution
CNG discrimination Pt/other
rating dementia levels
use of attribute sets: MA-A, MA-B
CNG and Ordinal CNG
Conclusion

3. Introduction Effects of the Alzheimer’s disease (AD) Can we detect it?
reduced communicative ability
deterioration of linguistic performance
Can we detect it?
Current methods rely on structured interviews
confrontation naming
single word production
word generation given context
word generation given first letter
picture description

4. Analysis of spontaneous speech
drawbacks of structured interviews:
sometimes insensitive to early signs of dementia observed by family
low scores are not reliable unless difficulty is observed in natural conversation
brake “natural speech” into components
subjective, i.e., designed by a researcher
alternative solution: objective automatic analysis of spontaneous, i.e., natural, speech

5. Speech characteristics in Dementia of Alzheimer Type (DAT)
frequent use of functional words (closed class)
less rich vocabulary
difficulty with constructing longer coherent phrases
more difficulties at lexical and morphological level than phonetic and syntactic levels

6. Related work: Bucks et al. (BSCW)
Bucks, Singh, Cuerden, Wilcock 2000, 2001:
Analysis of spontaneous conversational speech in dementia of Alzheimer type (DAT)
use eight linguistic measures to analyze transcribed spontaneous speech:
1) noun rate 5) clause-like semantic unit rate
2) pronoun rate (CSU)
3) verb rate 6) Brunet’s index (W)
4) adjective rate 7) token type ratio (TTR)
8) Honore’s statistic (R)

7. Bucks et al.: Experiment design
experiment with 24 participants:
8 patients and 16 healthy individuals
discriminating between demented and healthy individuals:
100% on training data
87.5% with cross-validation

8. Related work: Automated authorship attribution
Problem of identifying the author of an anonymous text
One of Text Categorization Problems
Spam detection
Language and encoding identification
Authorship attribution and plagiarism detection
Text genre classification
Topic detection
Sentiment classification

9. Related work (authorship attribution)
style analysis
using style markers (features)
relying on non-trivial NL analysis
Stamatatos et al
language modeling
Peng et al. 2003, EACL’03
Khmelev and Teahan 2003, SIGIR’03
N-gram-based text categorization
Cavnar and Trenkle 1994

10. Shortcomings of style analysis
difficult to automatically extract some features
feature selection is critical
language dependent
task dependent, i.e., does not generalize well to other types of classification

11. Character N-gram -based Methods
Text can be considered as a concatenated sequence of characters instead of words.
Advantages
1. small vocabulary
2. language independence
3. no word segmentation problems in many Asian languages such as Chinese and Thai

12. How do character n-grams work?
Marley was dead: to begin with. There is no doubt whatever about that. …
(from Christmas Carol by Charles Dickens)
n = 3
Mar
_th 0.015
L=5
arl
___ 0.013
rle
the 0.013
ley
he_ 0.011
sort by frequency
ey_
and 0.007
y_w
_an 0.007
_wa
nd_ 0.007
was
ed_ 0.006 …

13. How do we compare two profiles?
Dickens: A Tale of Two Cities
_th 0.016
Dickens: Christmas Carol ?
the 0.014
_th 0.015
he_ 0.012
___ 0.013
and 0.007
the 0.013
nd_ 0.007
he_ 0.011
Carroll: Alice’s adventures in wonderland
and 0.007
_th 0.017
___ 0.017 ?
the 0.014
he_ 0.014
ing 0.007

14. N-gram distribution
(From Dickens: Christmas Carol)

15. CNG profile similarity measure
a profile = the set of L the most frequent n-grams
profile dissimilarity measure:
weight

16. Authorship Attribution Evaluation

17. ACADIE Data Set 189 GAS interviews (Goal Attainment Scaling)
95 patients (2 interviews per patient, except 1 patient)
6 sites; 17 MB of data (3.2 million words)
interview participants:
FR – field researcher
Pt – patient
Cg – caregiver
other people

18. Experiment set-up classification improvement detection
preprocessing
patients divided into two groups
85 training group (169 interviews)
10 testing group (20 interviews)
patient speech in training group is used to build Alzheimer profile
non-patient speech in training group is used to build non-Alzheimer profile
two experiments:
classification
improvement detection

19. Classification
from each test interview patient and non-patient speech is extracted
this produces 40 speech extracts
each speech extract is labelled by the classifier as Alzheimer or non- Alzheimer
accuracy is reported

20. Experiment 1.1 training and testing part (90:10)
use all speakers to generate profiles
use both interviews

21. ACADIE: Classification accuracy

22. Improvement detection
improvement is detected by observing an increase in S value between the first and second interview

23. ACADIE: Detected improvement

24. Experiment 1.2
use only first interviews to create Alzheimer and Non-Alzheimer profiles

25. Exp. 1.2: Classification accuracy
Improvement detection:

26. Experiment 1.3 use only first interviews
only speech produced by patients, caregivers, and other (not field researchers)

27. Exp. 1.3: Classification accuracy
Improvement detection:

28. Some experiment observations
Alzheimer n-gram profile captures many indefinite terms and negated (e.g., sometimes, don’t know, can not, …)
the profiles captures reduced lexical richness
Alzheimer
n-gram frequency
non-Alzheimer
n-gram rank

29. Second set of experiments
rating dementia levels
implement method BSCW (by Bucks et al.),
analysis and extension
comparison with CNG
application of a wider set of machine learning algorithms

30. MMSE – Mini-Mental State Exam
MMSE – a standard test for identifying cognitive impairment in a clinical setting
17 questions, 5-10 minutes
introduced in 1975 by Folstein et al.
score range from 0 to 30
a variety of cut points suggested over years: 17.5, 21.5, 23.5, 25.5

31. MMSE Score Gradation we use the following gradation
four classes: severe moderate mild normal
two classes: low high

32. MMSE Score distribution in data set
severe moderate mild normal

34. Part-of-speech tagging, MA-A
following the BSCW method
applied Hepple from NL GATE and Connexor
Hepple is based on Brill’s tagger
Connexor performed better
set of attributes MA-A: attributes similar to BSCW:
excluded CSU-rate:
manually annotated
reported non-significant impact by BSCW

35. Morphological Attribute Set: MA-B
start with all POS attributes
regression-based attribute selection
7 POS attributes selected (conjunctions included)
add TTR and Honore statistics
Brunet statistic shown to be non-significant
use several machine learning algorithms with cross-validation, using software tool WEKA

37. Ordinal CNG Method use two extreme groups to build profiles
severe dementia level
normal level
profile
severe
profile
normal
CNG similarity:
Ssevere
Snormal
test speech
profile
classify according to

38. Ordinal CNG: Thresholds
range of values: [0,1]
0 corresponds to severe, 1 to normal
what are good threshold
interesting observation:
the optimal threshold is very close to the “natural threshold” – 0.5 (varies from 0.5 to 0.512)

40. Conclusions
extensive experiments on morphological and lexical analysis of spontaneous speech for detecting dementia of Alzheimer type
methods:
CNG and Ordinal CNG
extension of method proposed by use of POS tags as suggested by BSCW
positive results in classification and detecting dementia level:
100% discrimination accuracy (Pt and other)
93% - severe/normal
70% - two-class accuracy
46% - four-class accuracy

41. Future work improvement detection use of word CNG method
stop-word frequency-based classifier
syntactic analysis
semantic analysis