1. Cognitive Testing, Statistics and Dementia Ralph J. Kiernan Ph.D. 14 th May 2013

2. Slide 2 The Normal Distribution The idealized situation A representative population Well-defined variables Accepted methods of measuring each variable Broad range of variation within the population The Bell-Shaped Curve

3. Slide 3 The Bell-Shaped Curve

4. Slide 4 Standardized Intelligence Tests The Wechsler Intelligence Scales Each subtest yields a normal distribution Ill-defined variables Method based on factor analysis What does each subtest measure? Prediction of academic success (Binet)

5. Slide 5 The Meaning of Extreme Scores Predictive utility-the broadly average range Well-defined between – 2 SD’s and + 2 SD’s Poorly-defined at the extremes outside the above range: below the 2 % or above the 98 % Can we define the lowest 2 % as abnormal, i.e. not part of the normal population?

6. Slide 6 The DSM-V on Dementia A neurocognitive disorder Minor: between 1 and 2 SD’s below the mean and no loss of independent functioning Major: more than 2 SD’s below the mean and a loss of independent functioning A misunderstanding of cognitive testing statistics and their clinical utility

7. Slide 7 The 80 Year Old Population The average individual dies before age 80 50 % of 80 year olds have Dementia An unknown but substantial portion of the remaining half have mild cognitive impairment Who should be included in the normal 80 year old population? Is someone in the top half of this group with MCI unacceptable as normal?

8. Slide 8

9. Slide 9 The Function of a Cutting Score

10. Slide 10 The Problem with Cutting Scores Normal and abnormal populations are seen as separate yet overlapping Cutting scores provide the maximum separation between these two populations Diagnostic decision making is reduced to an either / or choice around a single score Complex cognitive performances are reduced to a single number

11. Slide 11 Deficit Measurement Tests Most neuropsychological tests are designed to assess the degree of cognitive deficit These tests don’t measure a broad range of skills within the normal population These tests don’t generate a bell-shaped curve Their means and standard deviations can no longer be interpreted in the usual way Interpretation depends upon clinical criteria

12. Slide 12 Cognitive Screening Examinations Typically generate a single global score No attempt is made to assess specific skills Attention is often over-valued An unacceptable number of false negative and false positive results are common, e. g. MMSE It is impossible to specify a cutting score on a test which misclassifies patients at both ends Uncertain results complicate the clinician’s job

13. Slide 13 Sensitivity and Specificity A single score cannot be generally sensitive without losing all specificity A single cutting score cannot be used as if “organicity” were a unitary factor Statistics based on group data cannot replace clinically based guidance Cognitive data must be shared with clinicians who can incorporate it into their diagnosis

14. Slide 14 The Cognistat Correction Domain specific – Ten independent measures – Combines neurology and neuropsychology Differentiated assessment – Generates meaningful cognitive profiles Clinically based scoring – Relies on clinical rather than statistical criteria – Provides a graded assessment of risk

15. Slide 15 Cognistat Created in 1979 to screen for neurological causes of acute psychiatric episodes Originally called the Neurobehavioral Cognitive Status Examination (NCSE) Modified in 1983 to its present form Uses a screen and metric approach Requires 15 to 20 minutes to administer Provides a domain specific assessment

16. Slide 16 The Cognistat Assessment System Briefly measures skills within ten distinct cognitive domains Presents a graphic profile of the test results Alerts clinicians to cautions in response to performance issues that arise during testing Provides information regarding factors and medications that may impact test results Generates an expert systems analysis of test scores including the MCI Index

17. Slide 17 The Cognistat Profile

18. Slide 18 Cognistat Statistics Standardization: Demonstrates only that most people in the normal population perform perfectly or nearly so on every subtest Normal population means and standard deviations have no use as impairment criteria Reliability has no meaning in tests that have no range of scores within the normal population Screen items were made difficult enough to fail 10% to 15 % of the normal population

19. Slide 19 Cognistat Validity Clinical scale ratings of mild, moderate and severe are based on clinical not statistical criteria Validity has been assured through Cognistat’s use in a wide variety of clinical populations and by direct comparison with comprehensive neuropsychological testing Cognistat has proven its utility in geriatric screening through the Palo Alto VA GREC, the Oregon Aging Study and the Wisconsin Alzheimers’ Institute

20. Slide 20 The Cognistat Expert System

21. Slide 21 The MCI Index Risk assessment for a dementia diagnosis Based primarily on Constructions and Memory Adjusted for age and education Takes note of Language variables A seven point index ranging from 0 (no impairment) to 6 (strongly suggests a dementia syndrome Increasing impairment results in increasing risk

22. Slide 22 The MCI Index is a seven point scale that reflects both the degree of cognitive impairment and the likelihood of an MCI diagnosis. MCI Index = 0: no indication of cognitive impairment MCI Index = 1: raises the question of MCI MCI Index = 2: suggests MCI MCI Index = 3: strongly suggests MCI MCI Index = 4: raises the question of a dementia syndrome MCI Index = 5: suggests a dementia syndrome MCI Index = 6: strongly suggests a dementia syndrome The MCI Index

23. Slide 23 Preliminary MCI Index Data MCI Index ratings from 0 to 6 are based on clinical rather than statistical criteria Although our clinical experience with Cognistat in geriatric patients is extensive, actual Index scores have only been compared to neuropsychological testing in a preliminary sample of 13 patients The Index identified 10 of 11 patients judged to have MCI with extensive cognitive testing

24. Slide 24 NeuroPsych Batt.Cognistat PxAgeM/FDxDateMCI IndexDateOutcomeComments 179FMCIMar-123May-17-12TP 261FSCIJan/Feb -120Jun-7-12TN 380MDEMMay-124May-5-12TP 482FDEM20126May-31-12TP Progressive Decline? 585FDEMNov-21-116May-29-12TP Mental Tracking? 677FADApr-122May-24-12TP MCI vs. AD? 779MDEMJan-126May-24-12TP 888MMCI20110May-17-12FNAge related “normal” profile? 983MDEMMar-126May-15-12TP Too severe for MCI? 1063FMCI20123May-15-12TP 1175FMCIFeb-122Apr-24-12TP 1260FOCDOct/Nov 20115Apr-19-12FP Inadequate memory registration? 1377FMCIOct 19 20112Apr-17-12TP TP = True Positive, TN = True Negative, FP = False Positive, FN = False Negative Summary of Cognistat Data