Rohling’s Interpretive Method: Use of Meta-Analytic Procedures for Single Case Data Analysis Martin L. Rohling, Ph.D. University of South Alabama L. Stephen Miller, Ph.D. University of Georgia

Introduction to the RIM Process Flexible battery (multiple measure) use: Is the most frequently cited model of assessment among neuropsychologists. Only 15% of neuropsychologists use a fixed battery (Sweet, et al, 2000, TCN) Regarding the suitability, practicality, and usefulness of any fixed battery: “We know of no batteries that fully satisfy these criteria.” (Lezak, Howieson & Loring 2004, Neuropsychological Assessment, 4th Edition, p 648.) 9/20/2018 Rohling & Miller NAN 2004

Advantages of Flexible Battery Approaches Dynamic. Cover 1 or many domains. “Flexible”, can be adapted for each patient. Can “oversample” domains. Well suited as a hypothesis-driven approach. 9/20/2018 Rohling & Miller NAN 2004

Potential Problems - Flexible Battery Approaches Inflated error rates. Multicollinearity. Weighting decision problems. Unknown veracity/reliability of sets of tasks. Human judgment errors. 9/20/2018 Rohling & Miller NAN 2004

Human Judgment Errors (Wedding & Faust, 1989, ACN) Hindsight bias. Confirmatory bias. Overreliance on salient data. Under-utilization of base rates. Failure to take into account co-variation. 9/20/2018 Rohling & Miller NAN 2004

RIM Potential Judgment errors can threaten reliability and validity of multiple measure test batteries. RIM was designed to reduce these effects. Based on meta-analytic techniques. Uses a linear combination of scores placed on a common metric. 9/20/2018 Rohling & Miller NAN 2004

RIM Potential A strategy that produces summary results analogous to those generated in a fixed-battery approach (e.g., HII, GNDS, AIR). Takes advantage of psychometric properties of same metric data: e.g. T-Scores. 9/20/2018 Rohling & Miller NAN 2004

Today’s Intent Present a set of procedures that allows for a quantitatively-based comparison of an overall battery of measures. Non-specific to battery measures themselves. Can be used for any individual patient. Demonstrate importance and practicality of use of established statistical indices. (e.g., alpha, beta, effect size). 9/20/2018 Rohling & Miller NAN 2004

Today’s Intent (cont’d) Present a data format for any set of measures to be inspected at: Global level (OTBM) Domain level (DTBM) Test measure level (ITBM) Present a series of calculations to assist in the generation of these indices. Present Steps in conjunction with clinical judgment from an informed position. 9/20/2018 Rohling & Miller NAN 2004

RIM Categories Symptom Validity (SV) Tests. Emotional / Personality (EP) Measures. Estimated Premorbid General Ability (EPGA). Test Battery Means. Overall (OTBM), Domain (DTBM), & Instrument (ITBM). Cognitive Domains: VC, PO, EF, AML, VML, AW, PS Non-Cognitive Domains. PM, LA, SP 9/20/2018 Rohling & Miller NAN 2004

Sample RIM: Summary Table 9/20/2018 Rohling & Miller NAN 2004 Rohling & Miller NAN 2004

Sample RIM: Graphic Display 9/20/2018 Rohling & Miller NAN 2004

Brief of RIM Steps: There are 24 steps to the RIM process 17 calculation steps: Advice on design of the battery Calculation of summary statistics Generation of graphic displays 7 interpretative steps. Detail a systematic procedure for use of the statistical summary table and graphic displays to: Assess and verify summary data. Identify strengths/limitations of current data. Obtain a reliable diagnosis. Develop tx plans based on sound judgments. We briefly review each step in just a moment. 9/20/2018 Rohling & Miller NAN 2004

Support for the RIM Process Rational support/reasoning: Reduce clinical judgment errors. The RIM is a Process, not a program. A way of formulizing your thinking and interpretation of your data. This is operationalizing what you already do. 9/20/2018 Rohling & Miller NAN 2004

Support for the RIM Process: Specific Advantages Psychometric properties at level with fixed, co-normed batteries, without their limitations. Flexibility of test selection. Flexibility of theoretical view of cognition (domain structure) 9/20/2018 Rohling & Miller NAN 2004

Support for the RIM Process: Specific Advantages Quantitatively support your conclusions and interpretations Statistical evaluation Measure of confidence in findings Measure of limitations of findings Ability to present data at different levels of interpretation Greater defensibility 9/20/2018 Rohling & Miller NAN 2004

The RIM as a Procedure of Specific Steps 9/20/2018 Rohling & Miller NAN 2004

RIM Steps 1-4: Summary Data Design & administer battery. Use well standardized recently normed tests. Estimate premorbid general ability. Use Reading (WTAR), Regression (OPIE-III), & academic records (rank, SAT, ACT). Convert test scores to a common metric. We recommend T scores, but z or SS OK too. Assign scores to domains. Factor analysis to support assignment (Tulsky et al., 2003) 9/20/2018 Rohling & Miller NAN 2004

RIM Steps 5-8: Summary Data Calculate domain M, sd, & n. Calculate test battery means (TBM). Overall TBM – All scores, large N & high power. Domain TBM – Avoids domain over weighting. (e.g., attention & memory). Instrument TBM – One score per norm sample. Calculate p for heterogeneity. Have you put “apples & oranges” together? Determine categories of impairment. Recommend using of Heaton et al. (2003). 9/20/2018 Rohling & Miller NAN 2004

RIM Steps 9-12: Summary Data Determine % of test impaired. Analogous to Halstead Impairment Index # scores below cutoff / total # of scores Calculate ES for all summary stats. Use Cohen’s d = (Me – Mc) / SD pooled Calculate confidence interval for stats. 90% CI = 1.645 x SEM Upper limit of performance for impair. Look for overlap between 90% CI of EPGA (lower) & Summary Stats (upper) 9/20/2018 Rohling & Miller NAN 2004

RIM Steps 13-17: Summary Data Conduct one-sample t tests. Use EPGA as reference point Conduct a between-subjects ANOVA. Looking for strengths & weaknesses Conduct power analyses. Only needed for those NS differences Sort scores for visual inspection. Graphically display summary statistics. 9/20/2018 Rohling & Miller NAN 2004

RIM Steps 18-20: Interpretation Assess battery validity. Examine the Symptom Validity scores. Caution in accepting low power results. Look at heterogeneity of summary stats. Normative sample unrepresentative of patient. Scores assigned to wrong domain. Inconsistent performance on construct measures. Examine influence of psychopathology. Examine scores for heterogeneity. Check OTBM, DTBM, & ITBM impairment. 9/20/2018 Rohling & Miller NAN 2004

RIM Steps 21-24: Interpretation Examine strengths/weaknesses looking at: Confidence intervals overlap. Results from one-sample t tests. Results of ANOVA. %TI show differences otherwise not evident. Determine if pattern existed premorbidly. Examine non-cognitive domains. Psychomotor, Lang/Aphasia, Sensory Percept Explore Type II errors –need more tests? Examine sorted T-scores Look for patterns missed by summary stats. 9/20/2018 Rohling & Miller NAN 2004

RIM Sample Case 1: Obvious TBI Reason for Referral: TBI in head-on boat accident. Propeller hit pt in right parietal-occipital lobe (LOC = 7 days; GCS = 3). Eval. to determine capacity for medical & financial decisions, parenting skills, occupational prognosis, & disability status. Significant emotional, behavioral, occupational, and social problems pre-TBI. Age: 37 Handed: Left Race: Euro-American Sex: Female Ed: 14 years Occup: Nursing Marital: Sep. 10 yrs Living: Camper in parent’s backyard 9/20/2018 Rohling & Miller NAN 2004

RIM Sample Case 1: Obvious TBI 9/20/2018 Rohling & Miller NAN 2004

RIM Sample Case 1: Obvious TBI 9/20/2018 Rohling & Miller NAN 2004

TBI Dose Response Curves Dikmen ES’s Meyers’ T Scores 9/20/2018 Rohling & Miller NAN 2004

Return to Work Study: OTBM’s for 4 Groups of TBI Survivors SD ES Disabled 17 32.8 6.4 -2.29 Unemployed 96 39.5 6.1 -1.01 Below Previous 32 43.3 4.6 -.36 At Previous 137 45.1 5.2 -.45 9/20/2018 Rohling & Miller NAN 2004

RIM Sample Case 2: Obvious TBI Normal Distribution of T Scores 9/20/2018 Rohling & Miller NAN 2004

RIM Sample Case 2: Subtle Diabetes Reason for Referral: 2 yrs dangerous work habits. Eval to see if atrial fib & Type II diabetes impairs cognition. Hospitalized “TIA-like” Sx. Admitted to problems for 20 yrs, cardiac dysrhythmia & bradycardia, pacemaker, blood sugar difficult to manage, & family Hx of heart disease & diabetes. Age: 55 Handed: Right Race: Euro-American Sex: Male Ed: 13 years Occup: Mechanic Marital: Married 20 yr Living: at home w/wife 9/20/2018 Rohling & Miller NAN 2004

RIM Sample Case 2: Subtle Diabetes 9/20/2018 Rohling & Miller NAN 2004

RIM Sample Case 2: Subtle Diabetes 9/20/2018 Rohling & Miller NAN 2004

RIM Sample Case 2: Subtle Diabetes Normal Distribution of T Scores 9/20/2018 Rohling & Miller NAN 2004

RIM Critiques: Concern 1 The method of calculating the standard deviations (SDs) for summary statistics and domain scores is incorrect. Since many of the remaining steps of the RIM depend on the use of these SDs, this error is magnified in the subsequent steps. SDs statistically can not exceed 9.99 and are more likely to be around 6.4 9/20/2018 Rohling & Miller NAN 2004

Response 1: RIM Means 4 Large Datasets 9/20/2018 Rohling & Miller NAN 2004

Response 1: Inter-Individual Ms & SDs Mn SD Dataset 1 Psych Pts WAIS-R 457 43.2 7.2 WAIS 150 45.0 9.1 Dataset 2 (Green) 904 44.8 7.3 Dataset 3 (Meyers) 1,734 42.0 Dataset 4 (HRB) 114 42.8 6.8 Total 4 Samples 3,359 43.1 7.4 9/20/2018 Rohling & Miller NAN 2004

Response 1: RIM SDs 4 Large Datasets 9/20/2018 Rohling & Miller NAN 2004

Response 1: Intra-Individual Ms & SDs Mn SD % > 9.99 Dataset 1 Psych Pts WAIS-R 457 6.8 2.0 7% WAIS 150 7.4 2.2 10% Dataset 2 (Green) 904 11.4 2.9 65% Dataset 3 (Meyers) 1,734 11.9 56% Dataset 4 (HRB) 114 10.6 2.4 61% Total 4 Samples 3,359 10.8 2.8 50% 9/20/2018 Rohling & Miller NAN 2004

RIM Critiques: Concern 2 More false-positives then clinical judgment. Palmer et al. (2004) expressed concern that We failed to distinguish “statistical” from “clinical” significance. This failure is a critical error that precludes the prudent clinician from using the RIM. 9/20/2018 Rohling & Miller NAN 2004

Response 2: RIM vs. Manual Detecting Differences – Overall % % of Total S’s MANUAL METHOD RIM t TEST VIQ-PIQ: NS VIQ-PIQ: Sig. Marginal M’s 54% 23% 78% 1% 21% 22% 55% 45% 100% 9/20/2018 Rohling & Miller NAN 2004

Response 2: RIM vs. Manual Detecting Differences – ES Means (SDs) MANUAL METHOD RIM t TEST VIQ-PIQ: NS VIQ-PIQ: Sig. Marginal M’s .38 (.30) .80 (.41) .50 (.39) 1.58 (.82) 1.70 (.86) 1.69 (.85) .40 (.37) 1.22 (.80) .90 (.71) 9/20/2018 Rohling & Miller NAN 2004

Response 2: RIM vs. Manual Detecting Differences Scaled Scores Means (SDs) MANUAL METHOD RIM t TEST VIQ-PIQ: NS VIQ-PIQ: Sig. Marginal M’s 3.9 (2.5) 13.2 (3.7) 6.7 (5.2) 6.7 (0.8) 19.0 (6.5) 16.9 (8.5) 4.0 (2.5) 15.9 (6.0) 9.3 (7.4) 9/20/2018 Rohling & Miller NAN 2004

RIM Critiques: Concern 3 Clinicians who use the RIM will: Idiosyncratically assign scores to cognitive domains. This will result in low inter-rater reliability in analysis & diagnosis. 9/20/2018 Rohling & Miller NAN 2004

RIM Critiques: Concern 4 Scores on domains are unit weighted, which introduces error. Willson & Reynolds (2004) said scores load on multiple domains. Assignment to domains & weights depend on: Battery of tests administered. Patients whose test scores are being examined. 9/20/2018 Rohling & Miller NAN 2004

Response 4: Cross-Validation Unit Wts Conducted 4 multiple reg. on 457 pts’ WAIS-R. Split sample in ½ - assess shrinkage. Regressed patients’ verbal subtests onto PIQ. Generated ideal weights for the 1st ½ of sample. Used wts to predict PIQs in the 2nd ½ of sample. Pre-PIQs regressed on actual PIQs 2nd ½ sample. Also, generated weights for the 2nd ½ of sample. Use Pre-PIQ’s regress on actual PIQs 1st ½ sample. Repeated, except performance subtests predict VIQ split sample ½ & generate same statistics as before. 9/20/2018 Rohling & Miller NAN 2004

Response 4: Cross-Validation Unit Wts Purpose of these procedures: How much variance in wts. is sample specific. Amount of shrinkage using cross-validated wts. Shrinkage error compared to error introduced by using “unit wts” vs. “ideal wts.” Results: 98% of the variance accounted for with unit wts. Compared to ideal weights. Support use of unit wts. Rather than ideal wts. Also see, Dawes, R. M. (1979). The robust beauty of improper linear models in decision making. American Psychologist, 34, 571-582. 9/20/2018 Rohling & Miller NAN 2004

RIM Critiques: Concern 5 Multiple measures used to generate composite scores: Results in less accurate estimates of the cognitive domains. 9/20/2018 Rohling & Miller NAN 2004

Response 5: Estimate FSIQ Using Scaled Score Means’s 9/20/2018 Rohling & Miller NAN 2004

RIM Critiques: Concern 6 A general ability factor is used to represent premorbid functioning for all domains. This not supported by the literature. This results in inaccurate conclusions regarding degree of impairment suffered by a patient in each cognitive domains assessed. 9/20/2018 Rohling & Miller NAN 2004

RIM Critiques: Concern 7 Norms used come from samples that are of undocumented comparability. Furthermore, even when norms used were generated from different but comparable samples, their format prohibits ready comparisons. 9/20/2018 Rohling & Miller NAN 2004

Response 7: Split-Half Reliability Analyze Dataset 2 - OTBMs from 42 DVs. Individuals’ data split into two sets 21 test variables for each OTBM (1 & 2). 2 independent OTBMs created for patient. Split DVs intentionally - separated so that no normative sample was included in both OTBMs. 9/20/2018 Rohling & Miller NAN 2004

Response 7: Split-Half Reliability Results r = .81, 66% of variance accounted Slope of the regression line was .82 (SE = .027) Intercept 9.2 (SE = 1.20). Mean OTBM-1 = 45.0 (SD = 7.3). Mean OTBM-2 = 43.6 (SD = 7.2). Results simulate worse case scenario. used an entirely different set of norms. Est. test-retest r for OTBM 42 DV’s increased r = .82 - .88 (Spearman-Brown correction). 9/20/2018 Rohling & Miller NAN 2004

Response 7: Split-Half Reliability No overlap in normative samples. Worst-case condition, generally administer instruments (e.g., WAIS-III) with OTBMs generated from “co-normed” variables. Meyers & Rohling test-retest reliability of .86. When different norms used, often gave same instruments (e.g., AVLT or RCFT). Our simulation, no instruments included in OTBM-1 included in OTBM-2. Heaton et al.’s (2001) - schizophrenic pts. Obtained a test-retest reliability of .97. Comparing 2 identical batteries, not worst-case. 9/20/2018 Rohling & Miller NAN 2004

RIM Critiques: Concern 8 The RIM will result in an undue inflation of clinicians’ confidence. Such overconfidence results in more error in a interpretation, not less. 9/20/2018 Rohling & Miller NAN 2004

RIM vs. Tulsky et al. (2003): Case 1 WAIS & WMS Battery Full Battery Index or Domain Scores Disc. Model RIM Model # S EPGA1 (WTAR) 103 102 1 106 5 U Overall TBM2 (FSIQ) 112 104 18 96 I 70 M Domain TBM2 --- 105 6 98 7 Inst TBM2 (FSIQ/GMI) 2 93 I 14 Verbal Comp (VCI) 120 * 118 * 3 110 Percept Organ (POI) 121 * 117 * 107 Attent/Work Mem (WMI) 95 I* 9 4 Process Speed (PSI) 93 ** 95 ** 86 I* Aud Mem & Learn (AMI) 97 ** 81I***** Vis Mem & Learn (VMI) 94 95 13 Executive Function (EF) 100 I* 15 8 Psycho-Motor (PM) 111 9/20/2018 Rohling & Miller NAN 2004

RIM vs. Tulsky et al. (2003): Case 2 WAIS & WMS Battery Full Battery Index or Domain Scores Disc. Model RIM Model # S EPGA1 (WTAR) 125 120 1 117 5 U Overall TBM2 (FSIQ) 119 103 18 96 I 53 M Domain TBM2 --- 105 6 95 I 7 Inst TBM2 (FSIQ/GMI) 106 2 93 I 11 Verbal Comp (VCI) 124 122 3 Percept Organ (POI) 95 97 Attent/Work Mem (WMI) 108 102 4 Process Speed (PSI) 98 92 83 Aud Mem & Learn (AMI) 111 110 99 I 9 Vis Mem & Learn (VMI) 104 89 I 10 Executive Function (EF) 94 I 14 8 Psycho-Motor (PM) 77 9/20/2018 Rohling & Miller NAN 2004

Summary of the Rohling Interpretive Method of Statistical Analysis of Neuropsychological Data 9/20/2018 Rohling & Miller NAN 2004

24 total steps to the RIM process Summary of RIM Steps 24 total steps to the RIM process 17 calculation steps: Battery Design Calculation of summary statistics Generation of graphic displays 7 interpretative steps. Use of summary table and graphic displays to: Assess and verify summary data Identify strengths/limitations of current data Obtain a reliable diagnosis Develop tx plans based on clinical judgments. 9/20/2018 Rohling & Miller NAN 2004

Summary of RIM Advantages Formulize thinking interpretation of data: Operationalize what you already do. Reduce judgment errors thru RIM Process. Take advantage of psychometric properties at level with fixed, co-normed batteries. Allows flexibility of test selection. Allows flexibility of theoretical view of cognition (e.g., domain structure) 9/20/2018 Rohling & Miller NAN 2004

Summary of RIM Advantages cont’d Gives Quantitative support for your conclusions and interpretations Statistical evaluation Measure of confidence in findings Measure of limitations of findings Ability to present data at different levels of interpretation Equals greater defensibility 9/20/2018 Rohling & Miller NAN 2004

Our RIM Cautions/Concerns Does not “replace” clinical judgment, rather, informs clinical judgment. This still means CJ errors are possible. Susceptibility T-Scores to distrib. deviance Process, not program Pre-morbid ability estimates Domain selection, test placement 9/20/2018 Rohling & Miller NAN 2004

Published Research Findings Using the RIM 1) RIM vs. HRB 2) Variance Accounted for by SVT 3) Effect of Depression on NP Results 4) Prediction of Employment after TBI

RIM of HRB: OTBM vs. HII Heaton et al.’s (1991) HRB norms for OTBM T Score (M=50, sd=10) OTBM r with HII = -.79 (p < .0001) 62% variance account. Over predicts low Under predicts high 9/20/2018 Rohling & Miller NAN 2004

RIM of HRB: OTBM vs. GNDS OTBM r with GNDS = -.87 76% variance acc. OTBM neither under nor over predicts across range of GNDS Intercept impairment is T Score = 46.0 Reitan & Wolfson (GNDS = 29) 9/20/2018 Rohling & Miller NAN 2004

RIM of HRB: OTBM’s Relationship to Global Indices INDICES OF FUNCTION Correlation Coefficient Halstead Impairment Index .79 Average Impairment Rating .90 Global Neuro. Deficit Scale .87 RIM: Domain TBM .99 RIM: Instrument TBM .95 RIM: % Tests Impaired .96 9/20/2018 Rohling & Miller NAN 2004

RIM of HRB: Diagnostic Classification Using the HII BR 65% Sens. Spec. PPV NPV % Corr. HII .64 .66 .77 .51 65% AIR .58 .78 .82 GNDS .63 .79 .62 73% OTBM .90 .32 .70 .65 69% ITBM .86 .37 .71 .60 %TI .85 .56 .68 74% 9/20/2018 Rohling & Miller NAN 2004

RIM of HRB: Cross-Validation of RIM using HRB in 2 Samples Regressed Dikmen & Meyers TBI data: Generated a predicted HII for pts in OK dataset. Correlation actual & predicted HII = .95 Sense = .60, Spec = .77, PPV = .78, NPV = .59 Overall % Correct Classification = 71% Predicted HII from MSB’s OTBM more accurate indicator of impairment than actual HII. 9/20/2018 Rohling & Miller NAN 2004

Factor Loadings of Domain Scores Genuine Normal Genuine Neuro Exag Normal Obj Perf Self-Report NPT .57 -.03 .64 -.01 .89 .11 SVT .58 -.08 .63 -.02 .87 .12 MCI .56 .04 .55 .33 .81 PSX -.07 -.06 .03 .91 Eigen 1.30 1.83 1.22 1.67 2.02 1.14 % Var. 33% 46% 30% 42% 51% 29% 9/20/2018 Rohling & Miller NAN 2004

Means & SDs of Composite Scores Genuine Exaggerate Normal Neurologic Neuropsych Test Scores .33 (.62) .19 (.64) -.60 (.80) -.79 (.65) Symptom Validity .51 (.38) .50 (.30) -1.25 (.94) -.50 (.52) Memory Complaints .14 (.93) .41 (.92) -.62 .49 (.54) Psychiatric Symptoms .10 (.95) .46 (.96) -.39 (.85) (.99) 9/20/2018 Rohling & Miller NAN 2004

Mean z Score on Objective Tests Small differences between Gen. Normal & Gen. Neuro on NPT. No differences between Exag Normal & Exag Neuro on NPT. Deficits for Exag Neuro were more modest than for Exag Normals on SVT. Interaction between Validity & Neuro Status. 9/20/2018 Rohling & Miller NAN 2004

Mean z Score Self-Report No differences between Gen Neuro & Exag Neuro on Memory Complaints. No differences between Gen & Exag Neuro on Psychiatric Sx. Deficits for Exag Normal on the Psych Sx & Memory Complaints; latter is larger. Interaction between Validity & Neuro Status. 9/20/2018 Rohling & Miller NAN 2004

Depression Study: Reference Rohling, M. L., Green, P., Allen, L. M., & Iverson, G. L. (2002). Depressive symptoms and neurocognitive test scores in patients passing symptom validity tests. Archives of Clinical Neuropsychology, 17, 205-222. 9/20/2018 Rohling & Miller NAN 2004

Mood Group Assignment Patients classified into 2 subgroups From entire sample, 420 passed all SVTs Sample split based on BDI Low-Depressed 25%ile on BDI (< 10) n = 178, M = 6 (3) High-Depressed 75%ile on BDI (> 25) n = 187, M = 31 (6) 9/20/2018 Rohling & Miller NAN 2004

Depression Study Participants All 365 patients referred for evaluation for compensation-related purposes All diagnostic groups included 53% Head injury referrals 22% Medical referrals 14% Psychiatric referrals 11% Other neurological Age = 42 (11); Ed = 13 (3); Sex = 64% males; Non-English = 18%; Handedness = 9% Left 9/20/2018 Rohling & Miller NAN 2004

Results Mood & Validity Status SVT Status Mood BDI Genuine Exaggerating 107 (30%) 68 (19%) 159 (44%) 27 (7%) 175 (48%) Depress 75%ile 186 (52%) NonDep 25%ile 266 (74%) 95 (26%) 9/20/2018 Rohling & Miller NAN 2004

Results: Sample Split by Validity -1.5 -1.3 -1.1 -.9 -.7 -.5 -.3 -.1 .1 .3 .5 .7 .9 1.1 1.3 High-Dep Low-Dep EPT MCI OTBM EPT MCI OTBM EPT MCI OTBM Total Sample Gen Pts Exag Pts Z-scores 9/20/2018 Rohling & Miller NAN 2004

Effect of Mood Depends on Effort Exaggerating patients accounted for 39% of High-Dep group 14% of Low-Dep group Mood & Effort used as IVs and Cognition DV Effect for effort, no effect for mood However, when Memory Complaints DV Effects for both effort and mood Also, when other Emotion/Personality DV 9/20/2018 Rohling & Miller NAN 2004

Effect of Mood Depends on Effort When both mood groups were included in regression analysis, as predicted: Memory ratings related to mood (r = .60; p < .0001) Mood not correlated with cognition (r = .10; p > .10) Memory ratings not related to cognition (r = .13, p = .06) 9/20/2018 Rohling & Miller NAN 2004

Gervais’ pain sample findings (n = 177) Mood Replication Gervais’ pain sample findings (n = 177) Exaggerating patients accounted for 55% of High-Dep; 33% of Low-Dep group Memory ratings related to mood (r = .55) Mood not correlated with cognition (r = .06) Memory ratings related to cognition (r = .15) Group means correlated with Green’s .94 all patient (High-D, Low-D, Gen, & Exag). 9/20/2018 Rohling & Miller NAN 2004

Effect if Pain on OTBM 9/20/2018 Rohling & Miller NAN 2004

Effect if Pain on OTBM 9/20/2018 Rohling & Miller NAN 2004

Return to Work after Injury Three main hypotheses using MSB-RIM OTBM will predict return to work level Cognitive domain that will be most predictive will be executive function Adding the Patient Competency Rating Scale will improve work prediction PCRS is by Prigatano (1985) 9/20/2018 Rohling & Miller NAN 2004

Return to Work: ANOVA of OTBM Group n M SD ES Disabled 17 32.8 6.4 -2.29 Unemployed 96 39.5 6.1 -1.01 Below Previous 32 43.3 4.6 -.36 At Previous 137 45.1 5.2 -.45 9/20/2018 Rohling & Miller NAN 2004

Logistic Regression Using OTBM Predicted Observed Disable Un-employ Below Prev At Prev % Corr Disabled 2 12 3 12% Unemployed 1 48 47 50% Below Previous 9 23 0% At Previous 25 112 82% 9/20/2018 Rohling & Miller NAN 2004

Return to Work: Summary OTBM differences between groups. Disabled /Unemployed not able to separate. Below/At Previous not able to separate. Collapsed groups result in 71% correct above base rate of 52% correct. 9/20/2018 Rohling & Miller NAN 2004

Return to Work: Domain Analysis Executive function not the most predictive. Most of variance carried by Perceptual Organization & Working Memory Using Cognitive Domains OTBM increases % Correct from 71% to 74% Incremental validity of PCRS very low. 7% of the variance 9/20/2018 Rohling & Miller NAN 2004

Return to Work: Domain Analysis By including premorbid variables, increases diagnostic accuracy; most helpful being: Premorbid IQ, level of occupation, & education Including acute measures increases accuracy; most helpful being: LOC group Time since injury 9/20/2018 Rohling & Miller NAN 2004

Depression Study Conclusions Memory complaints not synonymous with impairment in compensation sample Findings replicated Effort accounts for more variance in self-ratings of cognition & objective performance than mood 9/20/2018 Rohling & Miller NAN 2004

Questions & Comments Welcome! Rohling’s Interpretive Method: Use of Meta-Analytic Procedures for Single Case Data Analysis Martin L. Rohling L. Stephen Miller Questions & Comments Welcome! 9/20/2018 Rohling & Miller NAN 2004