1. Mark Meterko, PhD 1 Errol Baker, PhD 1 Kelly L. Stolzmann, MS 1 Ann Hendricks, PhD 1 Keith D. Cicerone, PhD, ABPP-Cn 2 Henry L. Lew, MD, PhD 3 Psychometric Assessment of the Neurobehavioral Symptom Inventory (NSI-22) This work supported by VA HSR&D Grant: SDR 08-405 1 VA Boston Healthcare System 2 JFK-Johnson Rehabilitation Institute 3 Defense and Veterans Brain Injury Center 1

2. Background 1: Postconcussive Syndrome? Studies of postconcussive symptoms have raised several issues:  Is there a postconcussive syndrome (PCS)?  If yes, is there a single cluster of symptoms, or several?  What symptom patterns distinguish among them? Previous studies suggest different answers, depending on:  Etiology of injury  Evaluation instrument  Target population  Statistical procedures 2

3. Background 2: The VA Context VA Policy & Process Regarding TBI  Screening  Comprehensive TBI Evaluation (CTE)  Includes Neurobehavioral Symptom Inventory (NSI-22)  Clinical evaluator’s overall judgment regarding history and course “consistent with a diagnosis of TBI” 3

4. Background 3: Prior Work Cicerone KD & Kalmar K, 1995 (JHTR) 22-item self-report inventory of symptoms 50 mostly vehicular accident patients Using cluster analysis, 17 items grouped into 4 factors:  Cognitive  Affective  Somatic  Sensory 5 orphan items 4

5. Background 4: NSI22 Recent Work Benge JF, Pastorek NJ & Thornton GM, 2009. Postconcussive symptoms in OEF-OIF Veterans: Factor structure and impact of posttraumatic stress. Rehab Psych, 54(3), 270-278. Exploratory factor analysis yielded 6-factor model: 1. Cognitive4. Sensory 2. Vestibular5. Headaches, Sensitivity to light 3. Affective6. Hearing, Sensitivity to noise Caplan LJ, Ivins B, Poole JH, Vanderploeg RD, Jaffee MS, Schwab K, 2010. The structure of postconcussive symptoms in 3 US military samples. JHTR, 25(6), 447-458. Exploratory & confirmatory factor analysis Three models supported: 2,3 and 9 factors Endorsed 3-factor solution 1. Somatic/sensory 2. Affective 3. Cognitive 5

6. Purpose: Study Aims 1. Examine the factor structure of NSI-22  In large sample of deployed veterans judged to have mTBI 2. Examine whether & how pain related to other symptoms in the NSI-22 3. Assess utility of the factor-based NSI-22 scales  Compare subgroups defined by:  Etiology of concussion  Presence/absence of PTSD 6

7. Methods 1: Sample 7 VA National CTE database for FY08 & most of FY09  N=36,919 Random split into 2 samples  Derivation(n=18,459)  Confirmation (n=18,460) Applied inclusion/exclusion criteria to both samples  Keep only those with “symptoms consistent with TBI” (n=18,649)  Drop those with either pre- or post-deployment concussion history (n=5945)  Drop cases missing on pain (n=663)  Drop duplicate and invalid (test case) entries (n=53) Final samples  Derivation (n=6001)  Confirmation(n=5987)

8. Methods 2: Analyses 8 Check success of randomization  Compare derivation & confirmation samples on  Demographics & etiology (chi-square)  NSI-22 items and pain item (MANOVA) Examine factor structure of NSI-22  Derivation sample  Exploratory factor analysis (EFA) – four runs  Empirical criteria for n of factors retained, NSI-22 only  Empirical criteria, NSI-22 plus pain  Specify 4 factors, NSI-22 only  Specify 4 factors, NIS-22 plus pain

9. Methods 3: Analyses 9 Confirm factor structure  Confirmation sample  Confirmatory factor analysis (CFA) Utility of proposed factor-based scales  Confirmation sample  Stratified respondents by:  Etiology  Blast, Non-Blast, Both (“Blast Plus”)  PTSD co-morbidity  Dichotomous based on clinical evaluator judgment during CTE  Two-way MANOVA  Grouping factors (IV): Etiology, PTSD, Etiology x PTSD  Dependent variables: NSI-22 factor scores, with and without pain

10. Results 1: Randomization Success 10 No significant differences, derivation vs. confirmation samples on:  NSI-22 symptoms  Pain  Blast injury exposure  TBI diagnosis  Marital status  Education Borderline exception (p=.07): Employment status  Derivation sample: 7.2% working part time  Confirmation sample: 7.9% working part time  Very small effect size (Cramer’s V =.02)

11. Results 2: EFA in Derivation Sample 11 Using empirical criteria for N of factors to retain Three criteria Percent variance accounted for Horn’s parallel analysis Velicer’s Minimum Average Partial (MAP) test 2- and 3-factor models emerged  Same results with and without pain Preponderance of evidence favored 3-factor model  Somatosensory (11 or 12 items)  Pain loaded cleanly here when included  Affective (6 items)  Cognitive (4 items) Orphan items (2 items)  Hearing difficulties (no loading >=.40)  Change in appetite (equal loadings <.40 on two factors)

12. Results 3: EFA in Derivation Sample 12 Specify 4 factors a-priori Three items from Somato-sensory form a separate, Vestibular factor  Loss of balance  Dizziness  Poor coordination/clumsiness Pain remained affiliated with Somato-sensory Orphans – same as before

13. Results 5: CFA in Confirmation Sample 13

14. Results 6: Utility Analyses in Confirmation Sample 14 Significant main effects for both Etiology and PTSD co-morbidity  Regardless of whether 3 or 4 factors were compared  Regardless of whether pain was/was not included No significant Etiology x PTSD interactions  Results for PTSD and no-PTSD respondents the same across Etiology groups

15. Results 7: Utility Analyses in Confirmation Sample 15

16. Results 8: Utility Analyses in Confirmation Sample 16

17. Conclusions ScalekαScale Content Affective6.88 Low frustration tolerance Irritability Anxiety/tension Fatigue Difficulties sleeping Depressed or sad Somatosensory8.81 Light sensitivity Noise sensitivity Vision problems Headaches Nausea Numbness/tingling Change in taste, smell Pain Cognitive4.89 Difficulties getting organized/can’t finish things Poor concentration Forgetfulness Difficulties making decisions Vestibular3.82 Loss of balance Feeling dizzy Poor coordination/clumsy Unassigned2NA Loss or increase in appetite Hearing difficulty 17

18. Conclusions 18 PCS for Veterans injured during deployment as measured by NSI-22 are multi-dimensional Pain associated with Somto-sensory factor in all solutions By technical criteria, no substantial difference between 3- and 4-factor models Prefer 4-factor model  In EFA: No dual-loading items in EFA  In CFA: Fit statistics marginally but consistently better  Interpretability and utility of 4-factor model  Increased potential for differentiation among clinical groups