1. Comparison of Self-Reported and Medical Record Health Care Utilization measures
Concurrent criterion validation of a domain on health care utilization as part of a QOL instrument by comparing it with charts from a centralized medical chart center for the area

2. Information not provided: Abstract 1992 Evaluative tool QOL Benign prostatic hyperplasia
Random selection of 110 men
in male across the span of severity of BPH (no to severe symptoms)
6 domains: symptoms, degree of bother, BPH specific interference with activities, psycho, worries-concerns, sexual satisfaction
tested for test-retest reliability, responsiveness to change,
internal consistency, validity

3. Measurement
Pre-existing self-reported questionnaire on QOL for benign prostate hyperplasia,
with a 5-item domain on health care utilization
domains: same 6 + health care use, medical Hx, smoking, sociodemo info

4. Proposed use of the measurement
Predictive tool
for research
for administrative (health care planning, policy) purposes

5. Research question Not clearly stated
Pop: male community-dwelling, Olmsted county
Intervention: self-administered questionnaire QOL
Outcomes: comparison with centralized medical records of the county,
# in-pt nights in last yr
# in-pt nights for mental reasons,
# visits to MD in last 2 wks,
# visits to MD in 1 yr,
visits to MD for any of 12 symptoms of HBP (yes/no)
Direction: ?

6. Aspect of validity
Concurrent criterion validity

7. Study design and methods
Question not clear
Pop: random sample (size 500,10% hospitalized in last yr)
exclusion criteria not clear
Meaningful criterion: yes
Appropriate comparison: independant, blind?
Tool:
ideally only measuring health care use
scaling of answers not mentionned, questions not stated
Statistical analysis: no pre-set kappa or phi

8. Bias Selection bias Information bias
Response bias (55% returned the quest)
Information bias
Misclassification related to extraction data chart
Recall bias
Social desirability (underreporting)
Reporting bias ( in-hosp for mental prob)

9. Answered the research question?
What was the research question?
They answered the question we guessed they addressed
Tool:
1 question: did not report any result
4 questions: reported as reliability!

10. Implications Administrative, planning, research use for male 40-79
validity mostly
for # nights in-pt in last yr
recalling seeing a MD in last 2 wks
Clinical use: no application foreseen

11. Analysis of criterion validity
Dichotomous: phi ()
Continuous: weighted kappa, Pearson, ICC
Ordinal: consider as if continuous or use Wilcoxon signed rank test

12. Analysis criterion validity
Dichotomous: based on 2X2 table
Φ=׀BC-AD׀ /[(A+B)(C+D)(A+C)(B+D)]1/2
a b
c d

13. In the last year, have you seen a physician for any of the above symptoms?
Computerized charts
Y N
Questionnaire Y N
 = 0.45
However, not reported in methods or results

14. Analysis of ordinal data
Wilcoxon signed sum rank

15. How many in-pt nights in last yr?
Quest
Charts
sign
signRank
add the positives: ( ) = 17.5
add the negatives: ( ) = -10.5
Ho sumpositive = sum negative
then use a table to find p-value or use an approximate z test then find p-value
appropriate use of Wilcoxon in the statistical analysis

16. Ordinal: treat as continuous
Choice of weighted kappa, Pearson, ICC
used Weighted kappa
kappa= (Po- Pe) / (1-Pe)
quadratic weight: use ( exact disagreement)2
Here, weight: use (exact agreement)2
wKappa= 1- {wij X Poij /(wij X Peij)} why use simple agreement, kappa and Wilcoxon?

17. a word on reliability Pearson vs ICC vs kappa
If use quadratic weight (disagreement)2 then kappa = ICC
kappa: only for dichotomous variables
Pearson: linear regression ANOVA
measures how well you may fit the obs on a straight line

18. Pearson vs ICC Pearson compares 2 obs at a time or only 1 ICC
if 10 raters: 45 Pearson correlation coefficients!
or only 1 ICC

19. References
Health Measurement Scales. A practical guide to their development and use
DL Steiner, GR Norman
Statistics The Bare Essentials
HRM 727 Measurement Tool, course package G Norman