Download presentation
Presentation is loading. Please wait.
Published byLea Delany Modified over 9 years ago
1
MEQ Analysis
2
Outline Validity Validity Reliability Reliability Difficulty Index Difficulty Index Power of Discrimination Power of Discrimination
3
Validity “evidence present to support or refute the meaning assigned to assessment results” face validity face validity content validity content validity criterion-related validity criterion-related validity construct validity construct validity
4
Face Validity high face validity high face validity seemed valid (only) ~ person knee jerk & nervous system knee jerk & nervous system for doctor => high face validity for lay people => low face validity MEQ => high face validity MEQ => high face validity
5
Content Validity ~ sample & population ~ sample & population MCQ => high content validity MCQ => high content validity MEQ => low content validity MEQ => low content validity
6
Validity “evidence present to support or refute the meaning assigned to assessment results” face validity face validity content validity content validity criterion-related validity criterion-related validity construct validity construct validity - don’t need the score - before using the test - score needed - after using the test
7
Criterion-Related Validity Predictive validity Predictive validity MEQ score & close observation score of real practice at ER Concurrent Validity Concurrent Validity MEQ score & VIVA score in the same topic Statistic = correlation coefficient Statistic = correlation coefficient
8
Construct Validity based on theory based on theory communication skill ~ leadership correlation OSCE : communication VS questionnaire : leadership correlation OSCE : communication VS questionnaire : leadership good ethics ~ beloved doctor MEQ : medical ethics VS questionnaire : beloved doctor? MEQ : medical ethics VS questionnaire : beloved doctor?
9
Reliability stability stability internal consistency internal consistency equivalent equivalent
10
Stability test-retest reliability test-retest reliability parallel form reliability parallel form reliability intra-rater reliability intra-rater reliability ~ scoring key statistics : correlation coeff. statistics : correlation coeff. 0-1
11
Internal Consistency [homogeneity] [homogeneity] item - item correlation item - item correlation item - total correlation item - total correlation split half correlation split half correlation
12
Item-Item Correlation each item each item Dichotomous Phi Correlation Phi Correlation Interval Pearson’s Product Moment Correlation Pearson’s Product Moment Correlation whole test whole test Mean of...
13
Item-Total Correlation each item each item Dichotomous Point Biserial Correlation Point Biserial Correlation Interval Pearson’s Product Moment Correlation Pearson’s Product Moment Correlation whole test whole test Mean of...
14
Spilt Half Reliability Dichotomous Dichotomous Kuder-Richardson 20 (KR 20) Interval Interval Kuder-Richardson 21 (KR 21) Cronbach’s alpha coefficient
15
Equivalent parallel item on alternate form reliability parallel item on alternate form reliability inter-rater reliability inter-rater reliability agreement kappa
16
Difficulty Index [p] [p] (mean H + mean L)/2(full score) (mean H + mean L)/2(full score) p = 1 => very easy p = 1 => very easy p = 0 => very difficult p = 0 => very difficult expecting p = 0.2-0.8 expecting p = 0.2-0.8 must => 0.7 should => 0.5
17
Power of Discrimination [r] [r] (mean H - mean L)/(full score) (mean H - mean L)/(full score) > 0.40 => very good item > 0.40 => very good item 0.30 - 0.39 => good item 0.30 - 0.39 => good item 0.20 - 0.29 => borderline 0.20 - 0.29 => borderline poor item poor item
18
Conclusion Validity Validity Reliability Reliability Difficulty Index Difficulty Index Power of Discrimination Power of Discrimination
Similar presentations
© 2024 SlidePlayer.com. Inc.
All rights reserved.