1. S519: Evaluation of Information Systems Social Statistics Inferential Statistics Chapter 16: reliability and validity

2. This week What are reliability and validity

3. Scales of measurement What is measurement: The assignment of values to outcomes following a set of rules The scales of measurement have four types: Nominal, ordinal, interval and ratio

4. Nominal level of measurement An outcome can fit into one and only one class or category E.g. gender, political affiliation The least precise level of measurement Categories should be mutually exclusive

5. Ordinal level of measurement Things are ordered E.g., a rank of candidates for a job

6. Interval level of measurement Equal intervals but zero does not indicate absence temperature in Celsius temperature in Fahrenheit

7. Ratio level of measurement The presence of an absolute zero on the scale In biological sciences, zero molecular movement, zero light Height, weight, etc. temperature in Kelvin

8. In sum Any outcome can be assigned to one of the four scales of measurement Scales of measurement have an order, from the least precise being nominal, to the most precise being ratio The “higher up” the scale of measurement, the more precise the data being collected, and the more detailed and informative the data are

9. In sum Characteristics scaleAbsolute zeroEquidistant points Ranked dataData in categorical Ratioyes Intervalyes Ordinalyes Nominalyes

10. Reliability Whether a test, or whatever you use as a measurement tool, measures something consistently Reliability = “consistency”

11. Some ways for reliability Test-retest reliability Want to examine whether a test is reliable over time Compute the Pearson correlation coefficient on scores from a test at Time 1 and Time 2

12. Validity The property of an assessment tool that indicates that the tool does what it says it does Validity = “on targetness” Content validity Validate through domain expert Criterion validity Validate your criteria with existing tests or criteria Literature support

13. Unreliable and Invalid Shooting

14. Reliable but Invalid Shooting

15. Reliable and Valid Shooting