1. EDU 8603 Day 6

2. What do the following numbers mean? 85 92 45 90 95 68 97 75 88 85

3. Educational Measurement Measurement: assignment of numbers to differentiate values of a variable Purpose of measurement for research Provide a standard format for recording observations, performances, or other responses of subjects and summarizing results GOOD RESEARCH MUST HAVE SOUND MEASUREMENT!!

4. Descriptive Statistics Statistics: procedures that summarize and analyze quantitative data Descriptive statistics: statistical procedures that summarize a set of numbers in terms of central tendency or variation Important for understanding what the data tells the researcher

5. Descriptive Statistics Statistics: procedures that summarize and analyze quantitative data Descriptive statistics Statistical procedures that summarize a set of numbers in terms of central tendency or variation Foundational for inferential statistics Important for understanding what the data tells the researcher

6. Measures of central tendency Mean (µ) Median Mode

7. Thought Question Consider the following scores on a test Adriane 85 Marco 90 Adriane 85 Linda 75 Christy 99 Chantelle 88 Jay 45 Remi 68 Marcus 97 Chi Bo 92 Donnie 85 Which measure of central tendency would Adriane use when telling her parents about her performance?

8. Thought Question If Jay scored an 85 instead of a 45, what changes?

9. Descriptive Statistics Frequency distributions (see Figure 6.2) Normal - scores equally distributed around middle Positively skewed - large number of low scores and a small number of high scores; mean being pulled to the positive Negatively skewed - large number of high scores and a small number of low scores; mean being pulled to the negative

10. Normal Distribution

11. An Extreme Example Consider the salaries of 10 people Group A – All are teachers. Salaries: $45,000$45,000$45,000 $50,000$50,000$50,000 $50,000 $55,000 $55,000 $55,000

12. An Extreme Example Consider the salaries of 10 people Group B – All are teachers; 1 won the lottery. Salaries: $45,000$45,000$45,000 $50,000$50,000$50,000 $50,000 $55,000 $55,000 $6,300,000

13. An Extreme Example What happens to the mean and median in these 2 examples? Does it change? What happens to the normal distribution?

14. Positive Skew

15. Negative Skew

16. Descriptive Statistics Variability How different are the scores? Types Range: the difference between the highest and lowest scores Standard deviation The average distance of the scores from the mean The relationship to the normal distribution ±1 SD = 68% of all scores in a distribution ±2 SD = 95% of all scores in a distribution

17. Variability

18. Standard Deviation

19. Variability Why does variability matter?

20. Descriptive Statistics Relationship How two sets of scores relate to one another Correlation (positive) Low.10 -.39 Moderate.40 -.69 High >.70

21. Example of Correlation

22. Validity and Reliability What’s all the fuss about?

23. Validity/Reliability and Trustworthiness Why do we need validity and reliability in quantitative studies and “trustworthiness” in qualitative studies? We can’t trust the results if we can’t trust the methods!

24. Thought Question On the ACT and SAT assessments, there is a definitive script that test administrators are required to follow exactly. What measurement issue are the test makers addressing?

25. Reliability of Measurement Reliability - The extent to which measures are free from error Error is measured by consistency

26. Reliability of Measurement Sources of error Test construction and administration Ambiguous questions, confusing directions, changes in scoring, interrupted testing, etc. Subject’s characteristics Test anxiety, lack of motivation, fatigue, guessing, etc.

27. Reliability of Measurement Reliability Measurement 0.00 indicates no reliability or consistency 1.00 indicates total reliability or consistency <.60 = weak reliability >.80 = sufficient reliability

28. Reliability of Measurement Types of reliability evidence Stability (i.e. test-retest) Testing the same subject using the same test on two occasions Limitation - carryover effects from the first to second administration of the test Equivalence (i.e. parallel form) Testing the same subject with two parallel (i.e. equal) forms of the same test taken at the same time Limitation - difficulty in creating parallel forms

29. Reliability of Measurement Equivalence and stability Testing the same subject with two forms of the same test taken at different times Limitation - difficulty in creating parallel forms

30. Reliability of Measurement Internal consistency Testing the same subject with one test and “artificially” splitting the test into two halves Limitations - must have a minimum of ten (10) questions Often see “Chronbach’s alpha” for reliability coefficient (ex – Learning styles)

31. Reliability of Measurement Agreement / Inter-rater reliability Observational measures Multiple observers coding similarly

32. Reliability of Measurement Enhancing reliability Standardized administration procedures (e.g. directions, conditions, etc.) Appropriate reading level Reasonable length of the testing period Counterbalancing the order of testing if several tests are being given

33. Validity of Measurement Validity: the extent to which inferences are appropriate, meaningful, and useful Current example – content tests and teacher licensure

34. Validity of Measurement For research results to have any value, validity of the measurement of a variable must exist Use of established and “new” instruments and the implications for establishing validity Importance of establishing validity prior to data collection (e.g. pilot tests)

35. Validity Content Predictive (criterion-related) Concurrent Construct

36. Thought Question Criticisms of standardized tests like the SAT claim that they discriminate against particular groups of students (especially minorities) and do not represent a broad enough domain of knowledge to adequately assess a student’s academic potential. What issue of validity is operating in these arguments?

37. Thought Question Other arguments against the SAT state that the tests do not adequately estimate an individual’s ability to succeed in college. What issue of validity is operating here?

38. Reader’s Digest version… Reliability The extent to which scores are free from error Error is measured by consistency Validity The extent to which inferences are appropriate, meaningful, and useful “Does the instrument measure what it is supposed to measure??”

39. Reliability & Validity of Measurement What is the relationship of reliability to validity? If a watch consistently gives the time at 1:10 when actually it is 1:00, it is ____ but not ____. ______ is necessary but not sufficient condition for _______. To be _____, an instrument must be ______, but a ____ instrument is not necessarily _____.

40. Midterm 3 parts Multiple Choice (50%) – terms and application Short Answer (25%) – application Essay (25%) – evaluate a research article. This part is take home.

41. Take Home Portion of Exam Schlosser Article Based on topics we have discussed in class and you have read about, critique the article based on the following: Introduction and research problem, including the researcher’s background and involvement Review of literature/ theoretical framework Methods of data collection (including participants) and data analysis Results and conclusions including issues of trustworthiness. Be sure to address whether we should trust the claims that the authors have made and why we should or should not trust the claims.