1. © 2006 The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Instrumentation Chapter Seven

2. © 2006 The McGraw-Hill Companies, Inc. All rights reserved. McGraw-HillInstrumentation Chapter Seven Instructions: Circle the choice that indicates your opinion. 1. Teachers’ unions should be abolished.Strongly agreeAgreeUndecidedDisagreedisagree (5)(4)(3)(2)(1) 2. School administrators should be required by law to teach at least one class in a public school classroom every year.Strongly agreeAgreeUndecidedDisagreedisagree (5)(4)(3)(2)(1) 3. Classroom teachers should be able to choose the administrators in their schools.Strongly agreeAgreeUndecidedDisagreedisagree (5)(4)(3)(2)(1)

3. © 2006 The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill What are Data? Data refers to the information researchers obtain on the subjects of their research. Data refers to the information researchers obtain on the subjects of their research. Demographic information or scores from a test are examples of data collected. Demographic information or scores from a test are examples of data collected. The researcher has to determine what kind of data they need to collect. The researcher has to determine what kind of data they need to collect. The device the researcher uses to collect data is called an instrument. The device the researcher uses to collect data is called an instrument.

4. © 2006 The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Key Questions The instruments and procedures used in collecting data is called instrumentation. The instruments and procedures used in collecting data is called instrumentation. Questions arise regarding the procedures and conditions under which the instruments will be administered: Questions arise regarding the procedures and conditions under which the instruments will be administered: Where will the data be collected? Where will the data be collected? When will the data be collected? When will the data be collected? How often are the data to be collected? How often are the data to be collected? Who is to collect the data? Who is to collect the data? The most highly regarded types of instruments can provide useless data if administered incorrectly, by someone disliked by respondents, under noisy, inhospitable conditions, or when subjects are exhausted. The most highly regarded types of instruments can provide useless data if administered incorrectly, by someone disliked by respondents, under noisy, inhospitable conditions, or when subjects are exhausted.

5. © 2006 The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Validity, Reliability, and Objectivity Validity is an important consideration in the choice of an instrument to be used in a research investigation Validity is an important consideration in the choice of an instrument to be used in a research investigation It should measure what it is supposed to measure It should measure what it is supposed to measure Researchers want instruments that will allow them to make warranted conclusions about the characteristics of the subjects they study Researchers want instruments that will allow them to make warranted conclusions about the characteristics of the subjects they study Reliability is another important consideration, since researchers want consistent results from instrumentation Reliability is another important consideration, since researchers want consistent results from instrumentation Consistency gives researchers confidence that the results actually represent the achievement of the individuals involved Consistency gives researchers confidence that the results actually represent the achievement of the individuals involved Objectivity refers to the absence of subjective judgments Objectivity refers to the absence of subjective judgments This should be eliminated with regards to achievement of the subjects, even if its difficult to do so This should be eliminated with regards to achievement of the subjects, even if its difficult to do so

6. © 2006 The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Usability An important consideration for any researcher in choosing or designing an instrument is how easy the instrument will actually be to use. An important consideration for any researcher in choosing or designing an instrument is how easy the instrument will actually be to use. Some of the questions asked which assess usability are: Some of the questions asked which assess usability are: How long will it take to administer? How long will it take to administer? Are the directions clear? Are the directions clear? How easy is it to score? How easy is it to score? Do equivalent forms exist? Do equivalent forms exist? Have any problems been reported by others who used it? Have any problems been reported by others who used it? Getting satisfactory answers can save a researcher a lot of time and energy. Getting satisfactory answers can save a researcher a lot of time and energy.

7. © 2006 The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Ways to Classify Instruments Who Provides the Information? Who Provides the Information? Themselves: Self-report data Themselves: Self-report data Directly or indirectly: from the subjects of the study Directly or indirectly: from the subjects of the study From informants (people who are knowledgeable about the subjects and provide this information) From informants (people who are knowledgeable about the subjects and provide this information)

8. © 2006 The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Types of Researcher-completed Instruments Rating scales Rating scales Interview schedules Interview schedules Tally sheets Tally sheets Flowcharts Flowcharts Performance checklists Performance checklists Anecdotal records Anecdotal records Time-and-motion logs Time-and-motion logs Observation forms Observation forms

9. © 2006 The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Excerpt from a Behavior Rating Scale for Teachers (Figure 7.4) Instructions: For each of the behaviors listed below, circle the appropriate number, using the following key: 5 = Excellent, 4 = Above Average, 3 = Average, 2 = Below Average, 1 = Poor. A. Explains course material clearly. 12345 B. Establishes rapport with students. 12345 C. Asks high-level questions. 12345 D. Varies class activities. 12345

10. © 2006 The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Excerpt from a Graphic Rating Scale (Figure 7.5) Instructions: Indicate the quality of the student’s participation in the following class activities by placing an X anywhere along each line. AlwaysFrequentlyOccasionallySeldomNever 1. Listens to teacher’s instructions. Always Frequently Occasionally Seldom Never 2. Listens to the opinions of other students. Always Frequently Occasionally Seldom Never 3. Offers own opinions in class discussions.

11. © 2006 The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Example of a Product Rating Scale (Figure 7.6)

12. © 2006 The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Interview Schedule (Figure 7.7)

13. © 2006 The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Sample Observation Form (Figure 7.8)

14. © 2006 The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Discussion Analysis Tally Sheet (Figure 7.9)

15. © 2006 The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Participation Flowchart (Figure 7.10)

16. © 2006 The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Performance Checklist Noting Student Actions (Figure 7.11)

17. © 2006 The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Time-and-Motion Log (Figure 7.12)

18. © 2006 The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Types of Subject-completed Instruments Questionnaires Questionnaires Self-checklists Self-checklists Attitude scales Attitude scales Personality inventories Personality inventories Achievement/aptitude tests Achievement/aptitude tests Performance tests Performance tests Projective devices Projective devices Sociometric devices Sociometric devices

19. © 2006 The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Example of a Self-Checklist (Figure 7.13)

20. © 2006 The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Example of Items from a Likert Scale (Figure 7.14)

21. © 2006 The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Example of the Semantic Differential (Figure 7.15)

22. © 2006 The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Pictorial Attitude Scale for Use with Young Children (Figure 7.16)

23. © 2006 The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Sample Items from a Personality Inventory (Figure 7.17)

24. © 2006 The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Sample Items from an Achievement Test (Figure 7.18)

25. © 2006 The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Sample Item from an Aptitude Test (Figure 7.19)

26. © 2006 The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Sample Items from an Intelligence Test (Figure 7.20)

27. © 2006 The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Example from the Blum Sewing Machine Test (Figure 7.21)

28. © 2006 The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Sample Items from the Picture Situation Inventory (Figure 7.22)

29. © 2006 The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Example of a Sociogram (Figure 7.23)

30. © 2006 The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Example of a Group Play (Figure 7.24)

31. © 2006 The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Item Formats Questions used in a subject-completed instrument can take many forms but are classified as either selection or supply items. Questions used in a subject-completed instrument can take many forms but are classified as either selection or supply items. Examples of selection items are: Examples of selection items are: True-false items True-false items Matching items Matching items Multiple choice items Multiple choice items Interpretive exercises Interpretive exercises Examples of supply items are: Examples of supply items are: Short answer items Short answer items Essay questions Essay questions

32. © 2006 The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Unobtrusive Measures Many instruments require the cooperation of the respondent in one way or another. Many instruments require the cooperation of the respondent in one way or another. An intrusion into an ongoing activity could be involved which causes a form of negativity within the respondent. An intrusion into an ongoing activity could be involved which causes a form of negativity within the respondent. To eliminate this, researchers use unobtrusive measures, data collection procedure that involve no intrusion into the naturally occurring course of events. To eliminate this, researchers use unobtrusive measures, data collection procedure that involve no intrusion into the naturally occurring course of events. In most cases, no instrument is used, however, good record keeping is necessary. In most cases, no instrument is used, however, good record keeping is necessary. They are valuable as supplements to the use of interviews and questionnaires, often providing a useful way to corroborate what more traditional data sources reveal. They are valuable as supplements to the use of interviews and questionnaires, often providing a useful way to corroborate what more traditional data sources reveal.

33. © 2006 The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Types of Scores Quantitative data is reported in the form of scores Quantitative data is reported in the form of scores Scores are reported as either raw or derived scores Scores are reported as either raw or derived scores Raw score is the initial score obtained Raw score is the initial score obtained Taken by itself, a raw score is difficult to interpret, since it has little meaning Taken by itself, a raw score is difficult to interpret, since it has little meaning Derived score are scores that have been taken from raw scores and standardized Derived score are scores that have been taken from raw scores and standardized They enable researchers to say how well the individual performed compared to others taking the same test They enable researchers to say how well the individual performed compared to others taking the same test Examples include: Examples include: Age and Grade-level Equivalents Age and Grade-level Equivalents Percentile Ranks Percentile Ranks Standard scores are mathematically derived scores having comparable meaning on different instruments Standard scores are mathematically derived scores having comparable meaning on different instruments

34. © 2006 The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Examples of Raw Scores and Percentile Ranks (Table 7.1) 95125100 9312496 8822392 8532184 7911872 7541768 7061352 652728 621520 581416 542312 50114 N = 25 RawCumulativePercentile ScoreFrequencyFrequencyRank

35. © 2006 The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Norm-Referenced vs. Criterion- Referenced Instruments All derived scores give meaning to individual scores by comparing them to the scores of a group. All derived scores give meaning to individual scores by comparing them to the scores of a group. The group used to determine derived scores is called the norm group and the instruments that provide such scores are referred to as norm-referenced instruments. The group used to determine derived scores is called the norm group and the instruments that provide such scores are referred to as norm-referenced instruments. An alternative to the use of achievement or performance instruments is to use a criterion-referenced test. An alternative to the use of achievement or performance instruments is to use a criterion-referenced test. This is based on a specific goal or target (criterion) for each learner to achieve. This is based on a specific goal or target (criterion) for each learner to achieve. The difference between the two tests is that the criterion referenced tests focus more directly on instruction. The difference between the two tests is that the criterion referenced tests focus more directly on instruction.

36. © 2006 The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Measurement Scales There are four types of measurement scales There are four types of measurement scales Nominal Scales Nominal Scales Ordinal Scales Ordinal Scales Interval Scales Interval Scales Ratio Scales Ratio Scales

37. © 2006 The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Four Types of Measurement Scales (Figure 7.25)

38. © 2006 The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Four Types of Measurement Scales (Table 7.2) Measurement ScaleCharacteristics NominalGroups and labels data only; reports frequencies or percentages. OrdinalRanks data; uses numbers only to indicate ranking. IntervalAssumes that equal differences between scores really mean equal differences in the variable used. RatioAll of the above, plus true zero point.

39. © 2006 The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Nominal Scales Considered the simplest form of measurement. Considered the simplest form of measurement. Researchers assign numbers to different categories. Researchers assign numbers to different categories. An example would be to assign a “1” to men, a “2” to women. An example would be to assign a “1” to men, a “2” to women. The advantage to assigning numbers to the categories is to facilitate computer analysis. The advantage to assigning numbers to the categories is to facilitate computer analysis. Involves groups and labels data only. Involves groups and labels data only.

40. © 2006 The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill A Nominal Scale of Measurement (Figure 7.26)

41. © 2006 The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Ordinal Scales Involves the use of numbers to rank or order scores from high to low. Involves the use of numbers to rank or order scores from high to low. An example would be the ranking of high to low scores on an examination. An example would be the ranking of high to low scores on an examination. Differences in rankings would not necessarily be the same with certain scores. Differences in rankings would not necessarily be the same with certain scores. Ordinal scales indicate relative standing among individuals. Ordinal scales indicate relative standing among individuals.

42. © 2006 The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill An Ordinal Scale: The Winner of a Horse Race (Figure 7.27)

43. © 2006 The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Interval Scales Involves the use of numbers to represent equal intervals in different segments on a continuum. Involves the use of numbers to represent equal intervals in different segments on a continuum. Very similar to an ordinal scale with the exception of the equal intervals of points. Very similar to an ordinal scale with the exception of the equal intervals of points. Assumes that equal differences between scores really mean equal differences in the variable measured. Assumes that equal differences between scores really mean equal differences in the variable measured.

44. © 2006 The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Ratio Scales A ratio scale involves the use of numbers to represent equal distances from a known “zero point”. A ratio scale involves the use of numbers to represent equal distances from a known “zero point”. A scale designed to measure height would be a ratio scale since the zero point represents the absence of height. A scale designed to measure height would be a ratio scale since the zero point represents the absence of height. Ratio scales are almost never encountered in Educational Research. Ratio scales are almost never encountered in Educational Research.