1. Fraenkel/Wallen, How to Design and Evaluate Research in Education, Fifth Edition. © 2000 by The McGraw-Hill Companies, Inc. All rights reserved. Excerpt from a Graphic Rating Scale Figure 7.6 38 Instructions: Indicate the quality of the student’s participation in the following class activities by placing an X anywhere along each line. 1. Listens to teacher’s instructions AlwaysFrequentlyOccasionallySeldomNever 2. Listens to the opinions of other students AlwaysFrequentlyOccasionallySeldomNever 3. Offers own opinions in class discussions AlwaysFrequentlyOccasionallySeldomNever

2. Fraenkel/Wallen, How to Design and Evaluate Research in Education, Fifth Edition. © 2000 by The McGraw-Hill Companies, Inc. All rights reserved. Interview Schedule (for Teachers) Designed to Assess the Effects of a Competency-Based Curriculum in Inner-City Schools Figure 7.8 39 1.Would you rate pupil academic learning as excellent, good, fair, or poor? a.If you were here last year, how would you compare pupil academic learning to previous years? b.Please give specific examples. 2.Would you rate pupil attitude toward school generally as excellent, good, fair, or poor? a.If you were here last year, how would you compare pupil attitude toward school generally to previous years? b.Please give specific examples. 3.Would you rate pupil attitude toward learning as excellent, good, fair, or poor? a.If you were here last year, how would you compare attitude toward learning to previous years? b.Please give specific examples. 4.Would you rate pupil attitude toward self as excellent, good, fair, or poor? a.If you were here last year, how would you compare pupil attitude toward self to previous years? b.Please give specific examples.

3. Fraenkel/Wallen, How to Design and Evaluate Research in Education, Fifth Edition. © 2000 by The McGraw-Hill Companies, Inc. All rights reserved. Discussion Analysis Tally Sheet Figure 7.9 40 Type of Remark

4. Fraenkel/Wallen, How to Design and Evaluate Research in Education, Fifth Edition. © 2000 by The McGraw-Hill Companies, Inc. All rights reserved. Participation Flowchart Figure 7.10 41 Source: Adapted from E.I. Sawin (1969). Evaluation and the work of the teacher. Belmont, CA: Wadsworth, p. 179.

5. Fraenkel/Wallen, How to Design and Evaluate Research in Education, Fifth Edition. © 2000 by The McGraw-Hill Companies, Inc. All rights reserved. Part of a Checklist Figure 7.11 42 1.Takes slide________ 2.Wipes slide with lens paper ________ 3.Wipes slide with cloth ________ 4.Wipes slide with finger ________ 5.Moves bottle of culture along the table ________ 6.Places drop or two of culture on slide ________ 7.Adds more culture ________ 8.Adds few drops of water ________ 9.Hunts for cover glasses ________ 10.Wipes cover glass with lens paper ________ 11.Wipes cover glass with cloth ________ 12.Wipes cover with finger ________ 13.Adjusts cover with finger ________ 14.Wipes off surplus fluid ________ 15.Places slide on stage ________ 16.Looks through eyepiece with right eye ________ 17.Looks through eyepiece with left eye ________ 18.Turns to objective of lowest power ________ 19.Turns to low-power objective ________ 20.Turns to high-power objective ________ 21.Holds one eye closed ________ 22.Looks for light ________ 23.Adjusts concave mirror ________ 24.Adjusts plane mirror ________ 25.Adjusts diaphragm ________ 26.Does not touch diaphragm ________ 27.With eye at eyepiece, turns down coarse adjustment screw ________ 28.Breaks cover glass ________ 29.Breaks slide ________ 30.With eye away from eyepiece, turns down coarse adjustment screw ________ Source: Adapted from R. W. Tyler (1930). A test of skill in using a microscope. Educational Research Bulletin 9(11): 493-496.

6. Fraenkel/Wallen, How to Design and Evaluate Research in Education, Fifth Edition. © 2000 by The McGraw-Hill Companies, Inc. All rights reserved. Example of a Self-Checklist Figure 7.13 43

7. Fraenkel/Wallen, How to Design and Evaluate Research in Education, Fifth Edition. © 2000 by The McGraw-Hill Companies, Inc. All rights reserved. Examples of Items from a Likert Scale Measuring Attitude toward Teacher Empowerment Figure 7.14 44 Instructions: Circle the choice after each statement that indicates your opinion. 1. All professors of education should be required to spend at least six months teaching at the elementary or secondary level every five years.Strongly agreeAgreeUndecidedDisagreedisagree (5)(4)(3)(2)(1) 2. Teachers’ unions should be abolished.Strongly agreeAgreeUndecidedDisagreedisagree (1)(2)(3)(4)(5) 3. All school administrators should be required by law to teach at least one class in a public school classroom every year.Strongly agreeAgreeUndecidedDisagreedisagree (5)(4)(3)(2)(1)

8. Fraenkel/Wallen, How to Design and Evaluate Research in Education, Fifth Edition. © 2000 by The McGraw-Hill Companies, Inc. All rights reserved. Example of the Semantic Differential Figure 7.15 45 Instructions: Listed below are several pairs of adjectives. Place a checkmark (  ) on the line between each pair to indicate how you feel. Example Hockey: exciting :_____:_____:_____:_____:_____:_____:_____:_____: dull If you feel that hockey is very exciting, you would place a check in the first space next to the word “exciting.” If you feel that hockey is very dull, you would place a checkmark in the space nearest the word “dull.” If you are sort of undecided, you would place a checkmark in the middle space between the two words. Now rate each of the activities that follow [only one is listed]: Working with other students in small groups friendly :_____:_____:_____:_____:_____:_____:_____:_____: unfriendly happy :_____:_____:_____:_____:_____:_____:_____:_____: sad easy :_____:_____:_____:_____:_____:_____:_____:_____: hard fun :_____:_____:_____:_____:_____:_____:_____:_____: work hot :_____:_____:_____:_____:_____:_____:_____:_____: cold good :_____:_____:_____:_____:_____:_____:_____:_____: bad laugh :_____:_____:_____:_____:_____:_____:_____:_____: cry beautiful :_____:_____:_____:_____:_____:_____:_____:_____: ugly

9. Fraenkel/Wallen, How to Design and Evaluate Research in Education, Fifth Edition. © 2000 by The McGraw-Hill Companies, Inc. All rights reserved. Sample Items from the Picture Situation Inventory Figure 7.22 46 Source: N. T. Rowan (1967). The relationship of teacher interaction in classroom situations to teacher personality variables. Unpublished doctoral dissertation, p. 68. Salt Lake City: University of Utah.

10. Fraenkel/Wallen, How to Design and Evaluate Research in Education, Fifth Edition. © 2000 by The McGraw-Hill Companies, Inc. All rights reserved. Example of a Sociogram Figure 7.23 47

11. Fraenkel/Wallen, How to Design and Evaluate Research in Education, Fifth Edition. © 2000 by The McGraw-Hill Companies, Inc. All rights reserved. Criterion-Referenced vs. Norm-Referenced Evaluation Instruments Page 158 48 Criterion-referenced: A student... spelled every word in the weekly spelling list correctly. solved at least 75 percent of the assigned problems. achieved a score of at least 80 out of 100 on the final exam. did at least 25 push-ups within a five-minute period. read a minimum of one nonfiction book a week. Norm-referenced: A student... scored at the 50th percentile in his group. scored above 90 percent of all the students in the class. received a higher grade point average in English literature than any other student in the school. ran faster than all but one other student on the team. and one other in the class were the only ones to receive A’s on the midterm.

12. Fraenkel/Wallen, How to Design and Evaluate Research in Education, Fifth Edition. © 2000 by The McGraw-Hill Companies, Inc. All rights reserved. More About Research: Some Tips About Developing Your Own Instrument Box 7, Page 131 49 1.Be sure you are clear as to what variables are to be assessed. Much time and effort can be wasted by definitions that are too ambiguous. If more than one variable is involved, be sure that both the meaning and the items for each variable are kept distinct. In general, a particular item or question should be used for only one variable. 2.Review existing instruments that measure similar variables in order to decide upon a format and to obtain ideas on specific items. 3.Decide on a format for each variable. Although it is sometimes appropriate to mix multiple-choice, true- false, matching, rating, and open-ended items, doing so complicates scoring and is usually undesirable. Remember: Different variables often require different formats. 4.Begin compiling and/or writing items. Be sure that, in your judgment, each is logically valid – that is, that the item is consistent with the definition of the variable. Try to ensure that the vocabulary is appropriate for the intended respondents. 5.Have colleagues review the items for logical validity. Supply colleagues with a copy of your definitions and a description of the intended respondents. Be sure to have them evaluate format as well as content. 6.Revise items based on colleague feedback. At this point, try to have about twice as many items as you intend to use in the final form (generally at least 20). Remember that more items generally means higher reliability. 7.Locate a group of people with experience appropriate to your study. Have them review your items for logical validity. Make any revisions needed and complete your items. You should have half again as many items as intended in the final form. 8.Try out your instrument with a group of respondents who are as similar as possible to your study respondents. Have them complete the instrument and then discuss it with them, to the extent that this is feasible, given their age, sophistication, and so forth. 9.If feasible, conduct a statistical item analysis with your tryout data (at least 20 respondents are necessary). Such analyses are not difficult to carry out, especially if you have a computer. The information provided on each item indicates how effective it is and, sometimes, even suggests how to improve it. See, for example, K. R. Murphy and C. O. Davidshofer (1991). Psychological testing: Principles and applications. Englewood Cliffs, NJ: Prentice Hall. 10.Select and revise items as necessary until you have the number you want.

13. Fraenkel/Wallen, How to Design and Evaluate Research in Education, Fifth Edition. © 2000 by The McGraw-Hill Companies, Inc. All rights reserved. Four Types of Measurement Scales Figure 7.25 50 SCALE Nominal Interval Ratio Ordinal EXAMPLE Gender Temperature (in Fahrenheit) Money Position in race

14. Fraenkel/Wallen, How to Design and Evaluate Research in Education, Fifth Edition. © 2000 by The McGraw-Hill Companies, Inc. All rights reserved. An Ordinal Scale: The Winner of a Horse Race Figure 7.27 51