1. Work Sample Seminar1 Developing a Pretest & Posttest for the Literacy Work Sample Portland State University

2. 2 Overview n Constructing a Pretest and Posttest –A. Identify a possible unit goal/objective –B. Consider Pre-skills –C. Select items that match the objectives –D. Decide on a format –E. Describe how it will be administered –F. Describe how it will be scored

3. A. Identify a possible unit goal/objective: Your unit goal/objective should be at the heart of your assessment Plan to create, assess, and revise your assessment and objectives several times

4. Developing a Pretest based upon the Unit Goal/Objective Draft Unit Goal: n Example: Given a narrative text written at the fourth grade instructional level, Jane will correctly write answers to comprehension questions focused upon story elements (i.e. character, setting, sequence of 3 main events) with 80% accuracy.

5. This objective helps students make progress toward the following Common Core State Standard: College & Career Readiness (CCR) Standard for Reading: n Key Ideas and Details n 1. Read closely to determine what the text says explicitly and to make logical inferences from it; cite specific textual evidence when writing or speaking to support conclusions drawn from the text.

6. Link to Common Core State Standards (CCSS) Grade Level Standard 4.RL. Key Ideas and Details n 4.RL.1 Refer to details and examples in a text when explaining what the text says explicitly and when drawing inferences from the text. n 4.RL.3 Describe in depth a character, setting, or event in a story or drama, drawing on specific details in the text (e.g., a character’s thoughts, words, or actions).

7. Developing a Pre test based upon the Unit Goal/Objective Draft Unit Goal: n Example: Given 20 single syllable words containing the vowel combinations ai, ow, ea, and oa in isolation and in the context of a sentence, students will read the words aloud with 95% accuracy by May 31, 2014 for three consecutive days as measured by beginning of class list word reading activities.

8. This objective helps students make progress toward the following Common Core State Standard: College & Career Readiness (CCR) Standard for Reading: n Range of Reading and Level of Text Complexity 10. Read and comprehend complex literary and informational texts independently and proficiently.

9. Link to Common Core State Standards (CCSS) Grade Level Standard 9.RL.10 Key Ideas and Details n 9.RL.10 By the end of grade 9, read and comprehend literature, including stories, dramas, and poems, in the grades 9–10 text complexity band proficiently, with scaffolding as needed at the high end of the range.

10. B. Consider Pre skills n What are the pre skills for this unit? –Read 4 th grade text –Conceptual Understanding: Time Place Sequencing events n How will you assess the pre skills?

11. C. Select Items to Assess the Objective – comprehension example n Match definitions: 1. Characterhow the problem is resolved 2. Settingwho the story is about 3. Problemhow the characters try to solve the problem 4. Attemptsthe time and place the story occurs 5. Solutionthe goal or challenge the characters face in the story n Student reads a passage and writes answers to open ended questions to identify: n Setting n 5. Time _____________ n 6. Place _____________ n 7. Main Character ___________ n Sequence of 3 events 8.Event 1 (Problem indicator) 9.Event 2 (Attempt to solve it) 10.Event 3 (Solution indicator)

12. C. Select items that match the objective(s) – phonics example PreskillsWay to Assess 1. Recognizing vowels from consonants Present 20 letters in random order, 5 of which are vowels. Have student circle the vowels. 2. Saying the long vowel sound for a, e, and o. Point to each circled vowel (above) and have student say its long sound. 3. Saying most of the consonant sounds Have students say the sounds for the consonants in the list. 4. Blending 3 to 4 letter sounds –> word Have students read these words: pal, rep, tog, mend, frat.

13. Select items that match the objectives – phonics example Target SkillsWay to Assess 1. Reading words with ai Present words with ai in list and a sentence. 2. Reading words with ow Present words with ow in list and a sentence. 3. Reading words with ea Present words with ea in list and a sentence. 4. Reading words with oa Present words with oa in list and a sentence.

14. C. Select Items to Assess the Objective – Vocabulary Example n Step 1: Preskills –Are prefixes at the beginning or end of a word? –Are suffixes at the beginning or end of a word? –How many syllables are in the word employment? List the syllables, if more than one. n Step 2: Circle prefixes –Advertise –Consultation –… n Step 3: Circle suffixes –Financing –Employer –… n Step 4: Read the words aloud to the teacher –Adept –Assiduous –Reference –… n Step 5: Write a definition, in your own words, for each word –Professionalism –budgeting

15. D. Decide on a format Example Since the unit objective specifies 20 words, select five words each for ai, ow, ea, and oa. Put four of each word type in a word list, randomly mixed among the words with other vowel combinations. Write four sentences that each have one of the vowel combination words. (check to make sure the student can read all the other words in the sentence!) To assess preskills, randomly list the vowels and consonants type the words pal, rep, tog, mend, and frat. Prepare a sheet for the student to read from and a scoring sheet.

16. E. Describe how it will be administered. 1. How will you introduce the test? 2. How will you assess the pre skills? 3. How you will pretest the sequence objectives?

17. How will you introduce the test? Example n The pre/posttest will be individually administered to each student in the group. n Introduce the test by saying, “this short activity will see which things you know how to do and which things we want to teach you to do. Some things will be easy for you and some things will be hard. Don’t worry about how you do, just do the best you can.”

18. F. Decide how you will score it: Example:  Give one point for every story element word the student can match to a definition (questions 1 – 5).  Give 0 - 2 points for every story element the student can correctly write after reading the story. 0 – no answer / incorrect answer 1 – partial answer 2 – fully correct answers

19. Rubric: Data on Learning Gains To meet criteria, assessment tools must:  match description of student performance on unit /step objective (conditions and behavior),  include administration procedures and directions to students,  include criteria for scoring acceptable/ incorrect responses,  have identical or comparable pre and post measures (difficulty, number of items, format).

20. Helpful Hints Order the test items in the same order as you will teach the skills. The test must include items that assess your desired outcome, as indicated in your unit objective(s). The test format should match the conditions part of your unit objectives

21. Helpful Hints The pre- and posttest must measure the same skills in the same way. n Two options: 1. Posttest = pretest 2. Different items or prompts, but the same in its format, length, response modes, and level of difficulty.

22. Pre test Design Questions: n Is the pre-test implied in the unit objective/does it match the objective? n Does the pre-test scoring match the unit objective criteria? n Is the pre-test reasonable to complete in a short period of time? n Does the pre-test measure meaningful skills? n Does the pre-test measure relevant pre-skills?

23. Closing n Read the pre-test assignment and draft a pretest to assess your unit objectives n Be prepared to administer and revise your pretest several times n Your students should perform in a range that indicates the objective is appropriate (scoring about 20%-70% on the pretest) Work Sample Seminar23