1 The National External Diploma Program Writing Assessment in the New NEDP Linda Taylor, CASAS Director of Assessment Development National External Diploma Program Council Conference October 2012, Providence, RI

2 The Future of NEDP Writing CASAS’ review of NEDP Competencies and Tasks revealed a need for increased rigor. In terms of Writing, there is a need for: –More writing –More challenging writing In current NEDP development work, CASAS is incorporating more writing. The CASAS Written Prompt has been piloted for use in Diagnostics.

3 Writing Assessment in the New NEDP Four general types of writing in NEDP Tasks: –Extended Response Includes the Written Prompt in Diagnostics Each Competency Area will have at least one Extended Response item –Short Constructed Response –Specific Applications – e.g., letters, resume, power point presentation –Fill Matrix

4 CASAS Written Prompt January June 2014 CASAS will phase in the CASAS Written Prompt to replace the current NEDP writing process for Diagnostics January June 2013 –Agencies may voluntarily transition to the CASAS Written Prompt July 2013 – June 2014 –Agencies to be trained will be assigned timeframes for training CASAS has developed a self-paced electronic training that will be available 24/7 Additional implementation support will be scheduled periodically

5 NEDP Written Prompt Background The CASAS Functional Writing Assessment (FWA) Picture Task does not assess the highest levels of ability The CASAS National Consortium Writing Committee identified a need to develop a Written Prompt Task to assess higher level writing ability The Singapore Workforce Development Agency (WDA) funded and worked with CASAS to develop this new writing assessment from 2006 – 2008 Tryouts were also conducted in CT in 2007 and from 2010 – 2012 to ensure appropriateness in the US

6 Background The Written Prompt Task is a new writing assessment in the CASAS Functional Writing Assessment series. The task is to write an extended response generated by a written essay topic. Takes 45 minutes to one hour Holistically scored on a scale of 1 – 5 Diagnostic feedback is provided Designed to be computer-delivered

7 Development Target population –Advanced adult ESL and ABE learners and workforce participants –GED preparation, high school programs, NEDP –Transition to post-secondary Content of written prompts must be appropriate for both employed and unemployed test takers Developed 10 prompts; selected 5 for US field testing

8 Written Prompt Testing Process –Test Directions embedded in the NEDP software –New spell check feature tutorial is included –Client enters writing into NEDP software –Two trained and certified NEDP assessors score independently. Can be two assessors or one assessor and one advisor –If they don’t agree, they discuss and come to consensus –Client must score 3 (out of 5) to enter General Assessment

9

10

11

12

13

14 Written Prompt Rubric Development CASAS researched academic essay rubrics Wanted continuity with CASAS Functional Writing Assessment Picture Task rubric Created/revised numerous versions Issues: Voice/style, key elements, formatting of rubric for holistic scoring Scored both Singaporean and US responses Trained Singapore staff and part-time scorers –good success rate

15 Providing Diagnostic Feedback Holistic scoring with targeted feedback on relevant aspects of the writing sample Drop-down menu items AND Open-ended comments Show example

16 Results from Phase 1 Pilot (2009 – May 2010) Collected information about the writing ability levels of the NEDP population Adjusted the Written Prompt rubric to distinguish more clearly between rubric score points Revised the wording in the directions to the examinee (“Write about the essay topic” instead of “Write three paragraphs…”)

17 Results from Phase 1 Pilot Collected writing samples for scorer training materials Realized that computer delivery lends itself to including a “Spell-check” feature which would make the writing assessment more realistic Converted to CASAS eTests software delivery platform to conduct more tryouts (2010)

18 Written Prompt -- Current Status Created and tried out self-paced scorer training materials (2011 – 2012) –Currently converting into Moodle platform Now embedding the test directions and delivery into the NEDP online software CASAS will work with State Departments of Ed to determine training schedule and who will score

19 Written Prompt Research Studies Inter-rater Reliability studies – Two local program scorers score the same responses Parallel forms –At least 100 test takers take a set of two prompts on the same day or within one or two days

20 Writing in Generalized Assessment Extended writing is included in all NEDP Competency Areas (except Health) These items require writing based on analyzing or summarizing information from higher level reading Scoring Guidelines are provided for each item that requires writing. Items requiring writing are judged on content and ability to clearly communicate in writing. Errors in mechanics are only considered if they interfere with clarity of communication.

21 NEDP Writing Foundation Competency 3 Write to: present, explain and examine information and ideas state a position, with clear and coherent supporting arguments describe experiences and events, real or created, and express thoughts, feelings and emotions, using writing techniques, organizational structures, word choices, English language conventions, and styles of expression that are effective and appropriate to the purpose.

22 Writing Assessment in the New NEDP Four general types of writing in NEDP Tasks: –Extended Response Includes the Written Prompt in Diagnostics Each Competency Area will have at least one Extended Response item –Short Constructed Response –Specific Applications – e.g., letters, resume, power point presentation –Fill Matrix

23 Extended Written Response Type of Writing Assessment PI Item Title Description of Writing Required Phase paragraphs The Legal System Write an argument for and against Miranda Rights Phase paragraphs Critique Live Performance Apply vocabulary; evaluate quality of cast, costumes, music, special effects

24 Short Paragraph Phase sentences 43.2 National Geography Explain how geographic feature has been affected by environmental factor Phase 1 2 or more sentences 2.0Read literary texts ̶ literature Apply vocabulary; analyze central ideas; evaluate literary elements; make inferences

25 Extended Narrative: The Legal System 4. Required: Client must write four or five paragraphs that cover components a, b and c, below. Assess this response for content only; mistakes in mechanics are to be considered only if they interfere with clarity

26 Extended Narrative: The Legal System Guideline: Check each box that is demonstrated. If PTA is required, the response must be rewritten; however, boxes that have already been checked will not require re-demonstration.  a.Identifies and summarizes the main idea behind the argument for the reading of the Miranda Rights to an individual under arrest

27 Extended Narrative: The Legal System  b. Identifies and summarizes the main idea behind the justification for not reading the Miranda Rights to a suspected terrorist  c. Cites two resources researched at end of the response from which main ideas were drawn for arguments for and against reading the Miranda Rights, using citation guidelines in R4  D  NDYou provided an:  incorrect  incomplete Written response describing circumstances under which the Miranda Rights should or should not apply to an individual, citing two sources of information.

28 Short Paragraph: Science Issues in the News 3.Required: Two sentences interpreting data in line graph of temperature change:  one sentence summarizing temperature movement between 1900 and 2000, and  one sentence that makes prediction for next 100 years; prediction may mention general trend or more specific temperature increase Guideline: Assess this response for content only; mistakes in mechanics are to be considered only if they interfere with clarity. For sample responses, see following box.  D  ND You provided an:  incorrect  incomplete statement on temperature changes displayed in graph and prediction of future temperature change.

29 Sample Responses 3.Sample response, acceptable: Between the years of 1900 and 2000, average temperatures have increased two degrees due to human effects. Based on this data, we can predict that average temperatures will increase a similar amount in the next 100 years. Sample response, not acceptable: This graph shows effect of natural forces and human effects. Temperatures are changing.

30 One or Two Sentences One or two sentences 53.1Elements of the Scientific Method Provide evidence for hypothesis, dependent and independent variables; identify correct steps in experiment

31 Specific Applications: Resume Resume 62.3Creating a Resume Design and write a resume for a job notice identified in earlier assessment (6 content criteria; 5 formatting criteria)

32 Specific Applications: Formal Letter Consumer Complaints Write a formal complaint letter (4 types of information; 6 elements of a formal letter) 62.4Employment Cover Letter Write an cover letter for job notice identified in earlier assessment (9 criteria including no errors in spelling, grammar and punctuation)

33 Formal Letter: Consumer Complaints 4a.Required:A complaint letter to Copper City Electronics with each of the four types of information below: 1) stating the problem 2) asking for specific action 3) including copies of all related documents, if applicable 4) providing contact information

34 Formal Letter: Consumer Complaints 4b. Required: All six elements of a formal letter, following guidelines for written statements and placement of each convention 1) Date 2) Salutation 3) Sender Address 4) Receiver Address 5) Closing 6) Signature Guideline: Check each box that is demonstrated. If PTA is required, the letter must be rewritten; however, boxes that have already been checked will not require re-demonstration

35 Formal Letter: Consumer Complaints Click Sample Letter Response and Letter Formats to guide evaluation. Any of the six displayed formats is acceptable.  D  NDYou provided a  incorrect  incomplete formal complaint letter containing four types of information and six formal elements.

36 Topics for Discussion Will writing mechanics (e.g., spelling, punctuation) be assessed whenever writing is required in a task? –No, in short written responses, the focus will be on content. Other Topics?

37 NEDP Contact Information Jim Harrison Director, National External Diploma Program , ext. 191 Melissa DaytonLinda Taylor NEDP CoordinatorCASAS Director of Assessment Development , ext , ext. 186 Christine O’Hara NEDP Operations , ext