1. Overview of the 2009-2010 Alternate Proficiency Assessment Administrator, Coordinator, and Child Study Team Training 12009-2010 NJ APA Administrator Training

2. Agenda APA Participation pages 3 – 19 Student Transfers, Medical Leave, and Excessive Absenteeism pages 20 – 24 Administrator Responsibilities including Pre-ID data submission, materials survey, etc., pages 25 – 31 APA Test Design pages 32 – 65 Common Errors and APA Revisions pages 66 – 70 Scoring Rubric Information pages 71 – 105 Sample Entry 106 – 112 Professional and Ethical Responsibility When Compiling and Submitting APA page 113 Score Reports and Universal Scoring Issues pages 114 – 125 Administrator APA Review and Other Tasks pages 126 – 136 Request for Teacher Assistance page 137 Contact Information pages 138 - 141 22009-2010 NJ APA Administrator Training

3. What is the Purpose of the APA? To measure performance of students with the most significant cognitive disabilities on the CCCS – the knowledge and skills that ALL New Jersey students are learning To ensure that ALL New Jersey students are included in the accountability system that is required by the No Child Left Behind Act 32009-2010 NJ APA Administrator Training

4. What is the APA? An alternate assessment that measures achievement of CCCS specifically designed for students with the most significant cognitive disabilities. A portfolio of student work that demonstrates a student’s performance of knowledge and skills that are LINKED to grade level knowledge and skills. ◦ This is a requirement of the USDOE. 42009-2010 NJ APA Administrator Training

5. Who are the Students who Participate in the APA? Students with the most significant cognitive disabilities Decisions about who participates in the APA are not based on: ◦ Student’s disability category ◦ Student’s educational placement ◦ On which assessment the student is most likely to score highest 52009-2010 NJ APA Administrator Training

6. Participation Decisions Students with the most significant cognitive disabilities will participate in: ◦ The APA in one/some content area(s) and the general assessment with accommodations in the other content area(s) or ◦ The APA in all content areas (LAL, Mathematics, and Science). 62009-2010 NJ APA Administrator Training

7. Determining Who Participates In The APA Decisions are made: ◦ By the IEP team ◦ Separately for each content area assessed by the general statewide assessment ◦ Using information about the student and the assessments  Test Specifications/Sample Test forms  Present Levels of Academic Achievement and Functional Performance/Goals and Objectives  Student Work Samples 72009-2010 NJ APA Administrator Training

8. High School Assessment YES SRA NO PASS? HSPA – IEP says student Is Expected to Pass- Student takes HSPA HSPA - IEP says Student Will Take HSPA but is Not Required to Pass for a Diploma – Student Takes Student takes HSPA HSPA (No Further Assessment) APA Student participates in APA (No Further Assessment) How will student participate in statewide assessments? 8 2009-2010 NJ APA Administrator Training

9. Participation Guidelines Information on APA participation guidelines is provided in your training packet. This checklist must be completed in order to determine which students are eligible for the APA. An administrator must sign the form and include it in the APA binder. Access to the Directory of Test Specifications and Sample Test for the general assessment is necessary in order to complete the checklist correctly. 92009-2010 NJ APA Administrator Training

10. Participation Criteria Checklist 1. Is the nature of the student’s cognitive disability severe? 2. Is the student’s cognitive disability so severe that the student is not receiving instruction in any of the knowledge and skills measured by the general statewide assessment? ◦ While all students are to be instructed in the Core Curriculum Content Standards, not all students with significant cognitive disabilities are instructed at the same level of skills assessed by the general assessment. ◦ Upon reviewing the test specifications and sample test forms, if the student is NOT receiving ANY instruction at the same skill levels, then answer YES to question 2. 10 2009-2010 NJ APA Administrator Training

11. Participation Criteria Checklist (continued) 3. Is the student’s cognitive disability so severe that the student cannot complete any of the types of questions on the assessment in the content area, even with accommodations and modifications? 4. Is the student’s Individualized Education Program (IEP) aligned to grade level New Jersey Core Curriculum Content Standards through modified expectations? 11 2009-2010 NJ APA Administrator Training

12. The answer to all four questions must be YES in order to assess the APA for a content area. A school or district administrator must sign the checklist as confirmation that the checklist process was done correctly. A teacher may not sign the form. A signed copy must be placed in the APA binder. 122009-2010 NJ APA Administrator Training Participation Criteria Checklist (continued)

13. Assessment and CCCS Materials Additional copies of all APA materials, including the teacher training materials, may be obtained on the Pearson website: http://pem.ncspearson.com/nj/apahttp://pem.ncspearson.com/nj/apa Copies of the Core Curriculum Content Standards, curriculum frameworks, test specifications, and sample tests for the general assessments can be obtained on the DOE website: www.state.nj.us/education ◦ Copies can be ordered for a fee by calling the DOE publications office at 609-984-0905. 132009-2010 NJ APA Administrator Training

14. What Must Be Written in Each Student’s IEP? If the student is taking the general assessment, list the assessment and the accommodations/modifications (from approved list) that will be provided. If the student is not taking the general assessment in one or more content areas, include a rationale for this decision. If the student is not taking the general assessment for LAL, Math, and/or Science, list that he/she will be participating in the APA. IEP model forms and information can be found on the DOE website http://nj.gov/education/specialed/iep_form_ann.pdf 14 2009-2010 NJ APA Administrator Training

15. What Grades and Subjects Are Assessed? Students in grades 3 – 8 and 11 must be assessed in language arts literacy and mathematics. Students in grades 4 and 8 must also be assessed in science. Students in high school must be assessed in science the year they receive instruction in biology (grade 9, 10, 11, or 12). ◦ High school students may have an APA submitted for science in a different year than the year the APA is submitted for language arts and math. 152009-2010 NJ APA Administrator Training

16. Grade 12 Students If a senior is new to the state this year and has not participated in either the APA or the HSPA, the IEP team must determine which assessment is appropriate and the student must participate in this assessment. Students who were juniors last year and should have participated in the APA, but did not, must participate in the APA this year. 162009-2010 NJ APA Administrator Training

17. Retained Students If a student has been retained in a grade due to poor academic performance the student must be reassessed in that grade. ◦ Students who remain in the same assigned grade twice due to reasons not related to academic performance must be assessed only once in that assigned grade, except for the circumstances described below. If a student received a score report with a void for illness or a void for no evidence provided in a subject area, and the student is retained in that grade, the student must be assessed again. ◦ If a student in 11 th grade received a void for illness or void for no evidence in language arts literacy or mathematics, the student must be assessed again in these content areas, either in 11 th grade if they remain in that grade, or as a 12 th grader. 17 2009-2010 NJ APA Administrator Training

18. Assigned Grade Level The grade level of the student must be marked on the Demographic Scan Sheet that is included in the portfolio. This is the assigned grade level used during scoring of the APA. ◦ If a record change for the student’s grade level is submitted in April 2010, the APA assigned grade level becomes the grade level provided during record changes. The assigned grade level will be matched against the grade level of the CPI Links selected for assessment during the scoring process. ◦ If the grade level of the CPI Links does not match the assigned grade level provided during data collection, the entry will receive a zero score.  Please note that students in high school (grade 9, 10, 11, or 12) are tested on the grade 12 CPI Links. 18 2009-2010 NJ APA Administrator Training

19. LEP LAL Exemption If a student has entered the United States after July 1, 2009, AND is currently enrolled in a language assistance program, this student is EXEMPT from the LAL assessment. The APA scan sheet delivered to districts in February 2010 must indicate the LEP LAL exemption. 192009-2010 NJ APA Administrator Training

20. Student Transferred to NJ District from Out of State after October 30, 2009 Students who require an APA, and enter your school after October 30, 2009, transferring from out of state, have missed the cut-off date for the development of the APA assessment. ◦ Submit a binder, a completed scan sheet, and a letter from an administrator that indicates from where the student transferred, the date of the transfer, and noting this as the justification for no APA evidence. ◦ The score report will indicate a Void 4, no portfolio evidence, when the student is a late registrant. 20 2009-2010 NJ APA Administrator Training

21. Student Transferred from One NJ District to Another NJ District The district that has the student during any part of a collection period must compile the portfolio. Once the student transfers within NJ, the district must send the portfolio to the new district. The new district must complete the portfolio and submit it for scoring. ◦ If a student is enrolled in a district after July 1, 2009, the administrator may bubble on the scan sheet Time In School and Time In District less than one year. This will remove the student’s results from the AYP report. ◦ If the student transfers out of the state prior to the end of the second collection period, the portfolio does not need to be submitted for scoring. It may be sent to the out-of-state school for use when reviewing the student’s educational program. 21 2009-2010 NJ APA Administrator Training

22. Medical Emergency ◦ Medical emergency is the occurrence of a severe medical or psychiatric condition or episode which requires medical attention or supervision during which time the student is not able to participate in state assessments. ◦ APA students who are not receiving instruction due to a medical emergency may be eligible for a Void Code due to Medical Emergency when there is not enough APA evidence. ◦ APA uses the term extensive sick leave/hospitalization interchangeably with the term medical emergency. Note that no other definition may be used when determining a void due to medical emergency. ◦ See the requirements for this void on the next slide. 22 2009-2010 NJ APA Administrator Training

23. ◦ When a student is out of school for an extended amount of time and not receiving instruction due to extensive sick leave or hospitalization, the portfolio may be eligible to receive a Void 1 (medical emergency code). Eligibility is based only on the info below.  If the student is receiving instruction for 10 days or less during a collection period, and  The student has an extended hospitalization or leave due to illness and is not receiving instruction, and  An official record documenting the student absences is available, then ◦ Include a letter from an administrator documenting the medical emergency. You must also include a record of the student’s absences. The portfolio will be voided due to extended illness during the collection period. ◦ A district administrator can call the Office of State Assessments if additional clarification is needed on the use of this exemption. 23 2009-2010 NJ APA Administrator Training Medical Emergency

24. Absenteeism - Non Medical Emergency If a student is receiving instruction for 10 days or less in a collection period (excessive absenteeism), and the reason for absence is unrelated to a medical emergency/extensive illness, then the school administrator should contact the Office of Assessments. 24 2009-2010 NJ APA Administrator Training

25. Test Coordinator Main Responsibilities Ensure APA participation guidelines are used correctly by child study teams. Ensure registration of APA students and Submission of Pre-ID file by the deadline. Periodically review teacher’s APA process, use of test specifications and links, etc. Complete Scan sheet (similar to front of test book) Complete Final portfolio review prior to shipping Ensure Timely shipment of portfolios Review APA record change rosters and correct/revise as needed 252009-2010 NJ APA Administrator Training

26. Materials Survey – Registration of APA Students Register the number of APA students in your district no later than October 2, 2009. ◦ Failure to register your students will result in a failure of delivery of APA materials to your district. ◦ Survey should be completed online (see memo). ◦ Do not register out of district students that are assessed in New Jersey. The receiving district will register these children. ◦ Register students that are sent by your district to an out-of-state placement. 26 2009-2010 NJ APA Administrator Training

27. Online Registration http://ncsschoolhouse.com Log On Using the User ID and Password Provided (blue sheet with Material Survey) Select Enrollments and Follow the Prompts (Select APA, etc.) Any questions? Please call our NJ APA Customer Service team at 888-705-9416 272009-2010 NJ APA Administrator Training

28. NJ Students Sent Out of State The sending district is responsible for communication to the out-of-state placement. This includes providing training materials to the school, ensuring that the CCCS is instructed and assessed, etc. Portfolios must be sent to the sending district. The sending district includes this portfolio in the district’s shipment. Registration of these students, completion of scan sheets, etc., is the responsibility of the sending school. 282009-2010 NJ APA Administrator Training

29. Electronic Pre-ID File Pre-ID File ◦ Student demographic information can be submitted electronically using the Pre-ID tab on the Pearson’s SchoolHouse web site http://ncsschoolhouse.comhttp://ncsschoolhouse.com  Pre-ID Instructions were sent with the Materials Survey in August.  Pre-ID information is printed on the scan sheet in lieu of gridding the information. ◦ Pre-ID information will be printed on scan sheets for those participating districts and sent with the Return Materials shipment in February 2010. Blank scan sheets will also be provided. ◦ Errors/omissions on the Pre-ID label may be corrected by gridding the info on the scan sheet. 29 2009-2010 NJ APA Administrator Training

30. Scan Sheet Student demographic scan sheets, along with directions on the completion of the sheet, are shipped to districts in February. Administrators will need to provide such info as student birth date, SE classification code, grade level, etc. A county district school code (CDS) must be provided for the home school or school student is attending. ◦ If the student is in a receiving school, a CDS code for the sending school must also be provided. The sending school must provide this code to the receiving school for inclusion on the scan sheet. If a Pre-ID file was submitted, check the preprinted information on the scan sheet for accuracy. ◦ You may correct information submitted in the Pre-ID file by bubbling in the correct information on the scan sheet. The only exception to this is the student name, which may only be corrected during the record change process. 30 2009-2010 NJ APA Administrator Training

31. Record Change Process A roster of APA student demographic information is shipped to all districts in April 2010. Both receiving and sending schools must review this information for accuracy. ◦ The receiving school or home school is responsible for correcting/revising all student demographic information that was provided on the scan sheet. ◦ A sending school must also review this information as errors (e.g. CDS code of sending school) will impact the sending school AYP report.  Corrections and revisions to the data must be submitted to the student’s receiving school at least several days prior to the record change deadline. The receiving school must submit these changes to Pearson during the record change timeframe. 31 2009-2010 NJ APA Administrator Training

32. APA What will this portfolio look like? What are the required components? 322009-2010 NJ APA Administrator Training

33. What is the Format of the APA? Portfolio: Paper based, submit in a 3-ring binder ◦ Binders are mailed to districts in October, based on the APA registration counts your administrator submitted in September. Content Areas: Mathematics, Language Arts Literacy, and Science ◦ Science is included only in grade 4, 8, and high school. 33 2009-2010 NJ APA Administrator Training

34. What is the Format of the APA? Entries relate to content standards, grade-level CPIs, and CPI Links. Each subject area will have four entries, each one reflecting the assessment of one CPI Link. Evidence documents educational instruction and student performance of skills. The instructional activity used at the beginning and at the end of the CPI Link instruction is submitted as the assessment evidence. 34 2009-2010 NJ APA Administrator Training

35. Test Design NOTE: NO entry should contain more than four pieces of evidence. This graphic represents a student who is being assessed in all three subjects with the APA. Not all portfolios will include all three subjects. 35 2009-2010 NJ APA Administrator Training APA Portfolio Table of Contents Language Arts 4 Entries Mathematics 4 Entries Science 4 Entries

36. Test Design Example - Language Arts (Same Structure for Each Content Area) NOTE: NO entry should contain more than four pieces of evidence. An entry containing more than four pieces of evidence will result in zero scores for all dimensions. 36 2009-2010 NJ APA Administrator Training APA Portfolio Table of Contents Language Arts Entry 1 Entry Cover Sheet reflecting 1 st Standard, Strand, CPI, and CPI Link 2 pieces of evidence Language Arts Entry 2 Entry Cover Sheet reflecting 2 nd Standard, Strand, CPI, and CPI Link 2 pieces of evidence Language Arts Entry 3 Entry Cover Sheet reflecting 3 rd Standard, Strand, CPI, and CPI Link 2 pieces of evidence Language Arts Entry 4 Entry Cover Sheet reflecting 4 th Standard, Strand, CPI, and CPI Link 2 pieces of evidence

37. Page Numbering A table of contents that references page numbers in the portfolio must be included Page numbering for evidence that is longer than one page should be numbered using a combination of page number and letters (i.e. 2, 2a, 2b, etc.)  For example, a student was given a reading comprehension test (matches the CPI Link) with 8 test items. The test was two pages long, but it is one activity/piece of evidence. The pages might be numbered page 3 and 3a. Entry Cover sheets get their own page numbers. 37 2009-2010 NJ APA Administrator Training

38. When Do I Collect Evidence? Dates on the evidence must include month, day, and year Be sure to plan early in the year when you will instruct and assess each CPI Link and content area. ◦ All students will need to be assessed on an initial activity within the correct time frame above, even if the majority of instruction will not occur until the winter. ActivityCollection Period Initial ActivitySeptember 1- November 13, 2009 Final ActivityDecember 14, 2009 - February 19, 2010 382009-2010 NJ APA Administrator Training

39. Four Entries per Content Area Based on Standard And Strand 1 Based on Standard and Strand 2 Based on Standard and Strand 3 Based on Standard and Strand 4 Select CPI & CPI Link Select CPI & CPI Link Select CPI & CPI Link Select CPI & CPI Link Provide instruction & document activities and results One Piece of Evidence from September 1 - November 13, 2009 One Piece of Evidence from December 14, 2009 - February 19, 2010 Provide instruction & document activities and results Provide instruction & document activities and results Provide instruction & document activities and results One Piece of Evidence from September 1 - November 13, 2009 One Piece of Evidence from December 14, 2009 - February 19, 2010 One Piece of Evidence from September 1 - November 13, 2009 One Piece of Evidence from December 14, 2009 - February 19, 2010 One Piece of Evidence from September 1 - November 13, 2009 One Piece of Evidence from December 14, 2009 - February 19, 2010 Entry One Entry TwoEntry ThreeEntry Four 392009-2010 NJ APA Administrator Training

40. Entry Cover Sheet 402009-2010 NJ APA Administrator Training

41. Contents of the Entry Each entry requires documentation (evidence) of student performance of the skill stated in the CPI Link. Includes two different instructional activities. Two pieces of evidence is the requirement for an entry. ◦ One piece of evidence from each instructional activity. Evidence may come from classroom work, community based instructional settings, or other places where student is working on grade-level CPI Links. 412009-2010 NJ APA Administrator Training

42. Contents of the Entry (continued) ◦ The first piece of evidence documents the initial, or very early, performance of the CPI Link. This evidence must capture what the student does/does not know related to the CPI Link.  This evidence must reflect the entire CPI Link in one activity, and be collected between September 1 and November 13, 2009.  In order for the CPI Link to be eligible for the student’s APA entry, the accuracy score on the activity can not be above 39%. 42 2009-2010 NJ APA Administrator Training

43. ◦ The second piece of evidence documents final performance of the CPI Link. This evidence should capture what the student has learned related to the CPI Link.  This evidence should assess the same CPI Link using a different instructional activity, and be collected between December 14, 2009, and February 19, 2010. 43 2009-2010 NJ APA Administrator Training Contents of the Entry (continued)

44. Evidence Requirements Evidence must document the entire CPI Link noted on the Entry Cover Sheet. Evidence should relate only to the CPI Link, not assess other skills as well. ◦ Providing evidence that documents multiple skills (skills not part of the selected CPI Link), or less than the entire CPI Link, will result in a low scoring entry. Have at least 5 items/questions/task elements included in each activity 442009-2010 NJ APA Administrator Training

45. How are CPI Links Related to the CCCS? Grade-level CPIs are the building block. Content Centrality establishes what concept must be assessed. ◦ Intent/Essence of the CPI Performance Centrality directs the level of skill statement. ◦ Complexity/difficulty  APA CPI Link can be less complex than the grade-level CPI and/or less difficult  Three types of links––Matched, Near, and Far––reflect the degree of complexity/difficulty when compared to the grade-level CPI. 452009-2010 NJ APA Administrator Training

46. CPI Link Grade Level CPI Essence Near Links Far Links Matched Links Links Links Links 462009-2010 NJ APA Administrator Training

47. Using the CPI Links for Instruction CPI Links should not be used in isolation. Read the Standard and Strand for the overall concept. Read the CPI and Essence. Make sure instruction is within the Essence of the Standard, Strand, and CPI. ◦ Evidence of instruction on CPI Link will be scored not only on the narrow Link but also within the Essence of standard and CPI ◦ For example, 3.1.3G3 is a reading standard with a Matched Link that states “Interpret information from graphs, charts, and/or diagrams found in text”. The evidence must assess the student’s interpretation of graphs/charts/diagrams within the context of reading text and not within the mathematical context of data analysis. 47 2009-2010 NJ APA Administrator Training

48. Each Piece of Evidence Must: Include the student’s name Include the full date (month, day, year) Be reflective of the student’s communication mode Assess one CPI Link completely and exclusively Include a percent score for performance (accuracy) Include a percent score for the number of items/questions/task elements the student completed independently Include a key (if acronyms, abbreviations, or symbols are used when scoring student work, explain what they mean) 48 2009-2010 NJ APA Administrator Training

49. Include the student’s name Include the full date (month, day, year) Include a percentage score for accuracy Indicate the accuracy of each response  Use + for correct and – for incorrect responses Include a percentage score for independence Indicate the independence level of each response ◦ I=Independent performance, V=Verbal prompt, G=Gestural prompt, M=Model prompt, P=Physical prompt Include a minimum of 5 test items that assess the complete CPI Link Evidence Must 492009-2010 NJ APA Administrator Training

50. Assess ONLY one CPI Link ◦ Remember, when viewing the CPI Links in the APA Procedures Manual, a CPI Link is only one bullet. Once you select the type of link you want the student to work on, i.e. Matched, Near, or Far Link column, then select only one bullet in that column. Assess the entire CPI Link Include the student’s response to the test item ◦ Must be completed in the student’s mode of communication ◦ Ensures that the student can review his own work  This is a good way to “test” if the evidence documentation includes the necessary components, but it is not a requirement that you document the student’s actual review of the work. 502009-2010 NJ APA Administrator Training Evidence Must

51. For CPI Links, an activity must include at least 5 test items. ◦ For example  A student will answer 5 comprehension questions about a completed job application.  A student will use multiplication to check the answers to 10 division problems.  Comparing and ordering decimals would require 5 instances of ordering decimals—not just that 5 decimals were ordered. A Minimum of 5 Test Items 512009-2010 NJ APA Administrator Training

52. Acceptable Evidence Student work samples serve as the best evidence to document all of the necessary components of the assessment activity. ◦ This type of evidence includes the test items/questions, response options (if applicable), and student responses. ◦ Teachers may need to scribe the student’s responses when necessary. The use of videotape, audiotape, data charts, and most photographs are not acceptable evidence. 52 2009-2010 NJ APA Administrator Training

53. Student Work Samples May Be A printout of computer screen of student’s response after hitting a switch A paragraph written using picture symbols A worksheet completed by the student A drawing to extend a pattern A printout of a sentence written using a picture writer (i.e. Writing with Symbols, PictWriter, etc.) A snapshot of work too large to fit into the portfolio ( a poster, a model, etc.) 532009-2010 NJ APA Administrator Training

54. Acceptable Evidence (continued) A series of captioned photographs that document the student performing the skills included in the CPI Link will only be accepted if ◦ each item/question/task element and the student’s response is clearly present in each photograph.  Must include a description for each photo  Must include running record of the items, questions, task elements performed  Must include score information (accuracy and independence level) for each item, question, task element A photograph of a large or bulky completed project will be accepted only if the picture captures the evidence requirements for the activity. ◦ A description of the activity must be included along with the documentation of the scoring, but the actual work must be clearly shown in the picture.  For example, a picture of a poster that displays the student’s work of classification of different shapes. 54 2009-2010 NJ APA Administrator Training

55. Acceptable Photographic Evidence A series of captioned photographs that document the student performing the skills included in the CPI Link will be accepted only if: ◦ Each item and the student’s response is clearly present in each photograph  Must include a description for each photo  Must include running record of the items performed  Must include accuracy and independence information for each item and include total percent scores for each.  Must meet the requirements for the Universal Scoring Rules 55 2009-2010 NJ APA Administrator Training

56. Acceptable Photographic Evidence (continued) A photograph of a large or bulky completed project will be accepted only if the picture captures the evidence requirements for the activity. A description of the activity must be included along with the documentation of the scoring, but the actual work must be clearly shown in the picture. The Universal Scoring Rules requirements must be met. 56 2009-2010 NJ APA Administrator Training

57. Scribing Work Samples Acceptable Scribing TechniqueUnacceptable Scribing Technique Uses the student’s communication mode to document the student’s answer Does not use the student’s communication mode Allows the student to review her/his work Does not allow the student to review her/his work Is used to record the student’s response on a work sample from eye gaze, a VOD, Augmentative Communication Device, reach and grasp, etc. Does not show the student performance of the skill (observational data) 57 2009-2010 NJ APA Administrator Training

58. Unacceptable Evidence Data charts Single photographs of student working on an activity Photographs of student work that could have been included 582009-2010 NJ APA Administrator Training

59. Has fewer than 5 items/questions/task elements Has no name, date, or incomplete date Is scored using anything other than percentages ◦ e.g., ratios (4/5), percent ranges (90-100%), letter grades (A,B+) Has percent scores that can not be replicated by scoring staff 59 2009-2010 NJ APA Administrator Training Unacceptable Evidence

60. Universal Scoring Rules must be met on two pieces of evidence for the entry to be scored based on the rubric. Otherwise, the entry will receive a score of zero for all dimensions. If any entry does not have at least two pieces of evidence OR has more than four pieces of evidence per entry, it will receive a score of zero.  Two pieces of evidence are all that is required for APA. 602009-2010 NJ APA Administrator Training Evidence

61. Universal Scoring Rules for Evidence 1.Student’s Name 2.Complete Date (month/day/year)  Evidence must be completed within the collection periods 3.Percent score for accuracy  First piece of evidence must be 39% or lower  All items must be marked as either correct (+) or incorrect (−) 4.Percent score for independence  All items must be marked as either independent (I) or with a prompt level (V, G, M, P) 5.At least 5 test items 6.Rubric is included when specified in a writing CPI Link and editing marks on writing sample 7.Appropriate format (student work sample, photographs, writing rubrics) 8.Student response must be evident 9.Evidence must match the Essence of the Standard and Strand (i.e., science must be doing a science activity). 10.The entire CPI Link must be assessed. 11.The same CPI Link and skills must be assessed in both pieces of evidence. 12.Evidence must not assess more than the CPI Link (this may be found in the student work or in a writing rubric) 61 2009-2010 NJ APA Administrator Training

62. For CPI Links, an activity must include at least 5 items. ◦ For example:  A student will answer 5 comprehension questions about a completed job application.  A student will use multiplication to check the answers to 10 division problems.  Comparing and ordering decimals would require 5 instances of comparing and ordering decimals—not just that 5 decimals were ordered. Universal Scoring Rule 5: Five Items 2009-2010 NJ APA Administrator Training62

63. Rule 5 : Example Five Items When the Link has an “and” in it, 5 items across the skills are required rather than 5 items for each part of the Link. ◦ For instance, 4.2.7E1 Near Link “Calculate the area of a triangle, rectangle and square” 63 2009-2010 NJ APA Administrator Training

64. Teachers Must Think about the Two Required Activities in Conjunction As they plan for instruction, they should not think about the initial and final activity in isolation. Things to think about: What type of activities might be easier and harder for the student? Do not use the easier activity as the final activity. What type of prompt, if any, might the student need on the final activity? Do not use a more intrusive prompt level on the final activity. How can I keep the student engaged in the various activities? How will the initial activity be different than the final activity, and still engage the student 64 2009-2010 NJ APA Administrator Training

65. Activities – Errors to Avoid The first piece of evidence should not include a more difficult application/context of the skill than the final activity/evidence. Examples of this type of error (a more difficult initial activity) are below: The initial activity has longer problems with more complex steps (e.g., open-ended essay questions vs. multiple choice) than the final activity. The initial activity has multiple-choice problems where no correct answers are offered as options. Fewer additional supports (steps, formulas, pictures, application, calculators, etc.) are given in the initial activity than the final activity. Regular grade-level text is used in the initial activity, but summarized text with picture cues and key details highlighted are used in the final activity No explanation of what to do is given for the initial activity, but directions are given for the final activity. Note: directions should ALWAYS be given during an activity. No prompts on the first piece of evidence with no answers written, or all answers are clearly off topic (answers are not at all related to concept). The initial activity is not in the student’s mode of communication, but the final activity is. 65 2009-2010 NJ APA Administrator Training

66. Common Mistakes found in 2008-09 Assessments Assessing the skills contained in the CPI Link without connection to the intent or big idea of the content and concepts of CPI, Strand, and/or Standard Assessing the skills contained within the CPI Link without correct understanding of the Content or concepts of CPI, Strand, and/or Standard Assessing more than one CPI Link in an entry Assessing different parts of a CPI Link in each piece of evidence Assessing only part of the CPI Link Assessing a grade level CPI Link different from the student’s assigned grade level 66 2009-2010 NJ APA Administrator Training

67. Changes for 2009-2010 CPI Links, the list of eligible skills for use in the APA, must be used when assessing a student on the APA. The revised CPI Links are included in the 2009-2010 Procedures Manual.  Some CPI Links were revised based on teacher feedback and information obtained during the scoring process.  The CPI Links related to assessment of spelling words had to be deleted since these did not link to the other assessment test specifications.  A few CPI Links were added based on information obtained during the scoring process. You must use the 2009-2010 updated CPI Links when assessing a standard, strand, and CPI for the APA. 67 2009-2010 NJ APA Administrator Training

68. Changes for 2009-2010 You may no longer write “All items done independently” on the top of the page and skip marking “I” for each item. You must mark every item/question with an I when an item was performed independently, even if 100% of the test items were completed in this manner. ◦ Many people forgot to write prompt levels next to the items last year, therefore you must now mark every item, regardless of independent performance or use of prompts. 68 2009-2010 NJ APA Administrator Training

69. Changes for 2009-2010 If an educator is assessing a writing skill that requires a rubric for scoring, the student’s writing sample must have editing/scoring notations that correspond with the rubric scores. ◦ Additional information on the use of rubrics for writing tasks is included later in this presentation. 69 2009-2010 NJ APA Administrator Training

70. Developing Standards-Based Entries 702009-2010 NJ APA Administrator Training

71. Developing an Entry Step 1: Select a CPI and one related CPI Link to be assessed. Step 2: Assess the student to get an initial piece of evidence (accuracy must be 39% or lower) to collect for the APA entry Step 3: Identify additional age- and grade-appropriate activities for use during instruction. Provide instruction on the CPI Link. Step 4: Determine when evidence can be collected to document the final instructional assessment of the CPI Link for APA purposes. Step 5: Based on student’s accuracy score and level of prompt information on the “final” activity, determine if additional instruction and collection of evidence needs to occur for the entry. Step 6: Review evidence to ensure that all required information related to test design requirements are included. 71 2009-2010 NJ APA Administrator Training

72. Entry Cover Sheet 72 2009-2010 NJ APA Administrator Training

73. How is the APA Scored? Each portfolio entry is scored based on specific scoring criteria described in the APA scoring rubric found in the Procedures Manual. The criteria are grouped into three categories called dimensions. 73 2009-2010 NJ APA Administrator Training

74. Scoring an Activity Scoring the activity correctly for assessment purposes is important. The evidence must include scoring information about ◦ a student’s accuracy when performing the skill, and ◦ the amount of time, if any, that the student performed the skill independently. Helping teachers to understand the difference between providing task directions, providing supports, providing indirect prompts (verbal, model, and gestural), providing physical prompts, and providing the answer (directly prompting the student with the answer to the question) will help ensure that the scoring information on the evidence is accurate. 74 2009-2010 NJ APA Administrator Training

75. APA Scoring Rubric Scoring Dimensions ◦ Complexity Link  Match, near, or far link selection ◦ Performance  Percentage of accuracy when performing skill ◦ Independence  Percentage of time skill was performed independently 752009-2010 NJ APA Administrator Training

76. Scoring Dimensions Dimension01234 Complexity Evidence provided is unscorable; all dimensions will receive a score of zero CPI Link was assessed, but there are major flaws in the evidence CPI Link is a Far Link to the grade-level indicator CPI Link is a Near Link to the grade-level indicator CPI Link is a Matched Link to the grade-level indicator Independence Evidence does not include percentage of time student was independent, is not clear, or percentage cannot be replicated. All dimensions will receive a score of zero. Student completed items/tasks independently 0-39% of the time Student completed items/tasks independently 40-59% of the time Student completed items/tasks independently 60-80% of the time Student completed items/tasks independently 81-100% of the time Performance Evidence is not scored, score is not a percentage, or score cannot be replicated; all dimensions will receive a score of zero Accuracy of work is 0-39% based on the last activity OR Second activity includes more intrusive prompt Accuracy of work is 40-59% based on the last activity Accuracy of work is 60-80% based on the last activity Accuracy of work is 81-100% based on the last activity 76 2009-2010 NJ APA Administrator Training

77. Complexity Dimension Evaluates the CPI Link assessed and how closely the complexity and difficulty (Matched, Near, Far) links to the Core Curriculum Content Standards (CCCS) and grade-level cumulative progress indicators (CPI). 772009-2010 NJ APA Administrator Training

78. 0 Evidence provided is unscorable; all dimensions will receive a score of zero 1 CPI Link was assessed, but there are major flaws in the evidence 2 CPI Link is a Far Link to the grade- level indicator 3 CPI Link is a Near Link to the grade- level indicator 4 CPI Link is a Matched Link to the grade- level indicator Complexity Dimension 782009-2010 NJ APA Administrator Training

79. An entry that demonstrates work in a CPI Link but has major flaws will score a 1 in complexity. A major flaw includes: Assessing only part of the CPI Link (this may be found in the work sample or the writing rubric) Using the same activity for both pieces of evidence Complexity: Major Flaws 792009-2010 NJ APA Administrator Training

80. What Makes an Activity Different? Two activities are required when submitting evidence for an entry. ◦ Both of these activities must assess the same CPI Link. ◦ The two activities must be different activities in order to address the Universal Scoring Rules. An activity is considered to be different than another when the context of the instruction of the CPI Link is different. An activity is also considered to be different from another when the application of the skill in the CPI Link is different. 80 2009-2010 NJ APA Administrator Training

81. Examples of two activities with different context: The student is working on following vertex-edge graphs 1.after whole class introduction and demonstration of vertex-edge graphs, the student completes a worksheet 2.after having a lesson on flight plans, the student uses a vertex edge graph to follow flight paths The student is working on completing input/output tables 1.in the context of a science experiment 2.in the context of figuring out how much money he/she could make for working 2 hours vs. 8 hours The student is working on writing a cover letter 1.after researching a job of a particular interest, write a cover letter that would be used when applying for the job 2.after listening to speakers about different careers, write a cover letter for a job of interest The student is working on identifying theme in a given work of fiction 1.by reading The Outsiders and answering multiple choice questions 2.by reading Flowers for Algernon and answering multiple choice questions What Makes an Activity Different? 81 2009-2010 NJ APA Administrator Training

82. Examples of two activities with different applications The student is working on classifying types of planets 1.by using a graphic organizer 2.by matching The student is working on addition and subtraction 1.by completing a worksheet with problems on it 2.by using a Smart Board to complete math problems The student is working on writing an essay 1.by using a computer or word processor 2.by using paper and pencil The student is working on identifying theme in a given work of fiction 1.by reading passage of The Outsiders and then answering multiple choice questions 2. by reading a passage of The Outsiders and then answering short answer questions What Makes an Activity Different? 82 2009-2010 NJ APA Administrator Training

83. Independence evaluates the extent to which the student completed items/tasks independently. Independence Dimension 832009-2010 NJ APA Administrator Training

84. 0 Evidence does not include percentage of time student was independent, is not clear, or percentage cannot be replicated. All dimensions will receive a score of zero. 1 Student completed items/tasks independently 0-39% of the time 2 Student completed items/tasks independently 40-59% of the time 3 Student completed items/tasks independently 60-80% of the time 4 Student completed items/tasks independently 81-100% of the time Independence Dimension 842009-2010 NJ APA Administrator Training

85. Independence scores must be summarized as a percent. A prompt level must be documented next to each item performed by the student. I = independent V= verbal G= gestural M= model P= physical If some other system is used there must be a key. Remember, if your prompt hierarchy is not the same as the one above, you must explain your prompt hierarchy clearly, giving the names of the prompts, hierarchical order, and letter key for the prompts. Even if the student performs with 100% independence, each item must be marked with an I. Reviewers must be able to recreate the score. If the score is not replicable the entry can not be scored, and will receive zero for all dimensions. Teacher Scoring for Independence 852009-2010 NJ APA Administrator Training

86. Scoring an Activity for APA Understanding the difference between the following provisions will help ensure that the scoring information on the evidence is accurate: providing task directions providing supports providing indirect prompts (verbal, model, and gestural) providing physical prompts providing the answer (directly prompting the student with the answer to the question) 86 2009-2010 NJ APA Administrator Training

87. Examples of Supports for Mathematics Use of manipulatives or pictures Use of a calculator Task-analyzed skills Number line Stamps to write numbers Word problems read to student Enlarged text Chunked problems 87 2009-2010 NJ APA Administrator Training

88. Examples of Supports for Reading Student reads by Following along with the text Following along with picture symbols Following along with objects Following along with tactile cues Adapted text Shortened text Chunked information Large font Color coding to emphasize important information 88 2009-2010 NJ APA Administrator Training

89. Examples of Supports for Writing Stamps Graphic organizers Write using pictures or objects Write by completing cloze sentences Scribing (work must use the student’s communication mode) Speech-to-text writing programs Large writing utensils or pencil grips Raised lined paper 89 2009-2010 NJ APA Administrator Training

90. Examples of General Supports Reading initial task directions to the student Redirecting the student to stay on task ◦ “you need to finish math now” Students must be given supports for both the first and second piece of evidence. 90 2009-2010 NJ APA Administrator Training

91. Verbal, Gestural, and Model Prompting For assessment purposes, only verbal, gestural, and model prompts that are indirect are allowed during an activity. These prompts lead or guide the student towards the correct answer. Indirect prompts do not directly give the student the answer. If any other type of prompt is used, a key must be provided that explains that system. 91 2009-2010 NJ APA Administrator Training

92. Prompting During Assessment - Allowable Prompts An indirect verbal prompt (V) can provide the student with a clue to try and spark the student’s recollection of the activity or lesson so that he/she can respond to the question (e.g., “Who was the story about?” or “Remember, the main character has red hair and pigtails. Point to the main character.”) An indirect gestural prompt (G) can Provide the student with a clue as to the general location of an answer (e.g., when looking up a word in the dictionary, the teacher may tap the page on which the word can be found but not exactly where the word is on the page) 92 2009-2010 NJ APA Administrator Training

93. Prompting During Assessment - Allowable Prompts An indirect model prompt (M) can provide the student with a clue through teacher demonstration of the skill (e.g., demonstrate how to carry in an addition problem then give the student a different problem) Provide the student with a clue through acting out a scenario (e.g., when presenting a choice of three pictures and asking the student which picture represents an unbalanced force, the teacher may make a sweeping or moving motion to represent an “unbalanced force”) 932009-2010 NJ APA Administrator Training

94. A physical prompt (P) requires physical contact between the student and teacher. may be a touch at the elbow prompting the student to move his/her hand to a particular answer, or it may be more invasive, involving the teacher physically moving the student’s hand to the correct answer If the student must be given a physical prompt to perform an item element, the answer must be marked wrong. This applies to full and partial physical prompts. 942009-2010 NJ APA Administrator Training Prompting During Assessment - Allowable Prompts

95. A direct verbal prompt provides the student with the specific answer to a question or item (e.g., “Remember, the main character was Pippi. Point to the picture of the main character.”) A direct gestural prompt points out the specific answer to the student (e.g., when presenting a choice of three pictures and asking the student which picture represents an unbalanced force, the teacher points to or taps the correct picture) A direct model prompt models the exact problem and answer that the student must perform (e.g., when sorting producers and consumers, the teacher says, “Remember, corn is a producer,” picks up the picture of corn, and places it in the producer column of a chart then asks, “Which one is a producer?”) Direct Prompting During Assessment - NOT ALLOWED 952009-2010 NJ APA Administrator Training

96. Prompting During Assessment When you are considering what type of indirect prompt, if any, to give for the initial activity, you must take into account what type of indirect prompt the student may need on the final activity. A student may not be given a more intrusive prompt on the final activity than he/she was given for the initial activity. ◦ For example, if the student is given verbal prompts on the initial activity, he/she may not be given gestural, model, or physical prompts (these are more intrusive) on the final activity. If the student is asked to complete the initial activity independently, then he/she must complete the final activity independently. 96 2009-2010 NJ APA Administrator Training

97. Scoring Evidence for Independence The total Independence score must be in the form of a percent. Take the number of items completed independently divided by the total number of items multiplied by 100 to calculate the total Independence score. When scoring for Independence, the information does not include accuracy. Example 6 items performed independently/10 items x 100 = 60% 97 2009-2010 NJ APA Administrator Training

98. This dimension measures the extent to which students show progress toward achieving the CPI Link. 982009-2010 NJ APA Administrator Training Performance Dimension

99. 0 Evidence is not scored, score is not a percentage, or score cannot be replicated; all dimensions will receive a score of zero 1 Accuracy of work is 0-39% based on the last activity OR Second activity includes more intrusive prompt 2 Accuracy of work is 40-59% based on the last activity 3 Accuracy of work is 60-80% based on the last activity 4 Accuracy of work is 81-100% based on the last activity Performance Dimension 992009-2010 NJ APA Administrator Training

100. Scoring Student Evidence for Accuracy If the student performs the item independently, then Score as correct (+) any item the student performs correctly. Score as incorrect (-) any item the student performs incorrectly. Any system other than +/- must include a key. If you used a different scoring system, such as x equals incorrect and + equals correct, you must explain that clearly on the evidence. 1002009-2010 NJ APA Administrator Training

101. If the student received an indirect verbal, gestural, or model prompt when performing the skill, then Score as correct (+) any item the student answered correctly. Score as incorrect (-) any item the student answered incorrectly. Remember to mark next to each item the prompt level used : V, G, or M (e.g., +V means that an indirect verbal prompt was given and that the student answered the item correctly.) 101 2009-2010 NJ APA Administrator Training Scoring Student Evidence for Accuracy

102. If the student received a physical prompt when performing the skill, then Score the item as incorrect (-P). Remember, if the student was given the answer, the item must be marked as incorrect. Each item must be scored for accuracy, even if the student scored 100% accurate. 102 2009-2010 NJ APA Administrator Training Scoring Student Evidence for Accuracy

103. An item is scored as correct (+) when An item is scored as incorrect (-) when The student performs the skill independently and correctly The student performs the skill independently but incorrectly An indirect verbal prompt is provided and the student performs the skill correctly A verbal prompt is provided but the student performs the skill incorrectly An indirect gestural prompt is provided and the student performs the skill correctly A gestural prompt is provided but the student performs the skill incorrectly An indirect model prompt is provided and the student performs the skill correctly A model prompt is provided but the student performs the skill incorrectly A physical prompt is provided (e.g. the teacher moves the student’s hand, wrist, etc. to place the sticker in the correct place on the grid) or the answer was given to the student. Performance Summary 1032009-2010 NJ APA Administrator Training

104. Scoring Student Evidence for Accuracy Total accuracy score must be in the form of a percent. The number of items correct divided by the total number of items multiplied by 100 equals the total accuracy score. Example 8 items correct/10 items x 100 = 80% 1042009-2010 NJ APA Administrator Training

105. Prompting can affect the reliability of the performance score if not implemented correctly.  Therefore, it is important to adhere to the prompt level hierarchy described in your 2009-2010 Procedures Manual.  If the second piece of evidence includes a more intrusive prompt level than the first piece of evidence, a score of 1 will be assigned for the Performance dimension. Performance 105 2009-2010 NJ APA Administrator Training

106. MATHEMATICS Sample Entry Fall 2009 Angie 106 2009-2010 NJ APA Administrator Training

107. Mathematics Entry Example 21 107 2009-2010 NJ APA Administrator Training

108. Angie9/9/09 20% accurate 60% independent Pg. 22 1082009-2010 NJ APA Administrator Training

109. Angie 2/1/10 Pg. 23 1092009-2010 NJ APA Administrator Training

110. Pg. 23a 1102009-2010 NJ APA Administrator Training

111. Pg. 23b 1112009-2010 NJ APA Administrator Training

112. Mathematics Evidence Index 8th Grade Mathematics Entry Entry 3 Standard 4.3 Strand A CPI 1 Near Link: Describe and extend a pattern involving rational numbers—must include negatives, decimals, or fractions Rubric DimensionScoreEvidence Complexity3 Pgs. 22–23b show the student working on the complete CPI Link from the “Near” column Pg. 22 (evidence 1) is a different activity than pgs. 23–23b (evidence 2) Different problems used Different application: fill in the blank vs. word problems using a table Independence3 Pgs. 23–23b student scored 60% Independence Each item is clearly marked Performance3 Pg. 22 student scored 20% (which is below 39%) on initial activity Pgs. 23–23b student scored 80% Each item is clearly marked Pgs. 23–23b do not include a more intrusive prompt than pg. 22 1122009-2010 NJ APA Administrator Training

113. Professional and Ethical Responsibility It is the responsibility of all contributors to a student’s portfolio to ensure that any and all documentation reflect authentic, accurate, and truthful information. ◦ Reminder, it is not appropriate to manipulate the date the evidence was created, prompt/independence levels, or scoring percentages. ◦ Include only evidence that reflects the student’s actual performance, even if the student did not make as much progress as you expected. Any student portfolio that is found to contain inauthentic documentation may result in professional consequences for school and district staff. ◦ If you have a V5 on an ISR from last year please contact me to discuss the security breach. 113 2009-2010 NJ APA Administrator Training

114. Pearson Scoring Center staff scored the portfolios during April and May. A standard setting was conducted in June 2009. Score reports were shipped in July. Most portfolios will be returned in September. Portfolios designated as security breaches (Void 5) are in process. Information regarding the breach consequences will be forthcoming. Status of 2008 -2009 Portfolios and Score Reports 114 2009-2010 NJ APA Administrator Training

115. Errors that Resulted in Zero Scores No name on evidence No date or incomplete date Inappropriate format (no student responses recorded, including pictures of worksheets instead of the worksheet, etc.) Submitted evidence where the initial activity had performance score higher than 39% Less than five test questions Evidence collected with dates that are outside the testing period Percent scores that we were unable to replicate Evidence with no percent score(s) 115 2009-2010 NJ APA Administrator Training

116. Evidence/rubric assessed more than the link Evidence/rubric did not connect to the essence of the CPI Evidence/rubric assessed multiple links Evidence/rubric did not assess link Not enough evidence was submitted 1162009-2010 NJ APA Administrator Training Errors that Resulted in Zero Scores

117. Contact Information For information on interpreting the APA score reports, please review the Score Interpretation Manual on Pearson’s website http://pem.ncspearson.com/nj/apa For score report interpretation please contact: Jo-Lin Liang – 609-633-6884 For missing score reports please contact: NJ Customer Service Team – 888-705-9416 1172009-2010 NJ APA Administrator Training

118. What is Proficient? New cut scores were set in June 2009 which determine how the total score for each subject area is classified. Total scores may be classified as advanced proficient, proficient, or partially proficient. ◦ The cut scores were determined based on the grade level and content being assessed, therefore the same number of total points may be classified as advanced proficient in one subject area for a grade level, and classified as proficient for a different grade level, due to differences in the CPI Links and developmental demands of students in different grades. ◦ This mirrors the general assessments where each grade level and subject area cut scores are determined independently. 118 2009-2010 NJ APA Administrator Training

119. How is a Total Score Calculated? The scoring formula was determined by reflecting on the purpose of the APA, which is to: ◦ assess the district and school grade-level instructional program provided to the student, ◦ assess a student’s attainment of knowledge and skills of the CCCS, and ◦ encourage high but appropriate instructional standards for all students, while providing necessary supports to access the curriculum and foster the highest level of independence possible for the student. The scoring formula uses the three dimension scores that each entry receives: ◦ Complexity ◦ Independence ◦ Performance 119 2009-2010 NJ APA Administrator Training

120. How is a Total Score Calculated? The total score for each content area is calculated by adding the scores for all three dimensions for each of the four entries. Each entry has a minimum possible score of zero and a maximum possible score of 16 points. The total score for each subject area has a minimum score of 0 and a maximum score of 64 points. 120 2009-2010 NJ APA Administrator Training

121. How is a Total Score Calculated? Each scorable entry will receive a score of 1 – 4 for each dimension. ◦ Unscorable entries receive a zero for each of the three dimensions. Each entry is scored by a minimum of two people (readers). ◦ First person is reader one (R1) and second person is reader two (R2). The performance dimension score for any entry is the sum of the two reader scores (R1+R2). ◦ The addition of the two reader scores gives more weight to this dimension in the overall total score than the other two dimensions. The complexity dimension score for any entry is the average of R1 & R2. The independence dimension score for any entry is the average of R1 & R2. 121 2009-2010 NJ APA Administrator Training

122. Accountability APA test results will be combined with the results from the general assessment for accountability purposes for state and federal reports. For accountability purposes, the APA is both a student assessment, and a school/district program assessment. APA test results are included in the Final AYP report. 122 2009-2010 NJ APA Administrator Training

123. Federal Results – 1% Cap The NCLB 1% cap limits the number of APA proficient and advanced proficient results that can be used in the AYP calculation. It does not limit the number of students who may participate in the APA. Districts legitimately testing more than 1% of the total grade population with the APA should apply for a waiver from the Office of Special Education Programs. The waiver, if approved, allows proficient and advanced proficient results greater than 1% to be used in the AYP calculations. ◦ This form is usually mailed by the department to districts in the winter. 123 2009-2010 NJ APA Administrator Training

124. AYP Reports Student data is collected on the APA scan sheet. This data is submitted to districts for verification. All requested revisions to student data must be made during the record change process. No changes may be submitted after this time frame. AYP reports are produced based on the student data on file as of the close of the record change process. Reports for sending districts include students assessed out of district, out of state, and in district. 124 2009-2010 NJ APA Administrator Training

125. AYP Information Up-to-date information on the calculation of AYP reports can be obtained on the Department of Education’s website www.state.nj.us/educationwww.state.nj.us/education ◦ Review the information provided from the Office of Title One. The Accountability Workbook provides specific details. If you have questions after reviewing the online information, please call the Office of Title One at 609-943-4283. 125 2009-2010 NJ APA Administrator Training

126. Main Responsibilities Complete scan sheet materials in February 2010. ◦ Work directly with administrators from the child’s sending or receiving school to ensure accurate information. Errors on the scan sheet may affect your AYP calculations. ◦ Several fields on the scan sheet must be completed in February since they are time sensitive and can not be completed during the Pre-ID process. ◦ A few fields on the scan sheet will need to be completed by the student’s teacher. Collect portfolios from teachers by end of day on February 19, 2010. ◦ Administrator’s may choose to collect them earlier in order to complete portfolio review process when high number of portfolios are produced. 126 2009-2010 NJ APA Administrator Training

127. Final Administrator Review Test Design Review ◦ Did the evidence address the required components?  Prescribed standards and grade-level CPIs  Correct amount of evidence  Meets the Universal Scoring Rules ◦ Conducting this review periodically will allow you to find and correct errors during the collection period. Security Breach Review ◦ Is white out being used to change information? ◦ Were prompt levels changed to independent performance? ◦ Were dates on the evidence changed? ◦ Were wrong answers changed to correct answers? ◦ Were wrong answers marked correct? ◦ Are the percentage scores accurate? ◦ Are there other indications of inauthentic information? 127 2009-2010 NJ APA Administrator Training

128. Final Administrator Review Review portfolios for accuracy and authenticity by February 26, 2010. ◦ Review the portfolios in light of the student’s assigned grade level. ◦ Review the portfolios in light of the current CPI Links. ◦ Review the portfolios in light of the test design requirements. ◦ Review the portfolio to ensure all evidence is authentic. 1282009-2010 NJ APA Administrator Training

129. Shipment of Portfolios Ship portfolios to contractor between March 1 - 5, 2010. ◦ Portfolios must be submitted on time in order to be processed. ◦ Shipping materials will arrive in chief school administrator’s office in February 2010. 1292009-2010 NJ APA Administrator Training

130. Administrator/Coordinator Activities General Responsibilities When information is sent to you, pass it on to district and school administrators and teachers. Review the time lines and request APA information from your chief school administrator when it is scheduled to be delivered. Review participation guidelines carefully. Ensure that all students who need to be assessed are identified (both in-district and out-of-district). 130 2009-2010 NJ APA Administrator Training

131. General Responsibilities (continued) Review school educational programs and ensure alignment with dimensions of the scoring rubric. Ensure that APA teachers register and attend training sessions. Review the APA Procedures Manual. Pass along questions and concerns to NJ DOE or contractor. 131 2009-2010 NJ APA Administrator Training Administrator/Coordinator Activities

132. Administrator/Coordinator Checklist Before Assessment Identify a School Coordinator for each school. Register number of students participating in APA and update mailing addresses using Material Survey no later than October 2, 2009. Arrange a teacher training schedule for your school. Ensure all materials are downloaded prior to the selected training day(s). Distribute materials to schools/teachers. 132 2009-2010 NJ APA Administrator Training

133. During Assessment Distribute materials to schools/teachers. Binders and manuals will be shipped to district/receiving school. Copies may be downloaded as of September 8, 2009 from the Pearson’s website http://pem.ncspearson.com/nj/apa. Click on documentation, then Procedures Manual.http://pem.ncspearson.com/nj/apa 133 2009-2010 NJ APA Administrator Training Administrator/Coordinator Checklist

134. During Assessment (continued) Gather information on students’ sending/receiving status and collect student demographic info. ◦ Work with the receiving/sending school to confirm student data. Check periodically that lead teachers are working on the APA. 134 2009-2010 NJ APA Administrator Training Administrator/Coordinator Checklist

135. After Assessment Notify lead teachers of timeframe for returning the completed APA portfolios. Collect and review all APA portfolios for accuracy and authenticity of information and inclusion of all required components. ◦ Be sure to review the portfolios for adherence to the universal scoring rules. Review the scan sheet information to ensure accuracy of all data. 135 2009-2010 NJ APA Administrator Training Administrator/Coordinator Checklist

136. After Assessment (continued) Package the APA materials securely and label them according to instructions. Contact the carrier designated for return of the portfolios to arrange pickup of the materials. Ensure all portfolios are submitted by March 5, 2010 deadline. 136 2009-2010 NJ APA Administrator Training Administrator/Coordinator Checklist

137. APA Teacher Involvement Additional sample activities for the CPI Links are needed. ◦ Upon review of your score reports, if you have instructional activities/evidence that successfully measured one of the CPI Links, please consider allowing this evidence to be placed on the Pearson website for use by other teachers. All confidential information such as student and school names, pictures of students and staff, etc., will be masked. Teachers are needed in March 2010 for a week to score portfolios. Please email Elizabeth Celentano if you are interested in assisting in these endeavors. Email address is Elizabeth.Celentano@doe.state.nj.us Elizabeth.Celentano@doe.state.nj.us If you have sample activities that you are willing to share with other teachers in the state, please either email them as an attachment, or mail clear copies to Elizabeth Celentano, Department of Education, P.O. Box 500, Trenton, NJ 08625-0500 137 2009-2010 NJ APA Administrator Training

138. Where To Get More Information? Educational Links, Information, and Resources on Assistive Technology and Universal Design for Learning, an excellent reference document for teachers and administrators, was updated in August 2009 and can be downloaded from the documentation section of the Pearson website: http://pem.ncspearson.com/nj/apa/ http://pem.ncspearson.com/nj/apa/ 138 2009-2010 NJ APA Administrator Training

139. Where To Get More Information? Other educational materials and resources can be accessed through the New Jersey Learning Resource Center Network website: http://www.nj.gov.education/lrc http://www.nj.gov.education/lrc 139 2009-2010 NJ APA Administrator Training

140. Where To Get More Information? Elizabeth Celentano Office of State Assessments New Jersey Department of Education 609-984-0107 elizabeth.celentano@doe.state.nj.us Office of Special Education Programs New Jersey Department of Education 609-292-0147 http://www.nj.gov/education/specialed/ Department of Education website: www.state.nj.us/education 140 2009-2010 NJ APA Administrator Training

141. Where to Get More Information? Pearson Amy Dombrowski amy.dombrowski@pearson.com Bruce Murphy bruce.murphy@pearson.com Customer Servicenjapacustsvc@support.pearson.com Phone: 888-705-9416 Pearson website: http://pem.ncspearson.com/nj/apa 141 2009-2010 NJ APA Administrator Training

142. Thank You Thank you for taking time to attend this session. If you have not had all of your questions answered, please see one of the presenters for more information. 1422009-2010 NJ APA Administrator Training