1. “Understanding the Elements of 21st Century Assessment”
Writing Roadmap 2.0:
Accessing and Using Reports
Good afternoon. For the next 15 minutes, I’m going to give you a brief overview of Writing Roadmap 2.0 and discuss just a few of the reports available in the program that might be useful to your school in helping your students improve their writing skills and prepare for WESTEST 2 Online Writing.
Principals Academy
July 23, 2009
Dr. Vaughn G. Rhudy, NBCT, Coordinator
Office of Assessment, Accountability & Research

2. What is Writing Roadmap 2
What is Writing Roadmap 2.0 and how can my school use it to help students improve writing skills?
The essential question here is: What is Writing Roadmap 2.0 and how can my school use it to help students improve their writing.

3. What is Writing Roadmap 2.0?
Writing Roadmap 2.0 is a Web-based, online essay-scoring tool that provides students with the opportunity for an unlimited number of writing practice sessions using teacher-created or program-provided prompts.
Writing Roadmap 2.0 is a Web-based, online essay-scoring tool that provides students with the opportunity for an unlimited number of writing practice sessions using teacher-created or program-provided prompts.

4. Who has access? Provided by WVDE at no cost to counties
Available to all students
Grades 3 – 12
Available to all counties, schools, teachers
The West Virginia Department of Education is providing this program at no cost to counties or schools. It is available to all students in Grades 3-12 and is available to all counties, schools, and teachers.

5. Why use it? To improve student writing
To emphasize writing on all grade levels
To prepare for WESTEST 2 Online Writing
Grades 3 – 11
RLA Standard 2 : Writing (30 pts.)
Online Writing scores to be included in RLA score on WESTEST 2 and will count in AYP
To practice for national assessments
2011 NAEP (computer-based for grades 8 and 12)
As we know, good writing skills are crucial to the demands of the 21st century global community. Writing Roadmap 2.0 is a valuable instructional tool that can help students improve their writing skills. Many companies today require prospective employees to submit writing samples when they apply, and companies are spending billions of dollars in providing remedial training to help their existing employees – include many CEOs – improve their writing skills. Remediation also is widespread on college campuses.
Previously, the annual Writing Assessment – now called WESTEST 2 Online Writing – was administered only in grades 4, 7 and 10. That changed this past year, and now all students in grades 3 through 11 complete Online Writing. This year the WESTEST 2 Online Writing score will be included in the Reading/Language Arts score and will count in AYP. The Online Writing score will account for 30 points of the total RLA scores. Added to that the multiple choice questions on WESTEST 2 that also assess objectives under the writing standard and you can see that writing has gained even more significance than ever before.
With this new emphasis on writing at all grade levels, the use of Writing Roadmap 2.0 to help students develop their writing skills and to practice for Online Writing is even more important. Additionally, the 2011 NAEP assessment for writing will be computer-based for grades 8 and 12, and NAEP is hoping to have the writing assessment for fourth graders on computer by 2019.

6. Are there changes coming soon to Writing Roadmap 2.0?
CTB/McGraw-Hill and Bookette Software, the state’s vendors for Writing Roadmap 2.0, constantly look for ways to improve the product and make it work well for West Virginia schools and teachers. Therefore, many people want to know: Are there changes coming soon to Writing Roadmap 2.0?

7. YES!
And the answer is a resounding YES!

8. WV Scoring Engine
A West Virginia Scoring Engine will be created to score the following:
All essays written to WRM 2 shelf prompts
All essays written to WRM 2 practice prompts
All essays written to teacher-created prompts
CTB and Bookette are in the process of customizing Writing Roadmap 2.0 to create a West Virginia Scoring Engine. Once created and trained, this scoring engine will score all essays written to 34 of the existing Writing Roadmap 2.0 shelf prompts as well as essays written to the prompts created by teachers.

9. WV Scoring Engine Scoring based on WV Rubrics Six-point scoring scale
Rubrics grades 3-11
Six-point scoring scale
Five analytic traits
Organization
Development
Sentence Structure
Word Choice/Grammar Usage
Mechanics
This new West Virginia Scoring Engine will base scores on the West Virginia Writing Rubrics for grades 3-11 instead of the more general rubric now used in Writing Roadmap 2.0. The scoring will be based on the six-point scoring scale and the state’s five analytic traits of Organization, Development, Sentence Structure, Word Choice/Grammar Usage and Mechanics.

10. WESTEST 2 Online Writing Practice Prompts!
Thirty-four of the passages/prompts remaining from the WESTEST 2 Online Writing Field Test will be available on WRM 2 for student practice
One per grade level/genre
WV hand-scored student essays from the field test will be used to train the engine
Student essays will be scored with the WV five-trait, six-point rubric
Additionally, thirty-four of the passages/prompts remaining from the WESTEST 2 Online Writing Field Test will be built into Writing Roadmap 2.0 for students to practice their writing. There will be one passage/prompt per grade level, per genre. West Virginia hand-scored student essays from the field test will be used to train the scoring engine, and the student essays will be scored with the state’s five-trait, six-point rubric.
This feature means that teachers will be able to assign customized passages and prompts that will give students a better idea of what students will encounter when they take WESTEST 2 Online Writing. Because the scoring engine is being trained using sample hand-scored student essays from the field test, the scoring will be more accurate, reliable and valid.

11. How do I access my school’s reports?
Web site address:
Site code
Username
Password
Writing Roadmap 2.0 has a number of reports available for county and school administrators, as well as teachers, that can be useful in identifying strengths and weaknesses of schools, groups, or individual students. To access these reports, users will need to go to the Writing Roadmap 2.0 Web sit at and then log in using their site codes, usernames and passwords. This information is provided for each of you on the label in your Writing Roadmap 2.0 folder.
Although there are a number of different reports available in Writing Roadmap, we are going to discuss a couple of reports that might be the most useful to you. In your folders you will find samples of these reports. We also are providing you with a handout that explains how you can access these reports.

12. County Name
School Name
The first report is called the Proficiency Summary. This can be obtained for the entire school, an individual teacher, or a specific group. The one shown here is for a school. If your school used Writing Roadmap 2.0 this past school year, you will find a report for your school in your folder; if your school did not use Writing Roadmap 2.0 this past year, you will find a sample report to show you what it looks like.
The number of assignments shown reflects the total number of assignments that have been completed at your school. When looking at this data, keep in mind that you might have some teachers that use the program more than others; therefore, the report only reflects the students who have completed assignments even if most of those students only came from one or two classes.
The proficiency levels that are shown are the proficiency levels pre-set in Writing Roadmap 2.0 and do not necessarily match the proficiency levels for WESTEST 2. The intent of this report is to give you a visual picture of where those students who have completed assignments fall. One possible use for this report is to print it during the early usage of the program and then to print another summary at a later time to see how the students are progressing over time.

13. School Name
Teacher Name and Group
This report is called the Writing Summary Report, and it shows a list of the teachers who are using Writing Roadmap 2.0 and how their classes are scoring in the dimensions of Ideas and Content, Organization, Voice, Word Choice, Fluency, and Conventions. This report allows you to see which groups are performing at a high level and which groups still need work.

14. School Name
School Name
This report shows the Writing Gain by School. The intent of the Writing Gain Report is to show how many points students overall gained (or dropped) from one assignment to a second assignment. This report works best if used in a pre-test/post-test manner. Because this report includes data from all students who took these two tests, it would be best to have all the students in the school to complete both tests to give the most accurate gains or losses for the school as a whole.

15. Teacher Name
Group Name
The Writing Gain Report also can be obtained for each specific group or class. This report allows you and your teachers to see how students in a particular class score from one assignment to a second assignment. Again, this works best in a pre-test/post-test situation and with all students in the group or class responding to the same two essays.

16. Student Name
Teacher Name
Group Name
The Writing Gain Report also can be obtained for individual students to record and track their growth in a pre-test/post-test situation. As you can see from this report, the student showed gains from the first assignment to a second assignment. This report is particularly beneficial to individual teachers, students and parents.

17. West Virginia Department of Education Dr. Vaughn G. Rhudy, Coordinator
Support/Contact Information
West Virginia Department of Education
Office of Assessment, Accountability & Research
Dr. Vaughn G. Rhudy, Coordinator
If you have any questions regarding Writing Roadmap 2.0 or WESTEST 2 Online Writing during the school year, please feel free to contact Vaughn Rhudy, WESTEST 2 Online Writing and Writing Roadmap 2.0 Coordinator.

18. KEY WEB ADDRESSES http://wvde.state.wv.us/
West Virginia Department of Education
Office of Assessment, Accountability & Research
CTB/McGraw-Hill
Writing Roadmap 2.0

19. “Understanding the Elements of 21st Century Assessment”
Using Acuity Reports
Principals Academy
July 23, 2009
Acuity is a web-based platform that provides schools with assessment, feedback, scoring and reporting. This program is offered free to all WV public schools and is designed for students in grades In our presentation today we will show how several of the Acuity reports can be used to effectively enhance instruction and improve student learning.

20. Using Acuity Reports Provide data regarding a particular assessment
Provide snapshot of Standards/CSOs covered by an assessment
Provide opportunity to assign Instructional Resources
Review student progress
These reports provide data on a particular assessment; they provide a snapshot of how students are performing on the Standards and on specific Objectives; from these reports Instructional Resources can be assigned to an entire class or to individual students; student progress can be viewed.

21. Acuity Reports School Summary Report School Assessment Report
(WVDE Benchmark 3)
School Item Analysis Report
Student Progress Report
In your folder you have the following reports that contain actual data from your school or WVDE Demo Data reports. The reports are the School Summary Report, School Assessment Report (WVDE Benchmark 3), Item Analysis Report (WVDE Benchmark 3), and a sample Student Progress Report.
Our office provided three benchmarks in RLA and Math during the last school year and several science benchmarks. In the upcoming school year, three benchmarks will be provided for RLA, Math, Science and Social Studies. The Science and Social Studies will be available when the window calls for that specific benchmark.

22. SCHOOL SUMMARY REPORT
The first report is the School Summary Report which shows all of the custom tests given in your school this past school year for Math. The WVDE Benchmark Assessments are considered “custom tests.” This same report could be available for Reading/Language Arts and selected science assessments. The first column shows the test name, grade level, content area and the date the assessment began and date completed. The second column shows the school’s average percent obtained on that particular benchmark. The third column shows the number of students who completed the benchmark and the number of students assigned that specific benchmark.
This is a nice snapshot of how your school performed on all the benchmarks in a specific content area. You may notice improvement from one benchmark to another or a significant drop in performance.
Let’s take a couple minutes so you can review your data.
Are there any questions about this report?

23. School Assessment Report
This second report, the School Assessment Report, shows the school’s performance compared to that of the district on the same assessment. Here you are seeing Benchmark 3 Math, Grade 6 and this is also what you are seeing in your folder if you have WVDE Demo Data. (With live data, elementary schools should have Grade 3, Benchmark 3 reports; middle schools, Grade 7; and high schools, Grade 10 if data was available for Benchmark 3).
Notice the dates test completed; the number of students who were assigned the test, and the number who completed. The Summary section shows the Average Percent of Points Obtained on that particular Benchmark and how the school performed compared to the District. The graph shows the distribution in “Tiers.” The definition of “tiers” in Acuity differs from that of RTI. Tier 1 in Acuity reflects the lowest performance; Tier 4 the highest. We are working with CTB to change this terminology
If you look to the bottom of that report you see the percentage breakdown of the Tiers: Tier 1 (0-25); Tier 2 (26-50); Tier 3 (51-75); Tier 4 (76-100).

24. School Assessment Report
Continued
The School Assessment Report is a multi-page report showing a school’s performance by Standard and by Objective on the specific Benchmark, showing the percentage of student performance by tiers. This shows WVDE Benchmark 3, Math, Grade 6 results. I believe you are only seeing the graph and one page of the CSOs. The actual report would show all math standards and objectives.
This report is interactive. By clicking on the percent, the names of the students in that particular range will display.

25. School Item Analysis Report
The third report is the School Item Analysis Report which is also available for each Benchmark or Custom Assessment. The greatest value in this report is the access to specific items and the answer choices. Notice the responses A, B, C, D, and the % of students that selected that answer is listed. The correct response is high lighted in green. For example, you may notice that for Item # 5 only 11% chose the correct answer and 62% chose Answer C. When you click on the item number, you see the item and the answer choices. Quite often, it is evident that students have the wrong concept. Also, by clicking on any of the percents, you can see the students who chose that answer.

26. What is 528,000,000 written in scientific notation?
Item 5 – M.O – Students will demonstrate an understanding of large numbers by converting and comparing numbers in scientific notation and standard notation (with and without technology).
What is 528,000,000 written in scientific notation?
A × 108 C × 106
B × D × 106
This is item 5 from the assessment. Clicking on the 5 in the report will produce a screen similar to this one. The item and answer choices are available for viewing as well as the depth of knowledge of the question, the p-value for the item and the answer choice distribution.
This item is a DOK level Recall of a simple fact or procedure. The item has a p-value of This means that about 11% of the students in the school answered this item correctly. Typically, we would expect the p-value of a DOK 1 item to be higher.
Similar items that addressed this CSO were also included in Benchmarks 1 and 2. The results were the same. As an administrator or teacher, we could assume that this concept was not addressed in the classroom yet. Since benchmark 3 was administered near the end of the year, it may be worth investigating if the concept was indeed taught or not. (Is it included in the textbook? Are the teachers comfortable with the topic? Etc.)
It is also possible that the concept was not taught correctly. Notice the number of students who selected answer choice “C”. Students simply counted the number of zeros and made that a power of 10. Scientific notation has two components – move the decimal so the first portion of the number is between 1 and 10, then make the number of decimal places moved a power of 10. In sixth grade, students are only required to convert large numbers to scientific notation. (Numbers less than 1 have negative exponents)
Principals may also want to examine this question by classroom reports to see if there is a difference.
Answer Choice A is the correct answer; Answer Choice B did not place the decimal in the correct position, Answer Choice C did not place the decimal in the correct position and made the number of zeros the exponent, Answer choice D has the decimal in the correct position but makes the exponent the number of zeros and not the number of decimal places moved.
The with and without technology portion – many calculators will convert numbers from standard to scientific notation and vice versa. However, if the students do not know how to input or read the numbers correctly using the calculator, their answers will also be incorrect.
Good news – since this concept needs further clarification and/or instruction – teachers can assign the instructional resource that aligns to this CSO to give the student additional practice and exposure. The instructional resource specifically addresses the common misconceptions made by students.

27. Principals may also want to examine this question by classroom reports to see if there is a difference.
Answer Choice A is the correct answer; Answer Choice B did not place the decimal in the correct position, Answer Choice C did not place the decimal in the correct position and made the number of zeros the exponent, Answer choice D has the decimal in the correct position but makes the exponent the number of zeros and not the number of decimal places moved.
The with and without technology portion – many calculators will convert numbers from standard to scientific notation and vice versa. However, if the students do not know how to input or read the numbers correctly using the calculator, their answers will also be incorrect.
Good news – since this concept needs further clarification and/or instruction – teachers can assign the instructional resource that aligns to this CSO to give the student additional practice and exposure. The instructional resource specifically addresses the common misconceptions made by students.

28. Correct Response:
converts a number into scientific
notation
Incorrect Response:
does not move the decimal all
the way over to make it a value between 1 and 10
only counts the number of zeros
not the number of spaces the
decimal moved

29. Review your data and complete a Team SMART Goal

30. Student Progress Report
The last report in your folder is a sample Student Progress Report. This particular report shows how the student performed on all three benchmarks (Test 1, 2 & 3) in Math. This was an actual student report and the school had used Acuity consistently during the year. Thankfully, this student showed improvement from Test 1 to Test 3. The information is graphed and also listed below the graph. This report can be created for any tests – benchmarks or school/teacher created assessments.

31. The second page of the Student Progress Report shows how the student performed on each standard in Math on Benchmark 1, 2, and 3. (Note – Algebra Standard – improvement (75% to 100%); however, in Number and Operations, notice the decline from 93% on Benchmark # 1 to 67% on Benchmark # 2, but a slight increase to 75% on Benchmark # 3. Looking at the items, of course, would be helpful but also noting that possibly this Standard: Number and Operations may not have been stressed as much at the end of the year as early in the year.

32. Review your data and complete a Team SMART Goal

33. Using Acuity Instructional Resources
Mini-lessons which include three sessions
Instruction
Guided practice
Mini-quiz
Easily assigned from reports or automatically from tests
ACUITY INSTRUCTIONAL RESOURCES -- one of the best features of acuity. The Instructional Resources are available to help prepare students for classroom, benchmark and summative assessments as well as to assist in instructing those students who need additional support for certain skill sets with the CSOs. These step-by-step resources guide students through a mini-lesson targeted to a variety of skill sets.
Mini-lessons which include three sections
Instruction
Guided practice
Mini-quiz
These can be easily assigned from reports or automatically from tests
In conjunction with the benchmark tests, these instructional resources can be assigned based on a student/class/district score or they can be assigned by CSO. It makes the activity “customized” to the student and/or the lesson plans of the teacher. It does not require the teacher to search through endless activities on other software programs to find the activities needed.
Screens that introduce and teach a given skill
Guided Practice
Mini-summative assessment to evaluate progress

34. WV Acuity Benchmarks Suggested Windows (2009 – 2010)
WVDE Benchmark # 1
October 19 – November 6
WVDE Benchmark # 2
December 14 – January 15
WVDE Benchmark # 3
March 1 – March 31
All three WVDE Benchmarks in RLA and Math (grades 3-11) will be available by August 10th. Science – Benchmark # 1 (grades 3-8 and physical science) Science – Benchmark # 3 (grades 3-8 and physical science) Science – Benchmark # 2 (will be available by Dec.). Social Studies -- Benchmark # 1 – available by Sept. ; Benchmarks 2 & 3 – available by the suggested window dates.
SUGGESTED TESTING WINDOWS:
WVDE Benchmark 1 – Oct. 19 – Nov. 6
WVDE Benchmark 2 – Dec. 14 – Jan. 15
WVDE Benchmark 3 – Mar. 1 – Mar. 31

35. Acuity Contact Information
Office of Assessment/Accountability
Sandra Foster, Lead Coordinator
Allegra Kazemzadeh, RLA/Social Studies Coordinator
John Miller, Technology Coordinator
Sonya White, Math/Science Coordinator
Please feel free to contact us regarding questions or concerns about Acuity.

36. Key Web Addresses http://wvde.state.wv.us/
West Virginia Department of Education
Office of Assessment, Accountability & Research
Acuity Web site
Writing Roadmap 2.0

37. THANK YOU!