1. STAR Basics

2. I’m thinking of a number between 1 and 100.
Guess what it is.

3. The Science of STAR Computer Adaptive
Difficult .3 .9 .6
RELIABILITY
Reliability of Student Score
10 min.
15 min.
**Click through three times and the 4th click will continue the next 22 items by itself.
Using computer adaptive technology we were able to make STAR extremely fast without sacrificing reliability. The test automatically adapts to the student’s ability level by administering a more difficult item after every correct response and an easier item after every incorrect response. The purpose of the test is to determine the student’s ability level on our vertical scale. With every additional item that’s administered, we become more certain about the student’s ability level and measurement error, shown here, narrows. The original STAR assessments administer 24 or 25 items and the new Enterprise tests administer 34.
Let’s now look at how test reliability increases as more items are administered. The vertical scale now represents reliability and the blue reliability curve shows how reliability goes up, but you will notice how it begins to level off after about 20 items. As we get much beyond that the gain in reliability becomes less and less significant. We found the optimal balance between reliability and test efficiency resulting in a reliable test that can be administered in as little as 10 to 15 minutes. There are other computer adaptive tests on the market but not with this balance of reliability and efficiency and the difference is pretty significant – some take 45 minutes or longer.
Easy
Items answered by student
Correct Response
Incorrect Response
Standard Error of Measurement

4. STAR Reading tests comprehension
Background
knowledge
Background
knowledge
Vocabulary
knowledge
Construct
meaning
from text
Comprehension
The items used in STAR Reading are designed to test students’ reading comprehension skills. The assessment items require students to call upon background information combined with vocabulary knowledge in order to construct meaning from text. These cognitive tasks are consistent with what researchers and educational experts describe as reading comprehension. In other words, background knowledge and vocabulary intersect to create meaning from text.
STAR Reading assesses comprehension directly with robust, holistic questions that tell you a lot about how all the reading skills relate to each other and work in concert.

5. STAR Early Literacy Measures proficiency in early literacy skills
General
Readiness
Phonemic
Awareness
Graphophonemic
Awareness
Phonics
Vocabulary
Comprehension
Structural
Analysis
STAR Early Literacy assesses seven literacy domains and 41 skills.
STAR Early Literacy can be used for many purposes. The two purposes related to AR are (1) to see if a student is an emergent, transitional, or probable reader and (2) to measure growth over time. Another primary purpose for STAR Early Literacy is to help teachers identify students’ strengths and weaknesses with specific skills so that they can plan instruction and intervention. STAR Early Literacy can also be used for screening and progress monitoring.
Teacher’s Guide pages 19-20
and pages 48-50

6. Choosing an assessment
Measures proficiency in early literacy skills
Measures reading comprehension
Administer STAR Early Literacy when a student:
Does not yet read Independently
Needs early literacy skills assessed
[For customers who also own STAR Reading.]
Many primary classrooms have students with varied reading abilities. STAR Early Literacy is used to measure proficiency with early literacy skills for students. Once students have mastered many of those skills, they will be classified as a Probable Reader on STAR Early Literacy. At this point, other assessment tools, such as STAR Reading, can be administered to measure reading comprehension.
In order to be able to successfully take a STAR Reading test, a student needs to have some general reading skills. Students who have a 100-word sight-word vocabulary or are classified as Probable Readers in STAR Early Literacy are generally able to complete a STAR Reading test. STAR Reading will measure the student’s comprehension and provide additional scores that STAR Early Literacy does not report, such as grade equivalent, instructional reading levels, and zone of proximal development.
Administer STAR Reading when a student:
Has a 100-word sight vocabulary
Has been classified as Probable Reader
on STAR Early Literacy

7. STAR Math Measures Math Competency

8. New Skills Items Enhanced Item Bank Extended Item Bank
FIVE DOMAINS
(Standards Based)
SEVEN DOMAINS
(Standards Based)
FOUR DOMAINS
(Standards Based)
Word Knowledge &
Skills Development
Comprehension
Strategies and
Constructing Meaning
Literary Analysis
Understanding
Author’s Craft
Analyzing Argument
and Evaluating Text
36 General Skills
Graphophonemic
Knowledge
General Readiness
Phonemic Awareness
Phonics
Comprehension
Structural Analysis
Vocabulary
41 Skill Sets
Numbers and Operations
Algebra
Geometry and Measurement
Data Analysis, Statistics and Probability
54 Skill Sets
This slide reviews the content for the ENTERPRISE versions.
Note the MAJOR changes resulting in STAR Reading being able to assess 371 skills in 5 domains.
STAR EL still assesses 130+ skill sets in 7 domains and received enhancements to its item bank.
STAR M now measures 556 skills sets in 4 domains (a grouping more closely aligned with major standards agencies). Instead of 215
Sel and math getting polished, reading rewritten and aligned with state test
Don’t start charging until all three are ready for your district.
Working on this for 2.5 years, just coming out due to common core standards final edition out of 50 states adopting
New SEL will be released in oct/nov
New Skills Items
Enhanced Item Bank
Extended Item Bank

9. Extended comprehension
Item Time Limits
Grades K - 2
3 minutes
per item
60 seconds
per item
Grades 3 +
STAR READING
K-2 60 seconds per item
Grades 3+ have 45 seconds per short comprehension item
Grades 3+ have 90 seconds per extended comprehension item
STAR MATH
All items have a 3 minute time limit
STAR EARLY LITERACY
All items have a 90 second time limit
ALL ASSESSMENTS
Testing time does not affect scores and there is no overall test time limit. They are just in place to maintain test security and keep the test moving.
Warning clock appears in upper right corner during last 15 seconds
When time limit expires, test continues to the next item and counts the timed out item as incorrect unless students have selected the correct answer before time runs out.
Control A will abort the test
Click on X at top and will pause the test
If timed out, a missed question
Need paper for math
Calculators not used, norming without calculators
Don’t override IEP’s. Put calculator on top of computer, try it without it.
45 seconds
Short comprehension
90 seconds
Extended comprehension
90 seconds
per item

10. Scaled Score (SS) What is the student’s actual score? 0 SS 1400 SS
656
A student’s scaled score is the raw score the student attained based upon the difficulty of the questions the student was given and whether or not they were answered correctly. All other STAR scores are derived from the scaled score.
The scale used is like a ruler, ranging from 0 to The harder the test question, the larger the number on the scale. As students work their way through the questions, they’ll move back and forth on the scale, depending on their answer choices. When the test is finished, the program will calculate their final scaled score.
As teachers become more familiar with scaled score, it will be easier for them to determine what scores are typical for students at the grade level they teach.
This student obtained an actual score of 656.

11. STAR Reading Reports
Diagnostic Report
Summary Report
Growth Report

12. STAR Early Literacy Reports
Summary Report
Score Distribution Report
Class Diagnostic Report
Student Diagnostic Report
Growth Report

13. STAR Math Reports
Diagnostic Report
Summary Report
Growth Report

14. Comparative (norm-referenced data)
Two Types of Data
Deciding which STAR scores will be meaningful to you will depend on your purpose for using STAR and the kind of information you need. There are two categories of data, actual and comparative.
Actual data tells us about the student’s performance without comparing him or her to other students. This is a numeric measure of a student’s ability. An example of this would be the student’s scaled score which we will look at a little more closely later today.
Comparative data looks at student performance as it compares to other students. For instance, a norm-referenced score such as the Percentile Rank can tell you how a student’s abilities compare to those of other students in the same grade.
Actual (raw data)
Comparative (norm-referenced data)

15. Scaled Score (SS) What is the student’s actual score? 0 SS 1400 SS
656
A student’s scaled score is the raw score the student attained based upon the difficulty of the questions the student was given and whether or not they were answered correctly. All other STAR scores are derived from the scaled score.
The scale used is like a ruler, ranging from 0 to The harder the test question, the larger the number on the scale. As students work their way through the questions, they’ll move back and forth on the scale, depending on their answer choices. When the test is finished, the program will calculate their final scaled score.
As teachers become more familiar with scaled score, it will be easier for them to determine what scores are typical for students at the grade level they teach.
This student obtained an actual score of 656.

16. What does the scaled score mean?
500
900
1200
1400
[You may want to ask paRTIcipants to create a chart of STAR Reading scaled scores. Ask them to draw the chart shown here, and fill it in during your discussion.]
(click) Write the number 0 on the left, and 1400 on the far right. This is our scale. The harder the test question, the larger the number on the scale. As students work their way through the 25 questions, they’ll move back and forth on the scale, depending on their answer choices. When the test if finished, the program will calculate their score.
(click) When the program was normed, Elementary school students scored in the first part of the scale, so label the section from 0 to 500, as Elementary, grades 1-5.
(click) Middle school students scored in the middle of the scale, in the range, so label that section Middle School, grades 6-8.
Keep in mind that these how most students scored. So some middle school students scored lower, and some higher.
(click) High School students tend to score in the range, so go ahead and label your scale about with 1200 and label that range as High School, grades 9-12.
Elementary
grades 1-5
Middle School
grades 6-8
High School
grades 9-12

17. What does the scaled score mean?
300
400
500
600
700
800
900
Pre-K K 1
In STAR Early Literacy the SS ranges from 300 to (PaRTIcipants may want to draw this chart for STAR EL.) Most Pre-K students scored between 400 and Kindergarten students generally scored between 500 and First grade students were between 700 and 800 SS, second grade between 750 and 850, and third grade between 800 and 850.
Scaled scores in STAR Early Literacy were designed to roughly correlate to age. A scaled score of 500 might be expected of 5-year-old students.
(SEL Technical Manual pg 41) 2 3

18. What does the scaled score mean?
1400
300
500
700
900
1100
Primary
grades 1-2
Intermediate
grades 3-5
In STAR Math, the scaled scores for each level look different from those in STAR Reading. (PaRTIcipants may want to draw this chart for STAR Math.) Primary students had scaled scores between 300 and 550, while most intermediate students scored between 500 and 750. Middle school students scored between 700 and 900. High school scores are found between 750 and 900. If you want to find out more about the scaled scores students generally attain, the conversion tables found within the STAR Math technical manual in the software may help you.
Middle
grades 6-8 HS
9-12

19. Percentile Rank (PR)
How does this student compare to other students at the same grade level?
1 PR
99 PR 58
Percentile Rank (norm referenced score)
Grade level specific (compares students within a grade to others in the same grade)
Gives the best measure of the student’s ability relative to peers
Ranges from 1 to 99, given in 1 percent increments
Example a PR of 58 means a student’s achievement is greater than 58% of same-grade students
PR is useful when comparing students at the same grade level. It is also helpful when looking at student growth relative to peers.
(Note: Although STAR Early Literacy is a criterion-referenced test, not norm referenced, it was field-tested with a large national sample of students who serve as a fixed reference group for normative purposes. As a result, STAR Early Literacy can report norm-referenced scores, such as percentile ranks. These percentile ranks are now provided as reference points for screening benchmarks and cut scores. However, percentile rank scores are not reported for individual students on STAR Early Literacy reports.)
A student with 58 PR has reading skills greater than 58% of same-grade students

20. To which grade level is this student’s performance similar?
Grade Equivalent (GE)
To which grade level is this student’s performance similar?
0.0 GE
12.9 GE
7.3
4.8
5.0
5.1
5.2
5.4
5.3
5.5
5.6
4.9
GE compares a student’s results to those of other students in the same grade who took the test
Score Ranges from 0.0 to Grade equivalent scores are a norm-referenced score, but it is often misinterpreted.
The GE score reports the grade placement for which that score would be considered average. In other words, it provides an estimation of the performance that an average student, at a particular grade level and month, would likely demonstrate on this test. So, for example, a student with a GE score of 7.3 scored the way an average seventh grader in the third month of school typically performs.
Keep in mind that a GE has nothing to do with how the student performs against the local school curriculum or standards for a particular grade. Our example student with the GE score of 7.3 may not be ready for seventh-grade work.
Looking at grade equivalent scores may help you see whether or not skills have increased, but do not tell you the significance of this change. GE scores are not the best option for measuring growth or making diagnostic and placement decisions.
Instead, a teacher could consider the scaled score and percentile rank to better gauge student performance.
(Note: Grade equivalent scores are not reported in STAR Early Literacy because SEL is not norm referenced.)
This student scored the same as a typical student in the third month of seventh grade.

21. Dangers of GE Scores
It’s about a performance level on this test, NOT the content a student has been exposed to or can handle currently.

22. Instructional Reading Level (IRL)
What level of instructional materials will best serve the student? PP P 1 2 3 4 5 6 7 8 9 10 11 12
PHS
IRL describes the reading level at which students can recognize words and comprehend written instructional material with some assistance. It is a criterion-referenced score, which means it measures student performance against a curriculum standard.
Describes student performance relative to a content domain or standard
Does not compare a student’s score with other students’ scores
Highest reading level at which the student can comprehend 80% or more of the words
Range: Pre-primer (PP) to post-high school
A student with a 5.3 IRL can recognize words and comprehend materials at the fifth-grade level with some assistance.

23. Zone of Proximal Development (ZPD)
What level of materials should a student use for reading practice?
Too easy Too hard
ZPD
The levels at which the most learning can take place
(for STAR Reading Enterprise customers only)
A ZPD score can be used as a starting point for reading practice. This score is used in the Accelerated Reader program to give students a range of levels from which to choose books for independent reading practice.
ZPD, short for the Zone of Proximal Development, is based upon a student’s Grade Equivalent score and represents the level of difficulty that is neither too hard nor too easy, and is the level at which optimal learning takes place.
By allowing students to choose reading materials throughout their ZPD range, not only can teachers be assured that students are practicing reading at levels most likely to improve skills, but they can also prevent the frustration that can occur when students read books that are too difficult for them and some of the boredom that occurs when books are too easy.

24. Fidelity of Implementation

25. “Fidelity of Assessment”
Quiet environment, free of distractions
Do not read aloud or explain questions
Ideal to test all students in a computer lab
“Fidelity of implementation” is a term that frequently comes up in the field of education.
Ask attendees what the phrase means to them.
Point out that this will be an underlying theme for the next four days.
What does
that mean to you?

26. Talking to Students About STAR
Try your best on the STAR Assessment.
Take it seriously.
Your goal is to get
a higher score each
time you take it.
Know that items get harder until you miss one.
Because STAR is a short, computer based assessment unlike traditional tests students have encountered, they sometimes have a tendency to not try their hardest or treat it like a game. Stress with students that it is important that they take the STAR assessment seriously. They need to attempt to do their best and their goal is to get a higher score each time they take the test.

27. Why is it important for students to test in the same way each time they test?
To accurately
track achievement
and growth
Why is it important for students to test in the same way each time they take the test?
Discuss. Click for text.
(You need to be sure that all students are given the test in the same way every time so you can accurately track achievement and growth. This is key to fidelity of implementation/administration.)

28. Purposes for Assessment
Handout 1
Universal
Screening
Forecasting State Test Proficiency
Differentiating
Instruction
STAR Enterprise can be used for multiple purposes, and you don’t need to determine just one. STAR Enterprise reports provide different ways to display data to help you determine a need for instruction or support, make instructional decisions or document results.
(click) STAR Enterprise can be used within an RTI framework for both universal screening and progress monitoring of students in an intervention.
(click) STAR Enterprise has been linked to state tests so you’re able to use STAR Enterprise early in the year to get a prediction about which students will met proficiency requirements on the state test.
(click) STAR Enterprise is aligned with state standards and common core state standards, which allows you to see how student performance compares to those standards.
(click) The Instructional Planning Report lists skills students are likely ready to learn next. By using the Learning Progressions, you’re able to find out more about the skills listed and see how those skills relate to each other. If you use Accelerated Reader, you’re also able to use a student’s ZPD for the suggested range of independent reading materials for the student.
(click) STAR Enterprise provides multiple ways to track and measure growth within a single school year or over multiple school years.
Mastery of State Standards and Common Core State Standards
Progress
Monitoring
Measuring Growth

29. Using STAR in a RTI Framework
Introduce yourself.
Discuss expectations of T of T and why it was created.
Over the course of the four-day session, attendees will explore STAR assessments and Accelerated Reader. The objectives are to learn best practices, understand the research behind them, and become comfortable with the software.
Attendees will be provided with materials and ideas that will help them deliver effective professional development to their staff.
There will be time to ask questions and discuss the unique needs of their school or district.

30. How does universal screening work?
Screening is the first step to using a data-driven decision making process. It helps you evaluate student achievement in order to plan for intervention and prioritize student need.
All students are given the STAR test to evaluate their achievement.
Then, look at the results in order to make instructional decisions. Students are grouped depending on their level of need. This helps you intervene early with the students most in need of assistance.
Performing Above
Benchmark
In Need of
Intervention
On Watch for
Intervention

31. Categories on the Screening Report
At or Above Benchmark
40 PR
On Watch
The STAR Screening Report displays all the students in rank order, lining them up from the highest performing student on the left to the lowest performing student on the right. All the students scoring above benchmark are shown in green. The students are categorized by color to make it easier to see how many students fall into each category to further assist in your planning and prioritization.
Intervention
Urgent Intervention

32. Screening Report: Student Details
The next pages of the STAR Screening Report show details about each student tested. You can select which groups, if any, you would like to see the details for when setting up the printing options for this report.
The scores shown on the detail pages vary depending on which STAR assessment was given.
For STAR Math Enterprise, the SS, PR, GE, and Recommended Accelerated Math Library are shown.
For STAR Reading Enterprise, the SS, PR, GE, Est. ORF and ZPD for use with Accelerated Reader are shown.
For STAR Early Literacy Enterprise, the SS, Est. ORF and Literacy Skills Classification are shown. (The SEL Screening report does not show the GE or PR because it is not norm referenced)
Next we’re going to talk about what those scores mean and how to use them to understand student performance.

33. A benchmark represents the minimum acceptable level of student achievement
District Benchmarks
School Benchmarks
State Benchmarks
Defaults are set based on expert recommendations
Can be set by administrators
Defaults are set based on expert recommendations
Can be set by administrators
Determined by linking study
Non-linked grades can be edited by administrators
Knowing the benchmark you want students to attain gives you a goal, or purpose. It sets a standard to ensure all students are meeting a minimum level of competency.
This is the very lowest score deemed acceptable for students to attain.
With STAR Enterprise you have the option of choosing from three benchmarks: school, district or state.
School and district benchmarks are set to default levels according to the recommendations of educational experts.
Administrators can edit these defaults, changing where the benchmark is set and the way scores are categorized.
State benchmarks are determined based on a linking study completed for each state that matches state performance levels to those in STAR Enterprise. State benchmarks are already available for some states, and additional states are continually being added. (See R44621 for availability.)

34. Screening Report: District Benchmark
950
900
850
800
750
700
650
600
550
At or Above Benchmark
40 PR
(District
Benchmark)
On Watch
Intervention
The STAR Screening Report graph displays all the students in rank order, lining them up from the highest performing student on the left to the lowest performing student on the right.
(Click) All the students scoring above benchmark would appear above the benchmark line, shown here at the district default cut score of 40 PR.
(click) Students with scores below the benchmark appear below the line.
(click) The students are further categorized based upon cut scores. The district default cut scores are 10 PR for the Urgent Intervention category, 25 PR for the Intervention category and On Watch students falling between 26 PR and 40 PR. With each category given a different color, it makes it easier to see how many students fall into each category to further assist in your planning and prioritization.
This graph shows students categorized according to district benchmarks. Changing to school or state benchmarks will change the way the graph looks.
Below Benchmark
Urgent Intervention
Students

35. Screening Report: State Benchmark
950
900
850
800
750
700
650
600
550
Exemplary
Exceeds Standards
Meets Standards
Approaches Standards
Here we have the same group of students and the same test event, only now we’ve applied the state benchmarks instead of district. You’ll notice that the shape of the graph is the same but the colors and benchmarks have changed. For this particular state, there are five proficiency categories instead of four. Three of those categories are above the proficiency benchmark and two are below.
Academic Warning
Students

36. Screening Report Comparison
Looking at these two graphs side-by-side, you can see that the shape of the graph is the same, but there are some differences. Note that the District Benchmarks has a higher proficiency level (where the black line is drawn). When you choose to run a Screening Report, you can choose which type of benchmarks to use. You may want to run the report using each type to better inform the decision-making process.
District Benchmark
State Benchmark

37. Steps for Progress Monitoring Goal Setting
Start and name an intervention. Set the goal end date. Select a start date (if necessary) Review the reference points. Choose a goal type and calculate goal. Choose a custom goal.

38. Steps for Progress Monitoring Goal Setting continued…
Editing a goal.
Change Duration or Goal
Starting a new intervention

39. On-the-spot goal setting guidance
By clicking on the Interpretations and Recommendations link in the Goal Setting Wizard, you can access a document that will walk you through the goal setting process and explain each step in more detail.

40. Goal star and Goal line Goal line: Desired growth
After setting up an intervention and goal in the software, the first page of the Student Progress Monitoring report will look something like this. The blue diamond represents the starting STAR test the student took. The vertical line shown here in red represents that a new intervention was started with the student. The software has calculated the student’s goal based on that test score.
Once you have set a STAR goal for a student, a gold star and green goal line appear on the Student Progress Monitoring report. The star shows the target the student is striving to reach. The green goal line connects the student’s current score to the goal. This provides a visual representation of what rate of growth the student will need to achieve in order to meet the goal. You’ll compare actual student growth to this ideal rate of growth to determine if the student is making progress.

41. Four scores make a trend line
Actual growth
The actual rate of growth is depicted by a trend line. It takes at least four STAR test scores for a trend line to appear on the Student Progress Monitoring report.
With each test score after the fifth the trend line will more accurately depict student growth.
The rate of growth shown by the trend line can be tracked for duration of the intervention so you can tell to what degree a student is experiencing growth.
STAR Reading can be given up to once per week to students for the purpose of progress monitoring. (Explain to customers who may have used STAR in the past that this is due to changes in the software that would enable a student to test more often without seeing an item for at least 60 days and increasing the number of questions in our database.)

42. Looking at trends over time
A trend line marginalizes score fluctuations and shows patterns of achievement
Individual student scores on the Student Progress Monitoring report do not make consistent, steady increases, even when the trend line shows growth is being made. The scores tend to fluctuate slightly with each successive test that is taken. With frequent testing, this often is the case, and every standardized test, if given multiple times within a short period of time will show fluctuation in scores.
A trend line marginalizes score fluctuations and allows you to see patterns of achievement.

43. Introduce yourself.
Discuss expectations of T of T and why it was created.
Over the course of the four-day session, attendees will explore STAR assessments and Accelerated Reader. The objectives are to learn best practices, understand the research behind them, and become comfortable with the software.
Attendees will be provided with materials and ideas that will help them deliver effective professional development to their staff.
There will be time to ask questions and discuss the unique needs of their school or district.

44. Access resources Getting the Most out of STAR Guides
Software: manuals, live chat
Phone: (800)
Renaissance Training Center:
Getting the Most out of STAR Guides for STAR Reading, STAR Math and STAR Early Literacy
Help in the software (Software and technical manuals, live chat)
Technical support by phone, , knowledge base
Renaissance Training Center: On-Demand Sessions
On-demand sessions
Implementation tips
Getting the Most out
of STAR Guides
Much more!