1. Welcome to the ELA Smarter Balanced Item and Task Types
November 13, 2014
West Virginia Department of Education
Office of Assessment and Research
While you wait for the webinar to begin, please be sure to turn on your speakers.
You can try this link if your speakers aren’t working.

2. ELA Smarter Balanced Item and Task Types Welcome
Stacey Murrell, Ed. D.
Interim Assessments Coordinator
Office of Assessment and Research

3. Balanced Assessment System

4. Smarter Balanced ELA Claims
ELA: Targets by Grade Level for each Claim

5. ELA Claims and Targets Claim 1 has Targets 1-14.
Targets were created by Smarter Balanced. When you read them, you can see that content standards are embedded in them. So a test question can assess multiple content standards within one target.
Claim 1 has Targets 1-14.
Claim 2 has Targets 1-9.
Claim 3 has Targets 1-4.
Claim 4 has Targets 1-4.

6. Claim 1 and Its Targets
Students can read closely and analytically to comprehend a range of increasingly complex literary and informational texts.
Targets 1–7 correspond with literary texts
Targets 8–14 correspond with informational texts
The assessment targets incorporate the content clusters from the Common Core State Standards
For Claim 1, there are fourteen assessment targets.
{+}
Assessment targets 1-7 focus on reading literary texts.
Assessment targets 8-14 focus on reading informational texts.
Assessment targets are linked to the content clusters within the Common Core State Standards. Now let’s look at a specific target for Claim 1.

7. Grade 5 Claim 1 Target 4
4. REASONING & EVIDENCE: : Make an inference or provide a conclusion and use supporting evidence to justify/explain inferences (character development/actions/traits; first- or third-person point of view; theme; author’s message).
Standards: RL-3, RL-3, RL-6
(DOK 3, DOK 4)
RL-3 Compare and contrast two or more characters, settings, or events in a story or drama, drawing on specific details in the text (e.g., how characters interact).
 RL-6 Describe how a narrator’s or speaker’s point of view influences how events are described.
 RL-9 Compare and contrast stories in the same genre (e.g., mysteries and adventure stories) on their approaches to similar themes and topics.
This grade 5 target focuses on reading literary text.
{+}
Notice that several standards are associated with this target. There is not a one-to-one correspondence between a given assessment target and a standard. Instead, assessment targets focus on skills and knowledge that cross a cluster of standards.
Also note that the depth of knowledge level that must be applied to achieve this assessment target is also specified.

8. Claim 2 and Its Targets
Students can produce effective and well grounded writing for a range of purpose and audiences.
Targets 1a, 3a, & 6a: Write Brief Texts
Targets 1b, 3b, & 6b: Revise Brief Texts
Targets 2, 4, & 7: Compose Full Texts
Target 5: Use Text Features
Target 8: Language & Vocabulary Use
Target 9: Edit
For Claim 2, there are nine assessment targets.
{+}
Targets 1, 3 and 6 focus on writing and revising brief texts.
Targets 2, 4, and 7 address composing full texts.
Target 5 focuses on the use of text features in writing.
Target 8 focuses on the use of language and vocabulary.
Target 9 focuses on editing text. Now let’s look at a specific target for Claim 2.

9. Grade 7 Claim 2 Target 1
1a. WRITE BRIEF TEXTS: Apply narrative techniques (e.g., dialogue, description) and appropriate text structures and transitional strategies for coherence when writing one or more paragraphs of narrative text (e.g., closure, introduce narrator or use dialogue when describing an event). 1b. REVISE BRIEF TEXTS: Apply narrative techniques (e.g., dialogue, description) and appropriate text structures and transitional strategies for coherence when revising one or more paragraphs of narrative text (e.g., closure, introduce narrator, or use dialogue when describing an event). Standards: W-3a, W-3b, W-3c, W-3d, and/or W-3e (DOK 3) W-3 a. Engage and orient the reader by establishing a context and point of view and introducing a narrator and/or characters; organize an event sequence that unfolds naturally and logically. b. Use narrative techniques, such as dialogue, pacing, and description to develop experiences, events, and/or characters. c. Use a variety of transition words, phrases, and clauses to convey sequence and signal shifts from one time frame or setting to another. d. Use precise words and phrases, relevant descriptive details, and sensory language to capture the action and convey experiences and events. e. Provide a conclusion that follows from and reflects on the narrated experiences or events.
This example shows Assessment Target 1 for Claim 2.
{+}
Target one focuses on writing and revising texts.
The several standards that are associated with this assessment target are listed.
This target also addresses Level 2 Depth of Knowledge.

10. Claim 3 and Its Targets
Students can employ effective speaking and listening skills for a range of purposes and audiences.
Language & Vocabulary Use
Clarify Message
3. Plan/Speak/Present
4. Listen/Interpret
For Claim 3, there are three assessment targets.
{+}
Target 1 focuses on use of language and vocabulary.
Target 2 focuses on adapting speech to a variety of contexts and tasks.
Target 3 addresses students’ ability to plan, speak and present information in an oral manner.
Target 4 focuses on listening and interpreting spoken information.
Now let’s look at a specific target for Claim 3.

11. Grade 11 Claim 3 Target 3
4. LISTEN/INTERPRET: Analyze, interpret, and use information delivered orally. Standards: SL-1, SL-2, SL-3 (DOK 1 DOK 3) SL-1 Initiate and participate effectively in a range of collaborative discussions (one-on-one in groups and teacher-led) with diverse partners on grades 11–12 topics texts and issues building on others’ ideas and expressing their own clearly and persuasively. SL-2 2 Integrate multiple sources of information presented in diverse formats and media in order to make informed decisions and solve problems, evaluating the credibility and accuracy of each source and noting any discrepancies among the data. SL-3 Evaluate a speaker’s point of view, reasoning, and use of evidence and rhetoric, assessing the stance, premises, links among ideas, word choice, points of emphasis, and tone used.
As seen earlier for Claims 1 and 2, assessment targets for Claim 3 address several standards, each of which are listed below the full description of the assessment target.

12. Claim 4 and Its Targets
Students can engage in research / inquiry to investigate topics, and to analyze, integrate, and present information.
1. Plan/Research
2. Interpret & Integrate Information
3. Analyze Information/Sources
4. Use Evidence
For Claim 4, there are four assessment targets.
{+}
Target 1 focuses on planning and conducting research.
Target 2 addresses student’s ability to interpret and integrate information.
Target 3 focuses on analyzing information and the sources of that information.
Target 4 focuses on the use of evidence to support an argument or position.
Now let’s look at a specific target for Claim 4.

13. Grade 4 Claim 4 Target 3
3. ANALYZE INFORMATION/SOURCES: Distinguish relevant-irrelevant information
Standards: RI-1, RI-7, RI-9, W-8, W-9 (PT: DOK 4) (CAT: DOK 2)
RI-1 Refer to details and examples in a text when explaining what the text says explicitly and when drawing inferences from the text.
 RI-7 Interpret information presented visually, orally, or quantitatively (e.g., in charts, graphs, diagrams, time lines, animations, or interactive elements on Web pages) and explain how the information contributes to an understanding of the text in which it appears. 
RI-9 Integrate information from two texts on the same topic in order to write or speak about the subject knowledgeably. 
W-8 Recall relevant information from experiences or gather relevant information from print and digital sources; take notes and categorize information, and provide a list of sources. 
W-9 Draw evidence from literary or informational texts to support analysis, reflection, and research.
This is a grade 4 assessment target for Claim 4. This target addresses two standards and requires a Level 2 Depth of Knowledge.
With these examples of claims and assessment targets, let’s now take a closer look at the evidence statements that are specified for assessment targets.

14. ELA/Literacy General Item and Task Specification

15. Smarter Balanced- ELA Response Types
Multiple-choice, single correct response
Multiple-choice, multiple correct response
Two part multiple choice
Matching Tables
Select Text
Order Text
Short text- constructed response
Long essay- performance tasks

16. Ways to Score Automatic with a single key Automatic with multiple keys
Multiple-choice, single correct response
Multiple-choice, multiple correct response
Two part multiple choice
Matching Tables
Select Text
Order Text
Short text- constructed response
Long essay- performance tasks
Automatic with a single key
Automatic with multiple keys
Automatic with machine rubric
Hand score with rubric

17. Considerations Accessibility Rubrics
Ensure use of most accessible response types to measure standards
Include careful review of graphic and language use in items
Rubrics
Include specific scoring rules to account for all correct responses
Carefully constructed for use by automated scoring
There are some important considerations in using new response types. Items must still be accessible to students with a wide range of abilities. Some response types may be difficult for students with visual impairments or other disabilities. Response types should be carefully selected based on the measurement construct; more accessible item types should be used wherever possible. Item specifications will provide guidance on the use of response types and specific accessibility considerations.
Item authors also need to provide carefully constructed rubrics for automated scoring. Specific scoring rules need to be created to ensure that students receive appropriate credit.

18. Development Guidelines: SR, CR, and PT
Selected response and constructed response primarily focus on one assessment target
Performance tasks written to multiple claims and targets
Items should not measure students’ feelings or values
Items should not provide advantage or disadvantage to a particular group of students
Items designed using universal design
Items coded to Smarter Balanced cognitive levels (pg. 68)
Vocabulary should follow Common Core 3-tier and academic vocabulary (pg )

19. Digging Deeper: SR Stems, Options, Claims, and Scoring
Stem presents a complete problem
Grades 3-5 stems usually in form a question
Grades 6-11 stems questions and also open-ended stems
Options must be plausible and a correct answer should be present
Length and syntax should be similar
Arrange according to a logical order
Item writers must include a distractor analysis to provide rationale common misunderstandings of a concept.
All 4 Claims have SR items
Scoring: Typically 1 point per item, unless more than one answer, then can be 2 points
Timing: Grade 3 allow for 3 minutes per item, Grade 4-11 allow 2 minutes per item

21. Digging Deeper: CR Stems, Options, Claims, and Scoring
Stems should provide the question and scoring information
2-point and 3-point CRs require an explanation with evidence that supports the explanation
Most CR items relate to a stimulus
All 4 Claims have CR items
Scoring: 2-point and 3-point rubrics are used that are unique to each item
Timing: 5-10 minutes

23. Sample Claim 1 CR Rubrics

24. Sample Claim 4 CR Rubrics

25. Sample Claim 2 Writing CR Rubrics
Note: This is a brief write rubric. The CR typically asks students to write a paragraph or two. This is not used to score essays.

26. Technology-Enhanced Items
All 4 Claims have TE items
Specialized interaction
Same requirements as selected and constructed response items
Students manipulate information
Defined responses, score points typically 1 or 2
Technology-enhanced items are computer delivered items that require
{+}
specialized interactions students must perform to produce a response. Responses produced by a technology-enhanced item require students to do something other than write text or select from among a set of options.
These items may also include digital media as the stimulus.
Technology-enhanced items should conform to the same essential requirements that have already been discussed for writing quality selected response and constructed response items.
The only difference is that they allow students to manipulate information in ways that are not possible with traditional selected response and constructed response items.
Like selected-response items, technology-enhanced items have defined responses that can be scored in an automated manner.

27. Technology-Enhanced Example Item
Below is a poem, a sonnet, in which the speaker discusses her feelings about a relationship. Read the poem and answer the question that follows.
Remember
When you can no more hold me by the hand,
Nor I half turn to go yet turning stay.
Remember me when no more day by day
You tell me of our future that you plann’d:
Only remember me; you understand
It will be late to counsel then or pray.
Yet if you should forget me for a while
And afterwards remember, do not grieve.
For if the darkness and corruption leave
A vestige* of the thoughts that once I had
Better by far you should forget and smile
Than that you should remember and be sad.
This is an example of an ELA technology-enhanced item for grade 8. For this item, students begin by reading a poem. Next, students find and
{+}
highlight lines in the text that reveal a change in the speaker’s message. Depending on how the item writer designed the item, students are able to select one or more blocks of text in the poem.
*vestige: a mark, trace, or visible evidence of something that is no longer present or evident.
In the sonnet “Remember,” which two lines reveal a change in the speaker’s message to her subject?

28. INTERACTION SPACE
Remember
When you can no more hold me by the hand,
Nor I half turn to go yet turning stay.
Remember me when no more day by day
You tell me of our future that you plann’d:
Only remember me; you understand
It will be late to counsel then or pray.
Yet if you should forget me for a while
And afterwards remember, do not grieve.
For if the darkness and corruption leave
A vestige* of the thoughts that once I had
Better by far you should forget and smile
Than that you should remember and be sad.
The notes for a summary need to be arranged correctly in the order in which the events occurred in the passage. Click on each sentence and move it to arrange the sentence into correct chronological order.
Summary of Events:
Maria laughs with the old women.
The guest and family eat dinner.
Maria’s mother asks the guests for a story.
Maria’s guests arrive.
Maria becomes sad.
The guests take turn telling stories.
There are several terms that are important to understand when developing technology enhanced items. The first term is Interaction Space. The interaction space is the area in which students interact with an item to produce a response.
{+}
This is the interaction spaces for an item that presents students with a poem and asks them to select blocks of text within the poem. The interaction space is the area in which students select text blocks.
This is a different item that presented a passage and is now asking students to order sentences that describe events in the passage. The interaction space allows students to select the sentences and then change their order.

29. TYPE OF INTERACTION SPACE
Below is a poem, a sonnet, in which the speaker discusses her feelings about a relationship.
Read the poem and answer the question that follows.
Remember
When you can no more hold me by the hand,
Nor I half turn to go yet turning stay.
Remember me when no more day by day
You tell me of our future that you plann’d:
Only remember me; you understand
It will be late to counsel then or pray.
Yet if you should forget me for a while
And afterwards remember, do not grieve.
For if the darkness and corruption leave
A vestige* of the thoughts that once I had
Better by far you should forget and smile
Than that you should remember and be sad.
*vestige: a mark, trace, or visible evidence of something that is no longer present or evident.
In the sonnet “Remember,” which two lines reveals a change in the speaker’s message to her subject?
When developing a technology-enhanced item, the task model will indicate which type of interaction space should be used to allow students to produce a response. {+}
As an example, the task model for this item specifies the use of the “select text” interaction space.
TYPE OF INTERACTION SPACE

30. Scoring Rule: Logic used to score student response
Who is the protagonist in the story?
A. Joe
B. Sue
C. Maria
D. The father
Scoring Rule:
If student response = C, then correct
Otherwise, incorrect
Scoring Rule:
If object 1 = B, 2 = A, 3 = B, 4 = A, 5 = A, then correct
Otherwise, incorrect
Classify each word below based on whether it is a verb or a noun.
Verbs Nouns
Doll Run Dog Swim Eat
All technology enhanced items have a scoring rule associated with them.
{+}
The scoring rule provides directions to the computer about the logic to be followed to score a response. In some cases, an item will have only one correct response. In other cases, there may be many possible correct responses. {+}
In reality, all items have a scoring rule. The scoring rule for a selected response item simply states that if the student’s response is identical to the correct response, then the student receives one point, otherwise the student receives zero points. {+} Similarly, for a drag and drop item, the scoring rule compares the location of each object with its correct location. The student is then awarded one or more points based on the number of correct categorizations made.

31. Technology-Enhanced Item Types
Common English Language Arts Technology-Enhanced item types
Dropdowns
Classification
Reorder text
Select and order
Select text
Below is a poem, a sonnet, in which the speaker discusses her feelings about a relationship.
Read the poem and answer the question that follows.
Remember
When you can no more hold me by the hand,
Nor I half turn to go yet turning stay.
Remember me when no more day by day
You tell me of our future that you plann’d:
Only remember me; you understand
It will be late to counsel then or pray.
Yet if you should forget me for a while
And afterwards remember, do not grieve.
For if the darkness and corruption leave
A vestige* of the thoughts that once I had
Better by far you should forget and smile
Than that you should remember and be sad.
*vestige: a mark, trace, or visible evidence of something that is no longer present or evident.
In the sonnet “Remember,” which two lines reveals a change in the speaker’s message to her subject?
The notes for a summary need to be arranged correctly in the order in which the events occurred in the passage. Click on each sentence and move it to arrange the sentence into correct chronological order.
Summary of Events:
Maria laughs with the old women.
The guest and family eat dinner.
Maria’s mother asks the guests for a story.
Maria’s guests arrive.
Maria becomes sad.
The guests take turn telling stories.
Classify each word below based on whether it is a verb or a noun.
Verbs Nouns
Doll Run Dog Swim Eat
There are many different types of technology-enhanced item types that can be used to write an item.
{+}
Most ELA items will use one of the following types;
dropdowns, classification, reorder text, select and order, and select text.
As just three examples, the TEI poem item discussed earlier is an example of a select text item. Students selected a block of text from a poem as their answer.
The item that asked students to order events in a story is an example of a reorder text item.
And the item asking students to classify words as verbs or nouns is an example of a classification item.
Additional examples are available in the Technology-Enhanced Item specifications.

32. Sample Technology-Enhanced ELA
Highlight text

33. TE: Mark the box

34. TE: Drag and Drop

35. TE: Select Text

36. TE: Matching tables

37. ELA Passages/Stimuli
Passages and Stimuli are found in all 4 Claim test questions

38. Claim 1 Stimuli Reading comprehension
Selected Response, Constructed Response, and Technology-Enhanced Items
Literary texts
narratives, dramas, poems
Informational texts
literary nonfiction, historical, scientific, and technical texts
On grade level
May include multiple texts
Different types of stimuli are required to collect evidence for each claim.
{+}
Claim 1 focuses on reading comprehension skills and collects evidence through
Selected response, constructed response, and technology-enhanced items.
Claim 1 stimuli can be literary texts, such as narratives, drama, and poems,
and informational texts, including literary nonfiction and historical, scientific, and technical texts.
Claim 1 stimuli should be on grade level.
Some Claim 1 items for certain targets can include multiple texts.

39. Claim 2 Stimuli Writing, revising, and editing
Selected Response, Constructed Response, and Technology-Enhanced Items
Literary and informational texts, audio/video presentations, texts needing editing/revising
Shorter than other stimuli
Should be one grade below grade level
Claim 2 targets focus on
{+}
writing, revising, and editing brief texts, and gather evidence through
Selected response, constructed response, and technology-enhanced items.
Stimuli for this claim include brief literary and informational texts, multimedia presentations, and texts requiring editing and/or revising.
Claim 2 stimuli are shorter than most other stimuli
and, since the focus is on writing skills and not reading skills, the stimuli should be one grade below the tested grade level.

40. Claim 3 Stimuli Speaking and listening
Selected Response, Constructed Response, and Technology-Enhanced Items
Audio presentations, 1-2 minutes in length
Claim 3 focuses on
{+}
speaking and listening skills
and uses selected response, constructed response, and technology-enhanced items.
Audio and video presentations are used for Claim 3 stimuli and can include speeches, lectures, movie clips, informational videos, and literary readings.

41. Claim 4 Stimuli Research skills
Selected Response, Constructed Response, and Technology-Enhanced Items
Includes literary text, informational text, audio presentations, statistical data, simulated web pages, and visual stimuli
May use multiple stimuli
Should be one grade below grade level
Claim 4 focuses on
{+}
research skills
and uses selected response, constructed response, and technology-enhanced items.
Claim 4 stimuli can include literary and informational texts, audio and video presentations, statistical data, simulated web pages, and visual stimuli, such as artwork, maps, charts, or graphs.
Items and tasks designed to collect evidence about Claim 4 may also present multiple stimuli.
The reading level of text stimuli for Claim 4 should be one grade below the tested grade level.

42. Performance Task Stimuli
Includes informational text, audio presentations, statistical data, simulated web pages, visual stimuli
Should be one grade below grade level
Multiple stimuli
Stimuli for performance tasks include
{+}
literary and informational texts, audio and video presentations, statistical data, simulated web pages, and visual stimuli, such as artwork, maps, charts, or graphs.
The readability of text stimuli should be one grade below the tested grade level.
Performance tasks will always use multiple stimuli.

43. Selecting Appropriate Materials
Interesting and grade appropriate topics
Free of bias and sensitivity issues
Note: speaking and listening passages and performance task sources have different length requirements.
Follow reading passage length guidelines:
Grade
Maximum Word Count 3
650 4
750 5 6
950 7 8
High School (9–11)
1,100
Selecting appropriate stimuli is an essential part of writing items and tasks.
{+}
English Language Arts stimuli must be interesting to students and cover grade-level appropriate topics. Appropriate topics may include animals, robots, space exploration, and historical people, documents, and events.
Stimuli must conform to the Smarter Balanced Assessment Consortium Bias and Sensitivity Guidelines.
Writers should use the passage length guidelines from the Smarter Balanced Assessment Consortium English Language Arts Stimulus Specifications. Stimuli must also be long enough to provide sufficient context for complex questions but short enough to be realistic for an assessment situation.

44. Text Complexity Quantitative and qualitative measures
The quantitative measures determine the grade band for a text (e.g., 6–8), while the qualitative measures help determine the appropriate placement within the band (e.g., 8)
Quantitative measures
Flesch-Kincaid index
Lexile
Word count
Range of difficulty
 As defined in the Common Core State Standards Appendix A: Research Supporting Key Elements of the Standards,
{+}
both quantitative and qualitative measures should be used to determine a text’s complexity.
Quantitative measures such as the Flesch-Kincaid index, Lexile level, and word count determine the grade band placement for a text. Each grade band should cover a range of difficulty.

45. Common Core Lexile Grade Level Ranges
Text Complexity Grade Band in the Standards
Lexile Ranges Aligned to CCR Expectations
K-1
N/A
2-3
4-5
6-8
9-10
11-CCR
This table displays recommended Lexile ranges for each grade level band. As is shown in the table, the Lexile ranges increase across the grade bands which is consistent with the expectation that students are able to read and comprehend increasingly more complex text as they develop college and career readiness.

46. Shortcomings of Quantitative Measures
Measures difficult to obtain for
poems and dramas
highly complex ideas
Quantitative readability measures can be helpful in determining grade band placement but they should never be used in isolation.
{+}
It is not possible to obtain accurate quantitative measures for some texts, such as poems and drama,
and some texts may have a low quantitative readability but contain highly complex ideas. Good judgment must be used to select complex, challenging, and appropriate texts.

47. Qualitative Measures of Text Complexity
English Language Arts State Collaborative on Assessment and Student Standards Rubrics
Meaning/Purpose
Text Structure
Language Features
Knowledge Demands
The English Language Arts State Collaborative on Assessment and Student Standards developed qualitative rubrics that provide a measure of literary texts and informational texts based on the following criteria:
{+}
meaning and purpose,
text structure,
language features,
and knowledge demands. These rubrics are found in the Smarter Balanced Assessment Consortium English Language Arts Stimulus Specifications.

48. Common Core State Standards Appendix A
Quantitative Measures – Pages 4-7
To learn more about text complexity, educators are strongly encouraged to review the information presented in the Common Core State Standards Appendix A. In particular, pages four through seven provides more information on Lexile measures and provides further guidance on the complexity of text for each grade level. Appendix A also provides more information about qualitative considerations for assessing text complexity.

49. Stimulus Metadata Title Author Source Copyright status Genre Topic
Length
Reading level
Level of complexity
To assist in examining whether the stimulus adheres to specifications and whether the stimulus is grade-level appropriate, several pieces of information must be associated with each stimulus. This information is referred to as metadata.
{+}
When developing or selecting stimulus materials, there are several pieces of metadata that should be assigned to a stimulus. This metadata includes information such as the title, author, source, copyright information, genre, topic, and readability. More information about assigning metadata will be presented in the module that focuses on the use of the item writing system.

50. URL Access
Here is the URL that takes you to Smarter Balanced Practice Tests and Training Tests. If you are using a web browser, only these work: Quick Access (any users): Any Internet-connected computer using a current Web browser including:
Mozilla Firefox
Google Chrome
Microsoft Internet Explorer 10
Apple Safari

51. Questions? Survey on this training session Go here: