1. English Language Arts Content Specifications, Item Specifications, and Depth of Knowledge
Welcome to the Smarter Balanced Assessment Consortium’s English Language Arts Content Specifications, Item Specifications, and Depth of Knowledge Training Module. This module introduces the consortium’s English Language Arts Content Specifications and Item Specifications documents. In addition, this module describes the Depth of Knowledge cognitive complexity taxonomy that will guide the development of items and tasks that are of appropriate difficulty and cognitive complexity.

2. Purpose of Training Module
Introduce primary source materials
Common Core State Standards for English Language Arts and Literacy
Smarter Balanced Content Specifications
Smarter Balanced Item and Task Specifications
Describe how these documents will be used for item writing
Define Depth of Knowledge
This training module focuses on three main topics.
{+}
This module provides an overview of the documents used to inform the development of items and tasks.
This module provides a description of how these documents are used by item writers to write high-quality items and tasks.
Finally, this module provides an introduction to the Depth of Knowledge taxonomy for cognitive complexity.
Let’s begin with an overview of the structure of each resource document.

3. Foundation for Item and Task Development
Items and Performance Tasks
Smarter Balanced Item and Task Specifications
Smarter Balanced Content Specifications
As will be explored in greater detail in this module, the Common Core State Standards
{+}
Provided the foundation for the Smarter Balanced Content Specifications.
The content specifications were then used to inform the development of the Smarter Balanced Item Specifications.
Item and task writers are expected to use all three documents to guide the development of Smarter Balanced items and tasks.
Common Core State Standards

4. Standards for English Language Arts 6–12
Structure of the Common Core State Standards for English Language Arts and Literacy
Standards for English Language Arts & Literacy in History/Social Studies, Science, and Technical Subjects K–5
Standards for English Language Arts 6–12
Standards for Literacy in History/Social Studies, Science, and Technical Subjects
The Common Core State Standards document has three main sections.
{+}
The first section focuses on English Language Arts standards and standards across content areas for kindergarten through fifth grade.
The second section focuses on English Language Arts standards for grades six through twelve.
The final section focuses on middle and high school standards for literacy in history, social studies, science, and technical subjects.

5. Structure of the Common Core State Standards
Strands
College and career readiness anchor standards
Grade-specific standards
Reading
Writing
Language
Speaking/Listening
Reading
Anchor Standards 1. …
10.
Writing
Language
Speaking/Listening
Grade 6
Grade 9
Grade 3
Each of the three sections of the Common Core State Standards are further divided into {+}
strands,
{+}
College and career readiness anchor standards,
and grade-specific standards.
The Common Core State Standards were developed by using the standards from various states to construct a focused and coherent set of standards that define what students should understand and be able to do. The standard, as well as the content specifications and item specifications, are rigorous, research-based, and consistent with college and career expectations.

6. Structure of the Content Specifications
The Smarter Balanced Assessment Consortium Content Specifications for English Language Arts and Literacy is a bridge document linking the Common Core State Standards to the Smarter Balanced assessment claims and targets.
Item Development Guidelines for all Selected Response and Constructed Response Items and Performance Tasks
Each SR and CR item should be written to focus primarily on one assessment target. Secondary targets are acceptable for some Claim 1 targets and should be listed in the item forms.
Each PT should be written to multiple claims and targets.
Items/tasks should be clearly written to elicit the desired evidence of a student’s knowledge, skills, and abilities.
Items/tasks should avoid measurement of a student’s feelings or values.
Items/tasks should not provide an advantage or disadvantage to a particular group of students. Items should not exhibit or reflect disrespect to any segment of the population in regard to age, gender, race, ethnicity, language, religion, socioeconomic status, disability, or geographic region.
Universal design principles must be followed. Universal design, as applied to assessments, is a concept that allows the widest possible range of students to participate in assessments and may even reduce the need for accommodations and alternative assessments by expanding access to the tests themselves.
All items/tasks must be coded to the Smarter Balanced cognitive levels as indicated on page 68 of this document.
Vocabulary items should follow the Common Core State Standards 3-tier words and academic vocabulary as described on pages of this document.
Guidelines for Developing Stems in SR Items
Stems should present a complete problem so that students know what to do before looking at the options; students should not need to read all options before knowing what is expected.
Grades 3–5 stems should be in the form of a question. An open stem, however, may be used if doing so will reduce wordiness and repetition.
Grades 6–11 permit open-ended stems, and there should be a variety of stem types. The decision for choosing open or closed stems should be based on ease of reading and clarity.
Guidelines for Developing Options in SR Items
All options should be plausible and there must be a correct or best answer.
All options, including the correct response, should be similar in length and syntax. Students should not be able to rule out a wrong answer or identify a correct response solely because it looks or sounds different from the other options.
All options should be arranged according to a logical order (e.g., vocabulary words, details, or quotations from the passage should follow the order used in the text).
All options must be followed by distractor analyses which help item writers think through students’ common misunderstandings of a concept.
The Common Core State Standards were used to create the The Smarter Balanced Assessment Consortium’s English Language Arts Content Specifications and Item Specifications. Clusters of the standards were used to inform the assessment targets that are defined in the content specifications.

7. Structure of the Content Specifications
Part I: Introduction and Background
Development Process for the Major Claims and Assessment Target
Part II: Content Specifications: Mapping Assessment Targets to Standards
Part III: Claims, Rationale, Evidence, Assessment Targets, Proposed Reporting
Categories
References & Appendices
Appendix A: Cognitive Rigor Matrix/Depth of Knowledge
Appendix B: Grade Level Tables for Reading Assessment Targets
Appendix C: Tools for Examining Text Complexity
The Content Specifications is divided into five main sections.
{+}
The first section describes the process used to develop the major claims and assessment targets based on the Common Core State Standards.
The second section maps the assessment targets to the standards. This section shows how each assessment target addresses a cluster of standards.
The third section provides further detail about the assessment targets for each claim and the evidence required for each assessment target.
The fourth and fifth sections provide additional details that may be useful for understanding the claims and assessment targets. Of particular interest to item and task writers is Appendix A, which provides detailed information about cognitive complexity and depth of knowledge. Before examining item and task specifications in greater detail, let’s take a moment to explore cognitive rigor and depth of knowledge in more detail.

8. Cognitive Rigor and Depth of Knowledge
The level of complexity of the cognitive demand.
Level 1: Recall and Reproduction
Requires eliciting information such as a fact, definition, term, or a simple procedure, as well as performing a simple algorithm or applying a formula.
Level 2: Basic Skills and Concepts
Requires the engagement of some mental processing beyond a recall of information.
Level 3: Strategic Thinking and Reasoning
Requires reasoning, planning, using evidence, and explanations of thinking.
Level 4: Extended Thinking
Requires complex reasoning, planning, developing, and thinking most likely over an extended period of time.
Smarter Balanced assessments will elicit evidence from students that is used to support claims about the extent to which students have the ability to integrate knowledge and skills across multiple assessment targets and are ready to meet the challenges of college and careers.
{+}
Items and tasks must be constructed at various levels of cognitive rigor. Smarter Balanced has defined four levels of depth of knowledge.
The first level focuses on recall and reproduction of facts and other types of information.
The second level focuses on basic skills and concepts that require cognitive processes that extend beyond the recall of information.
The third level focuses on strategic thinking and reasoning.
The fourth and final level requires extended thinking that includes complex reasoning, planning, development, and cognition that occurs over an extended period of time.

9. Cognitive Rigor Matrix
This matrix from the Smarter Balanced Content Specifications for ELA draws from both Bloom’s (revised) Taxonomy of Educational Objectives and Webb’s Depth-of-Knowledge Levels below.
The Cognitive Rigor matrix combines two common taxonomies that categorize levels of cognition and shows how the Smarter Balanced depth of knowledge categories relate to these taxonomies. As shown here, the concept of remembering information only relates to the first depth of knowledge level. In contrast, evaluation does not relate to either of the first two levels of depth of knowledge. This table, which can be found in the Smarter Balanced Content Specifications for ELA, is a useful aid for guiding the development of items at different depth of knowledge levels.
Now let’s examine the item specifications.

10. Evidence Statements Task Models Sample Items
Structure of the Smarter Balanced Assessment Consortium English Language Arts Item Specifications
Evidence Statements
Task Models
Sample Items
The English Language Arts Item Specifications were developed by Smarter Balanced to guide the development of items and tasks that support claims about the achievement of common core standards.
{+}
For each assessment target, the item specifications define the evidence to be collected to support claims about the target.
Each evidence statement has one or more associated task models. The task models describe specific characteristics of items that can elicit the evidence required to support a claim about an assessment target.
To make the task models more concrete, sample items are also provided.
With this overview of the common core state standards, content specifications, and the item specification, let’s examine an item specification table in greater detail.

11. Structure of Item Specification Table
The Item Specifications contain a large number of item specification tables. Each table focuses on a specific claim and assessment target or targets, and presents information that is used to guide the writing of items and tasks that collect evidence about that claim and assessment target. Let’s take a moment to examine several key parts of an item specification table.
{+} The top portion of a table indicates the content area, grade, claim, and assessment target addressed by the table. In this example, the content area is English Language Arts, the grade is eleven, and the table address claim one, assessment target 14.
{+} The first row restates the claim.
{+}
The second row states the assessment target.
And the third row indicates which standards are associated with the assessment target. All of this information is based on information provided in the Content Specifications. {+} The fourth row provides detailed information about the type of evidence required to support the claim about the assessment target. This information is useful for informing the contents of the prompt developed to elicit evidence from students.
The next row indicates the depth of knowledge level to which items or tasks should be written to provide this evidence.
The sixth row indicates the item types that can be used to elicit evidence from students. In this example, constructed response items are deemed appropriate.
Next, task models describe the characteristics of items and tasks that may be used to collect evidence. There will be more about task models in a few moments.
The eighth row provides information about any non-targeted knowledge or skills that may be required to respond to items based on the task models. In some cases, the skills or knowledge required to collect evidence about an assessment target require the application of related skills that are not the current focus of attention, and such skills are identified in this section.
The next section provides information about the types of stimulus materials that should accompany the item. In this example, informational texts are expected to be presented with the item prompt.
The tenth row details vocabulary that is associated with the assessment target and which can be used as part of the item’s content. This section provides guidance for making decisions about whether or not a term might introduce bias. Any term listed in this section is deemed appropriate for use within the context of the items or tasks developed for the claim and assessment target.
The eleventh row indicates which tools, if any, can be used while working on the item or to produce responses for the item. This information is intended to support decisions about potential accommodations and about resources that should be available to students as they work on this item.
Next, information about any specific attributes of the item is provided. In this example, information about the grade level of stimulus text is provided.
Any accessibility concerns that are specific to any of the task models is then documented. This information is used to help inform decisions about accommodations for the item or tasks built from the task model.
If a sample item exists, the next section will provide a link to it. Sample items provide models for what an item or task might look like.
Finally, information about scoring is provided. In this example, a three point constructed response scoring rubric must be provided for each item or task built for this assessment target.
With this overview of item specifications, let’s take a closer look at the claims and assessment targets for English Language Arts.

12. The Smarter Balanced Assessment Consortium English Language Arts Content Specifications
Claim 1: Students can read closely and analytically to comprehend a range of increasingly complex literary and informational texts. Reading, literary and informational text.
Claim 2: Students can produce effective and well grounded writing for a range of purpose and audiences.
Claim 3: Students can employ effective speaking and listening skills for a range of purposes and audiences.
Claim 4: Students can engage in research/inquiry to investigate topics, and to analyze, integrate, and present information.
The English Language Arts claims are derived from the Common Core State Standards and serve as the basis for the Consortium’s development of items and tasks. Each of the 4 claims has a number of targets and the targets detail the evidence required to support each claim. Now, let’s examine each claim.

13. Claim 1
Students can read closely and analytically to comprehend a range of increasingly complex literary and informational texts.
Targets 1–7 correspond with literary texts
Targets 8–14 correspond with informational texts
The assessment targets incorporate the content clusters from the Common Core State Standards
For Claim 1, there are fourteen assessment targets.
{+}
Assessment targets 1-7 focus on reading literary texts.
Assessment targets 8-14 focus on reading informational texts.
Assessment targets are linked to the content clusters within the Common Core State Standards. Now let’s look at a specific target for Claim 1.

14. Grade 5 Claim 1 Target 4
4. REASONING & EVIDENCE: Use supporting evidence to justify interpretations (theme, events, conflicts/challenges, setting, character development/ interactions, point of view) Standards: RL-2, RL-3, RL-6 (DOK 3) RL-2 Determine a theme of a story, drama, or poem from details in the text, including how characters in a story or drama respond to challenges or how the speaker in a poem reflects upon a topic; summarize the text. RL-3 Compare and contrast two or more characters, settings, or events in a story or drama, drawing on specific details in the text (e.g., how characters interact). RL-6 Describe how a narrator’s or speaker’s point of view influences how events are described.
This grade 5 target focuses on reading literary text.
{+}
Notice that several standards are associated with this target. There is not a one-to-one correspondence between a given assessment target and a standard. Instead, assessment targets focus on skills and knowledge that cross a cluster of standards.
Also note that the depth of knowledge level that must be applied to achieve this assessment target is also specified.

15. Claim 2
Students can produce effective and well grounded writing for a range of purpose and audiences.
Targets 1, 3, & 6: Revise/Write Brief Texts
Targets 2, 4, & 7: Compose Full Texts including essays and narratives
Target 5: Use of text features, e.g., headings, subheadings, etc.
Target 8: Language & Vocabulary Use
Target 9: Edit/Clarify
For Claim 2, there are nine assessment targets.
{+}
Targets 1, 3 and 6 focus on writing and revising brief texts.
Targets 2, 4, and 7 address composing full texts.
Target 5 focuses on the use of text features in writing.
Target 8 focuses on the use of language and vocabulary.
Target 9 focuses on editing and clarifying text. Now let’s look at a specific target for Claim 2.

16. Grade 7 Claim 2 Target 1
1. WRITE/REVISE BRIEF TEXTS: Apply narrative strategies (e.g., dialogue, description,) and appropriate text structures and transitional strategies for coherence when writing or revising one or more paragraphs of narrative text (e.g. closure, introduce narrator or use dialogue when describing an event) Standards: W-3a, W-3b, W-3c, W-3d, and/or W-3e (DOK 2) W-3 a. Engage and orient the reader by establishing a context and point of view and introducing a narrator and/or characters; organize an event sequence that unfolds naturally and logically. b. Use narrative techniques, such as dialogue, pacing, and description to develop experiences, events, and/or characters. c. Use a variety of transition words, phrases, and clauses to convey sequence, signal shifts from one time frame or setting to another, and show the relationships among experiences and events. d. Use precise words and phrases, relevant descriptive details, and sensory language to capture the action and convey experiences and events. e. Provide a conclusion that follows from and reflects on the narrated experiences or events.
This example shows Assessment Target 1 for Claim 2.
{+}
Target one focuses on writing and revising texts.
The several standards that are associated with this assessment target are listed.
This target also addresses Level 2 Depth of Knowledge.

17. Claim 3
Students can employ effective speaking and listening skills for a range of purposes and audiences.
Language & Vocabulary Use
Clarify Message
3. Plan/Speak/Present
4. Listen/Interpret
For Claim 3, there are three assessment targets.
{+}
Target 1 focuses on use of language and vocabulary.
Target 2 focuses on adapting speech to a variety of contexts and tasks.
Target 3 addresses students’ ability to plan, speak and present information in an oral manner.
Target 4 focuses on listening and interpreting spoken information.
Now let’s look at a specific target for Claim 3.

18. Grade 11 Claim 3 Target 3
3. PLAN/SPEAK/PRESENT: Gather and organize information compose and orally deliver short (e.g., summaries) and longer (presentations) for different purposes and audiences drawing from a range of digital media to enhance the message or intent Standards: SL-1 SL-4 SL-5 SL-6 (DOK 3 DOK 4) SL-1 Initiate and participate effectively in a range of collaborative discussions (one-on-one in groups and teacher-led) with diverse partners on grades 11–12 topics texts and issues building on others’ ideas and expressing their own clearly and persuasively. SL-4 Present information findings and supporting evidence conveying a clear and distinct perspective such that listeners can follow the line of reasoning alternative or opposing perspectives are addressed and the organization development substance and style are appropriate to purpose audience and a range of formal and informal tasks. SL-5 Make strategic use of digital media (e.g. textual graphical audio visual and interactive elements) in presentations to enhance understanding of findings reasoning and evidence and to add interest. SL-6 Adapt speech to a variety of contexts and tasks demonstrating a command of formal English when indicated or appropriate.
As seen earlier for Claims 1 and 2, assessment targets for Claim 3 address several standards, each of which are listed below the full description of the assessment target.

19. Claim 4
Students can engage in research / inquiry to investigate topics, and to analyze, integrate, and present information.
1. Plan/Research
2. Interpret & Integrate Information
3. Analyze Information/Sources
4. Use Evidence
For Claim 4, there are four assessment targets.
{+}
Target 1 focuses on planning and conducting research.
Target 2 addresses student’s ability to interpret and integrate information.
Target 3 focuses on analyzing information and the sources of that information.
Target 4 focuses on the use of evidence to support an argument or position.
Now let’s look at a specific target for Claim 4.

20. Grade 4 Claim 4 Target 3
3. ANALYZE INFORMATION/SOURCES: Distinguish relevant- irrelevant information (e.g., fact/opinion) Standards: W-8, W-9 (DOK 2) W-8 Recall relevant information from experiences or gather relevant information from print and digital sources; take notes and categorize information, and provide a list of sources. W-9 Draw evidence from literary or informational texts to support analysis, reflection, and research.
This is a grade 4 assessment target for Claim 4. This target addresses two standards and requires a Level 2 Depth of Knowledge.
With these examples of claims and assessment targets, let’s now take a closer look at the evidence statements that are specified for assessment targets.

21. Define the information or products elicited from students
Evidence Statements
Define the information or products elicited from students
Task models designed to elicit evidence
Sample items provide concrete examples of the characteristics specified in a task model
Evidence statements define the type of information or products that are collected from students to provide information about specific knowledge, skills, and abilities that are the target of assessment.
{+}
Task models guide the development of items that assess a particular target.
Sample items provide examples of how to write items for a specific assessment target. Not every assessment target has a sample item. Sample items provide an overview of some of the item types that will be used in the Smarter Balanced assessments.

22. Grade 5 Evidence Statement for Grade 5 Claim 1 Target 4
Evidence Required:
When reading literary texts, students
Determine the theme of a story and justify their interpretation by pointing to textual evidence from such elements as the story’s events, conflicts and resolutions, and the interactions between characters. Or
Compare and contrast characters, settings, or critical events from different parts of a story to form an interpretation justifying that interpretation through specific textual evidence. Or
Justify interpretations of how the author’s choice of narrator affects the theme, tone, or perspective on events of a text using textual evidence. 
This is an example of an evidence statement for the Claim 1 assessment target presented earlier. This statement identifies the evidence for the Claim 1 Target 4 assessment target and indicates that when reading literary texts students can
{+}
determine the theme of a story,
compare and contrast elements of the a story,
and justify interpretations of authors’ choices regarding theme, tone, or perspective.

23. Alignment to Claims and Assessment Targets, Primary Claims & Secondary Claims
Claim 1: Students can read closely and analytically to comprehend a range of increasingly complex literary and informational texts.
Target 4: REASONING & EVALUATION: Use supporting evidence to justify interpretations of such story elements as theme, events, conflicts/challenges, setting, character development/interactions, or point of view.
Secondary Target(s): Target 1. KEY DETAILS: Use explicit details and implicit information from the text to support answers or make inferences about information presented
All item specifications address a primary target.
{+}
For some item specifications, a secondary target is also addressed.
As an example, for Claim 1 the primary target is 4 and the secondary target is 1. This secondary target will always appear with this primary target and an item or task developed using this specification should address both the primary claim and the secondary claim.
Now let’s take a look at a task model.

24. Task Models
Model 1: 2-pt CR (DOK 3) – prompts students to make an inference or draw a conclusion about how the author develops the theme. Response uses relevant text evidence as support.
SR: Selected Response
CR: Constructed Response
ER: Extended Response
TE: Technology-Enhanced
PT: Performance Task
A task model describes the characteristics that should appear in an item that provides evidence for a given claim and assessment target.
{+}
As this example shows, a task model presents several pieces of information that should be used to inform the development of an item or task. Let’s take a moment to examine the several pieces of information contained in a task model.
The first piece of information in a task model indicates the model’s number. For each assessment target measured for claim one, one or more task model’s will be specified. In this example, task model number one is displayed.
Next, the number of score points that can be awarded for items or tasks built from the model is specified. In this example, the items built from this model are worth two points.
The third piece of information indicates the item type that is specified by the model. The abbreviation SR is used for selected response, CR for constructed response, ER for extended response, TE for technology-enhanced, and PT for performance task. For this example, the task model specifies a constructed response.
Next, the depth of knowledge level to which items or tasks based on the model are to be written is specified. In this example, the depth of knowledge of items based on this task model is 3.
Finally, the characteristics of items or tasks built from a model are described. In this example, the characteristics of items built from Model 1 indicate that students should be prompted to make an inference or draw a conclusion about how an author develops a theme and to use relevant evidence from the text as support.

25. Task Models Grade 11 Claim 1 Target 4
Model 1: 2-pt CR (DOK 3) – prompts students to make an inference or draw a conclusion about how the author develops the theme. Response uses relevant text evidence as support.
Model 2: 2-pt CR (DOK 3) – prompts students to analyze how point of view impacts the development of the plot. Response uses relevant text evidence as support.
Model 3: 3-pt CR (DOK 3) – prompts students to analyze the author’s choices regarding character development. Response uses relevant text evidence as support.
Model 4: 3-pt CR (DOK 3) – prompts students to analyze how the author uses irony in a text.
Let’s take a look at the task models developed for Claim 1, Target 4 at grade 11.
{+}
There are four task models specified for this target. Each task model uses a constructed response to elicit evidence from students.
For the first task model, items will prompt students to make an inference or draw a conclusion about how the author of a given text develops a theme. Items based on this model will also elicit responses that provide relevant evidence from the text to support their inference or conclusion.
In contrast, task model three specifies that items should prompt students to analyze the author’s choices regarding character development and that responses are expected to use evidence from the text as support.
It is important to note that each assessment target specifies one or more task models that can be used to collect evidence about the assessment target. The task models for each assessment target can be found in the item and task specifications.

26. Alignment to Claims and Assessment Targets, Primary Claims & Secondary Claims
Common Core State Standards: foundation for claims and assessment targets
Smarter Balanced Content Specifications: define claims and assessment targets
Depth of Knowledge: Level of cognitive processes applied by students
Smarter Balanced Item and Task Specifications: define evidence required and describe task models
As this module has explored, the Common Core State Standards provide a foundation for defining the claims and assessment targets that are the focus of the Smarter Balanced Assessment System.
{+}
The content specifications were developed by Smarter Balanced to provide a formal definition of the claims and assessment targets and provide a foundation for item specifications.
In addition, the content specifications indicate the depth of knowledge that students are expected to demonstrate for a given assessment target.
The item and task specifications were also developed by Smarter Balanced to define the evidence to be elicited from students to support claims about assessment targets and present task models that inform the development of items that are used to collect that evidence.
Collectively, this information is used to guide the development of items and tasks that form the Smarter Balanced Assessment System. To support the development of high quality items and tasks, item writers need to be familiar with the content and item specifications and are expected to reference them regularly when developing items and tasks.