1. Andrew C. Porter Vanderbilt University August, 2006
Aligned Instruction
Andrew C. Porter Vanderbilt University  August, 2006

2. Tools Teacher surveys of instruction Content analyses of
Daily logs
End-of-semester or end-of-year surveys
Content analyses of
Standards
Tests
Curriculum materials
Alignment indices—e.g, alignment between assessment and standards

3. Content Matrix

4. Vertical and Horizontal Alignment
Achievement
Instruction
District
Assessment
Standards
State
Assessment
Standards

5. Example Matrices to Measure Alignment
Cognitive Demand
Assessment
Standards .3 .1 .2 .2 .1
Topics
∑ |X-Y| 2
X=Assessment Cell Proportions
Y=Standards Cell Proportions
Alignment Index =1 -

6. Alignment of Assessments with Standards 7th-Grade Math:Goals Study
Average Within-State Alignment = .40
Average Between-State Alignment = .39
Average State-Test-to-NCTM Alignment = .39

7. Alignment of Instruction with Assessment 8th-Grade Math:SCASS Study
Average Within-State Alignment = .22
Average Between-State Alignment = .23
Average State-to-NAPE Alignment = .39

8. Alignment of Instruction with Instruction 8th-Grade Math: SCASS Study
Average Alignment = .69

9. 7th Grade Standards
State E
State F
NCTM

10. Quality of Data Response rates
Interrater agreement for content analyses
Validity of teacher self-report
Explaining between-teacher variance in alignment to NAEP
Predicting student achievement gains
[Note: The need for a reform-neutral language]

11. Uses of Tools Describing Instructional Practices
Research
Serve as dependent variable in teacher decision-making research
Describe the implemented curriculum
Measure implementation of new curricula
Assess the validity of transcript studies
Practice
Inform teacher reflections on their own instructional practices [Note: Should not be used for teacher accountability.]

12. Uses of Tools Describing Instructional Materials
Research
Research effects of textbooks on instruction
Assess the breadth and depth of content in instructional materials
Practice
Build tests
Write content standards
Develop national, state, or district indicator systems

13. Uses of Indices of Alignment
Research
Serve as a control variable
Serve as a dependent variable
Serve as a descriptive variable
Practice
Align state tests to state standards
Align instructional materials to standards or course frameworks

14. Increasing Validity and Value
Getting the content language right
Using time samples to describe instruction for an entire school year
Replicating the finding that alignment predicts student achievement gains
Identifying contexts in which teacher self-report on the content of instruction is more or less accurate
Improving the level and consistency of interrater agreement in content analyses
Understanding the distributional properties of the alignment statistics
Building powerful professional development programs for data-based decision making
Developing a content language for reading

15. Conclusions
Much progress has been made in recognizing the importance of instructional content as a variable in education research.
Some progress has been made in building tools for including content in education research.
There have been several innovative uses of these new tools in both research and practice, and more are on the horizon.
But there is much more work to be done.

17. 7th-Grade Standards--Close View
Number Sense and Numeration
State E
State F
NCTM

18. 7th-Grade Standards--Close View
Data Analysis and Probability
State E
State F
NCTM

19. Response Rates for Survey
Eisenhower Longitudinal
Wave %
Wave %
Wave %
Eisenhower Cross-Sectional %
Reform Up Close %

20. Interrater Agreement Assessment Mean Range Goals Study .51 .77 to .34
CCSSO Study to .37
Standards
Goals Study to .33
[Note: In each study, there was one outlier rater.]

21. Eisenhower Longitudinal Study
Longitudinal data on instruction alignment to NAEP yielded:
42% of variance explained by level (elementary, middle, high school) and subject
27% of variance explained by teachers in the same school
0% of variance explained by between school or between years

22. Alignment to Predict Achievement Gains

23. Index Intercorrelations