1. Measuring the Assessment Literacy of Educators ----
Using the MAC’s Assessment Literacy
Self-Assessments to Build
Assessment Literacy
Edward Roeber – Michigan Assessment Consortium
Denise Brady – Shiawassee RESD
Paul Stemmer – Michigan Department of Education

2. What is the MAC?
The Michigan Assessment Consortium (MAC) is a non-profit professional organization of educators who believe quality education depends on accurate, balanced, meaningful assessment.
MAC members work to advance assessment literacy and advocate for greater understanding about and use of assessment in Michigan.
The goal of the MAC is to help educators use assessment to improve their teaching and students’ learning.
A variety of MAC resources can be accessed at

3. MAC Members
Are individuals and organizations interested in quality assessment practices
Work in public and private schools, local and regional districts, government agencies, professional associations, and institutes of higher learning
Embrace high standards of excellence
Understand that quality assessment is key to quality learning and student achievement
Value professional learning and continuous improvement

4. Assessment Literacy Defined
An assessment literate individual is one who understands how student assessment can enable them to better carry out their role in education, believes that assessment can improve teaching and learning, and puts into place activities and behaviors to act on these beliefs

5. Who Needs to be Assessment Literate?
Everyone with a stake in education needs to be assessment literate, including:
Students
Parents/guardians
Teachers
Specialists at the district and building levels
Building administrators
Central office administrators
Policymakers at the local, state, and national levels
Public

6. Why is Assessment Literacy Needed?
Assessment literacy is needed for several reasons
Lack of understanding about assessment by those who adopt policy and laws, and govern our schools
Lack of understanding by those who teach our students or lead our schools, due to continued lack of pre-service preparation for educators
Misunderstanding by parents and students about how student assessment can promote high quality student learning
Increased volume and changes in types of student assessments
Increased stakes for students, educators, and school
Why is Assessment Literacy Needed?

7. Why is Assessment Literacy Needed?
Research has shown that students who are more involved in their own learning – and assessment – achieve more
Effective use of formative assessment practices requires teachers to understand how on-going instructionally-embedded assessment can help all student achieve at higher levels
Administrator involvement in school improvement activity is also related to higher student achievement
Why is Assessment Literacy Needed?

8. Standards Development Process The work was begun by the MAC Board
A review of the literature was carried out
Other sets of standards were examined
An initial draft of the ALS was created
Another draft of the ALS was produced and reviewed externally by Susan Brookhart, Carol Commodore, Margaret Heritage, Ken O’Connor, Jim Popham, and Rick Stiggins
An in-state review packet/survey was used to gather input from MI educators
Standards
Development
Process

9. Standards Development
Separate standards were developed for:
Students and Their Parents
Teachers
Building administrators
District administrators
Local and state policymakers
The MAC Assessment Literacy standards include:
Dispositions (i.e., beliefs)
Knowledge
Performance (i.e., skills)
Goal—provide a common basis for work in increasing assessment literacy
Standards Development

10. Sample Disposition Standards
Building-Level Administrators should believe that:
An effective assessment system must balance different purposes for different users and use appropriate assessment methods to measure different learning targets.
Multiple measures can provide a more balanced picture of a student or a school.
Quality assessments are a critical attribute of effective teaching and learning.
Sample
Disposition
Standards

11. Sample Knowledge Standards Building-Level Administrators should know:
A balanced assessment system consists of both of the following:
Different users have different assessment purposes.
Different assessment purposes may require different assessment methods.
The different types of assessment methods and when teachers should use each
Selected response
Constructed response
Performance
Personal communication
Sample
Knowledge
Standards

12. Sample Performance Standards
Building-Level Administrators should promote a culture of appropriate assessment practice by:
Promoting assessment literacy for self and staff
Building-Administrators should promote the use of assessment data to improve student learning through the alignment of curriculum, instruction and assessment by:
Using assessment results, including subgroup performance, to influence the school’s curriculum and instructional program.
Using multiple sources of data over time to identify trends in learning.
Sample
Performance
Standards

13. Sample Self-Assessment Question - Disposition
Which of the following is the most important characteristic of balanced assessment system? (I-B)
A. Each type of assessment is given the same number of times during the school year.
B. School accountability assessments are given the highest priority in the assessment system.
C. Classroom assessments are given the highest priority in the assessment system.
D. The information needs of all users are considered equally important and are met.
Sample Self-Assessment Question - Disposition
May not need this slide if sample assessment is given

14. Sample Self-Assessment Question - Knowledge
What is the definition of “formative assessment strategies?” (II-C3)
A. Short tests given multiple times during the school year. B. Sets of test items available in an item bank.
C. Activities used during instruction to determine whether students learned what they were just taught.
D. An assessment that is given at the end of each marking period for grading students.
Sample Self-Assessment Question - Knowledge

15. Sample Self-Assessment Question - Performance
What is the most effective way for a building administrator to interpret achievement results and create improvement goals with staff? (III-F)
A. Demand teachers improve instructional practices
B. Lead dialogues with staff in each of the content areas assessed
C. Tell teachers what they need to improve
D. Publically announce the high and low performing classrooms in the school
Sample Self-Assessment Question - Performance

16. www.socrative.com Student Log-in Room Name: MIALS
Take a Sample Assessment of Your Assessment Literacy
Student Log-in
Room Name: MIALS
Join the Room (You may put any name you choose!)

17. Three Important Questions
The MAC thinks about three questions:
If educators achieve these standards, what would be different in our students?
If policymakers achieve these standards, what would be different about assessment policies and legislation?
If this is a preferred future, what needs to be done to push this vision forward?
This set of questions drives the work of the MAC
Keep for after assessment

18. Assessment Development
Separate Assessment Literacy Self Assessments have been or will be created for:
Students/parents
Teachers
Building administrators
District administrators
Local policymakers
State policymakers
Talk about which have been developed at this point

19. Assessment Literacy Initiative: A New Frontier for NAEP
Cornelia Orr, National Assessment Governing Board (NAGB)
Holly Spurlock, National Center for Education Statistics (NCES)
Assessment Literacy Workshop
March 18, 2015

20. The Charge
December 2012: David Driscoll challenged the Board to use NAEP to make a difference
May 2013: Board member Jim Popham proposed increasing understanding of assessment literacy
August 2013: The Assessment Literacy Work Group was created to develop a communications plan to educate parents, policymakers, and students about assessments
Initial work group meetings determined that while this initiative would include NAEP, the campaign would be intentionally broader

21. NAGB Summit for Parent Leaders
This pre-planned event was a great opportunity to get feedback from one of the initiative’s target audiences
The Parent Summit collected feedback from parents on educational assessments
Key takeaways:
Parents want to know how to interpret their child’s test results
There is strong mistrust towards testing
Parents want to know how testing relates to learning
Parents are confused about different tests and purposes

22. NCSA 2014
Governing Board session at the National Conference on Student Assessment (NCSA)
Participants (including NAEP State Coordinators) provided feedback on the “Understandings”
Key takeaways:
Communication about achievement levels is important
Talk about the assessment quality and precision
Communicate the differences between a group and an individual assessment
Word understandings more positively

23. Target Audience
Primary audiences for the assessment literacy initiative are:
Parents
Policymakers
Students
Work group focuses on parents first
Important to educate parents on the value of student assessment
Governing Board’s existing plans to reach parents
Future focus on policymakers and students

24. Reaching Parents: Intermediaries and Influencers

25. Parent “Understandings”
“Understandings” are a set of concepts for the target audiences to acquire
“Understandings” will serve as content framework for the initiative
Five “Understandings”
Evidence for Interpretations and Uses
Accuracy of Test Scores
Fairness
Variations in Test Quality
Assessment to Improve Teaching and Learning

26. Focus Groups: Objectives
Gather feedback focused on clarity, relevance, appeal, and usefulness
Explore knowledge and attitudes about assessments
Explore parent preferences in communication channels
Use parent feedback to revise content for communication materials

27. Focus Groups: Topics Topic Areas
The Purpose of Educational Assessments
Types of Information from Educational Assessments
Using Assessment to Improve Teaching and Learning
Test Scores Are Not Perfect Estimates of Student Performance
Appropriate Use of Assessments
Quality Variation in Assessments

28. Focus Groups: Lessons Learned
“Understandings”
Findings
Evidence for interpretations and uses
Parents want more information about the specific purpose of each assessment and how the results will be used
Accuracy of test scores
Parents support having multiple measures to assess student performance
Fairness &
Accuracy
Parents are apprehensive about the fairness and accuracy of “high stakes” assessments
Variations in test quality
Parents rely on educators to determine if a test is high quality
Assessments to improve teaching and learning
Parents are not familiar with the term, but they value the concept of formative assessments

29. Strategies for Communicating with Parents
Core messaging principles to guide final message development and prioritization:
Assessment information must use plain, accessible language
Assessment information for parents must center on benefits for their children
Leading assessment messages must align with parent priorities

30. Prototypes: Objectives
Assessment literacy materials will:
Establish the Governing Board’s and NCES’ thought leadership on assessment literacy
Engage parents and their influencers
Serve a specific purpose to meet the various needs and interests of the diverse audience
Incorporate “plain language” messaging
Reach parents directly and indirectly through a variety of channels

31. Assessment Literacy 101 Video
Prototypes: Examples
Parent Toolkit
Assessment Literacy 101 Video
Microsite

32. Prototypes: Parent Toolkit
A toolkit of materials will allow parents, educators, and collaborators to use materials flexibly
Print materials can be adapted to accommodate a wide range of situations:
Distribute at school meetings
Featured at conferences
Could include fact sheet introducing the campaign and one-pagers that expand on the five “Understandings”

33. Prototypes: Video
Parent focus groups are interested in receiving assessment literacy information through videos
In response, the work group has:
Proposed a video—“Assessment Literacy 101”—with parent and teacher testimonials
Discussed a video that could spotlight each “Understanding” to provide deeper content
Storyboard in development on the topic of formative assessment

34. Prototypes: Microsite
Website will promote specific actions:
Learn about the “Understandings”
Download materials such as fact sheets or informational videos; review a sample test
Explore testing of all kinds in their community with an interactive map

35. Prototypes: Next Steps
Additional focus groups with parents are planned. Their feedback will be used to guide the refinement of the communication materials being produced
Continue to work collaboratively with the work group. This includes providing feedback from the focus groups
Determine the calls to action for parents. Clearly define our expectations of parents and the actions they should take

36. MAC Resources Supporting Assessment Literacy
To access the resources of the MAC, visit the following webpage:
Resources include Common Assessment Development modules,
white papers on assessment and other topics, access to a variety of
archived videoconferences, and other material

37. Michigan Learns Assessment!
First Steps on the Assessment Literacy Pathway
Creation of a comprehensive assessment-learning program designed for individuals who want or need to know more about how student assessments are developed and used.
The goal in creating the modules included in this introductory assessment-learning program is provide foundational knowledge about assessment dispositions, knowledge, and performances to teachers, administrators, and others, with the hope that the standards and the learning program will encourage individuals to seek to learn even more about assessment and therefore will use assessment more effectively

38. Michigan Learns Assessment! Vision:
Development of a more comprehensive assessment-learning system
A more advanced program of assessment learning might lead to:
An assessment specialty endorsement for administrators
MAC certification for teachers
Assessment specialists for each district and school in the state
Make Michigan the State of Assessment Literacy!

39. Michigan Learns Assessment! Modules:
Currently under development
Structured so that learning may occur:
Short or longer sessions - one or more days
Together or over time
Each module will contain:
A presentation including built-in individual and group learning activities
A discussion guide for use at the conclusion of each module to guide group discussion and deeper assessment understanding

40. Michigan Learns Assessment! Timeline:
Complete the drafts of the modules by January 2016
Conduct in-state reviews by April 2016
Revise and focus the presentations by June 2016
Field test the modules in schools/districts summer/fall 2016
Revise the modules by December 2016
Produce attractive final versions of the modules by May 2017
Use the modules to offer introductory assessment literacy efforts across the state by June 2017

41. Michigan Learns Assessment!
Proposed Eight MAC Introductory Assessment-Learning Modules:
1–Introductory Module – What is Assessment Literacy and Why is it Important?
Goal: To provide an overview of what assessment literacy means, why it is important, and how one can become assessment literate.
2–Standards and Assessment
Goal: Understand how different definitions of learning, types of standards, and instructional designs affect student assessment choices and designs.

42. Michigan Learns Assessment!
3–Developing a High Quality, Balanced Assessment Systems
Goal: Recognize the components of a balanced assessment system and the different types of measures used in it.
4–Criteria for High Quality Assessment
Goal: Understand (in non-technical terms) key metrics related to assessment quality
5–Developing or Selecting Appropriate Assessments
Goal: Know how to use the criteria for high quality assessment to select or develop needed student assessments

43. Michigan Learns Assessment!
6–The Formative Assessment Process
Goal: Describe the elements of the formative assessment process as well as how teachers would use them.
7–Making Meaning from Student Assessments
Goal: Know how to translate student assessment data usefully for teachers, students and their parents, and administrators
8–Using and Communicating Assessment Results to Promote Student Learning
Goal: Know how to effectively use and report assessment information to a variety of audiences in order to promote student learning

44. Developed a proposal for planning a larger grant proposal to create assessment literacy development strategies and resources
This initial grant will network MI professional organizations to think about what their members need to become more assessment literate—short-, medium-, and long-term
The work is envisioned as occurring over a number of years, on an on-going basis
Current Work of the MAC

45. The work will begin with a review of available resources
Assessment Literacy Self Assessments will be used to:
Gather data for proposal writing
Individual awareness about assessment literacy
Promote organizational work on assessment literacy
Goal is to provide resources to address the current state of assessment illiteracy
The MAC is committed to seeing this through – to continue as it can with available resources to push for greater assessment literacy
Current Work of the MAC

46. Use of the Assessment Literacy Standards
The goal in creating the Assessment Literacy Standards was to define what those affected by student assessment (of all types and for all purposes) should believe, know, and do regarding assessment
After defining these, it is the goal of the MAC (and an essential part of its mission) to help various users become more assessment literate
Becoming assessment literate is important, for a variety of reasons
The MAC’s work is important for both pre-service and in-service educators
Use of the Assessment Literacy Standards

47. Benefits of MAC Membership
Access to…
A rich library of assessment resources
Invitations to assessment professional learning opportunities
Video conferences and digitized content designed to enhance assessment literacy
An emerging catalogue of model, curriculum-embedded performance assessments

48. Edward Roeber
Assessment Director
Michigan Assessment Consortium
(517)
For More Information