The Smarter Balanced Assessment Consortium Training 2-Day 1 The Digital Library Review Process Using the Quality Criteria

Introductions and Success Stories Introductions by State Share one success from Training 1 Share one lesson learned from Training 1 Slide 2

Digital Library Resources Slide 3 Commissioned Professional Learning Modules Resources for educators, students and families Frame Formative Assessment within a Balanced Assessment System Articulate the Formative Assessment Process Highlight Formative Assessment Practices and Tools Assessment Literacy Modules Commissioned Professional Learning Modules Instructional coaching for educators Instructional materials for students Demonstrate/support effective implementation of the formative assessment process Focus on key content and practice from the Common Core State Standards for Mathematics and English Language Arts Exemplar Instructional Modules High-quality vetted instructional resources and tools for educators High-quality vetted resources and tools for students and families Reflect and support the formative assessment process Reflect and support the Common Core State Standards for Mathematics and English Language Arts Create Professional Learning Communities Education Resources

Our Successes So Far Formative Assessment Advisory Panel Developed Quality Criteria State Leadership Teams and State Networks of Educators Established, Involving Around 2,000 K-16 Educators Across the Country SLT Training 1 and SNE Training 1 Completed Over 450 Resources Submitted to the Digital Library Blueprints for Exemplar Instructional Modules and Assessment Literacy Modules Blueprints Drafted Matrices for the Exemplar Instructional Modules for ELA and Math and for the Assessment Literacy Modules Drafted Digital Library Software Versions 1.0 and 1.1 Released

What’s Coming Next SLT Training 2 in November and December 2013 Software Version 1.15 in February 2014 Short-Term SLT Trainings 3 (March 2014), 4 (May 2014), and 5 (August 2014) Software Version 1.2 in April 2014 First set of 23 Exemplar Instructional Modules and Assessment Literacy Modules in March 2014 Second set of Modules in May 2014 and third set of Modules by September 2014 Long-Term Digital Library Open to Consortium Members in April 2014 Milestone

Training 1 Follow-up SLT Webpage Updates – Updated Cover Profile Guide – Tips for Identifying the License Presentation Role Overview in the Digital Library – What can I view as an SLT member? SNE Submissions – What should be submitted by the end of November? Slide 6

SNE Individual Report Review Slide 7

Training 2: The Digital Library Review Process Using the Quality Criteria for Professional Learning and Instructional Resources Slide 8

SLT Training 2 Learning Goal SLT members will understand the Quality Criteria ratings and how to apply them consistently. Success Criterion SLT members will provide evidence of inter-rater reliability among the state SLT members and with other state SLT members in cross-state teams. Slide 9

Formative Assessment Advisory Panel Convened the Formative Assessment Advisory Panel to develop the Quality Criteria during three two-day meetings Page 10 Meeting 1 April 17 – 18 Meeting 1 April 17 – 18 Meeting 2 May 8 – 9 Meeting 2 May 8 – 9 Meeting 3 May 22 – 23 Meeting 3 May 22 – 23 Brainstormed initial Quality Criteria Determined structure of Quality Criteria Developed comprehensive list of potential criteria Discussed merits of checklist vs. rubric-based approach Tested criteria using sample resources Refined criteria Synthesized feedback Developed first draft of Quality Criteria Developed 2 nd draft; received feedback from panelists; developed present draft Slide 10

Members of the Formative Assessment Advisory Panel 1)Dr. Lynne Anderson-Inman (University of Oregon) 2)Dr. Robert Calfee (Stanford University, UC Riverside) 3)Dr. Bridget Dalton (University of Colorado) 4)Dr. Diane Heacox (St. Catherine University) 5)Dr. Joan Herman (UCLA – CRESST) 6)Dr. John Hill (Purdue University) 7)Dr. Yvette Jackson (National Urban Alliance for Effective Education) 8)Dr. Henry Kepner (University of Wisconsin - Milwaukee) 9)Dr. Katherine McKnight (National Louis University) 10)Valerie L. Mills (National Council of Supervisors of Mathematics) 11)Dr. James Popham (UCLA) 12)Dr. Lucinda Soltero-Gonzalez (University of Colorado-Boulder) Page 11 Twelve experts in the CCSS for ELA, the CCSS for Mathematics, the formative assessment process, adult learning, online professional learning, diverse learners, and urban and rural education comprised the Formative Assessment Advisory Panel. Slide 11

Parts of the Quality Criteria Cover Profile Gatekeeping Criteria Quality Criteria for Professional Learning Resources and Quality Criteria for Instructional Resources Governance Criteria Slide 12 N/A

Purposes of the Quality Criteria Page 13 Ensure that all resources in the Digital Library are of the highest quality, regardless of source Ensure that all resources reflect the Smarter Balanced vision of effective formative assessment practices Provide consistency in the review process for all resources Ensure that the resources in the Digital Library reflect the intent of the CCSS

Resource Posting Work Flow Page 14 Step 1: Resource Submitted Step 3: Quality Criteria Applied Step 4: Decision Step 2: Gatekeeping Criteria Applied SNE 1 SNE 2 SNE 3 SNE 1 Posted Sent to SLT Returned to Submitter Cover Profile

A Close Reading of the Quality Criteria Slide 15 Individually Read and Annotate the Quality Criteria – Pages 2-5: Quality Criteria for Professional Learning Resources – Pages 6-9: Quality Criteria for Instructional Resources Think About and Discuss with Your Table Group – What are the differences between the two sets of criteria? – How might these criteria help focus the review process? – Where will your state need more clarity and information for Training 2? Whole Group Discussion

Additional Resources SLT Webpage: balanced-slt balanced-slt – Quality Criteria FAQ – UDL – Professional Learning Slide 16

Break and Login Distribution Slide 17

Accessing the Digital Library in the Training Environment Page Open your Internet browser (Google Chrome typically works best, though Safari, Firefox, and Internet Explorer 9+ are also options). 2. Type the web address: Type your assigned address: 4. Type the password: password 5. Click “Log into the Digital Library”

Common IR Review Read the Cover Profile Read the PDF Materials Watch the Video (whole group) Individually Rate the Resource Discuss Ratings with Your State Team Determine Ratings, Comments, Posting Recommendation, Rationale as a Team Slide 19

Common IR Review Debrief On your poster paper: – Resource Title – Criteria Ratings – Posting Rec – Rationale Discuss with the whole group: – Talk us through your team’s process when discussing and deciding on the ratings. Slide 20

Reflection 1.What criteria will need more discussion/clarification with your SNE members? 2.What are the areas of confusion about the Quality Criteria for Instructional Resources? 3.What recommendations do you have for technology improvements? Slide 21

SLT Training Cycles SNE Payment Last Day of Training Cycle PCG Sends “SNE Member Payment Spreadsheet” to SLT Lead SLT Returns Spreadsheet to PCG SNE Payment Due Training 1 October 30, 2013 November 18, 2013November 26, 2013December 20, 2013 Training 2 December 31, 2013 January 20, 2014January 28, 2014February 21, 2014 Training 3 April 30, 2014 May 19, 2014May 27, 2014June 20, 2014 Training 4 June 30, 2014 July 24, 2014July 19, 2014August 22, 2014 Training 5 Sept. 30, 2014 October 20, 2014October 28, 2014November 21, 2014 Slide 22

Cross-State Teams Review the Cover Profile Review Materials Vet the Resource Against the Quality Criteria Slide 23

Slide 24 On your poster paper: – Resource Title – Criteria Ratings – Posting Rec – Rationale Discuss with the whole group: What were notable differences to apply to SNE trainings? Cross-State Debrief

The Smarter Balanced Assessment Consortium Training 2-Day 2 The Digital Library Review Process Using the Quality Criteria

Day 1 Recap & Your Feedback You all are amazing!! Thank you for your honest conversations and collaboration. We want to address feedback from Tickets Out the Door. Slide 26

Lunch and Temp Tuscan Buffet – Vegetarian Minestrone – Salad – Rolls – Cavatelli Pasta with Peas and Mushrooms in a Creamy Pesto Sauce – Chicken Breast Piccata in a Light Lemon Butter Sauce – Grilled Italian Vegetables – Tiramisu Indicate Veggie Request Slide 27

Who’s the Learner? Consistent Question Workgroup Discussion Update SLT Webpage with the Answer Slide 28

Help Me Help You Instructional vs. Professional Resources Cover Profile Guide – General Tab: Page 2/2 Close Reading: Quality Criteria FAQ What do you need to make this more clear? Slide 29

Copyright Updated Copyright Functionality in the Digital Library Copyright Clearance Form Updated Copyright Information in Cover Profile Guide Tips for Identifying the License Presentation Slide 30

State share of ideas on how we can help SNE members feel more connected and maintain morale? Slide 31 Send Thank You Cards Highlight Successes Recognizing SNEs with Principals/Providing Certificates Reassurance of completion and payment Recognizing SNE First Approved Resource Badges

Common Professional Learning Resource Review Login to the training environment: – training.smarterbalancedlibrary.org Select Common Professional Learning Resource: Formative Assessment Learning Strategies Click and Review: About This Resource Slide 32

Common Professional Learning Resource Review 20 minutes – Review About this Resource – Watch Video (whole group) 20 minutes – Review pdf files – Individually vet the resource using the Professional Learning Quality Criteria 15 minutes – Discuss and compare answers with your team 5 minutes – Create your poster Slide 33

Debrief What were the notable differences in reviewing a Instructional Resource versus a Professional Learning Resource? Additional questions, comments, concerns? Slide 34

Posting Decisions When does an SLT have to make a resource posting decision? Slide 35

20 SNE Recommendation Combinations Page 36 DecisionSNE Recommendation Software or SLT Decision? Post with Distinction Recommend with Distinction Software Recommend with Distinction RecommendSoftware Post Recommend with DistinctionRecommend Software Recommend Software SLT Member Judgment Recommend with Distinction Recommend with RevisionsSLT Recommend with Distinction Do Not RecommendSLT Recommend with DistinctionRecommendRecommend with RevisionsSLT Recommend with DistinctionRecommendDo Not RecommendSLT Recommend Recommend with RevisionsSLT Recommend Do Not RecommendSLT Not Post; Return to submitter Recommend with DistinctionRecommend with Revisions Software Recommend with DistinctionRecommend with RevisionsDo Not RecommendSoftware Recommend with DistinctionDo Not Recommend Software RecommendRecommend with Revisions Software RecommendRecommend with RevisionsDo Not RecommendSoftware RecommendDo Not Recommend Software Recommend with Revisions Software Recommend with Revisions Do Not RecommendSoftware Recommend with RevisionsDo Not Recommend Software Do Not Recommend Software Please note: The decision depends on the combination of recommendations; the order of the recommendations does not matter.

Recommendation Combinations- SLT Member Decision Page 37 An SLT member will make a posting decision given the following recommendation combinations: SNE Recommendation Recommend with Distinction Recommend with Revisions Recommend with Distinction Do Not Recommend Recommend with Distinction Recommend Recommend with Revisions Recommend with Distinction RecommendDo Not Recommend Recommend Recommend with Revisions Recommend Do Not Recommend

SNE Recommendation Recommend with Distinction Recommend with Revisions Recommend with Distinction Do Not Recommend Recommend with Distinction Recommend Recommend with Revisions Recommend with Distinction RecommendDo Not Recommend Recommend Recommend with Revisions Recommend Do Not Recommend SLT Decision Scenario Page 38

SLT Member Quality Criteria Review Log into the “real” Digital Library Software Demo Resource Review tab Slide 39

Training 2 SLT Arbitration Exercise Materials Resource Cover Profile Three SNE Quality Criteria Reviews – Recommend with Distinction – Recommend – Recommend with Revisions SLT Arbitration Form Page 40

When does an SLT have to make a resource posting decision? Find someone you haven’t spoken to from a different table Share your table’s recommendation and rationale. Whole Group Share – What did you hear that was interesting? – Any new thoughts? Slide 41

Evaluation 2 with the link by 12:00pm Slide 42