The Smarter Balanced Assessment Consortium Using the Individual Contributor and Reviewer Report

Agenda Page 2 Log into the Digital Library and navigate to the report. Select an SNE member and run a report. Interpret the results displayed in each section of the report.

Page 3

Logging into the Digital Library 4

Navigating to the Individual Contributor and Reviewer Report 5

Running a Report – Default Settings 6

Running a Report – Choose a Time Period 7

Running a Report for SNE Gabe Kotter 8

Three Sections – SNE Profile Information 9

Three Sections – Contributed Resources 10

Three Sections – Resource Reviews 11

Gabe’s SNE Profile 12

Contributed Resources Page 13

Interpreting Contribution Information Page 14 The Resources Contributed metric displays the number of resources that Gabe has submitted for review.

Interpreting Contribution Information Page 15 The Contributed Resources Posted metric displays the number of resources that have passed the Quality Criteria review process and are currently posted to the Digital Library.

Interpreting Contribution Information Page 16 The Contributed Resources Posted with Distinction metric displays the number of resources that have passed the Quality Criteria review process and are currently posted to the Digital Library with distinction.

Interpreting Contribution Information Page 17

Post Status SubmittedThe resource is awaiting review. In ReviewThe resource is currently being reviewed. PostedThe resource has been successfully posted to the Digital Library. Posted with DistinctionThe resource has been successfully posted to the Digital Library with distinction. ReturnedThe resource has been returned to the SNE for revisions. Page 18

Interpreting Contribution Information Page 19

Resource Reviews Tables Page 20

Interpreting Review Information Page 21

Interpreting Review Information Page 22 The Accepted Reviews column displays the number of resources that Gabe passed through the Gate-Keeping review, indicating they met all these criteria.

Interpreting Review Information Page 23 The Returned Reviews column displays the number of resources that Gabe did not pass through the Gate-Keeping review, indicating they did not meet all these criteria.

Interpreting Review Information Page 24 The Total Reviews column displays a sum of the Accepted Reviews and Returned Reviews.

Interpreting Review Information Page 25

Interpreting Review Information Accepted ReviewsReturned ReviewsTotal Reviews SNE completes Gate-Keeping review Returns the resource to the contributor Increases by 1 SNE completes Gate-Keeping review Accepts the resource and moves it forward Increases by 1 Page 26

Interpreting Review Information Page 27

Interpreting Reviewer Information Page 28

Interpreting Reviewer Information Page 29 The Reviews in Progress column displays the number of resources for which Gabe is currently conducting Quality Criteria reviews.

Interpreting Reviewer Information Page 30 The Completed Reviews column displays the number of resources for which Gabe has completed Quality Criteria reviews.

Interpreting Reviewer Information Page 31 The Average Consistency Rate for All Completed Reviewers shows the average recommendation consistency rate for all resources reviewed in the selected time period.

Interpreting Reviewer Information Page 32 Reviews in ProgressCompleted Reviews SNE starts a new Quality Criteria review Increases by 1 SNE cancels that same Quality Criteria review Decreases by 1 SNE completes that same Quality Criteria review and makes a recommendation Decreases by 1Increases by 1

Interpreting Reviewer Information Page 33

Interpreting Reviewer Information Page 34

Interpreting Reviewer Information Page 35

Interpreting Reviewer Information Page 36 R+ (Recommend with Distinction) R (Recommend) R- (Recommend with Revisions) NR (Do Not Recommend)

Interpreting Reviewer Information Page 37

Consistency Rate Calculation Example Recommend (R) Recommend with Distinction (R+) GabeMariaTony Do Not Recommend (NR) Recommend with Revisions (R-)

Consistency Rate Calculation Example Recommend (R) Recommend with Distinction (R+) GabeMariaTony Do Not Recommend (NR) Recommend with Revisions (R-) NR = 1 R- = 2 R = 3R + = 4 Numerical Values Do Not Recommend (NR) = 1 Recommend with Revisions (R-) = 2 Recommend (R) = 3 Recommend with Distinction (R+) = 4

Consistency Rate Calculation Example Recommend (R) Recommend with Distinction (R+) GabeMariaTony Do Not Recommend (NR) Recommend with Revisions (R-) Difference of 2 Difference of 1 Numerical Values Do Not Recommend (NR) = 1 Recommend with Revisions (R-) = 2 Recommend (R) = 3 Recommend with Distinction (R+) = 4 NR = 1 R- = 2 R = 3R + = 4

Consistency Rate Calculation Example Reviewer Rating Divergence from Gabe Divergence from Tony Divergence from Maria Total Divergence from Other Reviewers GabeR = Reviewer BNR = Reviewer CR- = Recommend (R) 4 Recommend with Distinction (R+) GabeMariaTony 1 Do Not Recommend (NR) 2 Recommend with Revisions (R-) Difference of 2 Difference of 1

Consistency Rate Calculation 100% – Total Divergence from Other Reviewers Maximum Possible Divergence from Other Reviewers ()

Consistency Rate Calculation 100% – Total Divergence from Other Reviewers = 3 Maximum Possible Divergence from Other Reviewers = 6 () Reviewer Rating Divergence from Gabe Divergence from Reviewer B Divergence from Reviewer C Total Divergence from Other Reviewers GabeR (3)0213 Reviewer BNR (1)2013 Reviewer CR- (2)1102 = 50%

Consistency Rate Page 44 Consistency RateExplanation 100% The review is highly aligned with the other two SNE reviewers. 83% 67% The review is somewhat aligned with the other two SNE reviewers. 50% The review is not very aligned with the other two SNE reviewers. 33% 17% 0% The review is not at all aligned with the other two SNE reviewers.

Interpreting Reviewer Information Page 45

Interpreting Reviewer Information Page 46

Interpreting Reviewer Information Page 47

Interpreting Reviewer Information Page 48

Looking at SNE Reviewers’ Comments Page 49

Looking at SNE Reviewers’ Comments Page 50

Looking at SNE Reviewers’ Comments Page 51

Looking at SNE Reviewers’ Comments Page 52

Reflecting on the Learning Page 53 Log into the Digital Library and navigate to the report. Select an SNE member and run a report. Interpret the results displayed in each section of the report.

Congratulations! You have completed the tutorial for using the Individual Contributor and Reviewer Reports. Page 54 Please contact the Help Desk, if you have any questions. Monday-Friday 7:00am-7:00pm (EST) (toll free)