Webinar for Activities Review Team On the Cutting Edge pre-GSA REVIEW CAMP Please mute your phone by pressing *6 Technical problems? Contact Krista at Webinar begins at: 1pm Eastern | 12 pm Central | 11 am Mountain | 10 am Pacific Review team instructions page at: September 25, 2013

THANK YOU! The Cutting Edge Review Process is: A tremendous community service; An opportunity for you to see a lot of great activities; An opportunity to see what goes into a quality activity to inform your own work; The realization of a decade-long goal of developing an online, peer-reviewed collection of activities

The collections  SERC: Science Education Resource Center at Carleton College  SERC office and staff helps develop and manage web resources for many projects through collaboration  Many different collections of activities, submitted for different projects  On the Cutting Edge: the first of the projects hosted by SERC  Most of the activities in the Cutting Edge collections were submitted in connection with workshops.

Cutting Edge Reviewed Collection  On the Cutting Edge is conducting a review of activities in the Cutting Edge collections  Each activity reviewed once and ranked:  Exemplary (Part of Reviewed Collection)  Pass (Part of Reviewed Collection)  Keep (Not part of Reviewed Collection)  De-accession

Cutting Edge Reviewed Collection  Activities ranked as “Exemplary”  Come up first in searches  Are designated on individual the ActivitySheet as being part of Exemplary Teaching Activities collection

Cutting Edge Reviewed Collection  Activities ranked as “Pass”  Come up second in searches  Are designated on individual ActivitySheet as being part of the Peer Reviewed Teaching Activities collection

Cutting Edge Reviewed Collection  Activities ranked as “Keep”  Come up last in searches  Have no designation on the ActivitySheet  Worth keeping as catalyst idea  Might be too local or items might be missing or has other problematic aspects

Review process  Activities that have been collected across the Cutting Edge collections  Each activity receives 2 reviews  Authors of “Exemplary” and “Pass” activities receive letters  Explains review process  Indicates activity rank  Indicates that reviewer comments are available on request if the author wishes to revise  If author does revise, the activity will be reviewed again

Plan for webinar  Explain the web interface  Clarify review criteria  Answer questions  As we go along, please post questions in chat

Your list of items  When you click on Review Tool on Review Team Instructions page, your login will take you to a page that lists only your items to review

Your list of items  If you have not yet completed the review, you will see:  Click the URL to go to the ActivitySheet and download the actual activity and any supporting materials.

The review tool  Click Review It to bring up the Review Tool

The review tool  You will evaluate the activity in five categories  Scientific accuracy  Alignment of goals, activity, and assessment  Pedagogic effectiveness  Robustness  Activity description

The review tool  For each category, questions plus rubric provide guidance for what to consider

The review tool  Summary score will tabulate automatically  Exemplary = 4; Very good = 3; Adequate = 2; Problematic = 1  Comments help the editors understand your ranking – please don’t leave these boxes blank!!

The review tool  At the end of the form, you will add your view about what it would take to raise the activity to Exemplary status if it fell short in your review  The editors will use your comments to respond to authors on request. Please phrase your comments in a collegial fashion.

Your list of items  Once you have submitted a review, your review list indicates completion for that item

Can you revise a review?  Yes – click on the Review It link and then the link to what you submitted previously  Your original rankings will come up, and you can change them and add to/change your comments.  Click submit when done.

Pause for questions  Anyone??

Your review  Review the activity in the context for which it was designed  Not just whether it’s good for a particular upper level course – many will be for other courses (e.g., intro geo)  Not everything has to be a full lab or major assignment (e.g., a back-of-the- envelope calculation could be Exemplary)  Not every activity needs to be usable by all instructors (e.g., a lab requiring specific software/math/expertise background)

Your review  Make a summary list of the activities that you reviewed  List both the total numerical score and your overall assessment  Exemplary  “Exemplary minus”  Pass  Keep  De-accession  Bring with you to review team meeting

Your review  Exemplary  Must have good science, good pedagogy, and all materials so that someone else can adapt/adopt, nothing “broken”  Can be “local” if it is also a good template or model for others to emulate  Does not need to have answer key or to provide an instructor with background  We have never required these so cannot ding someone for not including them  Scoring  Exemplary or very good in all categories  Exemplary in at least three of the five.  18 or higher.

Your review  Exemplary minus  Could be made Exemplary with only a small amount of work, such as:  fixing a URL  uploading the latest version of the assignment or adding instructor tips  fleshing out the ActivitySheet  This is not a formal category, but it would help us a lot to have your list of “Exemplary Minus” activities  These will be ranked as Pass, but knowing that they are “Exemplary minus” will help the editors craft feedback if authors request it.

Your review  Pass – these become part of the Reviewed Collection  Those that aren’t Exemplary but still have value to others  Must be more than just the germ of an idea  Must have all of the components  These must have no scientific errors.  If you think there are errors, confirm this with someone else on the review team.  Those with scientific errors should go into the Keep or De-accession category, depending on the severity of the problem.

Your review  Keep – no designation on ActivitySheet, will come up last in a search  Nucleus of a good idea  Insufficient info for someone to adapt or adopt or has scientific errors  Author does not receive a letter

Your review  De-accession  Not an activity or very fragmentary  Has truly egregious problems

Summary  Review each activity using rubric  Score the activity in each of 5 categories  Write a summary evaluation for each  Remember that these were submitted voluntarily to a community collection  Be kind but clear  Make a summary list  Exemplary  Exemplary minus (technically a Pass)  Pass  Keep  De-accession

Your assignment  Each team member has ~20activities to review  Reviews must be completed and submitted using review tool before the Review Camp  At Review Camp we will  Discuss issues  Resolve discrepant reviews  Arrive at final rankings  Hopefully do a few more reviews

Final Thoughts  Please write your reviews in the same spirit you would like to receive them  We all teach “out of field”—so don’t be dismayed if you have an activity not related to your immediate interests  Just use your best professional judgment.

Final Thoughts  The goal is to have a high quality, coherent, and comprehensive collection for use by all.  Acknowldege the great work that is available; credit and kudos to authors  Help authors improve their activities  Weed out those that are not ready for prime time

Thank you!  Dave or Krista with any questions    Have fun, stay in touch, post messages or questions to the listserv, and we’ll see you in Denver!