1. Activities Review Process On the Cutting Edge Workshop on Teaching Hydrogeology, Soils, and Low-T Geochemistry in the 21 st Century

2. HSG Review Management Team Devin Castendyk SUNY, Oneonta Managing Editor John McDaris Science Education Resource Center (SERC) Barbara Tewksbury Hamilton College Cutting Edge PI Maddy Schreiber Virginia Tech Associate Editor

3. The collections  SERC: Science Education Resource Center at Carleton College  SERC office and staff helps develop and manage web resources for many projects through collaboration  Many different collections of activities, submitted for different projects  On the Cutting Edge: the first of the projects hosted by SERC  Most of the activities in the Cutting Edge collections were submitted in connection with workshops.

4. Cutting Edge Reviewed Collection  On the Cutting Edge is conducting a review of activities in the Cutting Edge collections  Each activity reviewed twice and ranked:  Exemplary (Part of Reviewed Collection)  Pass (Part of Reviewed Collection)  Keep (Not part of Reviewed Collection)  De-accession

5. Cutting Edge Reviewed Collection  Activities ranked as “Exemplary”  Come up first in searches  Are designated on individual the ActivitySheet as being part of Exemplary Teaching Activities collection

6. Cutting Edge Reviewed Collection  Activities ranked as “Pass”  Come up second in searches  Are designated on individual ActivitySheet as being part of the Peer Reviewed Teaching Activities collection

7. Cutting Edge Reviewed Collection  Activities ranked as “Keep”  Come up last in searches  Have no designation on the ActivitySheet  Worth keeping as catalyst idea  Might be too local or items might be missing or has other problematic aspects

8. Review process  Any activity tagged with hydro, soils, low-T geochem, biogeochem, or ES that has not been reviewed yet  Each activity receives 2 reviews  Authors of “Exemplary” and “Pass” activities receive letters  Explains review process  Indicates activity rank  Indicates that reviewer comments are available on request if the author wishes to revise  If author does revise, the activity will be reviewed again

9. Plan for webinar  Explain the web interface  Clarify review criteria  Answer questions  As we go along, please post questions in chat

10. Your list of items  When you click on Review Tool on Review Team Instructions page, your login will take you to a page that lists only your items to review http://serc.carleton.edu/NAGTWorkshops/hydrogeo/HSG2013/review_team.html

11. Your list of items  If you have not yet completed the review, you will see:  Click the URL to go to the ActivitySheet and download the actual activity and any supporting materials.

12. The review tool  Click Review It to bring up the Review Tool

13. The review tool  You will evaluate the activity in five categories  Scientific accuracy  Alignment of goals, activity, and assessment  Pedagogic effectiveness  Robustness  Activity description

14. The review tool  For each category, questions plus rubric provide guidance for what to consider

15. The review tool  Summary score will tabulate automatically  Exemplary = 4; Very good = 3; Adequate = 2; Problematic = 1  Comments help the editors understand your ranking – please don’t leave these boxes blank!!

16. The review tool  At the end of the form, you will add your view about what it would take to raise the activity to Exemplary status if it fell short in your review  The editors will use your comments to respond to authors on request. Please phrase your comments in a collegial fashion.

17. Your list of items  Once you have submitted a review, your review list indicates completion for that item

18. Can you revise a review?  Yes – click on the Review It link and then the link to what you submitted previously  Your original rankings will come up, and you can change them and add to/change your comments.  Click submit when done.

19. Pause for questions  Anyone??

20. Your review  Review the activity in the context for which it was designed  Not just whether it’s good for a particular upper level course – many will be for other courses (e.g., intro geo)  Not everything has to be a full lab or major assignment (e.g., a back-of-the- envelope calculation could be Exemplary)  Not every activity needs to be usable by all instructors (e.g., a lab requiring specific software/math/expertise background)

21. Your review  Make a summary list of the activities that you reviewed  List both the total numerical score and your overall assessment  Exemplary  “Exemplary minus”  Pass  Keep  De-accession  Bring with you to review team meeting

22. Your review  Exemplary  Must have good science, good pedagogy, and all materials so that someone else can adapt/adopt, nothing “broken”  Can be “local” if it is also a good template  Does not need to have answer key or to provide an instructor with background  We have never required these so cannot ding someone for not including them  Scoring  Exemplary or very good in all categories  Exemplary in at least three of the five.  18 or higher.

23. Your review  Exemplary minus  Could be made Exemplary with only a small amount of work, such as:  fixing a URL  uploading the latest version of the assignment or adding instructor tips  fleshing out the ActivitySheet  This is not a formal category, but it would help us a lot to have your list of “Exemplary Minus” activities  These will be ranked as Pass, but knowing that they are “Exemplary minus” will help the editors craft feedback if authors request it.

24. Your review  Pass – these become part of the Reviewed Collection  Those that aren’t Exemplary but still have value to others  Must be more than just the germ of an idea  Must have all of the components  These must have no scientific errors.  If you think there are errors, confirm this with someone else on the review team.  Those with scientific errors should go into the Keep or De-accession category, depending on the severity of the problem.

25. Your review  Keep – no designation on ActivitySheet, will come up last in a search  Nucleus of a good idea  Insufficient info for someone to adapt or adopt or has scientific errors  Author does not receive a letter

26. Your review  De-accession  Not an activity or very fragmentary  Has truly egregious problems

27. Summary  Review each activity using rubric  Score the activity in each of 5 categories  Write a summary evaluation for each  Remember that these were submitted voluntarily to a community collection  Be kind but clear  Make a summary list  Exemplary  Exemplary minus (technically a Pass)  Pass  Keep  De-accession

28. Your assignment  Each team member has 10-11 activities to review  Reviews must be completed and submitted using review tool before the review team meeting in ABQ  Meeting in ABQ on June 4 at 8:30 am  Discuss issues  Resolve discrepant reviews  Arrive at final rankings

29. Thank you!  Email Barb with any questions  btewksbu@hamilton.edu