1. Calibrated Peer Review TM Orville L. Chapman, Arlene A. Russell, and Michael A. Fiore University of California, Los Angeles

2. CPR Design n CPR design is based on peer review of scientific proposals and manuscripts. n CPR permits only competent, calibrated peer review of student documents n CPR ensures careful reading for content. n CPR makes students conscious of style issues. n CPR enables learning by writing.

3. Motivation for Creating CPR n Science Literacy n Constructivist Learning n Critical Thinking

4. Science Literacy

5. Literacy n Discourse based on comprehension, understanding, organization, and logic.

6. Construction of Knowledge n Integration of new knowledge n Organization of knowledge n Critical evaluation of new information

7. Critical Thinking n Rigorous thinking n Testing ideas and data against existing intellectual models n Rejecting intellectual models n Creating new intellectual models

8. Bloom’s Taxonomy of the Cognitive Domain n Evaluation--highest level n Synthesis n Analysis n Application n Comprehension n Knowledge--lowest level * * * * * CPR

9. What Is CPR? n CPR is a network learning tool. n CPR is accessed through an Internet browser. n CPR is platform independent. n CPR enables writing assignments in any class, regardless of size or subject. CPR works even with the largest classes. n CPR enables shared writing assignments among institutions over the Internet.

10. Two factors Make CPR Special: n CPR is discipline independent. Art, anthropology, English, social science, history, economics, business, law, chemistry, physics, life science, and mathematics are using it. CPR can serve any curriculum. n CPR is level independent. It serves grades 6-12, colleges and universities, graduate and professional schools.

11. What Does CPR Comprise? n Student interface n Instructor interface n Assignment authoring tool n Reporting tool n Assignment management system n Assignment libraries

12. How Does CPR Work? n The instructor gives the student a writing assignment. n The student submits his or her document to CPR electronically. n CPR then gives the student three calibration documents for review. n When the student has completed the calibration, CPR provides a detailed report on the student’s performance.

13. How Does CPR Work? n After the student has successfully completed the calibration, CPR provides three peer documents for review. n CPR then gives the student his or her document for self-review. n Finally, CPR provides a detailed report of the peer review and the self-review.

14. The Review Process n The review process is the same for calibration, peer review, and self-review. n The student answers content questions for each document. n The student answers style questions for each document. n The student assigns a score to each document on a 1 to 10 basis.

15. Student Reports n summary of the calibration results summary of the calibration results

16. Instructor Reports n CPR provides a detailed report on each individual student’s work. n CPR delivers a detailed summary report for the class. n CPR reports are broken down among calibration, peer reviews, and self-review. n Students are scored on their writing and their reviewing.

17. Assignment Libraries n Molecular Science Library n CSUN Business School Library n Libraries enable frequent writing assignments with minimal instructor time.

18. Creating CPR Assignments n Identify important course topics. n Locate appropriate sources for student learning. n Compose appropriate guiding questions. n Write the calibration documents--one should be an exemplar. n Create carefully crafted content and style questions. n Have peers review your assignment.

19. Creating CPR Assignments: n Start by reading “Workshop Manual and Writing CPR Assignments,” by Arlene Russell. n To obtain a copy of the manual or to get advice on creating CPR assignments, contact Arlene Russell: russell@chem. ucla.edu.

20. Collaborative Library Building n Among institutions (NM CC’s; Other CC’s) n Within an institution (CSUN Business School) n Within a project (MSC consortium) n Within a school cluster (Narbonne-San Pedro School Cluster, ~24,000 students). n Publishing Companies n Consortia and individual partnerships

21. If You Create CPR Assignments, n You will change the way you teach. n You will sharpen the focus on what you want your students to learn. n You will become more effective. n You will be forced to ponder issues that you are currently ignoring. n You will be forced to define your assessment strategy first.

22. How We Use CPR n Create a scientific database. n Design an exploration of that database. n Give the students guiding questions that lead them to ask the questions that really matter. n Assign a writing task. n Create carefully thought out content and style questions. n Have colleagues criticize the content and edit the assignment.

23. A Useful Guide n John C. Bean, “Engaging Ideas: The Professor’s Guide to Integrating Writing, Critical Thinking, and Active Learning in the Classroom,” Jossey-Bass, San Francisco, 1996

24. What Does CPR Achieve? n Students learn science by writing. They construct meaning. n Students develop their writing skills. n Students learn to evaluate peer writing. n Students learn abstracting, reading for content, and self-evaluation. n Students think more deeply about important aspects of what they are studying.

25. Using CPR, Students Learn n Observation through description. n Comprehension and understanding through analysis and clear, precise expression. n Evaluation through critical assessment of peer work. n Synthesis and argument through organization and logical expression.

26. CPR Enables n Development of an instructor’s own writing assignments, at a cost of about 6-8 hours per assignment using the authoring tool. n Collaborative assignment development over the net. n Significant student writing with little instructor effort (about 5 minutes per assignment), using assignment libraries. n Cooperative distance-learning courses

27. An AP Chemistry Course, Arlene Russell n A one-year AP course, which bears credit, for rural high school students is being distributed as distance learning. n This project is funded by the Office of the President of the University of California. n This course is designed around use of learning objects from the Molecular Science Curriculum project and CPR.

28. Does CPR Work? n A study from CSUN Business School: n Two economics classes, each about 40 students, were selected for the study. n Both classes took the same examinations. n On each examination, two questions were taken from material not covered by CPR assignments.

29. Does CPR Work? n On each examination, two questions covered material that the test class had studied as CPR assignments and the control class had learned in lecture and discussion sections. n On the essay questions in which neither class had access to CPR assignments, the two classes were indistinguishable.

30. Does CPR Work? n On essay questions in which one class had used traditional methods and the other class used CPR, the CPR class scored 60% better than the traditional class. n These results were confirmed in a second test of the same design.

31. What Makes CPR Special? n CPR is discipline independent. n CPR is level independent. n CPR has been used by more than 10,000 students.

32. CPR Funding n The National Science Foundation through the initiative for reform in chemical education n The Howard Hughes Medical Institute, for applications in life science

33. If you have questions about CPR, Contact Orville L. Chapman, at chapman@chem.ucla.edu, or Arlene A. Russell, at russell@chem.ucla.edu. You may access the slides and text for this presentation at http://www.molsci.ucla.edu/chapman/cpr.htm.