University of Delaware Workshop presented at PBL 2006 Elaboration and Development of Problems and Scenarios in PBL Courses: Part II Deborah Allen Richard Donham & George Watson University of Delaware Workshop presented at PBL 2006 Lima, Péru 18 July, 2006

Workshop Overview Effective PBL Problems Grading “Grady” Levels of understanding Development of PBL Problems Step-by-step writing of your problem starter Sharing your ideas Some pointers to finish the problem Assessment in a PBL context Some Resources

What are the features of effective PBL problems? Take 5 minutes to discuss this question with your group.

Effective PBL Problems… relate to real world, motivate students require decision-making or judgments are multi-page, multi-stage are designed for group-solving pose open-ended initial questions that encourage discussion incorporate course content objectives, higher order thinking

Bloom’s Cognitive Levels Evaluation - make a judgment based on criteria Synthesis - produce something new from component parts Analysis - break material into parts to see interrelationships Application - apply concept to a new situation Comprehension - explain, interpret Knowledge - remember facts, concepts, definitions Bloom, B.S., ed. (1956) Taxonomy of Educational Objectives.

Bloom’s Taxonomy Revised Knowledge and cognitive domains combined; synthesis elevated above evaluation. Anderson and Krathwohl, eds. (2001). Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives.

Taxonomy of Significant Learning “Significant Learning” defined as lasting, important change in learner - not just cognitive Interactive rather than hierarchical levels L. Dee Fink (2003). Creating Significant Learning Experiences : An Integrated Approach to Designing College Courses

Six Facets of Understanding Sophisticated, knowledge-based justification of observations and ideas Explanation Subtle, thorough grasp of meaning of texts, data, events Interpretation Authentic use of ideas, processes in new/diverse contexts Application Critical analysis from different viewpoints Perspective Sensitivity to others Empathy Awareness of limitations of one’s knowledge, biases and habits Self-knowledge Wiggins and McTighe (1998). Understanding by Design.

Grading “Grady” Think back on the “Deflating Grady” problem. To what extent does it capture the features of an ideal PBL problem? Use the rubric to evaluate “Grady.” Discuss your evaluation with your group members and be prepared to report out.

Rubric to Evaluate PBL Problems Descriptors Criteria 3 2 1 Structure Progressive disclosure via multiple stages, builds on existing student knowledge. Staging does not flow well; transition could be improved. Too much or too little information provided at once; short cuts thinking/research. Questions Limited in number, short, and open-ended; encourage deeper understanding. Most are directive; preempt student-generated learning issues. Lead to “yes-no” answers rather than thoughtful discussion. Research Promotes substantive research using multiple resources. Research limited to textbook material. Limited necessity for research.

Writing Your Own Problem Starter

Step One: Identify Learning Objectives Think of a learning objective in your course. How do you usually address this learning objective? What kind of problem or activity do you usually assign? Typical end-of-chapter problem? A reading? Other?

Example from Physics: Solving Problems Using Conservation of Momentum Traditional examples: Pool balls colliding Bullets hitting blocks of wood

Example from Physics: Traditional End-of-Chapter Problem A 1500-kg car traveling east with a speed of 25 m/s collides at an intersection with a 2500-kg van traveling north at a speed of 20 m/s. Find the direction and magnitude of the velocity of the wreckage after the collision, assuming that the vehicles undergo a perfectly inelastic collision (ie, they stick together). Serway and Faughn. 3rd ed. College Physics, Saunders, 1992.

Example from Criminal Justice: Plea Negotiation Central concept in class on courts Most cases resolved through plea bargains Relates to other central course material Attorney and litigant roles, charging, sentencing Traditional approach Stand and deliver lectures Emphasis on facts of plea bargaining

Example from Biology Objectives: Compare and contrast the processes of respiration and photosynthesis, and connect these cellular events with their roles in global biogeochemical cycles Traditional approach: Separate chapters on respiration, photosynthesis, and global energy and carbon cycles; for majors, often taught in separate semesters

Types of Learning Objectives Content-oriented: subject specific Basic knowledge and understanding of specific concepts, techniques, etc. in the discipline Process-oriented: global skills Effective communication: oral and written Acquiring and evaluating information Working effectively with others Higher order, critical thinking

Problem Writing Step One: Identify Learning Objectives Think of a major learning objective that your problem can address. Traditionally, how might this learning objective be addressed? What kind of problem or activity would you assign? Typical end-of-chapter problem? A reading? Other?

Step Two: Identify Real-World Context Name a realistic application of the concept. Outline a scenario. Ideas: Add story-telling to end-of-chapter problem. Add motivation, require students to go beyond rote learning, do research. Include decision-making, analysis, or both. Other?

A Real Traffic Accident Based on police sketch Students need to make assumptions and approximations Information given gradually throughout problem

A Real Solution to Global Warming? John Martin discovered that high nutrient, low chlorophyll (productivity) ocean waters are missing an essential micronutrient: iron. “Give me a tanker full of iron, and I’ll give you an ice age.” Original decision: Should the government fund attempts to test if the “Geritol solution” can reduce the impact of excess CO2 emissions? Newest decision: Should for-profit companies be allowed to lease or own portions of the ocean to sell sequestered carbon? Who owns the rights to “the solution?”

Step Two: Identify Real-World Context Name a realistic application of the concept. Outline a scenario. Ideas: Add story-telling to end-of-chapter problem. Add motivation, require students to go beyond rote learning, do research. Include decision-making, analysis, or both. Other?

Two Schemes for Writing Problems? Think of one or more learning objectives in your course Name a realistic application of the concept(s). Outline a scenario. Think of a realistic scenario from the news, a videotape, or popular press article What learning objectives for your course are evident in the scenario?

Step Three: Draft the problem Outline the problem (create a ‘story board’) First consider, “What will be on the first page?” Suggestions: Good PBL problem has multi-page, multi-stage construction - leave students guessing! Not all information given in chapter or text - students look for resources. Challenge students to come to consensus, reach conclusions, and make judgments.

John Henry - Traffic Cop Stage 1: What questions need to be answered? What measurements, data? What physics principles? Then initial introduction to momentum. Stage 2: Sketch given, some information given; students analyze and ask questions. Stage 3: Outline procedure, make assumptions, Apply concepts. Stage 4: Make judgment and rationalize decision based on physics principles

Stages of Plea Negotiation Stage 1: Students are formed into groups, learn about case facts Stage 2: They choose roles, identify negotiation priorities and resources, research law Stage 3: Prosecution team makes initial charge, and negotiation begins Stage 4: Groups arrive at plea agreement, write up group report of negotiations

Consider the Following Problem Types: Explanation or Analysis Problems ‘What is going on here?’ Decision or Dilemma Problems ‘What would you do?’ ‘What do you think?’ Task-Oriented Problems Doing an activity or carrying out a project - for example, interviewing patients or designing a brochure

Another Scheme for Categorizing Problems - Based on Types of Knowledge Type of Knowledge Descriptive Explanatory Type of Problem Fact-finding Explanation Examples Following changes related to land use in Zimbabwe, many internal borders changed. People in the 15th century used to believe it was possible to fall off the earth. Example of Question What would a legal map look like? Explain why? Source: M. Savin-Baden and C. Major. 2004. Foundations of PBL. Berkshire, England: Open University Press.

Types of Problems Based on Type of Knowledge (cont.) Procedural Personal Type of Problem Strategy Moral dilemma Examples A 45-year old woman cannot lift her arm more than 45 degrees and she complains of pins and needles in her arm. A mother breaks into a drugstore to obtain expensive, life-saving drugs for her child. The next day she tells her physician what she has done. Example of Question If you were this client’s therapist, what would you do? What should the physician do?

Step Three: Draft the problem Outline the problem (create a ‘story board’) First consider, “What will be on the first page?” Suggestions: Good PBL problem has multi-page, multi-stage construction - leave students guessing! Not all information given in chapter or text - students look for resources. Challenge students to come to consensus, reach conclusions, and make judgments.

Problem Progress Report On overhead: Basic features of course Objectives for student learning Synopsis of problem Scenario for first page, including guiding questions What comes next?

Feedback To Presenters What did you like most about the problem? Does it challenge students to think and do research? Is the problem appropriate for the proposed course?

Writing PBL Problems Start with a Story Start with the Concepts Research Research What concepts are Look for the story to use involved? Research the Problem First Draft (Point of view, focus, appropriateness for audience, staging, objectives nature of the end-of-stage questions) Research, Draft II (refine) Teaching Draft III From C. F. Herreid, SUNY Buffalo & W. Welty, Pace University

A Final Consideration for Today: How would you assess to see if students met your original learning objectives?

What Is Assessment? “An assessment is an activity, assigned by the professor, that yields comprehensive information for analyzing, discussing, and judging a learner’s performance of valued abilities and skills.” Huba and Freed, Learner-Centered Assessment on College Campuses: Shifting the Focus from Teaching to Learning, 2000 In this definition we see roles for assessment beyond simply monitoring and judging the acquisition of knowledge: there is also a need for an analysis of student learning: How is it happening? Where are the barriers to full understanding? What is working well for the student - can those approaches be exploited more fully? What assessment activities are best suited for generating this type of information? Well-designed assessments can help us understand how our students are learning: the real audience for that information, however, should be the students themselves. One of the goals of problem-based learning, after all, is to develop in students the ability to become independent learners. They, even more than we as instructors, need to understand not only how they’re learning at present, but what they can do to improve that learning. This requires a discussion: feedback from student to instructor to diagnose the student’s learning, followed by feedback from instructor to student both about how well they’re doing and about how to improve in the future. We will look briefly at the practice of using rubrics both to set standards of expectation and to provide feedback to students.

First Principle for Assessing Student Learning Assessment of student learning should be directly connected to the learning objectives of your course.

How Will You Tell If They’ve Learned It? Summative assessment Traditional grading for accountability Usually formal, comprehensive Judgmental Formative assessment Feedback for improvement/development Usually informal, narrow/specialized Suggestive

Adapting the Traditional Exam Question to PBL: Probing Critical Thinking Skills Goal: to design a question that: goes beyond simple knowledge or comprehension uses novel situation or “real world” context involves multiple concepts requires recognition of concepts involved (analysis), their roles here (application), and how several ideas come together (synthesis)

Traditional Exam Questions: Chemistry Examples Calculate the vapor pressure of a solution of 5.8 g of NaCl in 100 g of water. Bloom Level: Knowledge Explain why a solution of NaCl will have a lower vapor pressure than pure water. Bloom Level: Comprehension

A Higher-order Exam Question The relative humidity inside a museum display case can be maintained at 75.3% by placing within the case a saturated solution of NaCl (containing excess solid NaCl). Explain, in molecular level terms, why the humidity remains constant - even when water-saturated air (100% humidity) diffuses into the case. Susan Groh, UD Chemistry & Biochemistry

Assess at Several Bloom Levels: Example from Chemistry Analysis of exam components # of points Total Points (Grade) +9 9 (F) +36 45 (D-) +22 67 (C+) +20 87 (A-) +9 96 (A) +4 100 Knowledge Comprehension Application Analysis Synthesis Evaluation Susan Groh, UD Chemistry & Biochemistry

PBL Problem Assignments You have drafted a PBL problem for your course. It addresses important concepts in a substantive context. Students will learn, understand, and remember. Why consider a PBL assignment separate from a PBL problem? Isn’t the problem the assignment?

Closure PBL problems are typically open-ended. Student within a single group often pursue different learning issues. Students frequently have difficulty knowing what the instructor thinks is important. Assignments focus on central issues and bring closure to PBL problems

Considerations for PBL Assignments Individual or group response? Course/problem content goals? General education and PBL goals? Graded or not? Difficulty? (Bloom’s Taxonomy) Interest level for students? Form of the response?

Types of PBL Problem Assignments Written assignments Write a 2-3 page paper on a learning issue you researched. Write a dialogue about the controversy presented by this problem. Transform an experiment described in this article into a laboratory exercise appropriate for undergraduates.

Types of PBL Problem Assignments Written assignments (cont.) Write a letter to the editor of your local newspaper on this issue. Compose a 200 word abstract for the article you have just read based on the Guidelines for Authors in Cell Biology. Research a topic related to the course and write a PBL problem for this course based on that topic.

Types of PBL Problem Assignments Student Presentations Prepare a case for your position and present it to the class as a debate. Prepare a 15-minute informative talk on this topic using PowerPoint slides.

Types of PBL Problem Assignments Written and visual assignment Create a website appropriate for high school students on this topic. Prepare a concept map that incorporates a major issue related to this problem.

What is a Concept Map? A pictorial representation that shows the relationships between and among a connected set of concepts and ideas

Purposes for Concept Mapping Generate ideas (brainstorming) Design complex structures (long texts, web-sites) Communicate complex ideas Aid learning by explicitly integrating new and old knowledge Assess understanding or diagnose misunderstanding

A Mini Concept Map photosynthesis consists of light-dependent reactions light-independent reactions that require Concepts Hierarchy Propositional linkages Also, cross-links water pigments light energy

A Map-Making Strategy: Think of all the words that apply, and write them in a list Prioritize, and transfer priority words to Post-Its Rearrange Post-Its to construct interrelationships, hierarchies Review and Discuss Establish propositional linkages, cross-links

Today’s Final Assignment Consider the course or problem you are working on and how you are going to assess student learning in light of the objectives you have. Write a synopsis of an idea for an exam question or assignment appropriate for the objectives of your problem starter from today.

Sources and Strategies for Writing Problems Newspaper articles, news events Popular press in the discipline Make up a story – based on content objectives Adapt a case to a problem Research papers Other?

PBL Clearinghouse An online database of PBL articles and problems. All material is peer-reviewed by PBL practitioners for content and pedagogy. All problems are supported by learning objectives and resources, teaching and assessment notes. Holdings are searchable by author, discipline, keywords, or full text. Fully electronic submission, review, and publication cycle. Controlled access by free user subscription, students excluded.

Reflections and Questions

Problem Writing Step 4 Continue to draft the storyline beyond a sketchy scenario – begin to write the first stage Think about point-of-view, nature of the end-of-stage questions, suitability for ‘audience,’ alignment with problem and course objectives Convene at to hear progress thus far at time announced.

Some Additional Considerations for Step 4 A good PBL problem leaves just the right information out! Challenge students to come to consensus, reach conclusions, and make judgments.