1. University of Delaware Workshop presented at PBL 2006
Elaboration and Development of Problems and Scenarios in PBL Courses: Part II
Deborah Allen
Richard Donham &
George Watson
University of Delaware
Workshop presented at PBL 2006
Lima, Péru
18 July, 2006

2. Workshop Overview Effective PBL Problems Grading “Grady”
Levels of understanding
Development of PBL Problems
Step-by-step writing of your problem starter
Sharing your ideas
Some pointers to finish the problem
Assessment in a PBL context
Some Resources

3. What are the features of effective
PBL problems?
Take 5 minutes to discuss this question with your group.

4. Effective PBL Problems…
relate to real world, motivate students
require decision-making or judgments
are multi-page, multi-stage
are designed for group-solving
pose open-ended initial questions that encourage discussion
incorporate course content objectives, higher order thinking

5. Bloom’s Cognitive Levels
Evaluation - make a judgment based on criteria
Synthesis - produce something new from component parts
Analysis - break material into parts to see interrelationships
Application - apply concept to a new situation
Comprehension - explain, interpret
Knowledge - remember facts, concepts, definitions
Bloom, B.S., ed. (1956) Taxonomy of Educational Objectives.

6. Bloom’s Taxonomy Revised
Knowledge and cognitive domains combined; synthesis elevated above evaluation. Anderson and Krathwohl, eds. (2001). Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives.

7. Taxonomy of Significant Learning
“Significant Learning” defined as lasting, important change in learner - not just cognitive
Interactive rather than hierarchical levels
L. Dee Fink (2003). Creating Significant Learning Experiences : An Integrated Approach to Designing College Courses

8. Six Facets of Understanding
Sophisticated, knowledge-based justification of observations and ideas
Explanation
Subtle, thorough grasp of meaning of texts, data, events
Interpretation
Authentic use of ideas, processes in new/diverse contexts
Application
Critical analysis from different viewpoints
Perspective
Sensitivity to others
Empathy
Awareness of limitations of one’s knowledge, biases and habits
Self-knowledge
Wiggins and McTighe (1998). Understanding by Design.

9. Grading “Grady”
Think back on the “Deflating Grady” problem. To what extent does it capture the features of an ideal PBL problem?
Use the rubric to evaluate “Grady.” Discuss your evaluation with your group members and be prepared to report out.

10. Rubric to Evaluate PBL Problems
Descriptors
Criteria 3 2 1
Structure
Progressive disclosure via multiple stages, builds on existing student knowledge.
Staging does not flow well; transition could be improved.
Too much or too little information provided at once; short cuts thinking/research.
Questions
Limited in number, short, and open-ended; encourage deeper understanding.
Most are directive; preempt student-generated learning issues.
Lead to “yes-no” answers rather than thoughtful discussion.
Research
Promotes substantive research using multiple resources.
Research limited to textbook material.
Limited necessity for research.

11. Writing Your Own Problem Starter

12. Step One: Identify Learning Objectives
Think of a learning objective in your course.
How do you usually address this learning objective? What kind of problem or activity do you usually assign?
Typical end-of-chapter problem?
A reading?
Other?

13. Example from Physics: Solving Problems Using Conservation of Momentum
Traditional examples:
Pool balls colliding
Bullets hitting blocks of wood

14. Example from Physics: Traditional End-of-Chapter Problem
A 1500-kg car traveling east with a speed of 25 m/s collides at an intersection with a 2500-kg van traveling north at a speed of 20 m/s. Find the direction and magnitude of the velocity of the wreckage after the collision, assuming that the vehicles undergo a perfectly inelastic collision (ie, they stick together).
Serway and Faughn. 3rd ed. College Physics, Saunders, 1992.

15. Example from Criminal Justice: Plea Negotiation
Central concept in class on courts
Most cases resolved through
plea bargains
Relates to other central course material
Attorney and litigant roles, charging, sentencing
Traditional approach
Stand and deliver lectures
Emphasis on facts of plea
bargaining

16. Example from Biology Objectives: Compare and contrast the
processes of respiration and photosynthesis,
and connect these cellular events with
their roles in global biogeochemical cycles
Traditional approach: Separate chapters on
respiration, photosynthesis, and global energy and
carbon cycles; for majors, often taught in separate
semesters

17. Types of Learning Objectives
Content-oriented: subject specific
Basic knowledge and understanding of specific concepts, techniques, etc. in the discipline
Process-oriented: global skills
Effective communication: oral and written
Acquiring and evaluating information
Working effectively with others
Higher order, critical thinking

18. Problem Writing Step One: Identify Learning Objectives
Think of a major learning objective that your problem can address.
Traditionally, how might this learning objective be addressed? What kind of problem or activity would you assign?
Typical end-of-chapter problem?
A reading?
Other?

19. Step Two: Identify Real-World Context
Name a realistic application of the concept.
Outline a scenario.
Ideas:
Add story-telling to end-of-chapter problem.
Add motivation, require students to go beyond rote learning, do research.
Include decision-making, analysis, or both.
Other?

20. A Real Traffic Accident
Based on police sketch
Students need to make assumptions and approximations
Information given
gradually
throughout problem

21. A Real Solution to Global Warming?
John Martin discovered that high nutrient, low
chlorophyll (productivity) ocean waters are missing an essential micronutrient: iron.
“Give me a tanker full of iron, and I’ll give you an ice age.”
Original decision: Should the government fund attempts to test if the “Geritol solution” can reduce the impact of excess CO2 emissions?
Newest decision: Should for-profit companies be allowed to lease or own portions of the ocean to sell sequestered carbon? Who owns the rights to “the solution?”

22. Step Two: Identify Real-World Context
Name a realistic application of the concept.
Outline a scenario.
Ideas:
Add story-telling to end-of-chapter problem.
Add motivation, require students to go beyond rote learning, do research.
Include decision-making, analysis, or both.
Other?

23. Two Schemes for Writing Problems?
Think of one or more learning objectives in your course
Name a realistic application of the concept(s). Outline a scenario.
Think of a realistic scenario from the news, a videotape, or popular press article
What learning objectives for your course are evident in the scenario?

24. Step Three: Draft the problem
Outline the problem (create a ‘story board’) First consider, “What will be on the first page?”
Suggestions:
Good PBL problem has multi-page, multi-stage construction - leave students guessing!
Not all information given in chapter or text - students look for resources.
Challenge students to come to consensus, reach conclusions, and make judgments.

25. John Henry - Traffic Cop
Stage 1: What questions need to be answered? What measurements, data? What physics principles? Then initial introduction to momentum.
Stage 2: Sketch given, some information given; students analyze and ask questions.
Stage 3: Outline procedure, make assumptions, Apply concepts.
Stage 4: Make judgment and rationalize decision based on physics principles

26. Stages of Plea Negotiation
Stage 1: Students are formed into groups, learn about case facts
Stage 2: They choose roles, identify negotiation priorities and resources, research law
Stage 3: Prosecution team makes initial charge, and negotiation begins
Stage 4: Groups arrive at plea agreement, write up group report of negotiations

27. Consider the Following Problem Types:
Explanation or Analysis Problems
‘What is going on here?’
Decision or Dilemma Problems
‘What would you do?’ ‘What do you think?’
Task-Oriented Problems
Doing an activity or carrying out a project - for example, interviewing patients or designing a brochure

28. Another Scheme for Categorizing Problems - Based on Types of Knowledge
Type of Knowledge
Descriptive
Explanatory
Type of Problem
Fact-finding
Explanation
Examples
Following changes related to land use in Zimbabwe, many internal borders changed.
People in the 15th century used to believe it was possible to fall off the earth.
Example of Question
What would a legal map look like?
Explain why?
Source: M. Savin-Baden and C. Major Foundations of PBL.
Berkshire, England: Open University Press.

29. Types of Problems Based on Type of Knowledge (cont.)
Procedural
Personal
Type of Problem
Strategy
Moral dilemma
Examples
A 45-year old woman cannot lift her arm more than 45 degrees and she complains of pins and needles in her arm.
A mother breaks into a drugstore to obtain expensive, life-saving drugs for her child. The next day she tells her physician what she has done.
Example of Question
If you were this client’s therapist, what would you do?
What should the physician do?

30. Step Three: Draft the problem
Outline the problem (create a ‘story board’) First consider, “What will be on the first page?”
Suggestions:
Good PBL problem has multi-page, multi-stage construction - leave students guessing!
Not all information given in chapter or text - students look for resources.
Challenge students to come to consensus, reach conclusions, and make judgments.

31. Problem Progress Report
On overhead:
Basic features of course
Objectives for student learning
Synopsis of problem
Scenario for first page, including guiding
questions
What comes next?

32. Feedback To Presenters
What did you like most about the problem?
Does it challenge students to think and do research?
Is the problem appropriate for the proposed course?

33. Writing PBL Problems Start with a Story Start with the Concepts
Research Research
What concepts are Look for the story to use
involved?
Research the Problem
First Draft
(Point of view, focus, appropriateness for audience, staging, objectives nature of the end-of-stage questions)
Research, Draft II (refine)
Teaching
Draft III
From C. F. Herreid, SUNY Buffalo & W. Welty, Pace University

34. A Final Consideration for Today:
How would you assess to see if students met your original learning objectives?

35. What Is Assessment?
“An assessment is an activity, assigned by the professor, that yields comprehensive information for analyzing, discussing, and judging a learner’s performance of valued abilities and skills.”
Huba and Freed, Learner-Centered Assessment on College Campuses: Shifting the Focus from Teaching to Learning, 2000
In this definition we see roles for assessment beyond simply monitoring and judging the acquisition of knowledge: there is also a need for an analysis of student learning: How is it happening? Where are the barriers to full understanding? What is working well for the student - can those approaches be exploited more fully? What assessment activities are best suited for generating this type of information?
Well-designed assessments can help us understand how our students are learning: the real audience for that information, however, should be the students themselves. One of the goals of problem-based learning, after all, is to develop in students the ability to become independent learners. They, even more than we as instructors, need to understand not only how they’re learning at present, but what they can do to improve that learning. This requires a discussion: feedback from student to instructor to diagnose the student’s learning, followed by feedback from instructor to student both about how well they’re doing and about how to improve in the future. We will look briefly at the practice of using rubrics both to set standards of expectation and to provide feedback to students.

36. First Principle for Assessing Student Learning
Assessment of student learning should be directly connected to the learning objectives of your course.

37. How Will You Tell If They’ve Learned It?
Summative assessment
Traditional grading for accountability
Usually formal, comprehensive
Judgmental
Formative assessment
Feedback for improvement/development
Usually informal, narrow/specialized
Suggestive

38. Adapting the Traditional Exam Question to PBL: Probing Critical Thinking Skills
Goal: to design a question that:
goes beyond simple knowledge or comprehension
uses novel situation or “real world” context
involves multiple concepts
requires recognition of concepts involved (analysis), their roles here (application), and how several ideas come together (synthesis)

39. Traditional Exam Questions: Chemistry Examples
Calculate the vapor pressure of a solution of 5.8 g of NaCl in 100 g of water.
Bloom Level: Knowledge
Explain why a solution of NaCl will have a lower vapor pressure than pure water.
Bloom Level: Comprehension

40. A Higher-order Exam Question
The relative humidity inside a museum display case can be maintained at 75.3% by placing within the case a saturated solution of NaCl (containing excess solid NaCl). Explain, in molecular level terms, why the humidity remains constant - even when water-saturated air (100% humidity) diffuses into the case.
Susan Groh, UD Chemistry & Biochemistry

41. Assess at Several Bloom Levels: Example from Chemistry
Analysis of exam components
# of points Total Points (Grade)
(F)
(D-)
(C+)
(A-)
(A)
Knowledge
Comprehension
Application
Analysis
Synthesis
Evaluation
Susan Groh, UD Chemistry & Biochemistry

42. PBL Problem Assignments
You have drafted a PBL problem for your course.
It addresses important concepts in a substantive context.
Students will learn, understand, and remember.
Why consider a PBL assignment separate from a PBL problem?
Isn’t the problem the assignment?

43. Closure PBL problems are typically open-ended.
Student within a single group often pursue different learning issues.
Students frequently have difficulty knowing what the instructor thinks is important.
Assignments focus on central issues and bring closure to PBL problems

44. Considerations for PBL Assignments
Individual or group response?
Course/problem content goals?
General education and PBL goals?
Graded or not?
Difficulty? (Bloom’s Taxonomy)
Interest level for students?
Form of the response?

45. Types of PBL Problem Assignments
Written assignments
Write a 2-3 page paper on a learning issue you researched.
Write a dialogue about the controversy presented by this problem.
Transform an experiment described in this article into a laboratory exercise appropriate for undergraduates.

46. Types of PBL Problem Assignments
Written assignments (cont.)
Write a letter to the editor of your local newspaper on this issue.
Compose a 200 word abstract for the article you have just read based on the Guidelines for Authors in Cell Biology.
Research a topic related to the course and write a PBL problem for this course based on that topic.

47. Types of PBL Problem Assignments
Student Presentations
Prepare a case for your position and present it to the class as a debate.
Prepare a 15-minute informative talk on this topic using PowerPoint slides.

48. Types of PBL Problem Assignments
Written and visual assignment
Create a website appropriate for high school students on this topic.
Prepare a concept map that incorporates a major issue related to this problem.

49. What is a Concept Map?
A pictorial representation that shows the relationships between and among a connected set of concepts and ideas

50. Purposes for Concept Mapping
Generate ideas (brainstorming)
Design complex structures (long texts, web-sites)
Communicate complex ideas
Aid learning by explicitly integrating new and old knowledge
Assess understanding or diagnose misunderstanding

51. A Mini Concept Map photosynthesis consists of light-dependent
reactions
light-independent
reactions
that
require
Concepts
Hierarchy
Propositional linkages
Also, cross-links
water
pigments
light energy

52. A Map-Making Strategy:
Think of all the words that apply, and write them in a list
Prioritize, and transfer priority words to Post-Its
Rearrange Post-Its to construct interrelationships, hierarchies
Review and Discuss
Establish propositional linkages, cross-links

53. Today’s Final Assignment
Consider the course or problem you are working on and how you are going to assess student learning in light of the objectives you have. Write a synopsis of an idea for an exam question or assignment appropriate for the objectives of your problem starter from today.

54. Sources and Strategies for Writing Problems
Newspaper articles, news events
Popular press in the discipline
Make up a story – based on content objectives
Adapt a case to a problem
Research papers
Other?

55. PBL Clearinghouse An online database of PBL articles and problems.
All material is peer-reviewed by PBL practitioners for content and pedagogy.
All problems are supported by learning objectives and resources, teaching and assessment notes.
Holdings are searchable by author, discipline, keywords, or full text.
Fully electronic submission, review, and publication cycle.
Controlled access by free user subscription, students excluded.

61. Reflections and Questions

62. Problem Writing Step 4
Continue to draft the storyline beyond a sketchy scenario – begin to write the first stage
Think about point-of-view, nature of the end-of-stage questions, suitability for ‘audience,’ alignment with problem and course objectives
Convene at to hear progress thus far at time announced.

63. Some Additional Considerations for Step 4
A good PBL problem leaves just the right information out!
Challenge students to come to consensus, reach conclusions, and make judgments.