Modeling and Assessing Online Discussions for Faculty Learning and Technology Development Janet de Vry and George Watson University of Delaware Barbara Frey University of Pittsburgh Copyright de Vry, Frey, Watson, This work is the intellectual property of the author. Permission is granted for this material to be shared for non-commercial, educational purposes, provided that this copyright statement appears on the reproduced materials and notice is given that the copying is by permission of the author. To disseminate otherwise or to republish requires written permission from the author.

Agenda Faculty Development Class Janet de Vry Problem Based Learning Context George Watson Assessment Barbara Frey Application to Your Campus

Workshop Goal: Experience Learning in an Online Discussion Participate in collaborative discussions Engage in critical thinking Interact at a high level

“The principal idea behind PBL is that the starting point for learning should be a problem, a query, or a puzzle that the learner wishes to solve.” —Boud (1985) What is Problem-Based Learning?

“…careful inspection of methods which are permanently successful in formal education…will reveal that they depend for their efficiency upon the fact that they go back to the type of situation which causes reflection out of school in ordinary life. They give pupils something to do, not something to learn; and if the doing is of such a nature as to demand thinking, or the intentional noting of connections; learning naturally results.” —John Dewey (1916) What is Problem-Based Learning?

PBL is a learning approach that challenges students to “learn to learn,” working cooperatively in groups, to seek solutions to real world problems. What is Problem-Based Learning?

PBL prepares students to think critically and analytically, to find and use appropriate learning resources, to communicate effectively, orally and in writing, to work well as members of a team.

What are the Common Features of PBL? Learning is initiated by a problem. Problems are based on complex, real- world situations. All information needed to solve problem is not initially given. Students identify, find, and use appropriate resources. Students work in permanent groups.

Problem-Based Learning Cycle Overview/ Assessment Problem, Project, or Assignment Group Discussion Research Group Discussion Preparation of Group “Product” Whole Class Discussion Mini-lecture (only when needed!)

A Typical Day in a PBL Course

Good PBL Problems… Relate to real world, motivate students Require decision-making or judgments Are multi-page, multi-stage Are designed for group-solving Pose open-ended initial questions that encourage discussion Incorporate course content objectives, higher order thinking, other skills

Workshop Design Issues Developing problem Gathering authentic data Simulating asynchronous world in a synchronous setting Developing questions, activities, staging Sequencing activities; both online and face to face

The Problem in 5 Stages 1.Identify problems with your students’ discussion 2.Formulate strategies for solutions 3.Post best solutions for entire class 4.Get expert advice 5.Develop an individual action plan

Initial Online Instructions

Authentic Data— Discussion Topics

Subject: War in Iraq I think that if you don't support the war, you don't support our troops. You are DISLOYAL! There are veterans and families who don't support the war. What do you think about that? >I think that if you don't support the war, you don't support our troops. >You are DISLOYAL! I agree!!!! You are just wrong, you idiot!!!!! Authentic Data — Sample Student Dialogue

We are in the classroom. Talking is permitted.

We are in the “Online World.” Please refrain from talking.

Activities Online Face to Face 1. Pre-class to analyze problem 1. Warm up exercise 2. Group to generate & evaluate strategies 2. Group to compare online experience to actual class 3. Individual to pose comments and develop questions for experts 3. Mini lecture on rubrics; presentation of action plan

Ongoing Activities Chat with experts Submit Resources Individual action plan

Asynchronous Discussions Strengths –Student interaction –Flexible format –Reflection time to develop thoughts –Permanent record of discussion Weaknesses –Time consuming –Multiple simultaneous threads –Linear presentation of postings –Late contributions are not developed –Single-pass reading for unread messages

Small Group Discussion Map Discussion Question: Develop a list of at least 10 different strategies to improve the quality of online discussions. Share your top 3 strategies with the whole class.

Analyzing Discussions Consider trigger or initial questions Review whole thread Consider depth vs. breadth of discussions Assess levels of thinking (Bloom’s Taxonomy) Reflect on unanswered questions and undeveloped topics Consider timeliness of responses

Evaluation Synthesis Analysis Application Comprehension Knowledge Bloom’s Levels of Cognitive Skill

Small Group Discussion Map

Advantages & Disadvantages of Discussion Map Easy to visualize where a posting is located within the structure of a thread Easy to see the complexity or simplicity of a thread Provides a mechanism for reviewing the postings – code by level of thinking, date or participant Time consuming to develop

Observations Postings were on-topic Subject lines did not change to reflect content or intent of posting Thread initiators usually responded to thread contributors Type of trigger question affected cognitive skill demonstrated in postings more than level/depth within thread Expectations woven into some discussion questions Some threads ended with unanswered question or comment

Recommendations Basic training and guidelines for participants should include guidelines to: –Change Subject Lines which act as advanced organizers –Take “ownership” of initiated threads; if appropriate, respond to questions and be responsible for summarizing the thread

Recommendations Facilitators should set clear guidelines and expectations. These should include: –Design good initiator or trigger questions –Keep topics specific and focused –Establish specific time frame –Provide models of well composed subject lines and postings –Encourage participants to summarize threads –Consider the facilitator’s role in discussions

Feedback from Participants What will be useful in your teaching? –“Managing online discussions” –“Rubrics for online discussion” –“How to moderate discussion” –“How to keep students on topic”

3 months later—what was the impact of workshop? “Defining topics is as important as setting guidelines” “Experience with what can go wrong with discussions” “Concrete ideas for implementation and evaluation” “I still had trouble figuring out how to use discussions.”

Questions?

Workshop website