Student Self-Review: Impacts on Future Class Discussion Jim Flowers Professor & Director of Online Education Ball State University Samuel Cotton Assistant.

Slides:



Advertisements
Similar presentations
Confirming Student Learning: Using Assessment Techniques in the Classroom Douglas R. Davenport Dean, College of Arts and Sciences Truman State University.
Advertisements

WHO Antenatal Course Preparing the new WHO eProfessors.
A GUIDE TO CREATING QUALITY ONLINE LEARNING DOING DISTANCE EDUCATION WELL.
Engaging Students in Online Discussion
Introduction to Grading Grading is one of the most important activities a faculty member does. Many problems in teaching arise because of grading issues.
Alternative Strategies for Evaluating Teaching How many have used end-of-semester student evaluations? How many have used an alternative approach? My comments.
Bridging Research, Information and Culture An Initiative of the Research and Planning Group for California Community Colleges Your Name Your Institution.
Briefing: NYU Education Policy Breakfast on Teacher Quality November 4, 2011 Dennis M. Walcott Chancellor NYC Department of Education.
Assisting Peers to Provide W orthwhile Feedback UC Merced SATAL Program.
Leadership Philosophy Style Practices Behaviors Characteristics Performance/ Success.
Providing Constructive Feedback
Increasing Preservice Teachers' Capacity for Technology Integration Through the Use of Electronic Models Melissa Dark Purdue University.
Does Schema-Based Instruction and Self-Monitoring Influence Seventh Grade Students’ Proportional Thinking? Asha Jitendra, University of Minnesota Jon R.
1 Carl Berger University of Michigan The Participant Perception Indicator: A Compentency Based Elementary Teacher Preparation Program.
David E. Walker, Ed.D. Bloomsburg University of PA.
UMCP Educational Technology Outreach On-line Course Assessment and Evaluation Model 2005 EDUCAUSE Mid-Atlantic Regional Conference Baltimore, Maryland.
Learners’ Internal Management of Cognitive Processing in Online Learning Chun-Ying Chen Department of Electronic Commerce Transworld Institute of Technology,
Jim Flowers, Ball State University.  Problem-based learning groups in Blackboard for an online graduate course  Technology Assessment: Health & Wellness.
Optimizing the Use of Discussion Board Forums Jim Flowers & Sam Cotton Department of Technology
1 Classroom-Based Research: How to Be a Researcher in Your Classroom Basic Skills Initiative Teaching and Learning Workshop October 2009 Darla M. Cooper.
OCTOBER ED DIRECTOR PROFESSIONAL DEVELOPMENT 10/1/14 POWERFUL & PURPOSEFUL FEEDBACK.
Writing a Research Proposal
Evaluation 101 Everything You Need to Know to Get Started Evaluating Informal Science Education Media Everything You Need to Know to Get Started Evaluating.
The Harold Schiffman Music Library Team 4 Needs Assessment David Covert, Mia Jordan, Rachel Koury, Lara Patterson, Savannah Whitaker.
Business and Management Research
Effective Use of Instructional Time Jane A. Stallings Stephanie L. Knight Texas A&M University.
Does the use of math journals improve students retention and recall of math facts and formulas?
An Action Learning Approach For Increasing Critical Thinking Skills In An Information Systems Capstone Course Alan Burns School of CTI DePaul University.
Implementing Active Learning Strategies in a Large Class Setting Travis White, Pharm.D., Assistant Professor Kristy Lucas, Pharm.D., Professor Pharmacy.
Asynchronous Discussions and Assessment in Online Learning Vonderwell, S., Liang, X., & Alderman, K. (2007). Asynchronous Discussions and Assessment in.
Interactive Notebooks in the middle school Social Studies classroom EDSS 8420 Lindsey Banos.
Making Sense of Online Learning: Frames, Rubrics, Tools & Coding Systems for Analyzing Asynchronous Online Discourse Theresa Flynn Pepperdine University.
Learners’ Attitudes and Perceptions of Online Instruction Presented by: Dr. Karen S. Ivers Dr. JoAnn Carter-Wells Dr. Joyce Lee California State University.
Strategies to Establish a Scientific Discourse Community Kaatje Kraft at Mesa Community College What is a Scientific Discourse Community? A scientific.
Maximizing Learning Using Online Assessment 2011 SLATE Conference October 14, /12/ P. Boyles, Assistant Professor, Chicago State University,
Prepared by: Liyana Binti Ahmad Afip Student ID: Prepared for: Associate Professor Dr. Hj Izaham Shah Ismail.
A Review of Research Methods for Assessing Content of Computer-Mediated Discussion Forums Marra, Rose. (2006). A Review of Research Methods for Assessing.
Teaching in a Web-Based Distance Learning Environment: An Evaluation Summary Based on Four Courses Charles Graham, Joni M. Craner, Byung-ro Lim, & Kursat.
Assessment Leadership October, Agenda Welcome and Norms Ice Breaker Activity Galileo Celebration Colorado Growth Model Data Driven Dialogue Lunch.
Leading (and Assessing) a Learning Intervention IMPACT Lunch and Learn Session August 6, 2014 Facilitated By Ozgur Ekmekci, EdD Interim Chair, Department.
Research Problem In one sentence, describe the problem that is the focus of your classroom research project about student learning: Biology undergraduates.
Gender and IT Education Conference, Indiana University, 2007 Gender & IT Education Recruitment and Websites What Do IT Program Websites Reveal about Woman.
Faculty Applications of Scholarship of Teaching and Learning Research Kathleen McKinney and Patricia Jarvis with the assistance of Denise Faigao.
Second Language Classroom Research (Nunan, D. 1990) Assoc. Prof. Dr. Sehnaz Sahinkarakas.
An Analysis of Three States Alignment Between Language Arts and Math Standards and Alternate Assessments Claudia Flowers Diane Browder* Lynn Ahlgrim-Delzell.
Using Journals to Build Information Literacy Skills Violet H. Harada University of Hawaii ACEI Annual International Conference April.
Personal Control over Development: Effects on the Perception and Emotional Evaluation of Personal Development in Adulthood.
Citizen of Edmonton Findings: Edmonton Public School Board Preference Measurement April 14, 2008 Public Presentation EPSB Board Meeting.
Evaluation of Learning Systems Bertram C. Bruce Graduate School of Library and Information Science University of Illinois, Urbana-Champaign.
ASSESSMENT OF TRUSTWORTHINESS OF ONLINE AUDITED REFLEXIVITY Frank LaBanca, EdD Director.
Chantira Chiaranai, RN, PhD Institute of Nursing Suranaree University of Technology Electronic Learning Portfolio: An Innovation Emphasizing Self-directed.
By Bundhun Amit Varma HMOA  Define Online Discussion  Recognise models of online discussions ◦ Synchronous ◦ Asynchronous  Distinguish three.
Personal Communication as Classroom Assessment. “What’s in a question, you ask? Everything. It is a way of evoking stimulating response or stultifying.
Taeho Yu, Ph.D. Ana R. Abad-Jorge, Ed.D., M.S., RDN Kevin Lucey, M.M. Examining the Relationships Between Level of Students’ Perceived Presence and Academic.
Presenters: Dr. David Collum Dr. Melanie Bishop Preservice Teachers Understanding of the Educational Needs of Diverse Learners: A Comparison of a Traditional.
Information Retention in e-Learning De Leon Kimberly Obonyo Carolyne Penn John Yang Xiaoyan.
Development of the Construct & Questionnaire Randy Garrison & Zehra Akyol April
Research Methods in Communication Cyndi-Marie Glenn.
Course Work 2: Critical Reflection GERALDINE DORAN B
指導教授:Chen, Ming-puu 報 告 者:Chen, Wan-Yi 報告日期:
Metacognitive Strategies: Are you using them in your Classroom?
SP_ IRS : Research in Inclusive and Special Education
Attitudes Towards Science and Science Teaching – Understanding How Coursework and Practicum Affect Preservice Elementary Teachers Dr. Victoria Rosin.
teacher-centered supervision
Problem Based Learning in an Online Course on Technology Assessment
Business and Management Research
Learning online: Motivated to Self-Regulate?
Student Self-Review: Impacts on Future Class Discussion
Trialling A Metacognitive Training Intervention
Evaluation Measures, Ongoing Improvements and Enhancement
Presentation transcript:

Student Self-Review: Impacts on Future Class Discussion Jim Flowers Professor & Director of Online Education Ball State University Samuel Cotton Assistant Professor Ball State University A full report on this research is currently being prepared for publication.

Concerns: Limited-quality discussion Need for more feedback Insufficient instructor time to give individual feedback on how to discuss Alternate strategies

Research Question What is the impact of student self-review on future cognitive dialog?

Weekly Blackboard discussions Week 1 discussions: Pre-Treatment Week 2 discussions: Pre-Treatment Week 3 discussions and students categorize their Week 1 contributions Week 4 discussions: Post-Treatment Week 5 discussions and questionnaire

Self review Survey Investigators analyzed discussions

Week 3 treatment: Self-categorization (analysis) each student identifies their 1.Total number of messages 2.Social messages 3.Other off-topic messages 4.On-topic messages, according to the instructor’s criteria:

Instructor’s criteria for high, fair, poor quality examples of on-topic dialog 1.Posing a relevant question 2.Offering unsolicited observation, opinions, advice related to the topic 3.Brief response to another’s question or statement 4.Lengthy response to another’s question or statement (Multiple counts per message were possible.)

Measures of the Quality of Cognitive Dialog Function Skill Level (Adapted from Henri, 1992; Henri & Rigault, 1996; Rose, 2002)

Functions 1.Cognitive 2.Metacognitive 3.Organizational 4.Social

Cognitive Skills 1.Inference 2.Analysis or Elaboration 3.Judgment 4.Cognitive Strategy 5.Elementary Clarification

Level 1.High (complex, long) 2.Low

Dependent Variables Quantity Total number of messages posted Quality of cognitive dialog Percent cognitive messages Percent analysis Percent inference Percent complex

Sample Single, online, graduate class (n=20) Students are practicing teachers/instructors 5-weeks, mid-May to mid-June, 2004

Quantitative Results: Discussion Analysis by Researchers

Unitizing Students looked at a message as the unit. Researchers looked at the sentence. This may have decreased the percentage of High Level units in both the pre- and post- treatment data. (Students had looked at a full message as the unit.)

Coding by Function, Skill & Level Two researchers as coders Third as a tie-breaker Discussions to resolve 3-way disputes

Intercoder Reliability Function: 88% Skill: 62% Level: 92%

What do you think? Effect of self-categorization on Total number of units Percent cognitive Percent inference Percent analysis / elaboration Percent high level

Results : Discussion Analysis by Researchers

Number of units Decreased 48.7 to 35.6 per person (Average unit length decreased from 17.3 words to 13.2 words.)

Function Cognitive: 90% Pre, 83% Post (p=.018*) Metacognitve: 7% Pre, 5% Post (p=.143) Organizational: 2% Pre, 5% Post (p=.003*) Social: 2% Pre, 7% Post (p=.001*)

Skill Inference: 10% Pre, 6% Post (p=.049*) Analysis: 18% Pre, 7% Post (p<.001*) Judgment: 16% Pre, 29% Post (p<.001*) Strategy: 1% Pre, 5% Post (p=.024*) Elem. Clarification: 55% Pre, 54% Post (p=.417)

Level High: 13% Pre, 8% Post (p=.007) Low: 87% Pre, 92% Post

Summary: Decreases in Quantity Total number of messages posted Quality of cognitive dialog Percent cognitive messages Percent analysis Percent inference Percent complex

Results: Students’ Analyses & Questionnaires Mixed, but showing some perceived benefit to this activity.

Student Comment “After reading my postings, I realized that I did not go as in depth as I would have liked to.”

Student Comment “…I feel my thought process is changing and my responses are improving in quality.”

Student Comment “I have a tendency to offer unsolicited observations…. Apparently I tend to tie almost everything into past experiences and I have become very opinionated… Hopefully this will change because of this reflection.”

Student Comment “In some ways, I find the rubric limiting in that I feel I must consciously pay attention to how much I am posting and of what type I am posting. I think I would post more if it were more free-flowing and not proscribed.”

Conclusions This self-categorization reduced the quantity and quality of cognitive dialog, likely due to an increased hesitancy brought on by an increased awareness that each message would be scrutinized.

Conclusions This activity should not be used to: improve cognitive dialog; or reduce instructor time for feedback without reducing quality of discussions.

Conclusions Some participants reportedly became aware of a need to change that did not manifest itself in their discussion.

Recommendations Attempts to improve cognitive dialog should focus on interventions that do not focus students’ attention on the assessment of their cognitive dialog.

Recommendations for Future Studies Future research may attempt to investigate this related to learning preferences, self- concept, open-mindedness, and language skills, to name a few.

References Rose, M. A Cognitive dialogue, interaction patterns, and perceptions of graduate students in an online conferencing environment under collaborative and cooperative structures. Ed.D. diss., Indiana University, Bloomington. Henri, F Computer conferencing and content analysis. In Collaborative learning through computer conferencing: The Najaden papers ed. A. R. Kaye, New York: Springer-Verlag. Henri, F., and C. R. Rigault Collaborative distance learning and computer conferencing. In Advanced educational technology: Research issues and future potential. Vol. 128, Computer and Systems Sciences ed. C. O’Malley, New York: Springer-Verlag.

Student Self-Review: Impacts on Future Class Discussion Jim Flowers Professor & Director of Online Education Ball State University Samuel Cotton Assistant Professor Ball State University A full report on this research is currently being prepared for publication.