Course Design: Pedagogical Approaches to Assessing Higher Order Cognitive Skills Presenters introduction Workshop introduction: It will be composed of activities, presentation, assessment Jennipher Murphy, M.S. Ed Education Technology Senior Administrator Academic Affairs jennipher.murphy@tufts.edu Tofool Alghanem, BDS, MS, DMD Assistant Professor, Public Health and Community Service Assistant Director of Advanced and Graduate Education tofool.alghanem@tufts.edu

Please login to: https://learningcatalytics.com/demo Welcome! Please login to: https://learningcatalytics.com/demo 1. Fill in your name and email. 2. Check “I agree”, then click “Start”. 3. Enter session ID 14865860, then click “Join”. Learning catalytics is a tool that we will use to collect pre- and post- discussions assessments. To participate, follow the instructions Wifi access: educauseair Password: educause

Pre-Presentation Check Point Please answer the 4 questions and submit. We will review the results at the end of our session Pre-Presentation Check Point Session ID: 30433411 5 Min It aims to assess the group understanding of certain topics in this workshop: Teamwork, leadership, critical thinking assessment, and feedback (assessment). This will help us decide on how much information we need to provide about each of these topics. One word about teamwork Leadership activities list How to assess critical thinking Rank feedbacks appropriateness

Agenda Activity #1- Check Point Introduction Cognitive skills review Pedagogical Approaches Taken Course Framework Assessment Strategies Implemented Course Outcomes Activity #2 Results Discussion 4:08 Tofool It seems that we in general know something about one or more of the terms used in the assessment, so we will not discuss them further and rather try to apply them Learning Catalytics: https://learningcatalytics.com/demo Session ID: 30433411

Higher Order Cognitive Skills Critical Thinking Leadership Communication Teamwork Self-assessment For the purpose of this presentation, we chose 5 standards to work with. The reason for choosing these standards is that they are the standards we focused on in our course. For example: a team will require a leader, to have the work done, you need communication which will stimulate critical thinking and assessment

Higher Order Skills Teamwork and Leadership Communication Foster ability to interact face-to-face Develop interpersonal and small group skills Create individual and group accountability Learn that your success is dependent on the contributions, inclusions, and success of others in your group Communication Communicate effectively with others while reasoning through problems Communicate in multi-cultural environment Effective provider-patient communication

Higher Order Skills (Cont.) Critical Thinking Identify problems and formulate questions clearly and precisely Gather and assess relevant information, weighing it against extant knowledge and ideas, to interpret information accurately and arrive at well-reasoned conclusions Test emerging hypotheses against evidence, criteria, and standards Self-Assessment Assists learners in developing the knowledge, skills, attitudes, and values considered important at their stage of learning (formative assessment) Self-assessment is indicative of the extent to which students take responsibility for their own learning

Traditional Pedagogical Approach “Sage-on-the-Stage”

Constructivist Pedagogical Approach Constructivism Traditional Responsibility for Learning Resides with the learner Rests with the faculty member Learning Motivation Intrinsic- authentic experiences Extrinsic- conform to standards and expected achievements Role of Faculty Facilitator- active learning Lecturer- passive recipients Interaction Learn from each other and an awareness of each other’s values Objective- faculty is the expert Collaboration Arrive at a shared understanding of the truth- “scaffolding” Achieve the content set in the curriculum Assessment 2-way process between faculty and student Process carried out by faculty member Modified from: Lam, B.H. (2011) Chapter 2 Constructivist perspectives of learning. Learning and Teaching in the Chinese Classroom. Hong Kong: HKU Press.

SPIRAL Pedagogical Approach Student-driven Pedagogy of Integrated, Reinforced and Active Learning (SPIRAL) EBD IPE Practice Management 1st Year 2nd Year 3rd Year 4th Year Normal Abnormal Clinical EBD Clinical Case Entry Graduation The SPIRAL approach offers learning that is not linear but provides opportunities for “deeper” learning and can be achieved through the spiral process. Ideas are repeatedly revisited and built upon. Revisiting requires students to develop their ability to reflect on their ideas and experiences. Dr. Mark Wolff & Dr. Andrew Spielman, New York University (NY U) School of Dental Medicine  

Basic Science/Clinical Science Spiral Seminar Series (BaSiCSsss) 2 Clinicians 2 Basic Scientists 1 Librarian Student Pod 1 Student Pod 2 Student Pod 3 Student Pod 4 Student Pod 5 Student Pod 6 Student Pod 7 Student Pod 8 Course Framework: 190 pods (800 students) 23 Faculty groups (72 faculty) Roles: 4th Year: Select case and coordinate team 3rd Year: Evidence-based dentistry (EBD) 2nd Year: Info pertinent to case i.e. medications, risk factors, pathology of dental issues 1st Year: Info pertinent to case i.e. biology, physiology, anatomy etc..

Active Learning & Assessment Strategies Allows faculty and librarian to stay connected and give feedback to team before the final presentation Online Collaboration Space Students define roles and responsibilities, expectations, conflict prevention, communication plan and timeline Teamwork Action Plan Dynamic, constructivist, supports the SPIRAL pedagogy Team Structure Reflection, constructive feedback Peer Review Reflection, leadership Team Leader Self-Assessment Critically investigate research with support from librarian Evidence Based Dentistry Form Team presents case study and Q & A with peers (other teams) and faculty Case Study Presentation

Patient Case Study and Presentation Assessment Breakdown Patient Case Study and Presentation Soft Skills= grammar, teamwork, eye contact, cooperation, critical thinking, persuasiveness, time management, interpersonal, cultural sensitivity Higher Order Skills= Critical Thinking, Biomedical Sciences, Behavioral Sciences, Clinical Sciences, Practice Management, Ethics and Professionalism Peer Review Softs Skills= grammar, critical thinking, interpersonal, cultural sensitivity, self-directed, reliability, relationship, communication Higher Order Skills= Critical Thinking, Self Assessment, Ethics and Professionalism

Assessment Breakdown Self Assessment Teamwork Action Plan Soft Skills= grammar, communication, relationship, cooperation, reliability, self-directed, critical thinking, motivation, time management, interpersonal skills, cultural sensitivity Higher Order Skills= Critical Thinking, Self Assessment, Ethics and Professionalism Teamwork Action Plan Soft Skills= grammar, communication, teamwork, cooperation, motivation, persuasiveness, time management, interpersonal Higher Order Skills= Practice Management and Health Care Systems, Ethics and Professionalism

Assessment Breakdown Faculty Mentors Evidence-Based Dentistry Form (PICO) Soft Skills= grammar, self-directed, critical thinking, persuasiveness Higher Order Skills= Critical Thinking, Clinical Sciences Faculty Mentors Soft Skills= grammar, communication, relationship, eye contact, reliability, cooperation, motivation, persuasiveness, interpersonal Higher Order Skills= Critical Thinking, Ethics and Professionalism

Outcomes “I didn't feel as though this seminar series helped me in any way, I have such a large course load that it was almost an inconvenience.” “as a first year student i really appreciated having this opportunity to communicate with the upper classman in my basically line and get a nice overview of how the basic science classes and operative will bridge over to clinic” “I never received any feedback or any comments for my presentation. That is unfair! We worked hard to put a presentation together I think we deserve some comments and feedback!” Tofool “Basics was awesome!! I learned so so much! I'm really glad that I had the opportunity to participate in this before my 4th year - very helpful.”

“It has an awful acronym.” “By far the most useful thing was coordinating group activity/research;” “This was actually pretty cool. I thought it would be a pain at first but I liked it. Good experience.” “The seminar is great as-is. Strongly promotes learning and communication between the building bridges team. Makes the student reflect back on a patient case and get a better understanding for the decisions they made, why they made them, and things they might do differently in the future. Very valuable experience.”

Challenges Students are not prepared with the foundational skills for participation in effective teamwork Students do not have quality experience in peer review or in giving feedback Faculty are more comfortable being lecturers then facilitating the learning Condensed curriculum and time According to a literature review by a panel of dental educators, “ none of the articles found fulfilled all the criteria for critical thinking and none referenced a learning model”. –Johnsen D.C. et al

Next Steps and Best Practices Improve Assessment: Faculty Calibration Enhance training and preparation for teamwork activity Define critical thinking competency Develop rubric for self-assessment Train faculty on facilitation techniques Jennipher Start simple then expand (one assessment not a whole course) Both Quantitative and Qualitative Realize that interactive teaching is hard Being Open-minded is a key for success Model life-long learning

Putting All Together Choose Curriculum/ Course Decide on Cognitive Skills Choose Pedagogical Approach/ Active Learning Strategy Design Activity for Integration How will it be implemented? How will the standard be assessed?

Activity #2 47882832 Click on “Join another session” Enter session ID# Part 1: Participants will work individually to create a learning activity/assessment for their course or institution that addresses a Higher Order Skill. (10 min) Part 2: Participants will then enter their assessment into Learning Catalytics for review with the group. (5 min) Click on “Join another session” Enter session ID# 47882832

Results & Discussion Pre-Presentation Check Point Assessment Activity What differences are we seeing?

Resources Accreditation standards for dental education programs. Commission on Dental Accreditation. Retrieved from: Berk, N.W. (2001) Teaching ethics in dental schools: Trends, techniques, and targets. Journal of Dental Education Vol 65 N.(8). Emerson A. & Williams, G.J. (2012) An innovative approach to encouraging spiral learning for third year undergraduates. Psychology Teaching Review Vol. 18 N(2) Johnsen, D.C. et al (2015). New dental accreditation standard on critical thinking: A call for learning models, outcomes, assessments. Journal of Dental Education, Vol. 79 N(10).

Resources Lam, B.H. (2011) Chapter 2 Constructivist perspectives of learning. Learning and Teaching in the Chinese Classroom. Hong Kong: HKU Press. Quieng, M.C., Lim, P.P. & Lucas, M.R. (2015) 21st century-based soft skills: Spotlight on non-cognitive skills in a cognitive-laden dentistry program. European Journal of Contemporary Education Vol. 11 N (1) pp. 72-81. Tsay, M. & Brady, M. (2010) A case study of cooperative learning and communication pedagogy: Does working in teams make a difference? Journal of the Scholarship of Teaching and Learning, Vol. 10 N(2), pp. 78-89. Yorke, M. (2003). Formative assessment in higher education: Moves towards theory and the enhancement of pedagogic practice. Higher Education Vol. 45: p.477-501.