1. A SSESSMENT OF COMMUNICATION PERFORMANCE IN SIMULATED VOLCANIC CRISES Jackie Dohaney, E. Brogt, T.M. Wilson, B. Kennedy Postdoctoral Fellow, Geoscience Education jdohaney@gmail.com

2. Why teach and assess Communication Skills? 1.We typically don’t teach communication skills 2.These are often adhoc, embedded, implicit (hidden) learning outcomes 3.Effective communication is central to Disaster Risk Reduction principles

3. Project Outcome: Build and assess curricula that improves students’ communication skills. Research Objective: Test & create evidence-based measures of communication. 1)‘Unpack’ Communication Performance 2)Variables/Proxies of Communication 3)Qualitative assessment of communication (via student pre-post interviews) 4)Compare proxies to interviews. A SSESSING C OMMUNICATION

4. 1)‘Unpack’ Communication Performance How can we measure communication?  Effectiveness of communication is highly contextual, and receiver-dependent.  Anecdotal, holistic, inherited and conflicting views on teaching communication. -> Little educational research is dedicated to the rigorous assessment of communication. A SSESSING C OMMUNICATION

5. Different perspectives

6. 1)‘Unpack’ Communication Performance -> Definition of “Best Practices” in Crisis Communication 7 C’s of Communication Comprehensible – simple, clear, jargon-free Contextualized – diversity, cultures, differences Captivating – engaging, relevant Credible – open, frank, acknowledges uncertainty Consistent – backed by evidence, confirmable Courteous – compassionate, empathetic, respectful (Addresses ) Concerns – empowers action/response Vivienne Bryner (University of Otago, NZ) http://www.youtube.com/watch?v=grhrLT8tfjg A SSESSING C OMMUNICATION

7. Curricula: Role-plays & Simulation A SSESSING C OMMUNICATION To acquire and perfect communication skills, research suggests students need to practice them in authentic scenarios. (Cox et al. 2012). Role-playing, modelling, coaching, rehearsal, reinforcement and feedback, enable individuals to improve their communication competence. (Richmond et al. 2013).

8. 2) Variables that contribute to communication? i.e., confidence A SSESSING C OMMUNICATION Student Interviews Actual performance

9. Variable: Communication Experience Measure: Communication Experiences Survey (Validation Process; Dohaney et al 2015) Experiences : Oral presentations Poster presentations Debates & Speeches In the classroom (Teaching) Group Discussions & Meetings Professional Media Setting/style: -At conferences, in your department, during your education -Provided, received or self-evaluative feedback for communication? A SSESSING C OMMUNICATION Estimated number of experiences None Few (1-10) Some (11-20) Many (21-30) A lot (>30) PLEASE COMPLETE THE SURVEY

10. Variable: Communication Efficacy Measure: Self-perceived communication competence SPCC Instrument (McCroskey and McCroskey 1988) Statements: 1. Present a talk to a group of strangers. 2. Talk with an acquaintance. 3. Talk/Discuss at a large meeting of friends. 4. Talk in a small group of strangers. 5. Talk with a close friend. 6. Talk/Discuss at a large meeting of acquaintances. 7. Talk with a stranger. 8. Present a talk to a group of friends. 9. Talk in a small group of acquaintances. 10. Talk/Discuss at a large meeting of strangers. 11. Talk in a small group of friends. 12. Present a talk to a group of acquaintances. A SSESSING C OMMUNICATION Likert Choices: Very strong ability Strong ability Average ability Poor ability Very poor ability

11. Can look at gender, age, nationality, curricula-type, etc to assess any factors that might influence pre- scores and “changes” (post- pre).

12. Perceptions of Crisis Communication: Content  Language & Terms  Amount of content  Level of detail  Equity & Diversity Delivery  Visual Aids  Media-type  Tone  Intent of message/Purpose Different Audiences  Scientists, Emergency Managers  The public and communities A SSESSING C OMMUNICATION

13. Variable: Science Communication Best Practices Measure: Perceptions of Science Communication Best Practices (Development process; Dohaney et al 2015/16) A SSESSING C OMMUNICATION Likert Choices: Stongly Agree Agree Neutral Disagree Strongly Disagree Examples of Statements: 1. In a crisis, scientific information presented to the PUBLIC should appear open and completely transparent. 2. In a crisis, scientific information presented to the PUBLIC should be comprehensive (including all scientific facts discussed). 3. In a crisis, when communicating to the PUBLIC, it is appropriate to show your emotions

14. A SSESSING C OMMUNICATION Perceptions: Better Communicators

15. A SSESSING C OMMUNICATION Perceptions: Scientific Responsibilities

16. 3) Qualitative assessment of communication (via student pre-post interviews) -> Direct Measure of Communication Performance Instrument- Presentation Skills Protocol for Scientists (2PS) Rubric to guide coding of pre- and post-interviews A SSESSING C OMMUNICATION

17. Pre-Interview Script: Question: How do scientists actually know that a volcano is going to erupt? “ That goes back to the monitoring that we are always doing. Our scientists know when something is happening, that is different from what we usually see. That’s when we will investigate further. And ya know, break it down into what that actually means.” A SSESSING C OMMUNICATION

18. Post-interview script: Question: How do scientists actually know that a volcano is going to erupt? “So we monitor the volcanoes constantly. And we have different methods of doing this. Scientists know what to look out for. We monitor it so frequently that, anything out of character, of the background level, so that we know something is a different kind of activity. So we will look into that further. And from there we can figure out if that is a normal process, or an increase in activity” A SSESSING C OMMUNICATION

19. Future Work: (4) Compare proxies to interviews. (5) Incorporating the qualitative coding into a robust rubric for assessment of communication performances that is appropriate to crisis contexts Excellent AveragePoor A SSESSING C OMMUNICATION

20. Thank You! Any Questions? Jackie Dohaney Postdoctoral Fellow, Geoscience Education jdohaney@gmail.com Perceptions (please see me!) Communication Experiences Survey http://canterbury.qualtrics.com/SE/?SID=SV_9Xrc0QfnzZnyN6d

21. SPCC Study Participants Student participants (n=37) were recruited from third and fourth year physical volcanology and hazards management courses which hosted the volcanic hazards simulation as part of their curricula. Two iterations of the simulation were tested; One was embedded at the end of a 7-day field course (took place in January, 2012; n=19) the other was embedded within a lecture-based course (took place in August 2012; n=18). They were mixed cohorts of American study-abroad students and New Zealand students who attended the University of Canterbury. Students ranged in gender (female (13) and male (24)), nationality (New Zealand (24), United States of America (11), Netherlands (1) and India (1)), and age (aged 19 – 22 (20) and  23 years old (17)).