Wednesday, December 5, 2018 1:00 – 2:00 PM, ET Learn How to Ask Better Questions with the Question Formulation Technique Wednesday, December 5, 2018 1:00 – 2:00 PM, ET

BEFORE WE BEGIN During the webinar… please provide feedback! After the webinar… survey, slides, recording, and more!

QUESTIONS? Use the chat at any time... Or wait until the end.

WEBINAR JOB AID Download it now… Use during and after the webinar!

Upcoming Webinars Teaching Metacognition to Help Students Own and Improve their Learning – A Two-Part Webinar Event Speakers: Patrick Cunningham (Rose-Hulman Institute of Technology), Holly Matusovich (Virginia Tech) Part 1: Weds. February 6, 2019 from 2:00 – 3:00 PM, ET Register now! Part 2: Weds. February 20, 2019 from 2:00 – 3:00 PM, ET Learn more: www.asee.org/webinars

Today’s Speakers Luz Santana Andrew P. Minigan Co-Founder and Co-Director Right Question Institute Andrew P. Minigan Director of Strategy, Education Program Right Question Institute

Where are you joining from today? Tell us in the chat pod! Luz and Andrew (Cambridge, MA) Alex (Astoria, NY)

Learn How to Ask Better Questions with the Question Formulation Technique (QFT) Luz Santana, Co-Director Andrew P. Minigan, Director of Strategy, Education Program

A Working Webinar

Poll: What are you interested in learning about? How the QFT can support my research. How the QFT can support my students’ research. How the QFT can support teaching and learning in my classroom.

Acknowledgments We are deeply grateful to The National Science Foundation and The Hummingbird Fund for their generous support of the Right Question Institute’s work in education. Thank you to Rocio Chavela and Alexandra Longo for their support and making possible this webinar.

Free Resources www.rightquestion.org visit www.rightquestion.org/events to find materials from today’s webinar

Overview Questions & Learning An experience in the QFT for research Preliminary results on the QFT for research Some students are not asking questions Examples of the QFT in the classroom A few closing words

Questions & Learning

“All learning must begin with the posing of a question.” – Richard Feynman Physicist, Nobel Laureate

“If you are a researcher you are trying to figure out what the question is as well as what the answer is.” “You want to find the question that is sufficiently easy that you might be able to answer it, and sufficiently hard that the answer is interesting.” – Edward Witten Physicist, Institute of Advanced Study

“There can be no thinking without questioning—no purposeful study of the past, nor any serious planning for the future.” - David Hackett Fischer University Professor Emeritus of History Brandeis University

The Right Question Institute

An Experience in the QFT for Research

Identify Research Topic Write down some topics you might consider for research. Please list them. Choose one topic you are considering for your research.

Rules for Producing Questions Ask as many questions as you can about your research topic Do not stop to answer, analyze, or judge questions Write down every question exactly as it comes to mind Change any statements into questions How might following these rules be different from how you usually produce questions?

Produce Questions Ask questions about the research topic you chose Follow the rules Number the questions as you produce them

Work with Closed and Open-ended Questions Definitions Closed-ended questions can be answered with a “yes,” “no,” or with a one-word answer. Open-ended questions require an explanation and cannot be answered with a “yes,” “no,” or with one-word. In the column titled “O/C,” label your closed-ended questions with a “C” and your open-ended questions with an “O.”

Advantages & Disadvantages Closed-ended questions Advantages Disadvantages

Advantages & Disadvantages Open-ended questions Advantages Disadvantages

Work with Closed and Open-ended Questions Change one question, either: one closed-ended question and reword it so it becomes open-ended. one open-ended question and reword it so it becomes closed-ended. Add this new question to the bottom of your list of questions. Closed Open Open Closed

Rework Questions Do you have questions that you would like to rework? If so, rework your questions and add them to the bottom of your list.

Prioritize Choose three priority questions you are most interested in using to guide a research project. Keep in mind the research topic you identified earlier Star your three priority questions in the column titled, “priority”

Prioritize Your rationale for why you chose these questions How many questions you asked

Further Explore Choose one priority question you would like to use as a guide for a research project. Produce questions about your priority question Follow the rules Number the questions as you produce them ? ? ? ? ? ? ? ?

Develop Research Questions and Research Agenda From all the questions you have asked so far, choose: one question you are glad you generated through the process one question that may help you advance your research Please star these questions in the columns titled, “Glad” & “Adv.”

Reflect What did you learn?

QFT for Research on One Slide Identified a research topic Produced questions Worked with open-ended and closed-ended questions Reworked questions Prioritized questions Produced questions on a priority question Developed research questions Reflected

Preliminary Results on the QFT for Research

Agreement on having strategies In considering what research questions… to pursue, I have specific strategies that allow me to generate good questions.*** to investigate, I have specific strategies that allow me to prioritize which questions are the best to pursue.*** ***p < .001

In considering what research questions to pursue, I have specific strategies that allow me to generate good questions. 1 = Strongly Disagree 2 = Disagree 3 = Slightly Disagree 4 = Slightly Agree 5 = Agree 6 = Strongly Agree

Beliefs about question formulation behavior Once I have a set of questions before me, I believe that it is worthwhile to take the time to revise and improve them.*** I am able to efficiently formulate research questions.*** ***p < .001

Confidence in question formulation skills I am confident in my ability to formulate quality research questions.*** In developing research questions, I believe that my skills are at a high level.*** It is easy to formulate quality research questions.*** ***p < .001

I am confident in my ability to formulate quality research questions. 1 = Strongly Disagree 2 = Disagree 3 = Slightly Disagree 4 = Slightly Agree 5 = Agree 6 = Strongly Agree

What did you notice about the quality of your questions? “Improved the quality, testability, and impact.” “Improved the quality and the quantity [of my questions.]” “It made them more specific and allowed me to pick the best from among several similar questions.” “Reworking my questions made my questions better and more complex.” “Improved the quality of the question but more importantly led to new questions.”

Some Students Are Not Asking Questions

College Presidents on What College Students Should Learn “The primary skills should be analytical skills of interpretation and inquiry. In other words, know how to frame a question.” - Leon Botstein, President of Bard College “…the best we can do for students is have them ask the right questions.” - Nancy Cantor, Chancellor of University of Illinois The New York Times, August 4, 2002

Yet, Only 27% of Graduates Believe College Taught Them How to Ask Their Own Questions Alison Head Project Information Literacy at University of Washington, 2016

But, the problem begins long before college...

Age Four: “The true age of inquisitiveness” James Sully dubbed age four, “the true age of inquisitiveness when question after question is fired off with wondrous rapidity and pertinacity.” Young children ask 10,000 questions per year before they begin formal schooling. Sully, 1896 Harris, 2012

Question Formulation by Adolescence Dillon, 1988, p. 199

Teach Students to Ask Their Own Questions

Examples of the QFT in the Classroom What does it look like when students learn to ask their own questions? Examples of the QFT in the Classroom

Classroom Example: College Biology Rachel Woodruff, Ph.D., Assistant Professor of Biology, Brandeis University Topic: Molecular Biology Purpose: To build students’ research skills and prepare them to develop their own a proposal later in the semester

Question Focus Students were assigned a complex molecular biology article Ask as many questions as you can about the reading

Next Steps Students generate questions about a complex article on their own Students discuss the key attributes of a good biological research question and compare to other types of questions Students form groups and improve their questions, based on these attributes

Classroom Example: College Biology Professor: Emily Westover, Ph.D., Brandeis University Topic: Post-Exam Review Purpose: To assess what the students were not understanding about the content. They had performed poorly on a particular question on an exam.

Question Focus Ask as many questions as you can about [the exam question]

Educator Reflections Students were thinking more deeply about the exam question Professor Westover was able to gather insight into why they may have had difficulty with the exam question

Classroom Example: Writing Fundamentals Sun Ezzell, Professor of English, Mt. San Antonio College Topic: Grades Purpose: Students use their questions to spark discussion on the article & write a response to article. Students consider whether they should switch from letter grade to pass/no pass.

Question Foci “the case against grades” (for students who read the article) or “grades” (for students who did not read)

Students’ Questions Why are students intimidated against grades? Is the grading process needed? How does the grading system affect students? Why do we have a grading system? Does the grading system need improvement? Why is it challenging to get good grades? Why do students think school is challenging? Why do students focus more on grading than learning? Does the grading system help students in life? What are your thoughts on our grading system? Do students focus more on grading than learning?

Next Steps After the activity, some students decided to switch from letter grades to pass/no pass Led to discussion about feedback and how they can use/ share feedback for learning. Sun gained insight into student thinking

Students’ Reflections “I discovered that by asking questions about the article it was easier to understand the article and the meaning behind it.” “I learned that by doing the question exploration it can help you not be stuck when you do not understand the material.” “The [QFT] was an amazing group activity...I loved that no one was excluded...” “When we come up with our own questions, we think more deeply.”

Reflect How can you use what you learned during this webinar?

A Few Closing Words

In the Age of Google “How should you respond when you get powerful new tools for finding answers? Think of harder questions.” – Clive Thompson Journalist and Technology Blogger

Thank you. Now, some time for your questions Thank you. Now, some time for your questions. We are eager to connect with you and explore how we can support your work: Andrew.Minigan@rightquestion.org Luz@rightquestion.org Register to access free resources over at: www.rightquestion.org

Thank you! Don’t forget: Complete the webinar survey Check your email for slides, recording, and job aids