1. PEER Module 4: Research & Evaluation Questions
Erika S. Trapl, PhD
October 21, 2015

2. The Perfect Question You won’t be writing it today.
You probably won’t write it tomorrow.
And once you start writing, the question you end up with probably won’t look much like the question you started with.
Don’t worry…you’re doing it right. This is a process.
So, where to start?

3. Research Question Defined
A clear, focused, concise, complex and arguable question around which you center your research
From
Could also:
Refer to the problem or phenomenon
Reflect the intervention in experimental research
Note the target group of participants
Inform the research methods

4. What makes a Q “good”? RELEVANT Contributes to the field
Adds to the theoretical foundation
Informs policy
Ask yourself “so what?”
Who cares about the question and why
What is the potential impact of answering your question

5. Getting started What topic are you interested in
What gets you really excited about it?
What are you curious about?
Can you narrow this down a little more?
How much do you already know?
Are there some questions you want to answer?

6. Getting started What topic are you interested in
What gets you really excited about it?
What are you curious about?
Can you narrow this down a little more?
How much do you already know?
Are there some questions you want to answer?
You are in pursuit of the Knowledge Gap. What next?

7. The Circle of Inquiry Identify Knowledge Gap
Search for Existing Information
Disseminate & Implement
Focus the Study Question
Analyze & Interpret Results
Collect Data
Design the Study
Adapted from: Nutting, PA, Stange, KC. Practice-based research: The opportunity to create a learning discipline. In: The Textbook of Family Practice, 6th Edition. Rakel RE (ed.), W. B. Saunders Company,

8. The Circle of Inquiry 1. Identify Knowledge Gap
2. Search for Existing Information
Disseminate & Implement
3. Focus the Study Question
Analyze & Interpret Results
Collect Data
Design the Study
Adapted from: Nutting, PA, Stange, KC. Practice-based research: The opportunity to create a learning discipline. In: The Textbook of Family Practice, 6th Edition. Rakel RE (ed.), W. B. Saunders Company,

9. Search for existing information
Literature Search:
(a) to help figure out what works;
(b) to pursue a topic, problem, or question of professional and/or personal interest;
(c) to pinpoint an area of further study;
(d) to provide a rationale/background for study;
(e) to survey or analyze research methodology

10. The Circle of Inquiry 1. Identify Knowledge Gap
2. Search for Existing Information
Disseminate & Implement
3. Focus the Study Question
Analyze & Interpret Results
Collect Data
Design the Study
Adapted from: Nutting, PA, Stange, KC. Practice-based research: The opportunity to create a learning discipline. In: The Textbook of Family Practice, 6th Edition. Rakel RE (ed.), W. B. Saunders Company,

11. The Circle of Inquiry 1. Identify Knowledge Gap
2. Search for Existing Information
Disseminate & Implement
3. Focus the Study Question
Analyze & Interpret Results
Collect Data
Design the Study
Adapted from: Nutting, PA, Stange, KC. Practice-based research: The opportunity to create a learning discipline. In: The Textbook of Family Practice, 6th Edition. Rakel RE (ed.), W. B. Saunders Company,

12. What makes a Q “good”? Contributes to the field
Adds to the theoretical foundation
Informs policy
RELEVANT
Ask yourself “so what?”
Who cares about the question and why
What is the potential impact of answering your question

13. How do you know you’ve got it
Tell your “story” and see if it sticks
Using the literature, provide a rationale
Explain what we already know
Identify the knowledge gap(s)
State your research question(s)
Describe the potential impact of your results

14. “If you can't explain it to a six year old, you don't understand it yourself.” - Albert Einstein

15. The Dance of Design What is your unique opportunity to generate new knowledge?
The question What is known What is feasible Some questions may be best answered in small, thoughtful steps. PEER is just the start of your research.

16. The Circle of Inquiry Identify Knowledge Gap
Search for Existing Information
Disseminate & Implement
Focus the Study Question
Analyze & Interpret Results
Collect Data
Design the Study
Adapted from: Nutting, PA, Stange, KC. Practice-based research: The opportunity to create a learning discipline. In: The Textbook of Family Practice, 6th Edition. Rakel RE (ed.), W. B. Saunders Company,

17. Design the Study Match the method to the question
Look for your particular opportunity to access meaningful data
How can you inform the knowledge gap?
Use the literature to begin to inform your design
What has already been done?
What were the strengths/flaws to designs?
What needs to be done next to answer the gap?

18. Measurement Observe measurement issues as you read the literature
Quantitative instruments
Items & scales
Validity, reliability, relevance
Existing scales or develop your own
Qualitative
Survey / interview
Direct observation
Case study
Etc.

19. Developing as a Researcher
Pay attention and reflect
In practice
In reading
In conversation
Learning opportunities
Seminars/Conferences
Reading
Doing your own studies
Joining others in their studies
Seek and draw lessons from feedback
Carry a way to record your thoughts
Let your mind wander
Constantly ask questions

20. Identifying a knowledge gap
Questions from practice
Questions from listening
Questions from reading
Evolving questions from
Discussion (develop a team)
Action/ reflection cycles
Openness
Making space
Write them down. Talk them out. (With anyone who’s willing.)

21. Search for existing knowledge
Sources
Google Scholar
PubMed
Social science literature databases
Ask colleagues & experts
Professional organizations
Look for
Holes/Gaps
Paradoxes that are ignored or explained away
(and…the “future studies” statements at the end of a journal article)

22. Focus the Question Person/Client perspective
To fill gaps in existing knowledge
Common holes to fill:
Person/Client perspective
Agency/provider perspective
Implementation
Boundary regions between disciplines

23. Evaluation (Research)
Evaluation is research and follows the same process
What is the (evaluation) question?
Is it RELEVANT? Who cares about this and why?
How will the results be used?
The audience for evaluation research may be more narrow
Typically examines aspects of a program
Could be process, could be outcomes

24. Questions & Discussion