1. S570: Session 11 Turning the Tables: Evaluating Research Proposals Jal Mehta November 15, 2012

2. Plan You are the COD (10:15 – 11:25) Debrief of COD experience (11:40 – 12:00) Mini lecture and discussion: Data collection and analysis (12:00 – 12:30) Reflection and workshop time: Data collection and analysis (12:30 – 1:00)

3. COD Exercise For each proposal: How did it fare by each of the criterion on the rubric? How compelling is the argument as a whole? Does it pass, pass with memo, or not pass?

4. COD Debrief What was particularly good about these proposals that you might want to emulate? In what ways were they not good, or at least not what you were trying to accomplish? Why might this be?

5. Mini Lecture: Data Collection and Analysis In one sense, these are technical questions: How do I design research that is unbiased? Valid? Reliable? See the wiki for more resources on how to address these questions (THANK YOU BECCA) But there are also some broader issues at stake

6. Where Should I Start? Starting points: Do you want to start from your data and work towards your question, or start from your question and work towards your data? Different disciplines have very different approaches to this; some privilege the data (economics), some privilege the question (sociology). You may want to iterate back and forth between these two possibilities.

7. The Data You Examine Is Your Study At the end of the day, you are what you research You are not what you write your lit review about If lost in the theory, trying to figure out what you might practically do – who you are going to talk to, what you are going to ask – can help to make it more concrete

8. Instruments Must Align With Theoretical Framework and Research Questions Can see this in the sample DPs for today – real effort to map how the data they were collecting would tie to the theoretical constructs they laid out. Specify, specify, specify. Justify, justify, justify. Can you actually get the data you need in the time you have? Two column exercise: What you actually want What you are going to ask to know (research qs) (interview questions)

9. Now or Later You don’t have to pin down everything at the proposal stage Can specifically leave open choice of some later research sites, research populations, etc., if you clearly explain what you are going to do first, and how that will likely lead to the information that will allow you to make an informed choice down the road.

10. Asking Good Questions Make your questions as simple as possible Try to eliminate assumptions from your questions, and instead begin as open-ended as possible Only ask questions that your informant is the best positioned person to answer Ask your respondents about ideas you are developing, and get their reactions. Play a little dumb – let them explain things to you

11. Mixed Methods Valuable only if there is a clear rationale for what each method will do for you Qual – quant – qual is a good pattern Initial qualitative pilot tells you what to look for Quantitative methods tell you how representative it is Return to qualitative to explain puzzling features of quant data

12. Reflection and Workshop Time: Focus on Data Collection and Analysis To reflect: Do I know what data I’m collecting? Do I know how I am planning on analyzing it? Is my data collection and analysis closely aligned with my research questions? How could I make it more so? To pose to the group: What questions do I have about my data collection and analysis strategy?