S EPTEMBER SESSION 1: R ESEARCH Q UESTIONS & S ELECTING M EASURES The first steps toward getting your EA research project underway

Y OUR EA R ESEARCH P ROJECT As an EA, you will be evaluating the impact of your SLC project Why? Because you need to know if it works Typical SLC projects do outreach to a group of students with a particular goal in mind, say, getting middle school students excited about computing. If you don’t evaluate it, you won’t know if it is effective!

E VALUATION Evaluation is the systematic acquisition and assessment of information to provide useful feedback about some object Questions about the value or worth of something Context of how value is measured The data measured The knowledge from the measured data 1. Research Question 2. What data can be collected 3. What data will be collected, when, how & by whom 4. Findings & Interpretations

R ESEARCH Q UESTIONS Questions require answers- They give you a way to evaluate evidence Clear, open-ended questions require research and critical thinking

STEP 1: I DENTIFY T OPIC What are you interested in? What is your SLC doing? What are you curious about? Example: an EA, is curious if doing a pair programming activity with students and parents will make parents think that computer science is a viable option for their kids

STEP 2: B RAINSTORM Start asking questions about the SLC outreach What are the goals of the SLC outreach How will you know that the goals are being met? Example: EA starts thinking about how he can answer his question; do parents enjoy the activity? Do they think computing is a viable career for their kids? Is there a larger effect on the kids attitudes when doing pair programming with their parents, versus a group of similar kids doing pair programming with peers?

S TEP 3: H YPOTHESIZE What do you expect to happen? What do you think the outcome will be? Or what is the intended outcome? Are there several expected outcomes? Prioritize What’s feasible in your timeframe? For our example: The ability to get groups of parents and a comparable group of kids will determine the questions and research design

S TEP 4: D ESIGN YOUR R ESEARCH What measurement tool(s) will you use? Student survey, knowledge test, interviews, observation Teacher survey, interviews Parents survey, interviews What will be the design? Quasi-experimental: Pre/post assessment (before activity/after activity) [repeated measures design] Post assessment only Qualitative- interviews, focus groups, open ended items on a survey [mixed-methods]; observations (documented) Experimental: require control group

C OMMON R ESEARCH D ESIGNS Descriptive Goal is to observe & describe what happened Methods used: Surveys about opinions, attitudes, behaviors (called self- report) Observation Example- Collect teacher observation of classroom environment (mood of students) following an SLC activity

C OMMON R ESEARCH D ESIGNS Quasi-Experimental (because you have no ‘control’ group) Goal is to determine cause, i.e. see if your SLC outreach (called an “intervention”) creates the outcomes you hypothesize Methods used: Surveys about opinions, attitudes, behaviors (called self- report) Tests of knowledge Can do pre/post design (before and after intervention) Example- Collect student attitudes (survey) about robots before an SLC activity and after- works best for long-term interventions, not one time workshop

C OMMON R ESEARCH D ESIGNS Experimental (has ‘control’ group) Goal is to determine cause, i.e. see if your SLC outreach (called an “intervention”) creates the outcomes you hypothesize Control group for comparison, who has no ‘intervention’ Methods used: Surveys about opinions, attitudes, behaviors (called self-report) Tests of knowledge Can do pre/post design (before and after intervention) Example- Collect student attitudes (survey) about robots before an SLC activity and after- works best for long-term interventions, not one time workshop; collect same survey from a classroom of students in same school and same grade who didn’t have the SLC activity

Q UALITATIVE Interviews Protocol- the questions you will ask Semi-structured vs Structured Focus Groups Small groups of 8-10 people Observations Identify what you will record Documentation (form, pictures, recordings, etc.)

W HAT WILL YOU DO ? Report back in the EA session what your initial thoughts are Your write up is your IRB