1. L9:Big Idea 1-Question and Explore-Aligning the Inquiry Approach, Design, and Method

2. Lesson 9: Focus
Describe what you think is meant by the following terms as they pertain to a student’s inquiry:
Approach
Design
Method
Revise your definitions after the lesson if necessary.

3. Misconception Alert #1
You may think you need to be an expert in every discipline-specific method of inquiry.
This is not true. You simply need to be able to access information, sources and evidence to help you choose or develop an aligned method of inquiry to collect and analyze data appropriate to the field of your topic of inquiry.
You also need to be able to use other’s studies and evidence (within the discipline) to provide rationale for your choices.

4. Misconception Alert #2
You may think that good research collects only primary data. This is not true.
Many great studies or meta-studies collect secondary data/information and add their own interpretation to that data in a new context, which is a valid form of research.
However, when collecting secondary data and analyzing such, you might mistake what is a meta-analysis, content analysis versus performing another literature review.

5. A Meta-Analysis involves Statistics
If a student says “I performed a meta-analysis” and it’s a summary of reports that are already known in the field….this is not evidence generated from a true meta-analysis. This will lead to an oversimplified conclusion made from a review of the literature of the field.
A meta-analysis is a statistical technique that allows the researcher to combine the results of several studies to determine whether or not they yield consistent, predictable results that can be generalized to a different population, using a different treatment, or in different setting.
This involves careful selection of the studies to be analyzed, their data, and the statistical technique to be used on all the data to show similarities or differences between the studies in addressing the same problem but maybe from different angles.
Students don’t really seem to know what is a meta-analysis and this misunderstanding is made bare in their academic paper but is made even more so in the presentation when they have to speak about the links between their claims and evidence.
So, when a student says “I have performed a meta-analysis”….you should see choices made about what “studies (not informational pamphlets) to look at, what variables within those studies to quote/unquote study, and what statistical analyses to run on those variables to determine if this conclusion emerges that can be translated to multiple populations, multiple settings or contexts.

6. A Content Analysis involves Coding and Emerging Themes
Some students may say “hey, I am doing a “content analysis” of all this literature in the field” and then they simply report back an oversimplified conclusion of what is already known in the field. This is not a content analysis—this is a literature review.
A content analysis is computerized technique that allows researchers to analyze text to classify terms or phrases that are present in frequency in order to identify themes that are common between texts.
This involves coding and interpretation of data. If you don’t see evidence of student telling you the content they are choosing, how they are coding it, and how they are interpreting it into emerging themes, then they are not backing up claims with evidence generated via this research method. Instead they are doing a literature review and this will generate an oversimplified, generalized conclusion, every time.
So- to be clear, in many student samples, you may see students reviewing a lot of literature of what is already known in the field and calling this a meta or content-analysis. Yet they do not show the steps of either of these methods nor do they present evidence generated from such in order to back their claims. Therefore, the claims being made will be generalized or oversimplified as they emerge simply from a review of the literature of the field and not a true research method.

7. Approach: Exploratory, Explanatory, or Create
Explore a phenomenon/event/thing and look for themes, variables, and trends (this usually takes place when variables, trends, and/or themes are not known or not well understood).
Explain connections between and patterns within phenomena, events, variables (this usually takes place when variables about a phenomena are known but the connections between them are not).
Create or design a solution to a problem (this could be a new understanding under “explore” or “explain” but in this sense, its meant too be a process, design, or additional piece of scholarly work as a result of the research).

8. Design: Experimental or Non-experimental
Experimental Design usually involves pre-test, post-test, identifying correlations between variables, random assignment, control groups etc.
Non experimental Design does not include a large measure of control and is usually assigned to case studies, interviews, and largely qualitative studies (although some qualitative studies may be highly experimental).
Regardless, both designs must be explicit and clear so as to be reproducible.
Experimental research:
manipulate the predictor variable and subjects to identify a cause-and-effect relationship
requires an experimental group, or the ones being manipulated, while the other is placed in a placebo group, or inert condition or non-manipulated group
a laboratory-based experiment gives a high level of control and reliability.
Non-experimental research:
cannot control, manipulate or alter the predictor variable or subjects, but instead, relies on interpretation, observation or interactions to come to a conclusion.
relies on correlations, surveys or case studies, and cannot demonstrate a true cause-and-effect relationship.
non-experimental research tends to have a high level of external validity, meaning it can be generalized to a larger population.

9. Research Method: Qualitative, Quantitative, or Mixed
Qualitative, quantitative, or mixed methods are explicit in:
what data/information is collected
Primary
Secondary
Text, numbers, images, etc.
how data/information is collected
how data/information is interpreted/analyzed
Quantitative research is empirical research where the data are in the form of numbers. Qualitative research is empirical research where the data are not in the form of numbers. (Punch, K; 1998)

11. When in Doubt, Check the Course and Exam Description
EK 1.5 B5-provides general descriptions of qualitative and quantitative methods.
Students don’t have to memorize these methods.
Students MUST know how to choose which method to govern their research and provide effective rationale for why that method is appropriate/aligned as opposed to others.

12. Comparing Qualitative and Quantitative Purposes
Focus is on objectivity
Explanatory
Focus is on interpretation
Descriptive/Exploratory
Data is counted/measured (i.e. uses numbers). It tends to come from lots of people.
Data is descriptive often textual. It tends to focus on a specific case or cases.
Pre-organised data categories
Open and flexible – data categories emerge through the process.
Focus on “facts” and the relationship between one set of facts and another
Focus on meanings
Aims to find generalizable conclusions
Aims to find insights into particular contexts

13. Comparing Data Collection Methods
Quantitative research methods include:
Surveys
Laboratory experiments
Questionnaires
Qualitative research methods include:
Case study research - a case study is an empirical enquiry that investigates a contemporary phenomenon within its real-life context
Ethnography- the ethnographer immerses her/himself in the life of people s/he studies and seeks to place the phenomena studied in its social and cultural context
Observations
Interviews
Focus Groups

14. Differences in Qualitative and Quantitative Analysis and Reporting Methods
Components
Qualitative Reporting/Analyses
Quantitative Reporting/Analyses
Data Type Emphasized
narrative, text
numbers
Categorization of data
Coded themes
Variables, statistical coefficients
Condensed presentation of data
Narrative examples
Graphs, charts, data tables
Rationale
Claims supported with contextual and narrative data
Claims supported with statistical data
Logical pathway to conclusion
Inductive
Deductive
Acknowledges Limits
Emphasis on bias, perspective, triangulation
Emphasis on correlation and measurement validity
General approach
Build a picture with rich triangulated data sources (Neuman, 2011).
Identify the strength of an assumed pattern within the big picture with statistical data

15. Primary and Secondary Research
Primary research involves collecting data through fieldwork, trials or experiments
Secondary research synthesizes existing research.
Primary Research
Secondary Research
Advantages
Disadvantages
Researcher can focus on both qualitative and quantitative issues
Can be very expensive in preparing and carrying out the research
Somebody has done all the legwork for you
Establishing credibility may be an issue
Addresses specific research issues
Requires the development and execution of a research plan
Others may have more resources at their disposal than you
Is it up to date?
Researcher controls the design to fit their needs
It takes longer to undertake primary research than to acquire secondary data
There is usually greater variety to choose from
Is it a fair representation?
Greater control of issues such as size of project, time frame, and goal
Low response rate has to be expected
It is usually quicker to find
It might not be an exact fit for your purpose

16. Example (Purpose, Data Collection, Data Analysis)
Amount of orange juice consumed associated with number of times you smiled in the morning (survey 900 people)
versus
Your explanation of how you felt in the morning associated with what you ate (interview 20 people)
Ask students to discuss the difference in method used for each of these questions as it pertains to purpose. Form of data collection, and data analysis procedures and whether or not the data collected should be primary or secondary data

17. Aligning Approach, Design, and Method in the Inquiry Process
After reviewing the different types of research approaches, designs, and general methods with your instructor, look at the Research Methods in a Nutshell table.
Use the description of the various methods to complete the data table below by indicating the approach, design, method, and type of data typically associated with each type of inquiry process.
Remind students that you could actually reason away how most of the methods could fall into any of the categories of approach, design, and/or method, depending on the type of data you collect, how you collect the data, and how you interpret it (or even what product you might create from it).

18. Inquiry Process
Approach
Design
Method
Primary/Secondary Data
Explore/Explain/Create
Exp/non-exp
Qnt., Qlt., Mxd.
1, 2, both
Case Study research
Causal‐comparative research
Content analysis
Correlational research
Descriptive research
Ethnographic
Experimental research
Grounded Theory
Hermeneutic research
Historical research
Meta‐analysis research
Narrative research
Phenomenology
Quasi‐experimental
True Experimental research

19. Inquiry Process
Approach
Design
Method
Primary/Secondary Data
Explore/Explain/Create
Exp/non-exp
Qnt., Qlt., Mxd.
1, 2, both
Case Study research
Explore
Non-Experimental
Qualitative
Usually 1°
Causal‐comparative research
Explain
Quantitative or Mixed
both
Content analysis
Correlational research
Explore/Explain
Quantitative
Descriptive research
Ethnographic
Experimental research
Experimental
Grounded Theory
Hermeneutic research
Usually 2°, sometimes both
Historical research
Meta‐analysis research
Qualitative/Quantitative/Mixed
Usually 2°
Narrative research
Phenomenology
Quasi‐experimental
Quantitative/Mixed
True Experimental research
Remind students that these are sample answers and that students don’t have to be tested on this information BUT they should be able to choose an aligned method and provide rationale for their choice.

20. Decision Tree for Methods
Research Question
Purpose: Explore
Non Experimental
Qualitative (usually)
Purpose: Create
Aesthetic/Engineer Processes
Workshopping, storyboarding, etc.
Purpose: Explain
Experimental
Quant/Mixed (usually)
Any well-informed reader should be able to identify the purpose of the study (explore, explain, and/or create) from the research question. The purpose of the study should LEAD you to the right design (experimental versus nonexperimental). Most explore studies are nonexperimental. THIS IS NOT THE ONLY way to make decisions about alignment of method (there are nuances). However, students must provide effective rationale for their decisions (based on the review of literature and what other studies have employed in their field of inquiry.

21. The Importance of Alignment
The purpose (approach) of the inquiry is the basis for the research question.
The narrow context/variables/scope of the research question as well as the purpose (approach) of the inquiry should point to the design of the inquiry process.
Finally, the design of the inquiry process provides the foundation needed for the students to choose and then implement an aligned method.
If any of these components are misaligned, it may be very difficult for the reader to interpret a researcher’s results and could even hamper the validity of the research as well as the credibility of the researcher.

22. Case and Point
If the purpose of the inquiry is to explore the variables and themes within a phenomenon and the design is experimental requiring a pre-test and post test of unknown variables, the research will ultimately fail (seeing as the variables to be tested are not even known to exist).
If the purpose of the inquiry is to create a new design or process yet the researcher wants to explain whether or not the new process/design was effective as an intervention…students need to know this is misaligned. An intervention strategy is an “explain” purpose with an “experimental” design.

23. Check Your Alignment
Your instructor will assign you one of the excerpts below. Use your assigned excerpt (1–4) to identify the research question, approach, design, and method.
Evaluate the extent to which the excerpt presents an aligned research question, approach, design, and method. Use the questions below to guide your evaluation:
1. Is the method clearly articulated? (Provide evidence.)
2. Is the method congruent/aligned with the approach inherent in the research question? (Provide rationale.)
3. Is the design aligned to the research question, approach, and method? (Provide evidence.)
4. What if any components are not clear or are missing from the excerpt in reference to:
a. a well-formed, focused research question;
b. a clearly articulated method for collecting data/information to answer the research question; and
c. an aligned approach, design, and method to the research question/purpose of the study
Provide time for groups to present their results

24. Reflection-Include your response in your Trello.
What criteria or rules will you use to determine whether or not the method you designed or chose for your research is aligned with the purpose of your question?
Where can you look to get ideas about aligned research methods to help you choose or develop your own?