Experimental Research Methods in Language Learning Chapter 2 Experimental Research Basics

Leading Questions Do you believe in a cause-effect relationship in language learning? Why or why not? What do you think are characteristics of an experimental research study? What kind of research questions do you think experimental researchers ask?

Experimental Research Design A useful research methodology for those studies that aim to address a causal-like relationship. It allows researchers to strictly control the influence of factors that are not of interest by setting them constant across groups, but to vary the degree of a factor under study across groups of learners in order to undestand a causal-like relationship.

Key Characteristics of Experimental Research Causal or causal-like relationships Research questions in experimental research Independent and dependent variables Dichotomous and continuous variables Measurement scales of variables Constructs in experimental research Manipulation and control in experimental research

Causal or Causal-like Relationships Researchers can examine whether their hypothesis about a causal-like relationship is supported by empirical data Reasons for using ‘ causal-like ’ in place of ‘causal’ are that there is no direct proof that an experimental finding indicates a causal relationship (i.e. We can only make inferences). Statistical analysis in experimental research is not a method to discover causes.

Examples of Research Titles Baralt and Gurzynski-Weiss (2011): Comparing learners’ state anxiety during task- based interaction in computer-mediated and face-to-face communication Park (2010): The influence of pretask instructions and pretask planning on focus on form during Korean EFL task-based interaction Takimoto (2006): The effects of explicit feedback on the development of pragmatic proficiency

Research Questions Research questions are used to frame a research focus and method to answer them Two types of research questions: Theoretical questions, and practical or pedagogical questions. Theoretical questions are connected with basic research which seeks empirical evidence that can inform a new theory, or refine and extend existing theories. Example: What is motivation in L2 learning?

Research Questions Practical/pedagogical research questions are connected with applied research which aims to address a practical/pedagogical problem in a particular setting or to apply relevant theories or recommendations to real practice. Example: How can teachers help students memorize a grammar rule?

Examples of Research Questions Do adult and child dyads respond differently to the amount of implicit negative feedback provided to NNSs [non-native English speakers] during task-based interaction? (Mackey, Oliver & Leeman 2003, p.44) To what extent is Lexical Focus-on-Form beneficial during a focus on meaning activity (such as listening comprehension) in terms of students’ receptive vocabulary learning? (Tian & Macaro 2012, p. 373)

Independent and Dependent Variables Variable is an aspect or characteristic of something that can take different values or scores. Examples: age, gender, first language, length of learning, intelligence, English language proficiency, motivation and anxiety An independent variable is a variable that exists freely, and is hypothesized to have an effect on other variables that are described as dependent variables

Independent and Dependent Variables Independent variables are factors that influence certain behaviors or psychological processes. A dependent variable is a variable that changes as the independent variable being examined changes. Example: Gass and Mackey (1999): The effects of task repetition [IV] on linguistic output [DV]

Dichotomous and Continuous Variables Categorical variables are used to group non- overlapping variables such as English proficiency levels (beginning, intermediate and advanced). A dichotomous variable is the simplest type of categorical variable. It has only two classes (e.g., male or female, pass or fail). Continuous variables can be arranged from lowest to highest (e.g. age, length of residency)

Measurement Scales of Variables Nominal scales use numbers to label or classify variables into categories Ordinal scales are rank-order scales; used for ranking some quality or ability (e.g. grade point average (GPA)). Interval scales have the features of both ordinal scales and equal distances or intervals (e.g. language test scores, personality scores). Ratio scales are measurements with all the properties of nominal, ordinal and interval scales, and also possess a true zero

Constructs in Experimental Research revisited Constructs are aspects or abstract concepts that researchers seek to understand (e.g. intelligence, language proficiency, memory, language aptitude). Constructs cannot be seen directly or be easily measured, thereby needing a sound theory to help us define and measure them.

Constitutive versus Operational Constructs A constitutive construct is one defined using the general definition of a term (e.g., motivation, self-regulation, language learning strategies, and self-efficacy). An operational construct definition is one defined by researchers for a particular study. It is often concerned with exactly what a researcher means in a research setting and how it can be measured using a designed instrument.

Manipulation and Control in Experimental Research Manipulation is related to how experimental researchers manipulate independent variables. It aims to rule out other plausible rival explanations of the research outcomes. Manipulation helps researchers control confounding variables as well as to systematically vary the independent variable for testing its effect.

Manipulation and Control in Experimental Research Confounding variables are those independent variables which are not of interest but can interact with the independent variable to co-affect the dependent variable. Manipulation can also take place when researchers hold several conditions for two or more groups of comparisons constant, to avoid the potential confounding effects.

Discussion What are the ways in which experimental researchers can produce evidence of causality in language learning? What is random assignment? Why is it essential for a true experimental study? Can you think of a situation in which you can do an experimental study? Think of a situation in which you can have control over your research. What would be the potential confounding variables that can influence your findings?