Research Methods of Applied Linguistics and Statistics (3) Types of research Constructing Research Designs.

Slides:



Advertisements
Similar presentations
Andrea M. Landis, PhD, RN UW LEAH
Advertisements

Defining Characteristics
Exploring Marketing Research William G. Zikmund
GROUP-LEVEL DESIGNS Chapter 9.
Experimental Research Designs
Chapter 9 Group-Level Research Designs. CHARACTERISTICS OF “IDEAL” EXPERIMENTS Controlling the Time Order of Variables Manipulating the Independent Variable.
SOWK 6003 Social Work Research Week 4 Research process, variables, hypothesis, and research designs By Dr. Paul Wong.
SOWK 6003 Social Work Research Week 4 Research process, variables, hypothesis, and research designs By Dr. Paul Wong.
Group-Level Research Designs
Chapter 13: Descriptive and Exploratory Research
Questions What is the best way to avoid order effects while doing within subjects design? We talked about people becoming more depressed during a treatment.
Educational Research by John W. Creswell. Copyright © 2002 by Pearson Education. All rights reserved. Slide 1 Chapter 11 Experimental and Quasi-experimental.
RESEARCH METHODS Lecture 19
9 Quantitative Research Designs.
6-1 Chapter Six DESIGN STRATEGIES. 6-2 What is Research Design? A plan for selecting the sources and types of information used to answer research questions.
Designs that allow testing of hypotheses.  Describe pre-experimental, experimental and quasi-experimental research designs.  Explain the types of conclusions.
Chapter 9 Experimental Research Gay, Mills, and Airasian
McGraw-Hill © 2006 The McGraw-Hill Companies, Inc. All rights reserved. Experimental Research Chapter Thirteen.
L1 Chapter 11 Experimental and Quasi- experimental Designs Dr. Bill Bauer.
Experimental Research
Formulating the research design
The Research Process. Purposes of Research  Exploration gaining some familiarity with a topic, discovering some of its main dimensions, and possibly.
Problem identification Research questions Constructs & Variables Research design.
Experimental Research Take some action and observe its effects Take some action and observe its effects Extension of natural science to social science.
Chapter 8 Experimental Research
RESEARCH DESIGNS FOR QUANTITATIVE STUDIES. What is a research design?  A researcher’s overall plan for obtaining answers to the research questions or.
I want to test a wound treatment or educational program but I have no funding or resources, How do I do it? Implementing & evaluating wound research conducted.
Intro to Computing Research
I want to test a wound treatment or educational program but I have no funding or resources, How do I do it? Implementing & evaluating wound research conducted.
Chapter 2: The Research Enterprise in Psychology
An Introduction to Research Methodology
Chapter 2: The Research Enterprise in Psychology
Research Tools and Techniques
Experimental and Quasi-Experimental Designs
McGraw-Hill © 2006 The McGraw-Hill Companies, Inc. All rights reserved. The Nature of Research Chapter One.
Research Design for Quantitative Studies
Quantitative Research Designs
Group Discussion Explain the difference between assignment bias and selection bias. Which one is a threat to internal validity and which is a threat to.
Methodology Describe Context & setting Design Participants Sampling Power Analysi s Interventions Outcome (study variables) Data Collection Procedures.
Power Point Slides by Ronald J. Shope in collaboration with John W. Creswell Chapter 11 Experimental Designs.
MARKETING RESEARCH CHAPTERS
Evaluating a Research Report
Copyright ©2008 by Pearson Education, Inc. Pearson Prentice Hall Upper Saddle River, NJ Foundations of Nursing Research, 5e By Rose Marie Nieswiadomy.
Group Quantitative Designs First, let us consider how one chooses a design. There is no easy formula for choice of design. The choice of a design should.
Experimentation Chapter 14 Cooper and Schindler. What is Experimentation? Causal method Allow the researcher to alter systematically the variables of.
Quantitative and Qualitative Approaches
1 Experimental Research Cause + Effect Manipulation Control.
Experimental Designs. Experiments are conducted to identify how independent variables influence some change in a dependent variable.
Question paper 1997.
Chapter 10 Experimental Research Gay, Mills, and Airasian 10th Edition
Chapter 10 Finding Relationships Among Variables: Non-Experimental Research.
EDU 5900 AB. RAHIM BAKAR 1 Research Methods in Education.
Experimental & Quasi-Experimental Designs Dr. Guerette.
SOCW 671: #6 Research Designs Review for 1 st Quiz.
Journalism 614: Experimental Methods Experimental Research  Take some action and observe its effects –Extension of natural science to social science.
Chapter Nine Primary Data Collection: Experimentation and
Experimental and Ex Post Facto Designs
 Describe the defining characteristics of quantitative research studies.  List and describe the basic steps in conducting quantitative research studies.
What is Research Design? RD is the general plan of how you will answer your research question(s) The plan should state clearly the following issues: The.
Research design By Dr.Ali Almesrawi asst. professor Ph.D.
School of Public Administration & Policy Dr. Kaifeng Yang 研究设计 : 实验研究的基本问题.
Educational Research Experimental Research Chapter 9 (8 th Edition) Chapter 13 (7 th Edition) Gay and Airasian.
Chapter 6 Selecting a Design. Research Design The overall approach to the study that details all the major components describing how the research will.
Chapter 9 Scrutinizing Quantitative Research Design.
© Yosa A. Alzuhdy - UNY © Yosa A. Alzuhdy – FBS-UNY 2b. HOW and WHY of RESEARCH Quantitative Research © Yosa A. Alzuhdy, M.Hum. English.
Part Two.
EXPERIMENTAL RESEARCH
DUET.
Chapter Eight: Quantitative Methods
Reminder for next week CUELT Conference.
Presentation transcript:

Research Methods of Applied Linguistics and Statistics (3) Types of research Constructing Research Designs

Research Types Data sources: Data sources: primary, secondary primary, secondary Objectives: Objectives: descriptive, exploratory, correlational, explanatory descriptive, exploratory, correlational, explanatory Time for data collection: Time for data collection: cross-sectional, longitudinal (panel, trend) cross-sectional, longitudinal (panel, trend) Data types: Data types: qualitative, quantitative (survey, experimental) qualitative, quantitative (survey, experimental)

Primary: studies based on primary, or original, data sources, such as classroom observations or real students, or their test scores, or their responses to a questionnaire. Primary: studies based on primary, or original, data sources, such as classroom observations or real students, or their test scores, or their responses to a questionnaire. Secondary: studies based on secondary sources such as other researhers ’ books and articles, including library research and literature reviews. Secondary: studies based on secondary sources such as other researhers ’ books and articles, including library research and literature reviews.

Descriptive: studies involving the collection of data in order to test hypotheses or to answer questions concerning the current status of the subjects of the study. Descriptive: studies involving the collection of data in order to test hypotheses or to answer questions concerning the current status of the subjects of the study. Exploratory: studies conducted into an issue or problem where there are few or no earlier studies to refer to. The focus is on gaining insights and familiarity for later investigation. Exploratory: studies conducted into an issue or problem where there are few or no earlier studies to refer to. The focus is on gaining insights and familiarity for later investigation.

Correlational: The basic question for descriptive research is - "What are the values of a number of variables for a given sample of subjects. The basic research question for correlation research is - What is the relationship between two or more variables for a given set of subjects. Notice that we said relationship between variables and not the effect of one variable on another variable. Correlational: The basic question for descriptive research is - "What are the values of a number of variables for a given sample of subjects. The basic research question for correlation research is - What is the relationship between two or more variables for a given set of subjects. Notice that we said relationship between variables and not the effect of one variable on another variable. In descriptive research we are just describing our subjects in terms of one or more variables, while in correlational research we are looking at the relationship between the variables. In descriptive research we are just describing our subjects in terms of one or more variables, while in correlational research we are looking at the relationship between the variables.

Explanatory: studies conducted in order to explain any behaviour in L2 learning and teaching. It could be done by using questionnaires, group discussions, interviews, etc. Explanatory: studies conducted in order to explain any behaviour in L2 learning and teaching. It could be done by using questionnaires, group discussions, interviews, etc. Cross-sectional: studies in which groups of participants of different ages are observed and compared at a given time Cross-sectional: studies in which groups of participants of different ages are observed and compared at a given time

Longitudinal: It is research that studies a person or group over a set period of time, normally to track the effect of some variable and as such permits causal pathways to be determined. For example, when trying to discover the effect of language aptitude on L2 proficiency, a researcher may track 50 children over a 30 year period in order to find a common factor between those children who are quicker and effective L2 learners. Longitudinal: It is research that studies a person or group over a set period of time, normally to track the effect of some variable and as such permits causal pathways to be determined. For example, when trying to discover the effect of language aptitude on L2 proficiency, a researcher may track 50 children over a 30 year period in order to find a common factor between those children who are quicker and effective L2 learners.

Constructing Research Designs Determining the design Determining the design Classification of designs Classification of designs

Research Design 1: Quantitative / Qualitative Quantitative research is empirical research involving analysis of numerical data Its aim is to determine the relationship between one thing (an independent variable) and another (a dependent or outcome variable) in a population. Quantitative research designs are either descriptive (subjects usually measured once) or experimental (subjects measured before and after a treatment). A descriptive study establishes only associations between variables. An experiment establishes causality. Quantitative research is empirical research involving analysis of numerical data Its aim is to determine the relationship between one thing (an independent variable) and another (a dependent or outcome variable) in a population. Quantitative research designs are either descriptive (subjects usually measured once) or experimental (subjects measured before and after a treatment). A descriptive study establishes only associations between variables. An experiment establishes causality.

Qualitative research is empirical research involving analysis of data such as words (e.g., from interviews), pictures (e.g., video), or objects (e.g., an artifact). Its aim is building theory. It is generally inductive in approach, is based originally on the naturalistic assumption that reality is mind dependent (i.e., can only be known as it is interpreted and has "meaning" for the observer), is usually of single-subject design, and generally deals with nominal data Qualitative research is empirical research involving analysis of data such as words (e.g., from interviews), pictures (e.g., video), or objects (e.g., an artifact). Its aim is building theory. It is generally inductive in approach, is based originally on the naturalistic assumption that reality is mind dependent (i.e., can only be known as it is interpreted and has "meaning" for the observer), is usually of single-subject design, and generally deals with nominal data

Philosophical base Phenomenology (quali.)Logical positivism (quan.) Data from personal involvement Data from experiment SubjectiveObjective EmpathyDistance Data in the form of string Data in form of number

Methodology SyntheticAnalytic From part to wholeFrom whole to part Internal structureExternal structure Process-orientedProduct-oriented Dynamic reality hypothesis Static reality hypothesis

Purpose InductionDeduction Hypothesis generatingHypothesis confirming Exploratory, descriptiveInferential, confirmatory, explanatory Result of pattern or hypothesis Formation of theory

Features for both designs High validity, low reliability Low validity, high reliability Preference of content to form Preference of form to content Low generalizing capacity High generalizing capacity High explanatory power Low explanatory power

Mixed design: Balanced

Mixed design: Unbalanced

Literature review of quantitative and qualitative studies by Lazaraton (2000) Current Trends in Research Methodology and Statistics in Applied Linguistics Current Trends in Research Methodology and Statistics in Applied Linguistics

Research Design: experimental research An experimental study is a study in which the researcher manipulates one or more independent variables and measures their effect(s) on one or more dependent variables while controlling the effect of extraneous variables. An experimental study is a study in which the researcher manipulates one or more independent variables and measures their effect(s) on one or more dependent variables while controlling the effect of extraneous variables.

Major components of experimental research The population The population A comparison which involves at least 2 groups or 2 conditions. A comparison which involves at least 2 groups or 2 conditions. The treatment The treatment A treatment in which one or more independent variables are manipulated. A treatment in which one or more independent variables are manipulated. The measurement of the treatment The measurement of the treatment The measurement of dependent variable(s) as the result of the treatment. The measurement of dependent variable(s) as the result of the treatment.

Letters standing for components X: an experimental treatment (teaching method, exposure to a material … ) X: an experimental treatment (teaching method, exposure to a material … ) - : no treatment - : no treatment O: measurement of the effects of the treatment or a test O: measurement of the effects of the treatment or a test R: randomization, or the random assignment of subjects to groups in order to control for extraneous variables. R: randomization, or the random assignment of subjects to groups in order to control for extraneous variables.

Types of experimental studies Pre-experimentalQuasi-experimentalTrue-experimental One-group posttest only study: X O 1 Posttest only nonequivalent groups: EG: X O 1 CG: O 2 Posttest only equivalent groups EG: R X O 1 CG: R O 2 One-group pre/post test study: O 1 X O 2 Pre/post test nonequivalent groups: EG: O 1 X O 2 CG: O 3 O 4 Pre/post test equivalent groups: EG: R O 1 X O 2 CG: R O 3 O 4 No control group; no randomization a control group; no randomization a control group + randomization

Pre-experimental: single group design One-shot design One-shot design One group pretest-posttest design One group pretest-posttest design Time-sampling design Time-sampling design

Example for one-shot design An English teacher used the immersion teaching method for teaching English communicative use. After 3 weeks, a test is administered to the class. The class performs well on the test. An English teacher used the immersion teaching method for teaching English communicative use. After 3 weeks, a test is administered to the class. The class performs well on the test. X O X O It is also called a pilot study design It is also called a pilot study design

One group pretest-posttest design It is also called repeated measures. It uses the subjects as their own controls and eliminates the need for a control group design. It is also called repeated measures. It uses the subjects as their own controls and eliminates the need for a control group design. Strength: It is efficient because some extraneous variables, attrition and loss of subjects controlled. Strength: It is efficient because some extraneous variables, attrition and loss of subjects controlled. Weakness: validity might be affected by history, attrition and maturation. Weakness: validity might be affected by history, attrition and maturation.

Time-sampling design It is also referred to as time-series designs. It is also referred to as time-series designs. O 1 O 2 O n X O 4 O 5 O n O 1 O 2 O n X O 4 O 5 O n O 1 X O 2  O 3 -- O 4  O 5 X O 6  O 7 -- O 8 O 1 X O 2  O 3 -- O 4  O 5 X O 6  O 7 -- O 8

Quasi-experimental: posttest only nonequivalent groups EG: X O 1 CG: O 2 O 1 -O 2 The treatment effect could also be attributed to a series of extraneous variables.

Posttest/posttest nonequivalent groups EG: O 1 X O 2 CG: O 3 O 4 (O 2 -O 1 ) – (O 4 -O 3 ) (O 2 -O 1 ) – (O 4 -O 3 ) Without randomization, the confounding effects cannot be fully controlled. Without randomization, the confounding effects cannot be fully controlled.

True-experimental: posttest only equivalent groups EG: R X O 1 CG: R O 2 O 1 -O 2 The design assumes that the 2 groups are identical on all relevant features, but it is still sensitive to selection bias, mortality, Hawthorn effects.

Pre/post test equivalent groups EG: R O 1 X O 2 CG: R O 3 O 4

Factorial Designs Factorial designs are similar to the true experimental designs and include all of the elements found in those designs, such as randomization, pre- and post-testing, and treatments. The difference is that the effects of independent variables and a moderating variable may be tested at the same time. Factorial designs are similar to the true experimental designs and include all of the elements found in those designs, such as randomization, pre- and post-testing, and treatments. The difference is that the effects of independent variables and a moderating variable may be tested at the same time.

Example for a factorial design In a study to test for the effect of language laboratory training on pronunciation, it is also decided to measure for the effect of such training on learners of different proficiency levels, as well as the effect of different kinds of language laboratory practice. In a study to test for the effect of language laboratory training on pronunciation, it is also decided to measure for the effect of such training on learners of different proficiency levels, as well as the effect of different kinds of language laboratory practice. Types of practice (contextualized / decontextualized drills), proficiency levels of the learner (beginners / advanced learners ) Types of practice (contextualized / decontextualized drills), proficiency levels of the learner (beginners / advanced learners )

Exemplary Figure for Factorial Design

Experimental study by Robb, et al. (1986) Salience of feedback on error and its effect on EFL writing quality Salience of feedback on error and its effect on EFL writing quality Salience of feedback on error and its effect on EFL writing quality Salience of feedback on error and its effect on EFL writing quality Types of feedback: complete correction, coded feedback; yellow highlighting of errors, listing the number of errors Types of feedback: complete correction, coded feedback; yellow highlighting of errors, listing the number of errors The compositions were graded on 19 features which were collapsed into 7 categories. The compositions were graded on 19 features which were collapsed into 7 categories.