Experimental Research I Day 4. For Tomorrow Email first draft of introduction by class tomorrow. –Follow framework from day 1 and in syllabus –Use published.

Slides:



Advertisements
Similar presentations
Educational Research: Causal-Comparative Studies
Advertisements

McGraw-Hill/Irwin © 2003 The McGraw-Hill Companies, Inc.,All Rights Reserved. Part Three SOURCES AND COLLECTION OF DATA.
Experimental and Quasi-Experimental Research
Defining Characteristics
Inadequate Designs and Design Criteria
GROUP-LEVEL DESIGNS Chapter 9.
Experimental Research Designs
Correlation AND EXPERIMENTAL DESIGN
Research Design and Validity Threats
Research Problems.
Two essential characteristics of an experiment:
Aaker, Kumar, Day Seventh Edition Instructor’s Presentation Slides
CHAPTER CHAPTER 7 Experimental and Quasi-Experimental Research.
Today Concepts underlying inferential statistics
Chapter 9 Experimental Research Gay, Mills, and Airasian
Experimental Research
McGraw-Hill © 2006 The McGraw-Hill Companies, Inc. All rights reserved. Experimental Research Chapter Thirteen.
Experimental Research
Nasih Jaber Ali Scientific and disciplined inquiry is an orderly process, involving: problem Recognition and identification of a topic to.
Basic Principles of Research Design
EVALUATING YOUR RESEARCH DESIGN EDRS 5305 EDUCATIONAL RESEARCH & STATISTICS.
Chapter 8 Experimental Research
Experimental Design The Gold Standard?.
I want to test a wound treatment or educational program in my clinical setting with patient groups that are convenient or that already exist, How do I.
Quantitative Research Designs
Group Discussion Explain the difference between assignment bias and selection bias. Which one is a threat to internal validity and which is a threat to.
Copyright © 2008 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved. John W. Creswell Educational Research: Planning,
Day 6: Non-Experimental & Experimental Design
Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved.
Copyright ©2008 by Pearson Education, Inc. Pearson Prentice Hall Upper Saddle River, NJ Foundations of Nursing Research, 5e By Rose Marie Nieswiadomy.
Chapter Eight. Figure 8.1 Relationship of Experimentation to the Previous Chapters and the Marketing Research Process Focus of This Chapter Relationship.
Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved.
Group Quantitative Designs First, let us consider how one chooses a design. There is no easy formula for choice of design. The choice of a design should.
Chapter 14 Cooper and Schindler
Experimentation Chapter 14 Cooper and Schindler. What is Experimentation? Causal method Allow the researcher to alter systematically the variables of.
Types of Research and Designs This week and next week… Covering –Research classifications –Variables –Steps in Experimental Research –Validity –Research.
Understanding Research Design Can have confusing terms Research Methodology The entire process from question to analysis Research Design Clearly defined.
1 Experimental Research Cause + Effect Manipulation Control.
Educational Research Chapter 13 Inferential Statistics Gay, Mills, and Airasian 10 th Edition.
Introduction section of article
Experimental Research & Understanding Statistics.
 Descriptive Methods ◦ Observation ◦ Survey Research  Experimental Methods ◦ Independent Groups Designs ◦ Repeated Measures Designs ◦ Complex Designs.
Chapter 6 Research Validity. Research Validity: Truthfulness of inferences made from a research study.
Chapter 10 Experimental Research Gay, Mills, and Airasian 10th Edition
Experimental Research Methods in Language Learning Chapter 5 Validity in Experimental Research.
Experimental Research Hanser and Wheeler. Principles Independent Variable Dependent Variable.
Chapter 11.  The general plan for carrying out a study where the independent variable is changed  Determines the internal validity  Should provide.
Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved.
Essentials of Marketing Research William G. Zikmund Chapter 9: Experimental Research.
11 Chapter 9 Experimental Designs © 2009 John Wiley & Sons Ltd.
Experimental Research I Day 3. Business For Tomorrow One article from Day 2 reading list. Prepare to summarize & comment Examine 1 Thesis proposal online.
© 2011 Pearson Education, Inc. All rights reserved. This multimedia product and its contents are protected under copyright law. The following are prohibited.
Chapter 18 Experimental and Quasi- experimental Research.
PS Research Methods I with Kimberly Maring Unit 9 – Experimental Research Chapter 6 of our text: Zechmeister, J. S., Zechmeister, E. B., & Shaughnessy,
Experimental Design. Talking While Driving: Are Cell Phone Conversations Different from Passenger Conversations? Research Problems Talking on a cellphone.
Research designs Research designs Quantitative Research Designs.
Educational Research Experimental Research Chapter 9 (8 th Edition) Chapter 13 (7 th Edition) Gay and Airasian.
Experimental and Quasi-Experimental Research
EXPERIMENTAL RESEARCH
Experimental Research Designs
Chapter 8 Experimental Design The nature of an experimental design
Understanding Results
Part Three SOURCES AND COLLECTION OF DATA
Ron Sterr Kim Sims Heather Cruz aka “The Carpool”
2 independent Groups Graziano & Raulin (1997).
Chapter 6 Research Validity.
Chapter 11 EDPR 7521 Dr. Kakali Bhattacharya
Types of Designs: R: Random Assignment of subjects to groups
Reminder for next week CUELT Conference.
Presentation transcript:

Experimental Research I Day 4

For Tomorrow first draft of introduction by class tomorrow. –Follow framework from day 1 and in syllabus –Use published articles as a guide –Feel free to cite authors that support your statements – Non-research articles are OK for this. –Before the purpose statement, explain the problem or the issues involved. This material will lead to your purpose statement.

Review Standard Error of the Mean Confidence –Level –Interval –Limits Upper bound Lower bound

Experimental Research Only type of research with an intervention A direct attempt to influence a particular variable Only method that can truly begin to untangle cause and effect hypotheses Directional Hypothesis = Theory statement predicting the outcome [directional] (There will be a significant difference…). Reflects researcher’s expectations. Bilingual 3 rd graders taught with the Kodaly method will demonstrate significantly higher musical achievement than bilingual 3 rd graders taught with a traditional eclectic method.

Null Hypothesis Null Hypothesis = Theory statement predicting the outcome stated in the negative [non- directional] (There will be no significant difference…) The statistical hypothesis that states that there are no differences between observed and expected data. Does not reflect researcher’s expectations (value free) –There will be no significant difference in musical achievement of bilingual 3 rd graders taught with the Kodaly methods and bilingual 3 rd graders taught with a traditional eclectic method. –The goal is to REJECT the Null Hypothesis based (on 95% Confidence level or above) –Cannot prove the null hypothesis (a negative)

Type I and Type II Error Type I Error is erroneously claiming statistical significance or rejecting the null hypothesis when in fact, it’s true (claiming success when experiment failed to produce results) –Possible w. incorrect statistical test Type II Error is when a researcher fails to reject the null hypothesis when it is in fact false (claiming failure when successful) –The smaller the sample size, the more difficult it is to detect statistical significance –In this case, a researcher could be missing an important finding because of study design

Group Comparisons Experimental Group –Receives a particular treatment specified by the researcher Control/Comparison Group –Does not receive that particular treatment Sometimes difficult in educational research to have a strict no-treatment, control –Example: Any instruction is likely to be more effective than no instruction

Randomization Random assignment to groups –Every individual has an equal chance of being in the experimental or control/comparison group Supposed to help eliminate extraneous sources of variance –For example… if the groups are sufficiently large, any differences between groups on extraneous variables are likely to be due to chance or randomly distributed among the groups Quasi-Experimental=non-randomized groups –Most ed. research –Intact classes & convenience samples –Impacts ability to generalize to whole population

Variables Independent variable (IV) –The experimental or treatment variable –This variable is manipulated by the researcher –Examples: instructional approach, environmental condition, the introduction of a particular musical element –Participant attribute Dependent variable (DV) – Compared b/w groups –The criterion or outcome variable –Examples: student attitudes, student achievement, teacher effectiveness as measure by ? Experiments can be expressed as “The effect of the ‘IV’ on the ‘DV’” Extraneous Variables –Those that are not specifically included in the study but never the less may effect the outcome –Object is to control for extraneous variables –The researcher may not know them all

Manipulating the IV Presence of the variable vs. absence of the variable –Kodaly instruction (treatment group) vs. Kodaly instruction (control group) One form of the variable vs. another –modeling vs. verbal music instruction (vs. control group) Varying degrees of the same variable –100% positive feedback, no negative feedback vs. 50% positive feedback, no negative feedback

Controlling for Extraneous Variables Best case scenario – all individuals are as similar as possible on all variables other than the independent variable Methods to control: –Randomization & large sample –Holding variables constant (freeze private lessons) –Build variable into the design (compare private lessons w/ no private lessons) –Matching pairs – one to control, other to exper. –Statistical control – analysis of covariance (ANCOVA)

Design and Experiment [Effect of Colored note heads on Music Reading] State Hypothesis and Null Hypothesis Select sample and assign to group (control and treatment). How many in each? Identify independent and dependent variables. Any possible extraneous variables? Describe experiment. What will you do w/ each group and for how long? How will you know what they already know?

Discussion of Projects On task Practice explaining your project –Background; State the problem –Purpose statement –Research questions –Methodology (research design)

Experimental Research Designs

Nomenclature/Abbreviations When looking at the symbols used to describe various experimental design approaches: –R = random assignment –O = testing (pre- or post-) –X = treatment –C = control/comparison –M = matched

Pre-Experimental Designs [Pilot Studies – Generally Weak] One Shot Case Study (X O) –No random assignment, No control/comparison, no pre- test One-Group Pre-test, Post-test (O X O) –No random assignment, No control/comparison group Static/Intact-group Comparison X O –No random assignment O Static/Intact-group Pre-test, Post-test –No random assignment, possible pre-test effects O X O O

True Experimental Designs Stronger – not always possible in educ. Randomized Post-test Only, R X O Control Group R O –Still not sure about pre-test levels Randomized Pre-Test, R O X O Post-test, Control Group R O O –Good checking whether smaller groups are actual similar at the start of the study and possible effects of pretest

Randomized Solomon Four-Group & Posttest Only, Control Group Solomon 4–Group controls for possible sensitization effects due to testing or maturation. 1. R O X O 2. R O O (maturation or pretesting?) 3. R X O (effect of pretest?) 4. R O (control group) In a successful experiment, what would we expect for each group? What if the Post Test scores for group 2 were as high as the Post Test for group 1? What if the Post Test scores for group 3 were lower than group 1? What if the Post Test scores for group 4 were the same as groups 1 & 2?

Counterbalanced Design (Latin Square) Order effect Ex: Introducing note values: 1=tatiti; 2=num/no subdiv; 3=num/sub div; 4 dotted notes

Quasi-Experimental Design So called b/c there is no randomization… Matching Only –Participants matched in pairs to control for an extraneous variable rather than randomly assigned Counterbalanced Design (next slide) –Multiple groups receive all treatment types in different order –Average post-test scores across groups are compared to determine effectiveness/effect of the treatment order –Vulnerable to multiple-treatment interference Time-series Design –Outcome measured several times before and after introduction of the treatment O O O O X O O O O

Quasi-Experimental Design Factorial Design (2 or more factors (IVs) at different levels) –Examining the effect of more than one independent variable –Allows for examination of attribute variables (i.e. gender, age) and interaction effects b/w combinations of IVs –Example: Kodaly vs. trad. instruction for bilingual and non-bilingual students Possible outcome showing interaction of two IV’s Non-bilingual students may do equally well w/ Kod. and traditional methods, while bilingual students may do better w/ Kod. vs. traditional methods. What if you had not separated these groups out?

Factorial Example (Two Way - 2x2) IVs = Language classification (biling. vs. non-biling.) & method (Kodaly vs. trad.) DV = musical achievement test? Groups (Six 3 rd gr. Sections-3 Kodaly; 3 Trad; Bilingual & Non-Bilingual in all groups.) –Bilingual & Kodaly –Non-Bilingual & Kodaly –Bilingual & Traditional –Non-Bilingual & Traditional

2 Way Factorial Designs (2 independent variables [often one manipulated, one attribute)

Internal Validity (Usefulness/Meaningfulness) - Control of Extraneous Variables: Time Bound Factors What happens within the experiment –History – What happens b/w pretest and posttest (private lessons, change in practice routine) –Maturation – is change result of treatment natural result of repetition and improvement over time?) –Mortality – Loss of participants may cause imbalance b/w groups

Internal Validity – Sampling & Measurement Factors Testing – pretest affect posttest. Ceiling and floor effects (eliminate outliers?) Instrumentation – changes in measurement or observers (judges at contest from one site to the next) Statistical regression – students who score extremely high (ceiling) or low (floor) on pretest may regress to the mean on posttest Selection – participants do not represent normal population (also affects external validity) Interactions – influence of a combination of the above factors

Internal Validity John Henry Effect –Control group performs beyond usual level because they perceive they are in competition with the experimental group

External Validity – Generalizability Population Validity –Extent sample is representative of the population to which the researcher wishes to generalize the results. Ecological –Study conditions and setting are representative of the setting in which the researcher would like to apply the findings Replication –Results can be reproduced (problem w/ Mozart effect) Detailed description of the sample needed in study –Important regardless of sampling method –‘Next best thing’ if not a large, random sample – often the case in music ed. research –Consider demographic questions in descriptive research

Other Threats to External Validity Effect or interaction of testing (testing will not occur in natural setting) Sample does not reflect population –Discuss in research report Reactive effects of sample –Hawthorne Effect Effects due simply to subjects’ knowledge of being in a study –Teacher or Researcher interactions different than in population Subconsciously encouraging or discouraging a group Research setting does not reflect typical settings (ecological validity) –A university lab school