Indawan Syahri indawansyahri_ump@yahoo.co.id RESEARCH PROCEDURES Indawan Syahri indawansyahri_ump@yahoo.co.id 2/16/2019

III. RESEARCH PROCEDURES Research Method Research Variables Operational Definition Population and Samples Techniques for Collecting Data Techniques for Analyzing Data 2/16/2019

A. Research Method Two types of methods in term of ‘treatment’ or ‘non- treatment’ Experimental method, the researchers do some experiments to verify or testify the tentative theories as formulated in hypotheses Descriptive method, the researchers describe some phenomena that are formulated in research questions. E.g., Error Analysis in ELT, the researchers finds the data in order to identify the types and sources of errors. 2/16/2019

Experimental Research Experimental research is unique in two very important respects: It is the only type of research that directly attempts to influence a particular variable. It is the only type of research that can really test hypothesis about cause-and-effect relationships. The researchers manipulate the independent variable and observe the effects of dependent variable. 2/16/2019

Types of Experimental Designs Weak Experimental Designs True Experimental Designs Quasi-experimental Designs Counterbalance Designs Time-series Designs Factorial Designs A-B-A Designs Multiple Baseline Designs 2/16/2019

Weak Experimental Designs (1) Do not have built-in controls for the threat to internal validity A number of explanation for any outcomes that occur Difficulties assessing the effectiveness of the independent variable. 2/16/2019

Weak Experimental Designs (2) The One-shot Case Design a single group is exposed to a treatment, and a dependent variable is subsequently observed (measured) in order to assess the effect of the treatment X O Treatment Observation (dependent variable) 2/16/2019

Weak Experimental Designs (3) The One-group Pretest-posttest Design A single group is measured or observed not only after being exposed to a treatment of some sort, but also before. O X O Pretest Treatment Posttest 2/16/2019

Weak Experimental Designs (4) The Static-group Comparison Design Two already existing or intact groups are used. These are static group Comparisons are made between groups receiving different treatments X1 O -------------------------------- X2 O 2/16/2019

True Experimental Designs (1) Subjects/samples are randomly assigned to treatment groups. The Randomized Posttest Only Control group design Both group are randomly assigned One group receive the experimental treatment while the other does not. Treatment Group R X1 O ---------------------------- Control Group R X2 O 2/16/2019

True Experimental Designs (2) 2. The Randomized Pretest-Posttest Control Group Design Both groups are randomly assigned Both groups are given pretest Both groups are also given posttest Treatment Group R O X1 O ------------------------------------- Control Group R O X2 O 2/16/2019

True Experimental Designs (3) 2. The Randomized Solomon Four-group Design Attempt to eliminate the possible effect of a pretest Random assignment of subjects/samples to four groups Two of the four groups are given pretest and two are not. Treatment Group R O X1 O ------------------------------------- Control Group R O X2 O ===================== Treatment Group R X1 O ------------------------------------------- Control Group R X2 O 2/16/2019

Quasi-experimental Designs (1) Do not include the use of random assignment Rely on other techniques to control (or at least reduce) threats to internal validity The matching Only Posttest-only control group Design Treatment Group M X1 O ---------------------------- Control Group M X2 O 2/16/2019

Quasi-experimental Designs (2) 2) The Matching Only Pretest-posttest Control Group Treatment Group O M X1 O -------------------------------------- Control Group O M X2 O 2/16/2019

16/02/2019 B. Research Variables An attribute of a person, a piece of text, or an object which varies from person to person, text to text, object to object, or from time to time. Variables can be very broad or very narrow. The broader the variable, the more difficult it may be to define, locate, and measure accurately. Variables can be assigned to groups of people as well as individuals. 2/16/2019 Indawan Syahri

Measurement of Variables 16/02/2019 Measurement of Variables When variables are of the all-or-nothing, yes/no sort, we cannot measure how much of the variable to attribute to the person, text, or object. So how? Variables will be quantified in different ways depending on whether we want to know how often an attribute is present or how much of the variables to attribute to the person, text, or text. 2/16/2019 Indawan Syahri

Nominal Scale Variables 16/02/2019 Nominal Scale Variables A nominal variable names an attribute or category and classifies the data according to presence or absence of the attribute. The classification numbers have no arithmetic value, e.g., in yes/no notation, 1 represents yes and 2, no. and in south Sumatra dialects, 1=Palembang, 2=Lahat, 3=Komring, 4=others. There is no reliable method of measuring the degrees, e.g., of how much happiness one possesses at any moment. 2/16/2019 Indawan Syahri

Ordinal Scale Variables 16/02/2019 Ordinal Scale Variables An ordinal scale variable shows how much the variable attributes a person, text, or object in term of degree. E.g., the variable happy can be measured in term of degree of happiness; very unhappy—unhappy—happy– very happy Most researchers prefer to use a 5-point, 7-point or 9-point scale You may use a Likert-type scale to tap students attitudes: The lesson were boring 1 2 3 4 5 6 7 2/16/2019 Indawan Syahri

Interval Scale Variables 16/02/2019 Interval Scale Variables An interval scale tell us how much of the variable to attribute a person, text or object which more precise measurement. The intervals of measurement can be described. Each interval unit has the same value so that units can be added or subtracted. E.g., the difference in intervals between 1 and 10 in a test of 100 items. 2/16/2019 Indawan Syahri

Frequency Data vs. Score Data 16/02/2019 Frequency Data vs. Score Data Another way to think about difference in measurement is to consider whether the study measures how much on an interval or ordinal scale or whether it measures how often something occurs– the frequency of nominal measurement. This distinction is important because it will determine the appropriate statistical analysis to use with the data. 2/16/2019 Indawan Syahri

Functions of Variables 16/02/2019 Functions of Variables To understand how the variables in a study relate to one another, we need to be able to identify their functions. Functions grow out of the RQ. Functions depend on the connection we believe exists between the variables we have chosen to study. One variables are expected to be related to one another or one affects the others. 2/16/2019 Indawan Syahri

Dependent Variable (DV) 16/02/2019 Dependent Variable (DV) The dependent variable is the major variable that will be measured in the research. E.g., the construct communicative competence (CC) of a group of students, the dependent variable is the construct and it might be operationalized as the students’ scores or ratings on some measure of CC. We expect the performance on the dependent variable will be influenced by other variables. 2/16/2019 Indawan Syahri

Independent Variable (IV) 16/02/2019 Independent Variable (IV) An independent variable is a variable that the researcher suspects may relate or influence the dependent variable. E.g., if we want to know something about CC of students, the dependent variable is the score for CC. We might believe that the male and females students differ on this variable. So we can assign gender as the independent variable. 2/16/2019 Indawan Syahri

Moderator Variable (MV) 16/02/2019 Moderator Variable (MV) Some researchers makes a distinction between IV and MV, others call them both IVs since they influence variability in the DV. Specifying variables as “independent” and “moderator” helps us study how moderating variables mediate or moderate the relationship between the IV and DV. E.g., we believe that gender as IV is the most important variable to look at L2 compliments, but we observe that length of residence suspected to influence the DV can be considered as MV 2/16/2019 Indawan Syahri

16/02/2019 Control Variable (CV) CV is a variable that not of central concern in a particular research project but which might affect the outcome. It is controlled by neutralizing its potential effect on the DV. E.g., if you assume that handedness can affect the Ss’ respond in many task, you can institute a control by including only right-handed Ss in your study Remember when you control in this way, you limit the generalizability of your study. 2/16/2019 Indawan Syahri

C. Operational Definition To define the key words in the title and variables The definitions should be specific to the study The definitions should be precisely clear, not ambiguous. The definitions are different from theoretical ones. 2/16/2019

D. Population and Samples A population is the group to which the results of the study are intended to apply. A sample is a group in a research study on which information is obtained. In almost all research investigations, the sample is smaller than the population, since the researcher rarely have access to all the members of the population. 2/16/2019

Why use samples? Although the best data comes from studying an entire population, samples are used because they are smaller and less unwieldy. It can be too time consuming and expensive to study an entire population 2/16/2019

Defining the Population Population: the universe of events from which the sample is drawn. Population: group that a researcher is trying to represent. Study a subset of the population because it is cheaper, faster, easier, and if done right, get same results as a census (study of whole population). Accessible population – the group you are able to realistically generalize to. 2/16/2019

Simple Random Sampling Forms of Sampling Random Sampling Simple Random Sampling Stratified Random Sampling Cluster Random Sampling Two-stage Random Sampling Non-random Sampling Systematic Sampling Purposive Sampling Convenience Sampling 2/16/2019

Simple Random Sampling A simple random sample is one in which each and every member of the population has an equal and independent chance of being selected. A B C D E F G H I J K L M N O P Q R S T V W X Y Z C F I L N Z Population Simple Random 2/16/2019

Stratified Random Sampling Stratified random sampling is a process in which certain subgroups, or strata, are selected for the sample in the same proportion as they exist in the population. A B C D E 25% F G H I J K L M N O P Q R S T 50% V W X Y Z C D F I L J W Z Stratified Random Population 2/16/2019

Cluster Random Sampling Cluster random sampling is similar to simple random sampling except that groups rather than individuals are randomly selected. A B C E F G H I J K L M N O P Q R T V W X Y Z Population Cluster Random 2/16/2019

Two-stage Random Sampling Two-stage random sampling is to combine cluster random sampling with individual random sampling. A B C E F G H I J K L M N O P Q R T V W X Y Z Population Sample of Clusters E. L. W. Sample of Individuals Two-Stage Random 2/16/2019

Systematic Sampling In a systematic sample, every nth individual in the population list is selected for inclusion in the samples A B C D E F G H I J K L M N O P Q R S T V W X Y Z B G L Q W Population Systematic Easily Accessible B G L Q W 2/16/2019

Purposive Sampling On occasion, based on previous knowledge of a population and the specific purpose of the research, investigators use personal judgment to select a sample. A B C D E F G H I J K L M N O P Q R S T V W X Y Z C G R W Population Purposive Equally Qualified C R G W 2/16/2019

Convenience Sampling A convenience sample is a group of individuals who (conveniently) are available for study. A B C D E F G H I J K LM N O P Q R S T V W X Y Z Q R S T X Y Population Convenience Easily Accessible 2/16/2019

E. Techniques for Collecting Data Observation Interviews Questionnaires Tests Documentation 2/16/2019

Instrumentation (1) The process of preparing to collect the data called instrumentation. It involves the selection of method/technique by which data will be collected, as well as the procedures and conditions for collecting them. Key questions in data instrumentation and measurement Where will data be collected? When will data be collected? How often will data be collected? Who will collect the data? 2/16/2019

Instrumentation (2) Validity – measures what it is supposed to (accurate) or the defensibility of the inferences a researcher can make from a study using an instrument. Reliability – a measure that consistently gives same reading (repeatable). Objectivity – absence of subjective judgment or the absence of subjective biases on the part of the researcher. Usability of instrument – consider ease of administration; time to administer; clarity of direction; ease of scoring; cost; reliability/validity data availability. 2/16/2019

F. Techniques for Analyzing Data The stages of analyzing data depend on what problems formulated of question arisen. E.g., in experimental studies, the researcher can do the following steps: Data distributions Frequency of pretest scores Frequency of posttest scores Comparisons between pretest and posttest scores Paired sample t-test Independent sample t-test 2/16/2019

Thank you indawansyahri_ump@yahoo.co.id 2/16/2019