Research Methods in MIS RESEARCH DESIGN Dr. Deepak Khazanchi

What is Research Design? A plan for conducting research which usually includes specifications of the elements and the procedures to be used. (R1, R3) The design is an activity- and time-based plan. The design is always based on the research question or hypothesis. The design guides the selection of sources and types of information (data collection) The design is a framework for specifying the relationships among the study’s variables The design outlines procedures for every research activity.

Classifications of Designs Category Options The degree to which the research question has been crystallized Exploratory study Formal study The method of data collection Monitoring Interrogation/communication The power of the researcher to produce effects in the variables under study Experimental study Ex post facto study The purpose of the study Descriptive study Causal study The time dimension Cross-sectional study Longitudinal The topical scope-breadth and depth-of the study Case study Statistical study The research environment Field setting Laboratory research Simulation The subjects’ perceptions of research activity Actual routine Modified routine

Degree of research question crystallization Exploratory vs. Formal study Essential difference: Degree of structure and immediate objective of study Exploratory studies: Loose structure and objective of discovering future research tasks Immediate objective: develop hypotheses or questions for further investigation Formal study: begins where exploration leaves off; starts with an hypothesis or research question Involves precise procedures and data source specifications Goal of Formal research designs: test hypotheses or answer research questions posed This dichotomy is not too precise All studies have some exploration in them, and few studies are completely uncharted.

Method of Data Collection Monitoring process Researcher inspects activities of a subject or the nature of some material w/o attempting to elicit responses from anyone E.g., “following” a computer through the repair process at Compaq or documenting each activity or interaction between Compaq’s call center employees and a customer complaint about a damager laptop. Interrogation-Communication process Researcher questions subjects and collects their personal or impersonal responses by Interview or phone conversations; Self-administered or self-reported instruments sent through the mail or electronic means; instruments presented before and/or after a treatment or stimulus condition in an experiment. Cooper and Schindler (1995) use the term communication to contrast with monitoring because collecting data by questioning is more encompasses more than the survey method.

Concept of Random Assignment The assignment of subjects to different experimental or control groups, using random procedures, for the purpose of equating groups prior to conducting the experiment. Goal: To provide an unbiased method for placing Ss in groups so that there is a high probability of characteristics such as age, education, etc. being about the same for each group. Important concept for understanding research design Random assignment is different from random selection, although both use random numbers.

Researcher control of variables Experimental vs. ex post facto Experimental: Researcher attempts to control and/or manipulate the variables in the study. Appropriate when one wishes to discover whether certain variables produce effects in other variables. Most powerful for a hypothesis on causation. Requires random assignment of subjects into groups. Ex post facto: Researcher has no control over the variables in the sense of being able to manipulate them. Researchers report on what is happening Results lead only to conclusions about associations, not cause and effect. Very limited ability to hold factors constant; mostly done through statistical means; Correlation is the key technique for analysis Also known as Nonexperimental design

The purpose of the study Descriptive vs. Causal studies Descriptive: Key purpose is to do research to describe phenomena or characteristics associated with a subject population (who, what, where, when, or and how much of a topic). Also, other objectives include: Discovers associations among different variables Estimates of the proportion of a population that have these characteristics Can be formal or exploratory in nature

The purpose of the study (cont’d) Causal: The purpose is to learn why--that is, how one variable produces changes in another. Causal studies try to explain relationships among variables. Discovery and measurement of cause-and-effect relationships among variables Generally formal in nature. Causation and Ex Post Facto Design (Nonexperimental): Have to be very careful in using ex post facto design to assess causation (no manipulation of variables is possible). Thorough testing, validating of multiple hypotheses, and controlling for confounding variables are essential. Post-Hoc Fallacy: The frequent problem of drawing unwarranted conclusions about causal relationships in ex post facto studies.

The Time Dimension Cross-sectional vs. Longitudinal Cross-sectional studies are carried out once and represent a snapshot of one point in time. Longitudinal studies are repeated over an extended period. Potential to track changes in variables over time. E.g., customer satisfaction with call-center responses measured across many months. Could use of cohort groups (different subjects, similar demographics) and panels (same subjects)

The Topical Scope Statistical study Case study Designed for breadth rather than depth Capture population’s characteristics by making inferences from a sample’s characteristics Hypothesis are tested quantitatively Generalizations about findings are presented on the representative ness of the sample and validity of the design Case study Emphasis on full contextual analysis of fewer events or conditions or interrelationships Hypothesis (often) are used; however, reliance on qualitative data makes rejection of more difficult Emphasis on detail can provide richer insights Leads to discovery of new questions

The Research Environment Field setting Study conducted under actual environmental conditions Laboratory setting Study under other conditions (lab; classroom) Simulation Studies that replicate the essence of a system or process. Used predominantly in operation research.

The Subjects’ Perceptions Subjects’ perception influence the outcomes of the research in subtle ways or occasionally, more dramatically. There are 3 levels of perception: Subjects perceive no deviations from everyday routines. Subjects perceive deviations, but as unrelated to the researcher. Subjects perceive deviations as researcher-induced. Researchers need to be vigilant to effects that may alter their conclusions.

Other Categories of Research Designs Not included in the previous classification scheme: QUASI-EXPERIMENTAL DESIGNS A research design that lacks the full control of a true experimental design. E.g., A researcher learns that the XYZ corp. will install a MIS in 6 months and designs a study to measure the flow of information before and after the MIS is installed. A control group from the STR corp. which does not have an MIS will be used as a comparison group. MUTLIPLE TIME-SERIES DESIGN, TIME-SERIES DESIGN, PACTHED-UP DESIGNS are examples of this type of design strategy

Other Categories of Research Designs (cont’d) Not included in the previous classification scheme: HISTORICAL DESIGNS A research design for which data or physical artifacts already exist and thus cannot be changed or manipulated. The researcher had not control over how or when or with what instruments the data were collected.