Research Design

How do we know what we know?

The way we make reasoning Deductive logic Begins with one or more premises, reasoning then proceeds logically from these premises toward conclusions that must also be true. All plants produce energy through photosynthesis (premise 1) Tulip is a type of plant (premise 2) Tulip produce energy through photosynthesis (conclusion) If the premise is wrong, we may get wrong conclusion but still it is logically valid.

For example, we might hypothesize that "The color of a mineral is determined by its crystal structure." If the color of a mineral is determined by its crystal structure; then all purple minerals should have the same crystal structure. But purple amethyst has a hexagonal structure and purple fluorite has an isometric structure (determined by observations). Therefore, the hypothesis is not supported or strengthened.

The way we make reasoning Induction reasoning Use specific instances or occurrences to draw conclusion. All observed crows are black; therefore, all crows are black. I see more traffic this year; therefore, the traffic is heavier this year than last year. Not considered as formal logical, hard to test the validity

Deductive Research Testing the theory Considered as the scientific method Universality Could be carried out by any competent person Replication Control Measurement Data can be measured

Deductive Research Features Carry out large scale literature review to gather enough theories or evidence as premises Use deductive logic to derive a set of hypotheses Systematically gather data relevant to the hypothesis Statistically test and interpret the data to see if they support the hypothesis

Examples of deductive research An empirical study of the effects of interactivity on web user attitude, International Journal of Human-Computer Studies The Tipping articles…

Inductive Research Building theory Start from the data, begin to detect patterns and regularities, formulate some tentative hypotheses, end up developing some general conclusions or theories Can be a meta-analysis…

Comparison: Kicking a Ball Deductive research Scientific principles Moving from theory to data The need to explain causal relationships The collection of quantitative data Highly structured approach Researcher independent of what is being researched The request for generalization Inductive research Gaining an understanding of the meanings humans attach to events A close understanding of the research context The collection of qualitative data Flexible structure Researcher is usually part of the research process Less concern with the need to generalize

How do we choose? Depends on your research topic Depends on the previous research Depends on the time Depends on the risks Something less practical… your own style

Different research strategies Experiment Survey Cross-sectional and longitudinal studies Case study Grounded theory Ethnography Action research

Experiment Classical deductive research Is generally used to find causal relationships Definition of a theoretical hypothesis Recruiting of subjects (not in large number) Allocation of subjects to different experimental conditions Introduction of planned change on one or more of the variables Measurement on small number of the variables Control of the other variables

Survey Usually associated with the deductive approach Allow the collection of a large amount of data from a sizable population in an economical way Questionnaire is the most popular data collection method Designing and testing questionnaire is the most importing part of a survey research

Research questions appropriate for a survey Self-reported beliefs or behaviors Ask many questions, measure many variables, and test several hypotheses in a single survey Behavior Attitudes/beliefs/opinions Expectations Self-classification Knowledge

Survey Types Cross-sectional design Collects data at one time Longitudinal Designs Takes place over time with two or more data collections

Case study Involves an empirical investigation of a particular phenomenon within its real life context using multiple sources of evidence Inductive research Data are collected from interviews, focus groups and so on. Study one case or a few cases ( comparative case studies )

Examples of a case study Phineas Gage Kanzi High-tech Company, Business Planning

Ethnography Firmly rooted in the inductive approach Comes from the disciplines of social and cultural anthropology Researchers have to immerse themselves in the life of people they study Main difference with case study is the extent to which the researcher immerses in the life of the social group under study A significant amount of data are collected through observation Very time consuming

Action Research very cool! Main features collaborative activity among colleagues searching for solutions to everyday, real problems experienced Collaboration between practioners and researchers Results should have implications beyond the immediate project

Using multi-methods Different methods can be used for different objectives in a study Enables triangulation The application and combination of several research methodologies in the study of the same phenomenon

Credibility of research findings How do I know what I know? Is the source reliable? Are the raw data reliable? Is the conclusion valid? Is it just some coincidence? Triangulation is a way help you make judgment Reliability & validity of the design Reliability mainly deals with “repeatibility” Validity is concerned with whether the findings are really about what they appear to be about

Validity and Reliability Neither Valid nor Reliable Reliable but not Valid Valid & Reliable Fairly Valid but not very Reliable Think in terms of ‘the purpose of tests’ and the ‘consistency’ with which the purpose is fulfilled/met

Validity Depends on the PURPOSE Measuring what ‘it’ is supposed to Matter of degree (how valid?)

Reliability Consistency in the type of result a test yields Time & space participants Not perfectly similar result but ‘very close-to’ being similar When someone says you are a ‘reliable’ person, what do they really mean? Are you a reliable person?

Validity of the methodology Internal validity The extent to which its design and the data that it yields allow the researcher to draw accurate conclusions about cause and effect and other relationships within the data External validity The extent to which the conclusions drawn can be generalized to other contexts Real-life setting A representative sample Replication In a different context