Data Analysis for Experimental Design Method Section & Data Analysis

Components of the Research Method of Experimental Design Method (Overview) Sample Design and experimental stimuli (manipulations) Pre-manipulation and post-manipulation measures Administration procedure

Independent Variables Setting the stage informed consent brief explanation of what is expected Types of manipulations straightforward staged to create a psychological state to simulate a real world situation use confederates

Dependent Measures Types of measures self-report rating scales behavioral reaction time error rate physiological GSR, heart rate

Dependent Measures (cont.) Sensitivity ceiling effect - too easy floor effect - too hard no effect Multiple Measures e.g. time perception Ethics of measures (e.g. privacy) Cost

Data Defects Missing data some statistics allow for missing data can replace with averaging techniques SPSS have several missing data options Extreme score or outliers techniques for discarding replace with averaging Appropriate Statistics!!!!!!!!!!!!!!!!!

Missing Data A Simple Example of a Missing Data Analysis Understanding the Reasons Leading to Missing Data Ignorable Missing Data Other Types of Missing Data Processes Examining the Patterns of Missing Data Diagnosing the Randomness of the Missing Data Process

Missing Data (Cont.) Approaches for Dealing with Missing Data Use of Only Observations with Complete Data Delete Case(s) and/or Variable(s)

Outliers Detecting Outliers Outlier Description and Profiling Univariate Detection Bivariate Detection Outlier Designation Outlier Description and Profiling Retention or Deletion of the Outlier

Measurement Scales Nominal (categorical) Ordinal Interval Ratio

Nominal Scales Mutually exclusive categories Labels only, no arithmetic properties Examples: Sex, location, marital status Statistical operations: Mode, frequencies, crosstabs

Ordinal Scales Implies more or less, but not how much more or less Can use median or mode, but not average Crosstabs

Interval/Ratio Scales Numbers in scale represent equal increments Scales have the most desirable statistical properties (The ratio scale is an interval scale with a fixed zero point, e. g. age and temperature)

The Multiple Choice Question Simple, versatile Single or multiple response No more than 8-10 so categories Mutually exclusive categories Pre-testing should show labeled alternatives cover 90% of answers

Single Response, Multiple Choice Questions Choice criterion must be clearly stated Choice criterion must define a single answer

Forced Ranking Scales

Forced Ranking Scales Options judged relative to one another Mimics actual decision making Disadvantage: interval between items not measured Limited number of items can be ranked

Guidelines for Using Forced Ranking Scales 10 Items or less Relative standing is focus Use single judgment criterion Analysis confined to frequencies, special procedures for ranked data

The Likert Scale Widely used scale 5 points Degree of agreement with statements Use several statements to measure a “construct” e.g. “customer satisfaction,” “loyalty,” “attitudes towards technology”

Developing Constructs

Curiosity I think learning about things is interesting and exciting. I am curious about things.  My spare time is filled with interesting activities.  I like to solve problems that puzzle me. I want to probe deeply into things. I enjoy exploring new places. New situations capture my attention. I feel inquisitive. The prospect of learning new things excites me. I feel like searching for answers.

Attitudes towards Technology I can usually figure out new hi-tech products and services without help from others. New technologies are often too complicated to be useful. I get overwhelmed by how much I need to know to use the latest technology. Technology gives people more control over their daily lives. People get too dependent on technology to do things for them. New technologies are mentally stimulating. Learning about new technologies can be as rewarding as the technology itself. The products and services with the newest technologies are more convenient to use. New technologies are not designed for use by people like me.

Customer Satisfaction (tailored for online purchases) If I had it to do over again, I’d make my most recent purchase at this website. I am sure it was the right thing to make my most recent online purchase at this website. I have truly enjoyed purchasing from this website. My choice to purchase from this website was a wise one. I am satisfied with my most recent decision to purchase from this website. I am happy I made my most recent online purchase at this website.

Loyalty (intentions) I encourage friends and relatives to do business with this website. I say positive things about the website to other people. I will do more business with the website in the next few years. I would recommend the website to someone who seeks my advice. I consider this website to be my first choice to buy the kind of product I most recently purchased online.

The Likert Scale Easy to use Easy to code Useful in statistical analyses Usually treated as interval level

The Likert Scale Keep in mind: Only use for statements that ask for degree of agreement Use for several items to use space well Items should be diverse enough to cover the issue

The Likert Scale Items for a construct can be summed Recode reverse scored items before summing

Likert-like Scales (NOT Likert-Scales)

Verbal Frequency Scale Indicates how often Likert-like, but usually not interval level

The Horizontal Numeric Scale Items judged on a single dimension Intermediate values are not labeled

The Semantic Differential Scale 7 point scale Bipolar adjectives/statements Can give quick comparative profile of objects

Purchase Intent Scales Often of interest in marketing Write them carefully

The Fixed Scale Rs allocate a fixed number of points (usually 10 or 100) to choices

Univariate Statistical Analysis What is hypothesis? Null and alternative hypotheses Significance level Chi-square test (test of a proportion) “Is the difference between the expected (hypothetical) distribution and the observed distribution statistically significant?” Illustration of an Example

Bivariate Statistical Analysis: Tests of Differences Chi-square test (a test of differences among groups) “Is the difference between the distribution statistically different?” Illustration of an Example T-test for Difference of Means “Is the difference between the means of men and women statistically different?”

ANOVA (Analysis of Variance) Determines if mean group scores are far apart relative to our uncertainty about the actual value of the means

Example Grand Mean= 11+8+5/3 = 8

Example There seems to be an overall average difference Questions: Is this difference statistically significant? If so, is the size of the difference managerially significant?

ANOVA (Analysis of Variance) Conceptually, ANOVA compares: difference among means/uncertainty OR explained variance/unexplained variance OR between treatment variance/within treatment variance

Analysis of Variance (ANOVA) F-test The larger F-value, the better result Determines whether there is more variability in the scores of one sample than in the scores of another sample) “Are the two sample variances different from each other or are they are from the same population?”

Logic of ANOVA Each observation is different from the Grand (total sample) Mean by some amount There are two sources of variance from the mean 1) That due to the treatment or independent variable 2) That which is unexplained by our treatment

Logic of ANOVA (cont.) The statistical test of significance is based on the relative size of the variance caused by the IV relative to unexplained variance Three quantities are calculated: Total variance Between treatment sum of squares (explained) Within treatment sum of squares (unexplained)

The F-ratio MS = mean square bg = between groups wg = within groups Numerator is the “effect” and denominator is the “error” df = # of categories – 1 (k-1)

F-Statistic Used in Factorial Designs Is an extension of the t-test. It is an analysis of variance that is a more general procedure than the t-test. When a study has only one independent variable and only two groups using an F or a t makes no difference. However analysis of variance (ANOVA) is conducted when there are more than two levels of an independent variable and when a factorial design with two or more independent variables is used.

One-Way ANOVA Partitions Total Variation Variation due to Random Sampling are due to Individual Differences Within Groups. Variation due to treatment Variation due to random sampling Sum of Squares Among Sum of Squares Between Sum of Squares Treatment (SST) Among Groups Variation Sum of Squares Within Sum of Squares Error (SSE) Within Groups Variation 89

Post Hoc Tests Vs Planned Comparisons Post hoc tests allow us to make all pairwise comparisons. These tests trawl the data, searching for significant differences and so are not very sensitive. Planned comparisons enable us to address specific questions with a higher degree of sensitivity. It is not appropriate to use both in order to answer the same question.

Statistical Issues Assumptions of the test are your data normally distributed is the sd about equal in each group Statistically significant difference vs. “real” difference Hypotheses null vs. research never “prove” anything

Statistical Issues (cont.) Scale attenuation effects floor and ceiling effects restricted range Regression to the mean Criterion choice the .05 “rule” Type I and Type II errors

Statistical Issues (cont.) Interpreting non-significant results poor design poor procedures poor instructions poor operational definitions power of the test N and effect size