Neuman & Robson Chapters 5, 6, & (some of) 12 (p. 332-8) CMNS 260: Empirical Communication Research Methods 4-Measurement (Part 1 of 2 slideshows) Neuman & Robson Chapters 5, 6, & (some of) 12 (p. 332-8) systematic observation can be replicated requires: construct (concept) operational measure/instrument/tool for making empirical observations

Today’s Lecture Review core notions: concepts, operational measures, empirical research, language of variables, hypothesis testing, errors in explanation, etc. Reliability & Validity: relationship, types Levels of Measurement Scales & Indices (if time– material for this in another slideshow)

Recall: The Research Process Babbie (1995: 101)

“Dimensions” of Research Purpose of Study Intended Use of Study Treatment of Time in Study Space Unit of Analysis Exploratory Descriptive Explanatory Basic Applied -Action -Impact -Evaluation Cross-sectional Longitudinal -Panel -Time series -Cohort analysis -Case Study -Trend study -dependent -individual -independent -family -household -artifact (media, technology) Neuman (2000: 37)

The Research Wheel The “Research Wheel” Steps in the research process Choose Topic Focus Research Question Inform Others The Research Wheel The “Research Wheel” Interpret Data Design Study Steps in the research process Collect Data Analyze Data Source: Neuman (1995: 12)

Developing research topics

Concepts Symbol (image, words, practices…) definition must be shared to have social meaning concepts with more than one possible value or attribute sometimes called variables

Concept Clusters Examples: Peer Group Role Model Broadcast Media Ethnic Identity Cultural trauma Collective memory Political economy

Measurement systematic observation can be replicated (by someone else) Measures include: Concepts (constructs), theories measurement instrument/tools Must recognize concept in observations (measures) ??(# of library holdings as a measure of quality of university?) MacLeans Magazine survey results, 2000.

From Concept to Measure Neuman (2000: 162)

Variables Must have more than one possible “value” or “attribute” Types: dependent variable (effect) independent variable (cause) intervening variable control variable

Causal Relationships proposed for testing (NOT like assumptions) 5 characteristics of causal hypothesis at least 2 variables cause-effect relationship (cause must come before effect) can be expressed as prediction logically linked to research question+ a theory falsifiable

Errors in Explanation

Propositions logical statement about (causal) relationship between two variables i.e. “Increased television watching leads to more shared family time and better communication between children & their parents”

Types of Hypotheses (note: plural form of Hypothesis) Null hypothesis predicts there is no relationship Direct relationship (positive correlation) more time spent studying leads to higher grades Indirect relationship (negative correlation) More time spent playing video games leads to lower grades

Hypothesis Testing

Possible outcomes in Testing Hypotheses (using empirical research) support (confirm) hypothesis reject (not support) hypothesis partially confirm or fail to support avoid use of PROVE

X Y Causal diagrams Direct relationship (positive correlation) Indirect relationship (negative correlation)

Spurious Association example

Causal Diagram S R D R= Racism against non-whites D= Discrimination against non-whites S=Intelligence Test Scores

Good & Bad Research Questions

Abstract to Concrete Concept to Measure

Reliability & Validity dependability is the indicator consistent? same result every time? Validity measurement validity - how well the conceptual and operational definitions mesh with each other does measurement tool measure what we think ?

Types of Validity

Content Validity measure represents all the aspects of conceptual definition of construct. how adequately a measure covers behavior representative of the universe of behavior the test was designed to sample. Love

Face & Expert Panel Validity judgement by group or scientific community that indicator measures the construct (conceptual def.) Examples: Socio-economic status (education, income & ?) Digital Divide (differences in access to computers, internet, broadband?...) Construct Measure ? Scientific Community

Criterion Validity The validity of an indicator is verified by comparing it with another measure of the same construct in which a researcher has confidence. Predictive : ex. Comparison of Aptitude test & performance measures concurrent validity: ex. Comparison of new measure with established one

Construct Validity A type of measurement validity that uses multiple indicators– the construct is a combination of measures of the same variable convergent : positive correlation with related measures discriminate: negative correlation with measures of different variables

Other Dimensions of Validity Internal Validity no error of logic internal to research design External Validity results can be generalized Statistical validity correct statistical methodology chosen ? assumptions fully met

Types of Reliability stability representative equivalence intercoder over time representative across different subgroups of a population equivalence multiple indicators intercoder type of equivalence reliability

Improving Reliability clearly conceptualize constructs increase level of measurement use pretests, pilot studies use multiple indicators : Dependent Variable Measure Independent Empirical Association? a2 a3 a1 b1 b2 A B Specific Indicators

Relationship between Measurement Reliability & Validity reliability necessary for validity but does not guarantee it “necessary but not sufficient” measure can be reliable but invalid (ex. not measuring what you think you are measuring) Source: Neuman (2000: 171)

Quantitative & Qualitative “Trustworthiness”

Creating Measures Measures must have response categories that are: mutually exclusive possible observations must only fit in one category exhaustive categories must cover all possibilities composite measures must also be: uni-dimensional

Levels of Measurement Levels of Measurement Categories (or attributes) must be exhaustive & mutually exclusive Relations between levels --can collapse from higher into lower, not vice versa

Nominal Measurement different categories (names, labels, images) not ranked attributes are mutually exclusive and exhaustive. Examples: What media do you use for finding out about news? Television Newspapers Radio Magazines Internet Other Babbie (1995: 137)

Ordinal Measurement different categories (mutually exclusive, exhaustive) rank-ordered attributes indicate relatively more or less of that variable. distance between the attributes of a variable is imprecise Example: “How important are newspapers as your news source?”

Interval Measurement different categories ranked in order can also tell amount of difference between categories Babbie (1995: 137)

Ratio Measurement different categories ranked in order amount of difference between categories also possible to state proportion (have a true zero) Example: “What was your income in dollars last year?”

Examples

Continuous & Discrete Variables Continuous variables: can have an infinite number of values interval and ratio levels of measurement Discrete variables: distinct categories nominal and ordinal levels of measurement

Composite Measures (continued in second slide series) Composite measures are instruments that use several questions to measure a given variable (construct). A composite measure can be either unidimensional or multidimensional. Ex. Indices (plural form of index) and scales