Chapter 2 Theoretical statement: Two concepts are theorized to be associated; this is the beginning of the research methods process We must agree about the concepts (their definitions) and how they are associated (what will be observed and measured). This process is called explication

Explication Explication is the process of attaching both meaning and measurement to an abstraction. Conceptualization is the process of attaching meaning to the abstract concept Operationalization is the process of attaching concrete measurement to the conceptual definition (variable)

Operational Definition A set of processes and procedures that allow for empirical observation Can be broken into two types: Qualitative observations Quantitative observations

Observation Observation implies a measurement of some kind Variables are the aspects of a concept that will be observed and measured

Validity Validity is a judgment that we agree upon regarding the degree of accuracy of a definition The validity of a definition is the degree to which the definition accurately represents the concept (conceptual definition) or variable (operational definition)

Types of Validity Validity of measurement Validity of a test Methodological validity

Validity of Measurement Meaning analysis: the process of attaching meaning to concepts Face validity Content validity Empirical analysis: how well our procedures (operational definition) accurately observe what we want to measure Construct validity

Construct Validity The degree to which our operational definitions accurately reflect the concepts they were designed to measure Four levels of assessing: Hypothesis testing Discriminant validity Convergent validity Multi-trait/multi-method validity

Construct Validity Hypothesis testing validity: the extent to which previous studies have successfully used the operational definitions in testing their hypotheses

Construct Validity Hypothesis testing validity Discriminant validity: the ability to differentiate, with measures, the differences between similar concepts

Construct Validity Hypothesis testing validity Discriminant validity Convergent validity: the extent that using different methods to explicate a concept or relationship produces similar results

Construct Validity Hypothesis testing validity Discriminant validity Convergent validity Multi-trait/multi-method validity: the process of assessing the levels of construct validity across time and studies

Validity of a Test Pragmatic (criterion) validity: the process of creating a test or assessment tool to measure and predict Concurrent validity: the ability of a test to measure a quality known to be in a group Predictive validity: the ability of a test to forecast the development of a certain quality in a group

Reliability Reliability and validity are related An operational definition may not be valid if it is reliable However, an operational definition must be reliable to be valid

Reliability Stability (over time) Equivalence (of form or coders) The consistency of an operational definition (the set of procedures agreed upon for observing some phenomenon) Stability (over time) Equivalence (of form or coders) Internal consistency of indices (homogeneity of an index)

Reliability Stability reliability: a test−retest reliability where we have a reliable measure to the extent that the instrument yields similar results over time

Reliability Stability reliability Equivalence reliability: the degree of agreement among different instruments designed to measure the same phenomena whether by forms or coders

Reliability Stability reliability Equivalence reliability Internal consistency: procedures that assess how well a set of items mathematically “go together” to allow them to be combined into a single measure

Measurement The terms measurement and observation are used interchangeably Empirical: another way to say observable or able to be experienced

Measurement Nominal Ordinal Interval Ratio Measurement is the set of procedures experts agree upon in observing a phenomenon Four types of measurement Nominal Ordinal Interval Ratio

Measurement Exhaustive: there must be a category for every observation Nominal: labels are used to differentiate categories The categories must be: Exhaustive: there must be a category for every observation Mutually exclusive: there is only one category for every observation

Measurement Nominal Ordinal: categories that can be ranked along some continuum

Measurement Nominal Ordinal Interval: equal distance between categories (numbers) Likert scales

Measurement Nominal Ordinal Interval Ratio: rankable categories that are equal distant from one another and have a true zero point; ratios can be calculated