Political Science 104 Wednesday, October 15 Agenda Reliability vs. Validity Groupwork: Applying what we’ve learned to the newspaper articles Assignment #3: Concept to Measurement (due in section next week!!!) Assigned Readings Best, Statistics, Chs. 5,6,7 Shively, Craft, Chs. 3,4,5 Salkin, Statistics, Ch. 6 Washington Monthly College Guide Data Quality Campaign and Information Alliance for Education Cartograms of 2004 presidential election (link)(link) Keiger, "The Number"

Validity vs. Reliability

Reliability Validity

Validity vs. Reliability Reliability Validity The property of an assessment tool that indicates that the tool does what it says it does (Salkind, 112). A measure is valid, if it actually measures what it purports to measure (Shively, 48). A measure is valid if it correctly measures what it says it does; provides us a way to measure the link between a concept and a measurement (Weatherford, 10/14).

Validity vs. Reliability Reliability Validity The property of an assessment tool that indicates that the tool does what it says it does (Salkind, 112). A measure is valid, if it actually measures what it purports to measure (Shively, 48). A measure is valid if it correctly measures what it says it does; provides us a way to measure the link between a concept and a measurement (Weatherford, 10/14). A test, or whatever you use as a measurement tool, measures something consistently (Salkind, 101). A measure is reliable to the extent that it gives the same result again and again if the measurement is repated (Shively, 45). A measure is reliable to the extent that it gives the same results again and again (Weatherford, 10/14).

Validity vs. Reliability Reliability Validity The property of an assessment tool that indicates that the tool does what it says it does (Salkind, 112). A measure is valid, if it actually measures what it purports to measure (Shively, 48). A measure is valid if it correctly measures what it says it does; provides us a way to measure the link between a concept and a measurement (Weatherford, 10/14). A test, or whatever you use as a measurement tool, measures something consistently (Salkind, 101). A measure is reliable to the extent that it gives the same result again and again if the measurement is repated (Shively, 45). A measure is reliable to the extent that it gives the same results again and again (Weatherford, 10/14). Reliability ≠ Validity Validity → Reliability

Different Forms of Validity (Lecture) Tests of Reliability (Book) Construct Validity Translation/Definitional Validity Face Validity Content Validity Criterion-Related Validity Predictive Validity Concurrent Validity Convergent Validity Divergent Validity Test-retest reliability: Testing whether a measurement tool or test works, simply by doing it twice. Parallel forms reliability: Using the same test twice, but using different content in both tests. Internal consistency reliability: Testing different items of a test to make sure that we are testing different items. Interrater reliability: Using two different raters to make sure their judgments are reliable.

Things to Remember Reliability vs. Validity: Reliability and Validity are two very different concepts that just happen to end in “ity”. Don’t confuse them on the test. Reliability vs. Consistency: Reliability is about a consistent measure while consistency is about your theory being consistent with previous research.

What types of things have we learned in this class that we can now generally apply to articles or studies that contain statistics? First, identify the theory that is being tested in the article. Can you identify the major concepts that are being tested? Does the article translate these concepts to variables? Is there enough information to do so? What is the independent variable/concept? What is the dependent variable/concept? Are control variables/concepts being used? If using variables, are the variables nominal, ordinal, or interval? Is the theory consistent, falsifiable, generalizable, parsimonious? Can we trust the statistics in the article? Robert Best: Are there missing numbers? Confusing numbers? Scary numbers? Authoritative numbers? Are the measurement tools reliable? (Do you think the test can be replicated reliably if someone wished to do so?) Are the measures valid? (Can you apply one of the different forms of validity in application to your question?)

Assignment #3 Here is a list of eight concepts appearing in political research. Pick two of these. For each of these, list one or more variables that might be used as an indicator of the concept, and briefly discuss the variables in terms of the validity and reliability with which they represent the concept. You might need several variables to capture the meaning of some of the concepts; if so, explain why and tell how each variables contributes to rounding out the interpretation of the concept. Economic developmentPolitical representation Party competitionRacial discrimination International tensionsLiberalism Political equalityTerrorism Select a newspaper article about some political trend or event in which you are interested. Devise a t least two explanations for it, stating the explanations as clearly as possible. List the concepts employed in each explanation, pair each concept with an observable variable that you could use to investigate your proposed explanation, and describe how your observations would allow you to distinguish which explanation is correct.