1. Validity & Reliability Trustworthiness
EDUC 894
Week 6

2. Plan for Today Validity & Reliability Trustworthiness
Dinner Break
Group-work
Planning for Data Collection
Informed Consent & Procedure Approval
For Next Week: Data Collection (No Class)
For Next Next Week
Creswell Chapter 9 Glaser Article on Qualitative Data Analysis

3. Thorndike & Guba
What is the overall issue that the concepts of reliability & validity or trustiworthiness are concerned with? Why are these issues critical to research?
How does the idea of evaluating the plausibility of rival hypothesis occur in the naturalistic paradigm? How does it occur in the rationalistic paradigm?

4. The Overarching Issue
Creating systematic safeguards that can give us confidence in knowing how we can rely on the results of our inquires.
Table Source: Guba (1981) Criteria for Assessing the Trustworthiness of Naturalistic Inquiries

5. Thorndike
What is the relationship between reliability & validity? Can you have reliability without validity? Can you have validity without reliability? Explain why or why not.
Why can’t we talk about the reliability of an instrument in general? Why can’t we talk about the validity of an instrument in general? (note that the answers to these questions are different)
Why don’t the concepts of reliability and validity “make sense” for researchers coming from a constructionist or subjectivist stance?
How will you document reliability & validity in your study?

6. Instrument Reliability & Validity
The accuracy or precision of a measurement procedure
X = T + E
Often thought of in terms of consistency
Measured by correlation coefficients
Validity
The degree to which scores provide information that is relevant to the inferences to be made from them
Content-related validity evidence
Criterion-related validity evidence
Construct related validity evidence

7. Study Validity Internal Validity External Validity
The degree to which the particular situation studied represents the theoretical / conceptual structure that of interest
External Validity
The degree to which the particular situation studied generates findings that are relevant to other situations

8. Guba
Why don’t the concepts of reliability and validity “make sense” for your research stance?
What is the difference between validity and credibility?
What is the different between generalizability and transferability?
What is the difference between reliability and dependability?
What is the difference between neutrality and objectivity? Which one will you show in your study and why?
How will you document credibility, transferability, dependability and confirmability in your study?

9. Trustworthiness The Naturalistic Treatment
Credibility (Results are Plausible)
Triangulation
Member Checks
Prolonged Engagement / Persistent Observation
Coherence / Discussion of alternative explanations
Establish referential adequacy
Transferability (Results are Context Relevant)
Thick description
Purposeful sampling
Dependability (Results are Stable)
Audit trail
Overlapping Methods / Step-wise replication
Confirmability (Results are Investigator-free)
Reflexivity