Download presentation
Presentation is loading. Please wait.
1
Objectives of Session Three Focus Group and Survey Assignment Formative Design Proposal Construct Validity Reliability in Data In-class Data Exercise
2
Focus Group/Survey Project Select topic and moderator(s) ½ hour – no penalty if you do not finish your guide Short ice-breaker, background on topic, and questions Be prepared with follow-up questions Handout relating to survey design
3
Sources for Formative Design Proposal Conduct an internet search using keywords for your program area of interest and evaluation terms Points of Light Foundation - http://www.pointsoflight.org/ http://www.pointsoflight.org/ United Way - http://national.unitedway.org/ http://national.unitedway.org/ Electronic Policy Network - http://movingideas.org/ http://movingideas.org/ Evaluation Center - http://www.wmich.edu/evalctr/index.html http://www.wmich.edu/evalctr/index.html Michigan Non-profit link - http://comnet.org/index.html http://comnet.org/index.html Community Foundation of CNY - http://www.cnycf.org/nonprofit/grantees.cfm http://www.cnycf.org/nonprofit/grantees.cfm
4
Construct Validity Does the measure capture the construct of interest? How well does the measure capture the process at hand? -Face validity -Content validity -Convergent v. Discriminant validity
5
Achieving Construct Validity Propose different possibilities for valid measures or operationalizations of the following concepts: –Quality of an individual’s diet –Customer satisfaction with visit to the Dept. of Motor Vehicles –Racial or ethnic identity –Physical limitations
6
Reliability Free of error Consistency in measurement and in coding Data cannot be valid unless it is reliable, but reliable data can be invalid.
7
Internal and External Validity Internal Validity: Does the design allow us to reach causal conclusions? External Validity: Are your findings generalizable?
8
Improving Validity and Reliability of Data Pretest your questions or data collection techniques Use widely trusted measures Correlations between coders/respondents Develop multiple measures Use qualitative findings to check quantitative findings and vice versa
9
In-Class Data Exercise Data collected from Open Cupboard, nonprofit that opened 10 food pantries 6 months ago What can the data tell us about service delivery? Different measures of initial implementation and service delivery -- no impact or outcomes Frequencies Means and variance Correlation Which measures are most valid? Most reliable?
10
Readings for Next Time: Weiss, Chapters 4 and 7
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.