1. Objectives of Session Seven Complete in-class survey Discuss question formats and ordering Case study: face-to-face interviews v. self-administered questionnaires Formative Design Project (Proposal Due on Monday, Feb. 10 th)

2. Developing Survey Questions Structure –Closed- v. open-ended questions –Skip patterns, valid choices, logical flow Wording –Avoid misunderstandings, need to make assumptions –Questions that different people will understand in the same manner Ordering

3. Mothers’ Well-Being Survey Basic Demographics Emotional and Mental Well-being Alcohol Use Violent or Stressful Life Events Drug Use Service Utilization Relationships, Abuse, Domestic Violence Work and Program Participation Raising Children Living and Home Arrangements

4. Steps in Instrument Development Precisely define the information you need Method and Sampling Strategy Pilot testing Conduct Interviews Analysis and presentation of findings

5. Case Study What was the population of interest here? Sampling frame? Sample? Ordering of questions explicitly discussed Response and completion rates Conclusions –Format affects participation and response rates –Format can also affect nature of responses

6. Improving Response Rates Techniques for limiting non-response Reward respondent Reduce cost to respondent Establish trust in advance

7. Observation Techniques Utility of method Way to gather information on program characteristics or implementation Think about your research questions in advance – develop a protocol that guides your observation activities Train observers, coders – ensure reliability Participant observation Subject reactivity

8. Stages in Developing an Evaluation Stage One –Defining the Evaluation –Program goals, implementation theory, type of evaluation, stakeholders Stage Two - Develop Analysis Strategy –Tracking implementation and outcomes –Proper data for the question(s) of interest –Sampling, validity, and reliability –Mix of methods, qualitative versus quantitative, sources of data Stage Three - Implementation of Data Collection Stage Four - Analysis, Report Writing, and Presentation

9. Readings for Next Time Bengston and Fan. “An Innovative Method For Evaluating Strategic Goals In A Public Agency,” Evaluation Review, 23(1): 77-100, February 1999. Handout for Data Assignment – Due on Wed. Feb. 19th Individual Formative Design Proposal Due on Monday