Program Evaluation M9205 November 28, 2000

Columbia University School of NursingM6920, Fall, 2000 Program evaluation: a special case l Often done by “interested outsiders” rather than curious researchers l Often associated with decisions about continuation or funding l Often conducted in public view l Often conducted with other agendas in mind

Columbia University School of NursingM6920, Fall, 2000 Sunset laws as stimulus l Enacted by states in the 70’s and 80’s l Intended to provide a mechanism for ending programs that were outdated l Bring the executive and legislative branches of government and external interests into dialogue if not conflict

Columbia University School of NursingM6920, Fall, 2000 What is included? l How well is this program doing in meeting its goals? l How effectively are we using our resources? l How efficiently are staff working? l How does this compete with other programs in this area?

Columbia University School of NursingM6920, Fall, 2000 Steps in program evaluation* l engage stakeholders l describe the program l focus the evaluation design l gather credible evidence l justify conclusions l ensure use and share lessons learned *CDC Standard

Columbia University School of NursingM6920, Fall, 2000 When does program evaluation begin? l Should be anticipated in program design l Should answer the question ‘how will I know if I have succeeded?’ l Baseline data are critical l Program data and record keeping should be designed to answer evaluation questions

Columbia University School of NursingM6920, Fall, 2000 Evaluation design l Usually a combination of methods l Usually involves both existing program records and new data collection l Usually a compromise between ideal research design and practical possibilities

Columbia University School of NursingM6920, Fall, 2000 From Nsanje evaluation: l Objective 80% of WCBA will have received instruction regarding nutrition during infancy and childhood l Evaluation no direct indicator data collected “80% of VHV received this training by 5/31/94”, with no training records

Columbia University School of NursingM6920, Fall, 2000 Nsanje, cont. l Objective 80% of VHCs will have received instructions regarding the importance of and methods of family planning l Evaluation no data are available on what percent of VHCs received this training.

Columbia University School of NursingM6920, Fall, 2000 Nsanje, cont. l Objective 80% of 0-11 month old children will participate in a monthly growth monitoring program l Evaluation survey data: 91% 0-11 month olds had growth cards, 65% weighed during the last month all weighted within past two months

Columbia University School of NursingM6920, Fall, 2000 Tacoma Syringe Exchange Program l Retrospective case control study l Use of surrogate markers (HBV and HBC) to assess reduction in HIV transmission l Controls from methadone or HIV testing center

Columbia University School of NursingM6920, Fall, 2000 Tacoma, cont. l Odds ratio: the ratio of one odds to another, in this case the ratio of the odds of SE use in cases compared to the controls

Columbia University School of NursingM6920, Fall, 2000 Wagner, et al Community Program evaluation

Columbia University School of NursingM6920, Fall, 2000 Lipson et al: Safety Net l Public hospitals, community health centers, local health departments