Government evaluations - Views from the supplier perspective James Wunsch – Research Director – Colmar Brunton

Some context – who am I? Research Director with Colmar Brunton 19 years in research & evaluation for government as supplier and buyer Have conducted major evaluations across employment, immigration, mental health & education View research & evaluation as complementary fields

Tips for running evaluation tender Outline Program or Project Logic – how is the intervention supposed to work in theory? “In scope” vs. “out of scope” What data do you have (be specific) Is ethics clearance required? Will project team be “hands on” or “hands off”? Specify budget Don’t be too prescriptive regarding methodology

Where things work well Scope is clearly defined Clear, ongoing communication Key stakeholders have buy in from the outset Addressing challenges as they arise Both sides being aware of and working within key constraints (logistical, political, etc.) Proper framing of the project within the Department (including all key stakeholders)

Key challenges Changing personnel/changing expectations Silos can mean no-one has end-to-end perspective of a program or process Lack of data, or data not useful (e.g. administrative/point in time, not longitudinal) Unrealistic timeframes to observe meaningful change Providing answers Department doesn’t want to necessarily hear

Adopt an evaluation mindset Ensure budget is available for evaluation Develop Program Logic Model Identify key data likely to be needed and ensure this is captured and recorded appropriately Can you identify a control group for evaluation purposes Hard to “reverse fit” an evaluation if project or program aims & objectives not clearly defined at the start