1. 1 SOCIAL RESEARCH ASSOCIATION IRELAND/IRISH EVALUATION NETWORK SEMINAR GOOD PRACTICE IN COMMISSIONING RESEARCH AND EVALUATION 10 TH JANUARY 2006

2. Contents of Presentation 2  a “supplier” perspective  nature of evaluation  typical stages in procurement process  common challenges, needs  some issues for future

3. Aspects of “Supplier” Perspective 3  evaluation more than research  external supplier perspective (not internal procurement)  a commercial consultancy perspective (not research grants)  multiple types, contexts  a small country perspective  emphasising common issues

4. Multi-dimensional Nature of Evaluation 4 Managing the team Stakeholder Involvement Research, Analysis Time, resources, budget Client, user relations EVALUATION A Balancing Act

5. Typical Stages in Procurement 5 5. Managing, Undertaking, Analysing

6. Common Challenges, Good Practice 6 1. Policy issue, topic, need  lack of clarity  lack of client consensus  clarity, consensus  spell out objectives 2. Terms of Reference, Brief  problems of no.1 flow in here  over-specifying methodology  “piece of string” problem  don’t rush the ToR  focus on objectives 3. Procurement, Tendering  poor procedures  overly formalised competitive procedures which prevent dialogue 4. Selection, Contracting, Inception  disproportional procedures  rush to get started  proportionality  don’t rush the Inception ChallengesGood PracticeStage

7. Common Needs, Challenges and Good Practice (Cont’d) 7 5. Managing, Undertaking, Analysing  unrealistic deadlines  absence of data  data collection v analysis  multiple stakeholders  “scope creep”  good project management  role of Steering Committees?  avoid surprises 6. Reporting  emphasis on written “tomes”  difficulty finalising reports  “reversing” compromises into reports  separation of consultants’ recommendations from policy decision ChallengesGood PracticeStage

8. Procurement Procedures, Tendering (balancing competition and the need for dialogue) 8  need for better procurement skills, e.g. training, guidelines  inviting too many bidders/overuse of e-tenders website?  need for guidance on scale  circulation of all replies to all bidders!  ability, availability of commissioner representatives  see also www.fitzpatrick-associates.com (tendering)

9. Wider Issues for the Future 9  extent of evidence-informed, evaluation culture  the need for research, evaluation to “speak” to policy makers  need for more basic, neutral data collection  balance between “independence” and “relevance”  emphasis on costs of research/evaluation v costs of poor policy decision  more inter-disciplinary research, evaluation (e.g. “economic” v “social”)  over-evaluation of some areas, under-evaluation of others