1. CATS Pilot Project Peer Evaluation Process
Lindsay Perrin, CWBL,
University of Greenwich

2. Peer Evaluation Process
Consists of 2 elements:
Impact Evaluation Model
Monitoring and evaluation plan
The University of Greenwich and Linking London CATS Pilot Project teams recognised the importance of assessing impact to justify value for money, to determine the direction and consequences of projects and to show outcomes in achievements, products and processes. They have therefore developed and agreed a Peer Evaluation Process by which both project teams will monitor the process and outcomes of the other’s project through a shared ‘virtual’ portfolio of evidence and produce an evaluative commentary to accompany the Project Final Report.
It was agreed that the process should be simple, clear and sufficiently robust but not so onerous as to divert time and resources away from other important project activities.
The Peer Evaluation Process consists of 2 elements:
An Impact Evaluation model which ensures consistency and provides a common framework to enable each project team to monitor each others’ progress against their work plan
A monitoring and evaluation plan which underpins the IEM by confirming the process by which each project team will monitor and evaluate

3. Impact Evaluation Model (UoG)
Impact Evaluation Model (IEM)
The Impact Evaluation Model is based on a model previously developed by the Kent and Medway Lifelong Learning Network, led by the University of Greenwich to help its partner institutions measure the impact of their LLN-funded projects. In turn based on a model developed by the Training and Development Agency for Schools (TDA) it did not claim to offer a brand new approach to evaluation, but attempted to provide an accessible and user-friendly framework to assist their activity.
The fact that it brings together qualitative and quantitative evidence and allows us to look not only at data but also the real people involved and the impact the project could have on communities (work-based learners, employers and learning providers) makes it particularly applicable to the CATS project. The model is designed to help the project team build up a picture of how they expect the project to work and by being clear about the outcomes of the project.
Initially, the project teams worked together to identify the inputs, outputs, intermediate outcomes and final outcomes that their project was trying to achieve and to populate the model with these. This enabled them to articulate how they expected their project to have an impact. The model was then used to identify the evidence needed and from there, to identify the potential sources of evidence.
By working through the model each project team was able to demonstrate the links between the various stages of project delivery, from planning all the way through to the impact on individual users of the activities and products being delivered and on the overall project aims.
From past experience we were aware that the model is most effective when put in place early in the project. By completing the model at this stage the project teams knew what data and evidence we needed to collect at the outset and were able to clarify for each other any uncertainties about the aims of our projects. Articulating the underlying assumptions of the projects and identifying the potential evidence sources at an early stage have informed the project work plans and have been explored as delivery has progressed.

4. Impact Evaluation Model (LL)

5. Monitoring and Evaluation Plan
Method
Purpose
Frequency
Face-to-Face Meetings
To update and question each other on progress
To agree strengths and weaknesses
To share knowledge and best practice
To discuss issues and propose solutions
Beginning (Sep 12)
Mid-term (Jan 13)
End (May 13)
Dropbox
To store and share records, documents and other evidence and make available to each other
Ongoing – as records and documents are produced s
To update each other on events, dropbox updates, developments
To share ideas and deal with issues arising between meetings
As and when required between meetings
Evaluative Commentary
To provide an evaluative commentary to inform final report
Once – end of project
The above methods have been used to monitor the progress of each others’ project:
Face to Face Meetings
The principle method to support the monitoring and evaluation process, three meetings of Project Directors and Managers
Sep 12 – to agree impact evaluation model and agree process
Jan 13 – to report progress to date and agree outcomes for draft interim report
May 13 – to report progress to date and agree outcomes for final report
Dropbox
We have used Dropbox as a virtual repository for all documents, publications, records and other evidence identified on the Impact Evaluation Model. Each project will have its own folder, each containing four sub folders - Inputs, Outputs, Intermediate Outcomes and Final Outcomes (as per IEM). Evidence will be filed in the appropriate sub-folder.
Folders and sub-folders have been shared and accessible to each other.

6. Evaluative Commentary
To what extent did the project meet its aims and objectives?
What, if any, are the additional benefits delivered by the project?
What lessons did we learn that we could apply to future projects?
Each project team will provide a brief evaluative commentary (approx. 500 words) on the others’ project to support the final report.

7. Benefits of Peer Evaluation
Assuring the quality of the University of Greenwich and Linking London Projects by providing a common framework by which to monitor progress against work plan
Enabling project teams to meet as ‘critical friends’ to identify common issues, propose solutions and share best practice, without prejudice
The peer evaluation process has assured the quality of the University of Greenwich and Linking London Projects. Meetings between the two teams acting as ‘critical friends’ have provided valuable opportunities to discuss common issues, propose solutions and share best practice, without prejudice.