Evaluating the Impact of Continuing Professional Development National CPD coordinators network meeting 1 st February 2008 Gillian Brydson

"Not for the first time, Larry asked himself why he had picked a goat for a coach"

Why is evaluating CPD important ‘Over the years, a lot of good things have been done in the name of professional development. So have a lot of rotten things. What professional developers have not done is provide evidence to document the difference between the good and the rotten.’

leading to…. ‘False or exaggerated claims of success are the basis of many school reform strategies – in part because we lack better and more timely evaluations of new practices and programmes’ Consortium for Policy Research in Education (1996)

2 key questions…  What is the quality of the CPD?  Has the money and time invested in a particular CPD activity made a difference to pupils?

Why is evaluating CPD difficult? Discussions of evaluation issues often are seen as unwelcome and unnecessary intrusions into the important work that needs to be done. Thomas Guskey

Why is evaluating CPD difficult? 1. What is quality?

‘What is required for success in professional development is a clear and compelling vision of the improvements needed, combined with explicit ideas on the organisational characteristics and attributes necessary for success’ Guskey (1999)

Why is evaluating CPD difficult? 2. What is ‘evidence’? Did the earth move for you?

Behaviour Instruction Institution Psychomotor Affective Cognitive Content Organisation Method Facilities Cost Pupil Teacher Head Specialists Family Community So many variables…..

 What is your general impression of our evaluation of CPD?  What types of information do you consider particularly important or meaningful in an evaluation? (Very Quick )Group Discussion

What are the limitations of our evaluations… Too brief  documentation not evaluation Too shallow  need to probe deeper Too quick  expect too much too soon

Not a self-contained exercise Integral to self-evaluation being undertaken by teacher, school and local authority Should be explicit and planned from the outset Begin with the end in mind. Overcoming the limitations?

Begin with the end in mind… If you don't know where you are going you will probably end up somewhere else. The Peter Principle.

It’s hard but….  We understand what CPD is…  We know the major models of professional development…  We have clear purposes and goals for Scottish Education  We agree on our outcomes…

Naomi Model

Kirkpatrick Model 1. reaction 2. learning 3. behaviour 4. results

Guskey Model 1. Participant reaction 2. Participant learning 3. Organisational support and change 4. Use of new knowledge and skills 5. Student learning outcomes

Tools to gather information?  Reaction – post course questionnaire  Learning – reflections / portfolio  Organisational Change – school self- evaluation,focus groups, stakeholder surveys  Use of new knowledge and skills – interviews, observations  Pupil Learning – achievement, participation, attainment

Professional Discussion  Consider these models in providing an evaluation of the impact of a CPD programme. What questions are addressed? What tools could you use? What measure is assessed? How is the information used?

 Is an evaluation framework useful?  If so, to what level?  What measures of outcome are already out there that are relevant to the evaluation of CPD?

Recommendations  Paper considered by CPD coordinators  Work taken forward by national group

 When you discover that you are riding a dead horse, the best strategy is to dismount. Dakota tribe proverb Professional Courage

However…in government, education and the corporate world, more advanced strategies are often employed, such as:  Giving horse and rider a good bollocking  Buying a stronger whip.  Changing riders.  Appointing a committee to study the horse.  Arranging to visit other countries to see how other cultures ride horses.  Convening a dead horse productivity improvement workshop  Reclassifying the dead horse as living-impaired.  Hiring outside contractors to ride the dead horse.  Outsourcing the management of the dead horse.

 Harnessing several dead horses together to increase speed.  Doing a productivity study to see if lighter riders would improve the dead horse's performance.  Declaring that as the dead horse does not have to be fed, it is less costly, carries lower overhead and therefore contributes substantially more to the bottom line of the economy than do some other horses.  Rewriting the expected performance requirements for all horses.  Providing additional funding and/or training to increase dead horse's performance… and the highly effective...  Promoting the dead horse to a supervisory position.

Thank you