1. Ms. Shivaun O' Brien, QFI National Co-ordinator Quality Framework Initiative for Youthreach and Senior Traveller Training Centres Embedding a Culture of Evaluation October 4 th 2007

2. Ms. Shivaun O' Brien, QFI National Co-ordinator Why a QFI for Youthreach and STTCs? Demand from the centres (Youthreach 2000) Education Act (1998) - Centres were designated as Centres for Education in 2004 YR/ STTCs both work with children, ie. under 18, also sixteen year olds/ those not completed 3 yrs post primary school DES responsible for the quality of education provided in Centres for Education as for schools Schools have SDPI YR/ STTC have QFI SDPI and QFI similar in process and resources

3. Ms. Shivaun O' Brien, QFI National Co-ordinator PHASETIMEFRAME Exploratory PhaseNov ‘00 – Feb ‘01 Consultation PhaseMay ’01- April ‘02 Development PhaseMay ’02 – July ‘03 Pilot PhaseSept ’03 – July ‘04 Roll out PhaseMarch ’05 - present Development of the Quality Framework Initiative- Timeframe

4. Ms. Shivaun O' Brien, QFI National Co-ordinator The Quality Framework QUALITY FRAMEWORK

5. Quality Standards – 27 Quality Areas Organisational Management Personnel and Development Learning Environment Programme 1. Ethos11. Staff Team15. Social Environment19. Prog. Development and Delivery 2. Planning12. Staff Recruitment and Induction 16. Code of Behaviour20. Recruitment of Learners and Admission 3. Evaluation13. Staff Dev. & Train.17. Equality21. Initial Ass. Ind. & Review 4. Commuic. & Links with the Community 14. Staff Support18.Interculturalism22. Learning Ass. & Certification 5. Transparency, Acc. and PR 23. Support Structures for Learners 6. Admin. & Financial Mgt. 24. Literacy & Numeracy 7. Record Keeping25. Soc. Personal & Health Ed. 8. Health & Safety26. Work Exp. 9. Premises27. Transfer & Progression 10. Equipment

6. Ms. Shivaun O' Brien, QFI National Co-ordinator How ICE happens (practicalities) ICE session takes place over two days Centre closed Some preparation required in advance QFI Facilitator booked in advance/ TES rates Structure of session prescribed/ guidelines Important that each activity is useful/ time well spent Facilitator types up outcomes of the session Centre Co-ordinator/ Director prepares annual report for VEC following session Check implementation of actions from year to year

7. Ms. Shivaun O' Brien, QFI National Co-ordinator What can be evaluated? Any of the 27 quality areas – select 9 areas to start Specific Programmes/ activities of the centre Implementation of the Centre Plan Combination of these

8. Who is involved? Core people (attend 2 days) Centre Co-Ordinator/ Director Centre Staff incl. admin Key - but involved to varying degrees Learners (evaluation workshops & questionnaires) Regional Co-Ordinator Adult Education Officer (AEO) Education Officer (EO) Chief Executive Officer (CEO) Board of Management Reps Involve where relevant (questionnaires/interview/ focus groups) Community Reps. e.g if evaluating links with community Parents/ Employers e.g. if contact with parents is being evaluated May attend part of 2 day session or complete questionnaire/ interview in advance

9. Ms. Shivaun O' Brien, QFI National Co-ordinator Internal Centre Evaluation

10. Ms. Shivaun O' Brien, QFI National Co-ordinator Purpose of the ICE Primary purpose: To examine centre practice, identify strengths and weaknesses, identify & implement actions to improve the service. Accountability: annual reports to VEC, evidence for DES inspectors But is that all?

11. Ms. Shivaun O' Brien, QFI National Co-ordinator Evaluation can have process outcomes “Process refers to and is indicated by individual changes in thinking and behaviour and programme or organisational changes in procedures and culture, that occur among those involved in evaluation as a result of the learning that occurs during the evaluation process” Michael Quinn Patten (1993) Utilization Focused Evaluation UFE :Concern with the usefulness of the evaluation

12. Ms. Shivaun O' Brien, QFI National Co-ordinator Process Outcomes Enhancing shared understanding ( NOT teachers teach and managers manage), staff concerned with overall service not just the subjects they teach Teambuilding - regularity NB Capacity Building – more capable more confident Increasing staff engagement, sense of ownership and self determination ( participatory and empowerment evaluation) Increased intentionality – end up with plan, a sense of direction, commitment to making progress

13. Ms. Shivaun O' Brien, QFI National Co-ordinator Building in process outcomes Bring staff together Clear communication – facilitator clarifies and challenges Everyone talks/ listens – equally valued See the service through each others’ eyes – opportunities for personal reflection Safety – speaking openly, evaluation of systems not people, controlled & kept focused Learning- new knowledge and skills: from gathering and discussing evidence, learning to solve problems, learning to evaluate (double loop learning), becomes integral to programme Making decisions - deciding on actions that are actually carried out

14. Ms. Shivaun O' Brien, QFI National Co-ordinator Evidence Evidence can be Quantitative or Qualitative Lists of evidence pg 23-26 ICE guidelines

15. Ms. Shivaun O' Brien, QFI National Co-ordinator Evaluation by learners Resources See QFI Guidelines for Internal Centre Evaluation Chapter 3 ICE guidelines- reflect current good practice www.youthreach.ie www.sttc.ie Hearing Young Voices www.youth.iewww.youth.ie Participation Spice it Up Dynamix UK Explain the reasons for evaluation to learners Approach depends on learners age, ability Decide the focus of the learner evaluation Plan session/s EVALUATION SHOULD BE A POSITIVE and USEFUL EXPERIENCE FOR LEARNERS

16. Ms. Shivaun O' Brien, QFI National Co-ordinator Evaluation by learners ctd. How will learners receive feedback Important to show learners that their opinion was considered and that appropriate action was taken Evaluation (reviews) of programmes by learners should happen every time a programme is delivered- feed into FETAC QA requirement SEE HANDOUTS

17. Ms. Shivaun O' Brien, QFI National Co-ordinator Follow-up to ICE Session Facilitator will type up outcomes of 2 day session and return it to centre Co-ordinator/Director Staff and VEC mgt get copy of evaluation report Ensure that action plan is implemented Keep action plan on agenda at staff meetings Co-ordinator/ Director responsible for ensuring that actions are implemented/ time allocated ICE following year – check progress

18. Ms. Shivaun O' Brien, QFI National Co-ordinator Contact : Shivaun O’Brien Quality Framework Co-Ordinator C/O Co Louth VEC King St Drogheda Co. Louth 041-9839930 shivaunobrien@eircom.net For QFI Resources www.youthreach.ie www.sttc.ie