Gathering, analysing and presenting information for evaluation through the prism of the TY Programme Cornelius Young (087)

Professional Development Service for Teachers Assessment for Learning Oral Language Problem Solving Active Learning Portfolio Assessment Inclusion Differentiated Strategies Learning Styles ICT in Teaching & Learning Classroom Management Cooperative Learning Practical Maths Mental Maths Writing Reading Reading Recovery School Planning Maths Planning EAL Tús Maith Child Protection Spreag an Ghaeilge le Spraoi Post-primary Languages Post Primary Sciences Leaving Certificate Applied Transition Year Cultural & Environmental Subjects Business & Enterprise Subjects Leadership Development Reflective School Transfer from Primary to Secondary Leading Planning

How do you evaluate Transition Year in your School?

Evaluation Evaluation is: “A process of obtaining information and using it to form judgements which in turn are to be used in decision making” Evaluation may be : External - conducted by Inspectorate Internal - conducted by the School

What is School Self- evaluation?  “School self-evaluation (SSE) is a collaborative, reflective process of internal school review whereby the principal, deputy principal and teachers in consultation with the board of management, parents and students engage in reflective enquiry on the work of the school”.  It involves: –An evidence-based process –Making judgments about the work of the school –Bringing about improvements in students’ learning. School Self-evaluation Guidelines, 2012

School self-evaluation cycle Gather evidence Implement and monitor improvement plan Make judgements about strengths and concerns Write school self- evaluation report Devise school improvement plan

Internal Evaluation of T.Y. “The programme should be regularly reviewed and evaluated internally by the coordinating team in close co-operation with school management, staff, students, parents, work providers and community interests.” (Transition Year Guidelines , Dept. of Education)

School Self-evaluation How effective are you as a school organisation? How do you know this? Evidence Where do we want to be? Targets How do we get there? Action Plans How do we know we got there? Evidence

School Self-evaluation How effective is you Transition Year Programme? How do you know this? Evidence Where do we want to be? Targets How do we get there? Action Plans How do we know we got there? Evidence

Aims of theT.Y. Programme  Provide students with opportunities to develop their personal, intellectual, spiritual, social, physical and emotional life in a caring, stimulating and varied learning environment.  Provide a transition between Junior and Senior cycles.  Encourage students to achieve their potential in subject areas and learning experiences within the programme.  Provide a broad and balanced curriculum including the provision of additional support in the core subjects where possible.  Provide students with opportunities for personal and social development. Where do you start?

Evaluation of the TYP  Mandatory part of ‘Writing the T.Y. Programme’  Affirms staff members –individually/collectively  Provides vital feedback and data  Encourages learning from experience  Facilitates continual improvement  Necessary for on-going course design  Identifies strengths and weaknesses  Initiates the reflective process.  Avoids staleness

Evaluation of the TYP  When to evaluate?  Who should be involved?  What should I evaluate?  What becomes of the Evidence?

PDST Samples  Evaluating a Transition Programme  Student End of Year Evaluation.  Student Evaluation of Specific Event  Student Evaluation of Work Experience  Teacher’s self /subject evaluation.  Parents Evaluation of T.Y.

VenueDate Athlone Ed Centre08/03/2013 Blackrock14/03/2013 Carrick on Shannon07/03/2013 Cork Education Support Centre15/03/2013 County Wexford Education Centre14/03/2013 Donegal13/03/2013 Drumcondra Education Centre11/04/2013 Dublin West16/03/2013 Dunmanway, West Cork07/03/2013 Ennis08/03/2013 Galway Education Centre14/03/2013 Kildare12/03/2013 Kilkenny Education Centre21/03/2013 Limerick06/03/2013 Mayo06/03/2013 Monaghan Education Centre21/03/2013 Navan Education Centre05/03/2013 Portlaoise Education Centre22/03/2013 Tralee15/03/2013 Data Gathering Workshops to support School Self- evaluation

Gathering and analysing data Evidence in schools

Data analysis  Think about analysis EARLY  Start with a plan  Code, filter, count  Analyse  Present/Report  Reflect – evaluate the evaluation

Often, it is better to use more than one method….  Mixed methods Log of activities and participation Self-administered questionnaires completed after module In-depth interviews with key informants Observation of workshops Survey of parents

Common data collection methods  Survey  Case study  Interview  Observation  Group assessment  Expert or peer reviews  Portfolio reviews  Minutes of meetings  Testimonials  Tests  Photographs, videotapes, slides  Diaries, journals, logs  Document review and analysis  Rating Scales  Focus groups  Document Review

Very often Quite often Only some- times Rarely or Never Learn a lot Learn quite a lot Learn a little Learn Nothing Answering the teacher’s questions Doing experiments Working on the computer Watching a video Listening to a tape Acting out or doing a role-play Taking notes while the teacher talks Taking notes from a book or worksheet Making things (like models)

Learning Preference Frequency AB DC E.g.: Project work

FocusingDevelopingEstablishingEnhancing Teachers promote positive attitudes to students and to learning There is an expectation on the part of teachers that all students will take an active part in learning Teaching uses a range of approaches that ensures all students take part. There is a culture of collaborative learning There is an expectation on the part of both teachers and students that learning is important and enjoyable and that everyone can improve Learning objectives are clear and appropriately challenging, and teaching is well focused A ‘can do’ culture is promoted Students know where they are in their learning and what they need to do to improve Teachers or practitioners are ambitious for students and expectations of learning are high Display emphasises and supports learning The layout of the classroom supports an inclusive, interactive teaching approach Learning and teaching resources promote student’s involvement and shared learning The learning process is valued – there is a supportive atmosphere that allows students to make and learn from mistakes. There is an emphasis on shared learning, analysis and discussion Learning objectives are explicitly shared with students Display reflects the learning process as well as the content and the product of Student’s learning Students get feedback on their learning in a range of ways. They know what ‘good’ looks like and are motivated to be ‘the best that I can be’ There are opportunities for reflection and discussion throughout the day and also, if appropriate, in the plenary of a lesson Opportunities for reflection at different points throughout the lesson, and throughout the day, support for students in peer and self-assessment Students talk as part of their learning and about their learning Students talking with adults and other students is a key feature of the learning process, and teaching approaches promote and guide this

1234 I concentrate on what I am expected to learn. I take notes of key points. I make an effort to understand rather than just get through it. I think through the problem carefully. I explain it to myself to understand it better. I identify blocks or difficulties with learning. I ask myself questions about what I’m learning. If there’s a problem I leave it and come back to it later. I look for someone who can help me. I explain to others what I am learning. I work hard at overcoming difficulties. I relate what I am learning to my experience. I am able to ignore distractions. I try to relate what I’m learning to things I’ve learned before. I work in a place where I am most comfortable but also alert. I try to make the subject matter interesting. I play background music while I am studying. I experiment with ways of memorising things well.

TIDY12345UNTIDY WARM12345COLD PARENT-FRIENDLY12345PARENT-UNFRIENDLY COLOURFUL12345DRAB AUTHORITARIAN12345DEMOCRATIC COMFORTABLE12345UNCOMFORTABLE ORDERLY12345DISORDERLY SENSITIVE12345INSENSITIVE STRICT12345EASY-GOING HIGH STRESS12345LOW STRESS PESSIMISTIC12345OPTIMISTIC TENSE12345RELAXED HELPFUL12345UNHELPFUL COMPETITIVE12345UNCOMPETITIVE FORMAL12345INFORMAL REACTIVE12345PROACTIVE LIKES CHANGE12345DISLIKES CHANGE STIMULATING12345BORING PUPIL-FRIENDLY12345PUPIL-UNFRIENDLY INFLEXIBLE12345FLEXIBLE CLEAR VALUES12345NO CLEAR VALUES AVOIDS CONFLICT12345 RESPONDS WELL TO CONFLICT ADVENTUROUS12345CAUTIOUS USES TIME WELL12345TIME USED BADLY RISK-TAKING12345AVOIDS RISKS OPEN TO NEW IDEAS12345SCEPTICAL OF NEW IDEAS IDEALISTIC12345PRAGMATIC PURSUES LONG-TERM GOALS12345 PURSUES SHORT-TERM GOALS LOOKS TO THE PAST12345LOOKS TO THE FUTURE

Data entry by computer  By Computer –Word (charts) –Excel (spreadsheet) –Google Drive

Why do I need an analysis plan?  To make sure the questions and your data collection instrument will get the information you want  Think about your “report” when you are designing your data collection instruments

Consider…  Resources available: Time, money, and staff to design, implement, and analyse the information.  Type of information you need. Numbers, percents, comparisons, stories, examples, etc.  Intrusions to work or participants. Which method is likely to be least onerous?  Advantages and disadvantages of each method.  The need for credible and authentic evidence.  The value of using multiple methods.  The importance of ensuring cultural appropriateness.

Analysing qualitative data “Content analysis” steps:  Highlight quotes/terms and note why important  Code quotes according to margin notes  Sort quotes into coded groups (themes)  Interpret patterns  Describe these patterns

Questionnaire design − Considerations  Kind of information: What do you want to know? Is the information already available?  Wording of questions and responses  Formatting the questionnaire  Pre-testing  Cover letters and introductions  When/where will the questionnaire be distributed?  How will returns be managed? How will the data be analysed?  Who is responsible for each task?

Kinds of information – What do you want to know?  Knowledge − what people know, how well they understand something  Beliefs − attitudes, opinions  Behaviors − what people do  Attributes/Demographics − what people are and what people have

Some key questions to ask:  Which pupils and pupil groups in your subject/teaching group are achieving well?  Are all pupils, including disadvantaged groups, fulfilling their potential or should they achieve more?  Do pupils, who did well in Junior Certificate, lose momentum and fail to make progress?  Are the most disadvantaged pupils, or those with SEN, making sufficiently rapid progress to catch up with their peers?  Are the most able pupils stretched and challenged sufficiently?  Are overall results at the end of TY concealing poor progress?

Questions? Cornelius Young   (087) 