PROGRAM EVALUATION 2013 R&D, FEBRUARY 12, 2014 DBEEPEDML

ABOUT THE PE2013 PROJECT A project of the SFL  DBE  DML  EPE Carried out by R&D (Zeynep Akşit, Defne Akıncı Midas / Hale Kızılcık) - interested researchers invited (FLE)

PROGRAM EVALUATION ---a systematic collection and analysis of information necessary to  improve a curriculum,  assess its effectiveness and efficiency, and  determine participants’ attitudes within the context of a particular institution. Brown, 1995

PURPOSE OF EVALUATION  Summative judgment  Learning and formative improvement  Accountability  Monitoring  Development  Knowledge generation

BENEFITS OF EVALUATION  Accountability  Information on specific strengths and weaknesses of the curriculum and its implementation  Critical information for strategic changes and policy decisions  Input for improved learning and teaching  Indicators of monitoring  Students (& their families)  Instructors  School Administration  University Board  Society  International Accreditors

OBJECTS OF EVALUATION Needs Analysis Objectives Testing Materials Teaching Brown, 1995

WHAT DO WE EVALUATE?  Curriculum design  The syllabus & program content  Classroom processes  Materials  The teachers  Teacher training  The students  Monitoring of ss progress  Learner motivation  The institution  Learning environment  Staff development  Decision making

WHAT ARE THE STEPS OF EVALUATıON?  What information do we need?  What will the results be used for?  Is evaluation necessary (do we already have this information)?  How much time is available?  What kind of information will be gathered?  Who will be involved in/support the evaluation?  How will we gather information (and who will be involved)?  How do we present the findings?

OBJECTS OF EVALUATıON Needs Analysis Objectives Testing Materials Teaching Evaluation Brown, 1995 Objectives

USEFUL TO WHO?  Stakeholders  Students  Instructors  Test Writers  Teacher Educators  Academic Coordinators  Administrators  Faculty Members

HOW DO WE EVALUATE?  Technical Procedure  Objectives – what do we intend to achieve?  Variables – what type of information is needed? (context, processes, outcomes)  Framework – what are the conditions for data gathering? (survey, environment analysis)  Instrumentation – what instruments will be used to collect data?  Data Collection – how will it be implemented?

IMPETUS MACRO  School-wide attempt at re-defining program outcomes ( METU Strategic Plan)  A need to coordinate the programs at the DBE & DML

IMPETUS MICRO  The last curriculum project – 2003  The last evaluation process – 2005 – problems indicated

SURVEY OF EXISTING CONDITIONS  The existing curriculum & syllabus  The materials in use  The teacher population  The learners  The resources of the program Dubin & Olshtain, 1990

WHAT DO WE WANT TO ACHIEVE?  To what extent the current program effectively addresses students’ needs?  Curriculum / Syllabus documents  Materials  Teachers  Test batteries  Environmental factors

EVALUATıON & CURRıCULUM RENEWAL

INITIAL PHASE Evaluation & Needs Assessment Analysis Results & Decision Making

SECOND PHASE Needs Goals & Objectives Content & Sequencing StrategiesMaterialsAssessment Review / Revise Present Situation & Target Situation Analyses Pilot Group

INıTıAL PHASE – STAFF ıNVOLVEMENT Needs Assessment Analysis Results & Decision Making R&D Core Group Voluntary Ts R&D Core Group Voluntary Ts R&D Core Group Voluntary Ts R&D Core Group Voluntary Ts R&D Core Group Ts as Decision-Makers R&D Core Group Ts as Decision-Makers

TENTATıVE SCHEDULE InputOutput1 st Stage2 nd Stage Focus Group InterviewsDBE Ss Questionnaire09 December25 Dec (+review) Administration of the Q13-17 Jan Ss Q Initial AnalysisSs Interviews17-24 Jan Feb Classroom Observations24 Feb.-03 Mar Teacher Questionnaire03-16 Mar (+review) Teacher Q Initial AnalysisTeacher Interviews17-30 Mar Apr Qual & Quan Data AnalysisPresentation of Results11 Apr.-15 May May 2014

WHERE ARE WE NOW?  Review of previous studies, literature, research reports,  Project outline / documentation  Focus group interviews  Ss Questionnaires

WHERE DO WE GO FROM HERE?  Data collection cont’d  DBE / DML Students  DBE / DML Instructors  Faculty Interviews  Faculty Documents  Data Analysis  Reporting

Project framework Tools Data Collection Data Analysis Report (Tentative) Program Revision Fall Spring Fall

Graves, 2000