1. PROGRAM EVALUATION 2013 R&D, FEBRUARY 12, 2014 DBEEPEDML

2. ABOUT THE PE2013 PROJECT A project of the SFL  DBE  DML  EPE Carried out by R&D (Zeynep Akşit, Defne Akıncı Midas / Hale Kızılcık) - interested researchers invited (FLE)

3. PROGRAM EVALUATION ---a systematic collection and analysis of information necessary to  improve a curriculum,  assess its effectiveness and efficiency, and  determine participants’ attitudes within the context of a particular institution. Brown, 1995

4. PURPOSE OF EVALUATION  Summative judgment  Learning and formative improvement  Accountability  Monitoring  Development  Knowledge generation

5. BENEFITS OF EVALUATION  Accountability  Information on specific strengths and weaknesses of the curriculum and its implementation  Critical information for strategic changes and policy decisions  Input for improved learning and teaching  Indicators of monitoring  Students (& their families)  Instructors  School Administration  University Board  Society  International Accreditors

6. OBJECTS OF EVALUATION Needs Analysis Objectives Testing Materials Teaching Brown, 1995

7. WHAT DO WE EVALUATE?  Curriculum design  The syllabus & program content  Classroom processes  Materials  The teachers  Teacher training  The students  Monitoring of ss progress  Learner motivation  The institution  Learning environment  Staff development  Decision making

8. WHAT ARE THE STEPS OF EVALUATıON?  What information do we need?  What will the results be used for?  Is evaluation necessary (do we already have this information)?  How much time is available?  What kind of information will be gathered?  Who will be involved in/support the evaluation?  How will we gather information (and who will be involved)?  How do we present the findings?

9. OBJECTS OF EVALUATıON Needs Analysis Objectives Testing Materials Teaching Evaluation Brown, 1995 Objectives

10. USEFUL TO WHO?  Stakeholders  Students  Instructors  Test Writers  Teacher Educators  Academic Coordinators  Administrators  Faculty Members

11. HOW DO WE EVALUATE?  Technical Procedure  Objectives – what do we intend to achieve?  Variables – what type of information is needed? (context, processes, outcomes)  Framework – what are the conditions for data gathering? (survey, environment analysis)  Instrumentation – what instruments will be used to collect data?  Data Collection – how will it be implemented?

12. IMPETUS MACRO  School-wide attempt at re-defining program outcomes (2011-2016 METU Strategic Plan)  A need to coordinate the programs at the DBE & DML

13. IMPETUS MICRO  The last curriculum project – 2003  The last evaluation process – 2005 – problems indicated

14. SURVEY OF EXISTING CONDITIONS  The existing curriculum & syllabus  The materials in use  The teacher population  The learners  The resources of the program Dubin & Olshtain, 1990

15. WHAT DO WE WANT TO ACHIEVE?  To what extent the current program effectively addresses students’ needs?  Curriculum / Syllabus documents  Materials  Teachers  Test batteries  Environmental factors

16. EVALUATıON & CURRıCULUM RENEWAL

17. INITIAL PHASE Evaluation & Needs Assessment Analysis Results & Decision Making

18. SECOND PHASE Needs Goals & Objectives Content & Sequencing StrategiesMaterialsAssessment Review / Revise Present Situation & Target Situation Analyses Pilot Group

19. INıTıAL PHASE – STAFF ıNVOLVEMENT Needs Assessment Analysis Results & Decision Making R&D Core Group Voluntary Ts R&D Core Group Voluntary Ts R&D Core Group Voluntary Ts R&D Core Group Voluntary Ts R&D Core Group Ts as Decision-Makers R&D Core Group Ts as Decision-Makers

20. TENTATıVE SCHEDULE InputOutput1 st Stage2 nd Stage Focus Group InterviewsDBE Ss Questionnaire09 December25 Dec. 2014 (+review) Administration of the Q13-17 Jan. 2014 Ss Q Initial AnalysisSs Interviews17-24 Jan. 201417-27 Feb. 2014 Classroom Observations24 Feb.-03 Mar. 2014 Teacher Questionnaire03-16 Mar. 2014 (+review) Teacher Q Initial AnalysisTeacher Interviews17-30 Mar. 20141-11 Apr. 2014 Qual & Quan Data AnalysisPresentation of Results11 Apr.-15 May 201430 May 2014

21. WHERE ARE WE NOW?  Review of previous studies, literature, research reports,  Project outline / documentation  Focus group interviews  Ss Questionnaires

22. WHERE DO WE GO FROM HERE?  Data collection cont’d  DBE / DML Students  DBE / DML Instructors  Faculty Interviews  Faculty Documents  Data Analysis  Reporting

23. Project framework Tools Data Collection Data Analysis Report (Tentative) Program Revision 2013 2014 Fall Spring Fall

24. Graves, 2000