1. Reflection and evaluation
Subject specialism

2. Models of Reflection
Different models of reflection can inform your own reflective writing/reflective journals.
Tripp’s critical incidents
EDAR
Schon
Kolb
Brookfield
Gibbs

3. Tripp’s Critical Incidents
A critical incident is one which causes a person to pause and contemplate the events that have occurred to try to give them some meaning. This may be a positive experience or a negative one. Using a critical incident as a way of reflecting involves the identification of behaviour deemed to have been particularly helpful or unhelpful in a given situation (Hannigan, 2001).
st=UUg_4z_8AM7x9wmZ6F1vJTcw
Tripp’s Critical Incidents

4. Experience – a significant event or incident you would like to change or improve
Describe – aspects such as who was involved, what happened, when it happened and where it happened
Analyse – consider the experience deeper and ask yourself how it happened and why it happened
Revise – think about how you would do it differently if it happened again and then try this out if you have the opportunity
(Gravells & Simpson 2008:90)
EDAR

5. KOLB’s learning cycle Engage with/observe experience
Reflection – feelings about the story/previous stories
Theorise – What does the ‘story’ mean? Enable user to explore the context of experience
Plans and actions – Identify goals, plans and services. What’s the next chapter in the story?
Kolb D. (1984).

6. Schon (1983)
Reflection in action – immediate response/change at time of incident, may be unconscious.
Reflection on action – reflection after an incident/session, more conscious, time to reflect consult others or evidence before making changes.

7. Reflection in Action
This is the teacher ‘thinking on his or her feet’, being spontaneous, creative and unique. The professional builds up a repertoire of knowledge and skills through reflection in action. There will be an ongoing purposeful reflection in action. Schön, D. (1983)

8. Gibb’s reflective cycle (1998)

9. Brookfield’s Critical Lenses
Brookfield HO

10. Stephen Brookfield Lenses on own practice….. -Our own eyes
Our students’ eyes
Our colleagues’ eyes
Research and theory.
(Brookfield, 1995)
Question: What opportunities do we have to look through each of these lenses?

11. OUR OWN EYES
Student evaluations Assessment answers and results student journals, student focus groups interview responses

12. STUDENTS’ EYES Critical incident questionnaire One minute paper
Muddiest point
Clickers

13. COLLEAGUES’ EYES Team teaching Critical reflection groups
Reciprocal peer review

14. LITERATURE
Teachers who research literature Teachers who present literature Teachers who publish literature

15. Analysis and evaluation of models
In groups, analyse the reflective model you are given and evaluate its potential strengths and weaknesses for you and your teaching and learning context.
Analysis and evaluation of models

16. Self Lens: Teachers may focus on their experiences as a teacher in order to reveal aspects of their pedagogy that may need adjustment or strengthening.
Student Lens: Engaging with student views of the learning environment can lead to more responsive teaching. Evaluations, assessments, journals, focus groups and/or interviews can each provide cues to improve teaching and learning.
Peer Lens: Peers can highlight hidden habits in teaching practice, and also provide innovative solutions to teaching problems. Further, colleagues can be inspirational and provide support and solidarity.
Literature Lens: Teaching theory provides the vocabulary for teaching practice, and offers different ways to view and understand your teaching. Here you'll find ways to utilise scholarly literature in your teaching and critical reflection.

17. Transformative Learning
This involves the learner in reinterpreting an old experience or a new one from a new set of expectations.  Transformative learning results in transformed meaning schemes through assessing assumptions.  (Mezirow, 1991)

18. Transformative learning continued
An important part of transformative learning is for individuals to change their frames of reference by critically reflecting on their assumptions and beliefs and consciously making and implementing plans that bring about new ways of defining their worlds. This process is fundamentally rational and analytical.

19. Transformative learning- complex?
At its core, transformative learning theory is elegantly simple. Through some event, which could be as traumatic as losing a job or as ordinary as an unexpected question, an individual becomes aware of holding a limiting or distorted view. If the individual critically examines this view, opens herself to alternatives, and consequently changes the way she sees things, she has transformed some part of how she makes meaning out of the world.

20. Why colleges succeed
A distinction ‘of these very good teachers is that their practice is the result of careful reflection …They themselves learn lessons each time they teach, evaluating what they do and using these self-critical evaluations to adjust what they do next time. (Why Colleges succeed, Ofsted 2004, Para 19)
What do you think of the link between between very good teachers and reflection?

21. References
MOON J (1999) Reflection in Learning and Professional Development: theory and practice London. Kogan Morgan 
Cowan, J (1999) On Becoming an Innovative University Teacher: Reflection in Action . SRHE and Open University Press
Mezirow, J (1997). Transformative learning: Theory to practice. New Directions for Adult and Continuing Education. Jossey Bass.
Biggs, J (1999) Teaching for quality learning at university. Buckingham: Open University
Schön, D. (1983) The Reflective Practitioner. How professionals think in action, London: Temple Smith.

22. References continued
Smith, M. K. (2001) 'Chris Argyris: theories of action, double-loop learning and organizational learning', the encyclopedia of informal education, update: May 29, 2012
Argyris, M. and Schön, D. (1974) Theory in Practice. Increasing professional effectiveness, San Francisco: Jossey-Bass.
Kolb D. (1984). Experiential learning: experience as the source of learning and development. Englewood Cliffs, New Jersey: Prentice Hall.
Brookfield, Stephen. Becoming a Critically Reflective Teacher. San- Francisco: Jossey-Bass. 1995

23. Useful web links
David Kolb on experiential learning: Gibbs reflective cycle bs_Reflective_Cycle.pdf Brookfield Critical lenses sydney.edu.au/arts/teaching_learning/.../Brookfield_summary.pd Brookfield critical incident questionnaire: ent_Questionnaire_files/CIQ.pdf

24. Useful web links continued
Explanation of the one minute paper:
Explanation of the muddiest point: s/muddiest-point.html also:
Why colleges succeed: collegessucceed

25. Reflection and Quality assurance
“Brilliant teachers and trainers make time for reflective practice.” (LSIS, 2010, p10)
‘...teaching is a complex activity, often contingent, and in which it is difficult to be prescriptive’ (Avis et al (2010: p188)
Key elements of overarching standards
AS 4: ‘reflection and evaluation of their own practice and their continuing professional development as teachers’
Rationale for reflective practice; can we challenge any of this?
Reflection and Quality assurance

26. Criticisms of reflective practice
According to Avis et al (2010: p196-7), the following criticisms have been levelled against reflective practice:
Doubts about the effectiveness of reflective practice (Cornford, 2002; Parker 1997; Husu et al 2008) due to lack of empirical evidence (ie data)
The context in which reflective practice takes place is often overlooked in the research/academic literature (Suter 2007). In less supportive environments, teachers may fear exposure if voice concerns about own practice.
Wider socio-economic context may not support reflection
Danger of reflection becoming routinized and superficial, especially due to time constraints; ‘strategic compliance’ (Shain & Gleeson 1999)
Criticisms of reflective practice

27. Gravells, A. & Simpson, S. (2008)Planning and Enabling Learning in the Lifelong Learning Sector Exeter: Learning Matters
Schon, D (1983) The Reflective Practitioner: How professionals think in action. London: Temple Smith
Gibbs, G. (1988) Learning by doing: a guide to teaching and learning methods. Oxford: Further Education Unit.
Brookfield, S. (1995) Becoming a Critically Reflective Teacher. San Francisco: Jossey Bass
Sutter, J (2009) ‘Planning and Assessment: Reflection, evaluation and the learning cycle’ In Teaching Adult ESOL Berkshire: OUP pp
References

28. Hannigan, B. (2001)A discussion of the strengths and weaknesses of reflection in nursing practice and education. Journal of Clinical Nursing 10:
Hughes, N. & Schwab, I. (2010) Teaching Adult Literacy: Principles and Practice Berkshire: Open University Press Chapters 8 & 11
Schellekens, P (2007) The Oxford ESOL Handbook Oxford: OUP Chapters 6 & 8
Paton, A & Wilkins, M (2009) Teaching Adult ESOL Berkshire: Open University Press Chapter 9
Reading

29. Evaluation: some concepts and principles
SESSION 12 • HANDOUTS
Evaluation: some concepts and principles
DIK/DHK7230 • LEEDS THOMAS DANBY • PGCE/Cert Ed • SKRBIC

30. Evaluation: Concepts and Principles
SESSION 12 • HANDOUTS
Evaluation: Concepts and Principles
Where do you stand?
Trainees discuss Evaluation: concepts and Principles statements in pairs and feedback key ‘findings’
DIK/DHK7230 • LEEDS THOMAS DANBY • PGCE/Cert Ed • SKRBIC

31. SESSION 12 • HANDOUTS
What does C.E. Beeby’s definition imply about the process of evaluation?
the systematic collection and interpretation of evidence, leading, as part of the process, to a judgement of value with a view to action Beeby, C.E. (1977) “The Meaning of Evaluation” in Current Issues in Education, No.4. Willington: Department of Education.
Trainees discuss this in small groups and write ideas on handout – then discuss as whole class
DIK/DHK7230 • LEEDS THOMAS DANBY • PGCE/Cert Ed • SKRBIC

32. provides information for improvement provides information on outcomes
Scriven, M. (1967). “The methodology of evaluation”. In R.W. Tyler, R.M. Gagne and M. Scriven (Eds.), Perspectives of Curriculum Evaluation, pp.39-83, Chicago, Il: Rand McNally.
Formative
Summative
ongoing
terminal
provides information for improvement
provides information on outcomes
prompt
longer term judgement
enabling

33. "When a cook tastes the soup, it is formative evaluation; when the dinner guest tastes the soup, it is summative evaluation."
Harvey, J. (Ed.) (1998) Evaluation Cookbook. Edinburgh: Learning Technology Dissemination Initiative of the Institute for Computer Based Learning. Heriot-Watt University, Edinburgh, Scotland.

34. SESSION 12 • HANDOUTS
Task Evaluation can take many forms according to who it is done by. How is evaluation different in the following instances and what distinguishes internal from external forms of evaluation?
Group A
If the evaluator is the organiser and/or those to whom they are accountable.
Group B
If the evaluator is the teacher.
Group C
If the person undertaking the evaluation is the learner.
DIK/DHK7230 • LEEDS THOMAS DANBY • PGCE/Cert Ed • SKRBIC

35. External
Internal
infrequent
frequent
organiser
teacher
funding
informs reflection
business objectives
informs CPD

36. SESSION 12 • HANDOUTS
Banking and dialogical models of evaluation Rowlands, J. (1991) How do we know it is working? The evaluation of social development projects.
DIK/DHK7230 • LEEDS THOMAS DANBY • PGCE/Cert Ed • SKRBIC 36

37. Activity: evaluation practices
SESSION 12 • HANDOUTS
Part 1: Complete the matching activity.
Part 2: Which evaluation practices do you use or have used? How effective were they?
Activity: evaluation practices
Trainees are given methods of evaluation and descriptions of these methods to match. Give HO to check answers and then discuss.
DIK/DHK7230 • LEEDS THOMAS DANBY • PGCE/Cert Ed • SKRBIC 37

38. Text reconstruction activity
SESSION 12 • HANDOUTS
Text reconstruction activity
With a partner, reconstruct Kirkpatrick’s four-part model.
DIK/DHK7230 • LEEDS THOMAS DANBY • PGCE/Cert Ed • SKRBIC 38

39. Kirkpatrick’s Learning Evaluation Model
SESSION 12 • HANDOUTS
Level 4 Results
Level 3 Behaviour
Level 2 Learning
Level 1 Reaction
Kirkpatrick’s Learning Evaluation Model
Level 1: Evaluate reaction. Level 2: Evaluate learning. Level 3: Evaluate behaviour. Level 4: Evaluate results.
Kirkpatrick, Donald L. (1994),
Evaluating Training Programs: the Four Levels San Francisco: Berrett-Koehler Publishers.
DIK/DHK7230 • LEEDS THOMAS DANBY • PGCE/Cert Ed • SKRBIC

40. SESSION 12 • HANDOUTS
Map the evaluation practices that you pinpointed from the earlier activity to Kirkpatrick’s model: do particular levels dominate?
Activity
Look at HO from matching ex and relate to hierarchy
DIK/DHK7230 • LEEDS THOMAS DANBY • PGCE/Cert Ed • SKRBIC 40

41. Thinking point What limits educational evaluation?
SESSION 12 • HANDOUTS
Thinking point
What limits educational evaluation?
DIK/DHK7230 • LEEDS THOMAS DANBY • PGCE/Cert Ed • SKRBIC

42. Limitation 1: non-generalisability
Limitation 2: currency
Limitation 3: impact

43. BIS Nov 2010 The Quality Assurance System for post-16 education and training provision Overarching Framework

45. Beeby, C.E. (1977) “The Meaning of Evaluation” in Current Issues in Education, No.4. Willington: Department of Education.
BIS Nov 2010 The Quality Assurance System for post-16 education and training provision Overarching Framework
Harvey, J. (Ed.) (1998) Evaluation Cookbook. Edinburgh: Learning Technology Dissemination Initiative of the Institute for Computer Based Learning. Heriot-Watt University, Edinburgh, Scotland.
Kirkpatrick, Donald L. (1994), Evaluating Training Programs: the Four Levels San Francisco: Berrett-Koehler Publishers.
References

46. LLUK New Overarching Professional Standards for teachers, tutors and trainers in the lifelong learning sector
Rowlands, J. (1991) How do we know it is working? The evaluation of social development projects.
Scriven, M. (1967). “The methodology of evaluation”. In R.W. Tyler, R.M. Gagne and M. Scriven (Eds.), Perspectives of Curriculum Evaluation, pp.39-83, Chicago, Il: Rand McNally.
References