1. Portfolio Use for Learning and Assessment Dr Val Klenowski

2. Portfolio Use for Learning and Assessment
Aims of the session are to:
Analyse the formative and summative purposes of portfolios
Analyse the assessment processes involved in formative and summative portfolio assessment
Consider key issues related to validity and reliability.

3. Portfolio Use for Learning and Assessment
Examine a process for using criteria to assess portfolios.
Discuss the guidelines for implementation of portfolios for formative and summative purposes.

4. Assessment
Assessment is the purposeful, systematic and ongoing collection of information as evidence for use in making judgments about student learning.
(Education Queensland 2001, Years 1-10 Curriculum Framework for Education Qld Schools, Department of Education, p.13)

5. Assessment Purposes Promote, assist and improve students’ learning
Inform teaching and learning
Provide data that can be communicated to a range of people about the progress and achievements of individual students or groups of students.
(Education Queensland 2001, Years 1-10 Curriculum Framework for Education Qld Schools, Department of Education, p.13)

6. Formative and Summative Purposes
Assessment of learning equates to summative assessment. Process of summing up or checking what has been learned at the end of a particular stage of learning.
Assessment for learning equates to formative assessment. Assessment that helps students learn.
(Weeden, Winter and Broadfoot, 2002)

7. Assessment of Learning
Adds procedures or tests to existing work
Involves only marking and feedback of grades or marks to students
(The Assessment Reform Group.

8. Assessment for Learning
Involves students in self-assessment
Provides feedback that leads to students’ recognition of their next steps and how to take them
Is underpinned by confidence that every student can improve
Involves both teacher and students in reviewing and reflecting on assessment data
(The Assessment Reform Group.

9. Assessment for Learning
Is embedded in a view of teaching and learning of which it is an essential part
Involves sharing learning goals with students
Aims to help students know and recognise the standards for which they are aiming
(See: Department of Education and Training, Schools of Isolated and Distance Education, Culture of the Dreaming, 2003)

10. What is a portfolio?
… a purposeful collection of student work that tells the story of the student’s efforts, progress or achievement in (a) given area(s). This collection must include student participation in selection of portfolio content; the guidelines for selection; the criteria for judging merit; and evidence of student self-reflection.
Arter and Spandel, 1992: 36

11. What is a portfolio?
“are sources of evidence for judgements of … achievement in a range of contexts, from classroom monitoring of student performance to high stakes summative assessment.
… contain ‘pieces of evidence.’ The more relevant the evidence, the more useful it is for inferring a student’s level of achievement in a learning area.”
Forster & Masters (1996:2)

12. What is a portfolio?
A portfolio is a collection of work that can include a diverse record of an individual’s achievements, such as results from authentic tasks, performance assessments, conventional tests or work samples.
A portfolio documents achievements over an extended period of time.

13. Why use portfolios?
Produces an accurate and holistic portrait of the student.
Involves students in decisions about the choice of inclusions and quality of work completed.
Allows students to exhibit difference e.g. multiple intelligences, cultural diversity

14. Why use portfolios?
Process oriented portfolios tell a story about the growth of the learner. They document the processes of learning and creating, including earlier drafts, reflections on the process and barriers to learning throughout the course/programme.

15. Why use portfolios?
Product oriented portfolios require a student to document and reflect on the quality and range of accomplishments.
Improve the quality of teaching by integrating assessment and teaching/learning.
Facilitates the accountability of teachers and schools. (Queensland Government, Education Queensland, 2003)

16. Purposes and Processes
What are the key assessment concepts with which you need to be familiar if you are to use portfolios for:
formative and
summative assessment purposes?

17. Summative Portfolios
Focus on learning outcomes and contain evidence that shows the range and extent of students’ skills. A summative portfolio demonstrates learning outcomes rather than the process of learning. If the intention is to assess a student’s skills or knowledge then the assessment is summative.

18. Formative Portfolios
The main role of a formative portfolio is to show the processes of learning in which a student has engaged. If the assessment is concerned with the learning process then assessment should be formative. (

19. Formative and Summative
When both summative and formative assessment are developed in tandem, the result is more effective assessment and more effective teaching and learning.
(The State of Queensland, Department of Education, 2002)

20. Research Evidence
Black and Wiliam’s review of research on formative assessment in 1998 revealed that much of it was concerned with issues such as: peer- and self-assessment, role of feedback in a pedagogy focused on learning.
This finding refocused ‘formative assessment studies away from systems, with its emphasis on the formative-summative interface, and relocated it on classroom processes’ (Black and Wiliam, 2003:628)

21. Research Evidence
Formative assessment raises standards and that current practices are under-developed. (Black and Wiliam, 1998)
Difficulties uncovered:
Teachers’ tests encourage rote and superficial learning
The questions and methods used are not discussed with other teachers in the same school and they are not critically reviewed in relation to what they actually assess
Primary teachers in particular emphasise quantity and presentation of work to the neglect of quality in relation to learning

22. Research Evidence Negative impact on learning from:
The emphasis on the giving of marks and grades rather than useful advice that emphasises the learning function.
The use of comparison that students perceive to be for competitive purposes rather than for personal improvement.
Negative impact of assessment feedback for students with low attainments is that they believe that they lack ‘ability’ and are de-motivated thinking that they are not able to learn.

23. Research Evidence
The managerial role of assessments such that teachers’ feedback was used for social and managerial purposes rather than focusing learning.
The push to collect marks for record purposes was given priority over the analysis of students’ work to identify their learning needs. (Black, Harrison, Lee, Marshall & Wiliam, 2003)

24. Self-Assessments
Students need to engage with the criteria prior to self-assessment and for formative purposes.
Use of assessment language in own self-assessments.
Not a grading but a reflection on the process.

25. Understanding criteria for formative and summative purposes
Consider the criteria and standards for assessment.
Criteria are the “the various characteristics or dimensions on which the quality of student performance is judged”
Standards are the ”levels of excellence or quality applying along a developmental scale for each criterion”. (Maxwell, 1993:293)

26. Criteria and Standards
Examine the assessment criteria for the summative assessment.
Identify the criteria.
Analyse the criteria.
Discuss with members of your group.
Reach a consensus of understanding.
Be prepared to report back to the larger group your findings.

27. Process Consider criteria and standards for assessment.
Analysis of the criteria individually.
Discussion with others to integrate understanding.
Application of understanding.
Discussion, debate, clarification, affirmation and integration.

28. Self-assessments Explicit Criteria
Minimum criteria: grasp of key concepts and issues, structure, coherence, logical argument, reflection on professional practice, wide range of reading; academic conventions, referencing, presentation

29. Self-assessments Implicit Criteria
Analysis, synthesis, evaluation, critical reflection, exploration of theory-practice links

30. Portfolio Assessment Processes
Generally, the individual identifies work from an accumulated collection to illustrate achievement and to demonstrate learning for a particular purpose, such as summative or formative assessment
Careful critical self-evaluation is an integral process and involves judging the quality of one’s performance and the learning strategies involved.
For examples see:

31. Portfolio Processes
The individual’s understanding of what constitutes quality in a particular context and the learning processes involved is facilitated by discussion and reflection with peers, teachers, lecturers or tutors during interview, substantive conversation, exhibition or presentation of learning.
The development of the portfolio involves documentation of achievements, self-evaluations, process aretfacts and analyses of learning experiences, strategies and dispositions. (Klenowski, 2002)

32. Processes
The identification of the purpose of the portfolio, together with the criteria and guidelines, can assist the student in selecting work for inclusion in the portfolio.
By involving students in the assessment process and by making clear to students that self-evaluation is important for demonstrating what has been learnt, and also how this has been achieved, an important message about what is valued in assessment is being communicated.

33. Processes
The developmental nature of learning is promoted by the integration of assessment with curriculum, teaching and learning.
The development of the a portfolio of work involves key processes such as self-evaluation, substantive conversation, reflective thinking and practice.

34. Processes
These learning processes and associated pedagogic practices, when used to develop a portfolio of work, foster metacognitve development which promotes knowing how, when, where and why one learns.

35. Using Portfolios for Learning & Assessment
What are the key issues, major tensions, and constraints?

36. Summative Purposes Administrative considerations
time efficient
manageable
inexpensive
adequate level of reliability
consistency of approach (guidelines)
consistency of grading
consistency of standards

37. Summative Purposes Administrative considerations
To ensure consistency of standards need to provide teachers with:
Exemplars
Commentary to explain the grade assigned
Grade descriptors
Grading instructions
(See: Culture of the dreaming e-learning)

38. Using a portfolio for summative purposes requires specification of:
standards
contents
by external agency, local authority, academic department, teacher to achieve formal assessment and monitoring.

39. Ensuring Reliability Documented, field tested scoring guides
Specified criteria
Annotated examples of all score points
Ample practice and feedback for raters
Multiple raters with agreement prior to scoring
(See

40. Ensuring Reliability Periodic reliability checks throughout
Retraining if necessary
Arrangements for the collection of suitable reliability data
Forster & Masters (1996:43)

41. Using Portfolios for Learning and Assessment
What have been some of the problems when you have used portfolios for learning and assessment purposes?

42. Problems and Pitfalls
Most attempts to design, develop and implement a portfolio for assessment system have encountered problems that have resulted from:
unhelpful policy decisions
conceptual confusions
practical problems
technical problems
psychological or social barriers.

43. Problems and Pitfalls
Major problems associated with new assessment forms such as the portfolio arise from:
a mismatch of purpose to paradigm
curriculum design
pedagogic practices.

44. Problems and Pitfalls
The prevailing understanding of the nature and purpose of assessment can militate against the success of portfolio use by distorting the associated teaching and learning processes.
Teachers need to engage in professional development that will support the integration of new assessment systems into the curriculum and their teaching practice, the policy context needs to be supportive of such change

45. Problems and Pitfalls
Students need specific teaching and support to develop the cognitive processes of critical reflection and self-evaluation that help them to develop a portfolio of work.
Technical problems encountered in the assessment of the portfolio include:
validity of inferences
generalisability
reliability and
resourcing

46. Problems and Pitfalls
In the assessment of the portfolio the threats to validity are:
Construct under-representation
ie omission of important given criteria when
assessing the portfolio of work
Construct irrelevant variance
ie use of irrelevant or idiosyncratic criteria not
intended or included in the assessment
guidelines

47. Using Portfolios for Learning & Assessment
What advice should be given about portfolios in guidelines for their use?

48. Implementing Portfolios for Assessment and Learning
Decide on the purpose of the portfolio
Determine the outcomes to be achieved and how the portfolio might be used to demonstrate them
Create the means by which teachers an students can collaborate in determining the contents of portfolios
Allow time for students to reflect and engage in self-assessment

49. Portfolio Building Process
Identify the areas of skills that the student is to develop
From these skill areas, develop specific learning outcomes for the student to achieve.
Identify performance indicators that establish that the student has achieved their learning outcomes and indicate the evidence that the student needs to collect.
Collect evidence that demonstrates the student has met the performance indicators.
Organise this evidence in a portfolio so assessors can easily understand how the evidence relates to each performance indicator. (

50. Guidelines for Using Portfolios
Include:
Suggestions for introduction and implementation of portfolio process
Framework for portfolio
Learning outcomes and suggestions of evidence suitable to address these outcomes

51. Guidelines for Using Portfolios
Advice regarding self-evaluation and reflective statements
Criteria to be used in the assessment of the portfolio
Grade descriptors
Exemplars that illustrate standards

52. References
Arter, J. A, & Spandel, V. (1992) ‘Using Portfolios of Student Work in Instruction and Assessment’, Educational Measurement: Issues and Practice, Spring, pp
Assessment Reform Group.
Black, P. and Wiliam, D., (2003) ‘In Praise of Educational Research: Formative Assessment,’ British Educational Research Journal, 29,5, pp
Black, P, Harrison, C. Lee, C. Marshall, B. & Wiliam, D. (2003) Assessment for Learning Putting it into practice, Maidenhead: Open University Press

53. References
Department of Education and Training, Western Australia, Schools of Isolated and Distance Education, Culture of the Dreaming, 2003
Education Queensland 2001, Years 1-10 Curriculum Framework for Education Qld Schools, Department of Education.
Forster, M. & Masters, G. (1996) Portfolios, Victoria: Australian Council for Educational Research.

54. References
Klenowski, V. (2002) Developing Portfolios for Learning and Assessment: Processes and Principles, London: RoutledgeFalmer.
Maxwell, G. (1993) ‘Criteria and Standards Based Assessment in Applied Statistical Mathematics’, in J.Izard and M.Stephens (eds) Reshaping Assessment Practices in the Mathematical Sciences Under Challenge, Proceedings of the First International Conference on Assessment in the Mathematical Sciences, Geelong, October, 1992, Victoria: Australian Council for Educational Research.
Weeden,P., Winter, J. and Broadfoot, P. (2002) Assessment (What’s in it for schools?), London: RoutledgeFalmer.