1. What can the key competencies contribute to assessment for learning?
Presentation to U Learn Conference: Auckland, October 2007
Rosemary Hipkins
New Zealand Council for Educational Research

2. Quick check of origins New Zealand Curriculum DeSeCo (OECD)
Functioning in socially heterogenous groups
Acting autonomously
Using tools interactively
New Zealand Curriculum
Relating to others
Participating and contributing
Managing self
Using language, symbols and text
Thinking T H I N K G

3. What’s special about KCs?
Why do OECD and curriculum developers think they are important? (Are they just a passing fad?)
What makes them different (e.g. from the “essential skills” of the current curriculum)?
In what ways might they change learning and teaching?
Do we need to rethink assessment, as well as learning?

4. What makes them different?
Holistic - integrate knowledge, skills, attitudes and values.
Demonstrated in authentic contexts and in interaction with others.
Demonstrated when we adapt the competency to use appropriately in a new setting.
Consistent with situated and socio-cultural views of learning.
Focus on dispositions

5. What’s involved in developing KCs?
Student identity and agency
Well informed decision making
Resourcefulness and a "can do" attitude
Recognising different points of view
Making connections
Balancing rights, roles, responsibilities
Questioning information, experiences, ideas
Using and creating knowledge
Interpreting representations in a range of contexts
Communicating in appropriate ways etc….
This is a list of phrases taken from the curriculum document where the KCs are described.
How can we avoid them just becoming meaningless labels?

6. What might KCs and assessment for learning have in common?
Assessment for learning has a focus on next learning steps
The learning intention/broad goal is clearly signalled
Evidence of successful learning is modelled
Learner is involved in judging their own learning where possible
Key competencies focus on building dispositions for lifelong learning
The development of greater autonomy/agency as a learner is an important goal
Student voice is valued and active participation in learning is a key competency in itself

7. But here’s a challenge…
Both key competencies and assessment for learning can contribute to making traditional models of teaching and learning more effective
Or, both can be transformative, bringing learning into a new alignment with the demands of the twenty-first century
So, how can we work to transform the curriculum and our practice?
What role might ICT play in this?

8. Traditional curriculum outcome
Basic literacy and numeracy
Enriched outcomes when a key competency focus is added
Multi-modal communication adds “new basics”
Ways to use various tools and representations is an explicit focus of learning
Multiple literacies are developed as learning makes broader and deeper connections to disciplines

9. Reading multi-modal text
What "message" about the greenhouse effect is the artist giving us?
Compare your explanation of this message with those sitting near you.
ARB item PE 9535
Original source: MAF website and now an ARB item

10. What can we learn from this assessment?
Which key competencies did you need to use to do this task?
How did you use them?
Would students find this easy to do?
How much practice do they get at using their competencies in these ways?
Are there aspects here that can be taught?

11. National trial information
What the trial showed: Of the 244 Year 10 students in the trial:
12% could not read images;
(Students misinterpreted the circles, the arrows and/or the colour.)
21% showed misconceptions in their content knowledge.
Misinterpreting the circles
A number of students thought the larger circles in each cartoon depicted the sun.
The artist is making it clear that the more pressure you put on the sun the less heat it makes.
It is showing how the sun gets heat from the greenhouse's effects. But the enhanced greenhouse effects will be too much hard work for the sun.
Misinterpreting the arrows
Some students thought the arrows represented gases rather than energy.
The message about the greenhouse effect is that we will all not be happy when the greenhouse effect is enhanced as it means that more gases will be entering and exiting the atmosphere so it really isn't a good idea.
Misinterpreting the use of colour
Some students interpreted the colour difference, intended by the artist to show a hotter atmosphere for the enhanced greenhouse effect, to be a representation of a visible pollution-related change in the atmosphere:
If we have a relatively clean environment the earth will not get too hot because the sun's rays will go out of earth's ozone, but if we have a dirty environment the earth will get hotter and hotter.
Could we adapt tasks like this to “unpack” images in a shared conversation (e.g. in a wiki space?)

12. The “something extra” in traditional assessment tasks
Task is set in an authentic context (the graphic was created to convey an important idea about global warming to users of MAF’s website)
The students’ knowledge and active participation are fore-grounded - they are invited to make a personal interpretation of the image
The teacher uses their subject expertise to interpret students’ responses and determine what else they might need to learn
There is also an explicit focus on “unpacking” the construction of the text

13. Being much more specific about conventions
Why do the arrows go that way?
What do they actually represent?
Who said they had to be like that?
Why are the colours only on one side?
What do they actually represent?
Who said they had to be like that?

15. What does the photograph show that the diagram doesn't?
Our second “Kick Starts” resource includes strategies for exploring multi-modal communication and systems thinking in the context of learning about the water cycle.
Google images is a great quick source of materials for students to find and use for this sort of task.

16. Systems thinking: Kick Starts resource two

17. Reinforcing important KCs messages
ULST is about much more than basic literacy – it challenges us to pay attention to both really big ideas and small details that we could take for granted
Thinking is indeed integrated into the other KCs (and they are integrated with both “content” and “context”)
Systems thinking is important!
Metacognition is really important – you have to be aware of the KCs you have used – and how you used them – to recognise the next situation where they might be useful
Knowing the “rules of the game” of how knowledge is constructed in different disciplines is important

18. New challenges for assessment
Traditional pencil and paper tests are not sufficient to provide the range of evidence that can demonstrate these additional outcomes.
What are the implications of making greater use of ICT for assessment purposes? (Formative? Summative?)

19. Traditional curriculum outcome
Gaining knowledge in a range of learning areas
Enriched outcomes when a key competency focus is added
Learning involves the use of knowledge to carry out meaningful tasks – there is a focus on creating and critiquing knowledge, not just “having” it in case it is needed at some future time
Key competencies focus on making links and hence the whole learning context .
Integrated and inquiry learning models are potential pedagogies

20. Traditional curriculum outcome
Developing a range of skills, again as appropriate to different learning areas
Enriched outcomes when a key competency focus is added
Skills are integrated with knowledge, attitudes and values in ways that direct attention to dispositions – the focus is on being ready, willing and able to use skills and knowledge in appropriate ways, at relevant and appropriate times.
The development of a disposition as a lifelong learner is a valued outcome.

21. What sort of knowledge is needed for ‘authentic’ participation?
Philosopher Bruno Latour distinguishes between ‘matters of fact’ and ‘matters of concern’
What else, in addition to systems thinking, could help students learn for an interconnected world?
What does it take to become an informed and active citizen?
fim_popup.php?id=39&title...

22. Content-context translations are tricky – you need to “know how” to be able to “do”
The debate
Some people say glaciers worldwide are getting smaller and that this is a sign that global warming really is happening. But some glaciers, such as Franz Josef in New Zealand, have got bigger in recent years. So other people say this proves global warming is not happening!
To make an informed decision about something it is important to gather enough evidence. The very first step is to work out what questions to ask.
Use each piece of information about glaciers in the table below to write a question you would need to ask about Franz Josef before you could make an informed decision about whether its recent changes are evidence either for or against global warming. (The first one is done for you)

23. Your turn to try
Fact
Question
i) There are many glaciers in the South Island and elsewhere in the world.
Is Franz Josef changing in the same way as other glaciers?
ii) Glaciers in steep, narrow valleys are less affected by small overall temperature changes than wider glaciers.
iii) Some glaciers have a layer of rubble on top. It makes them look dirty but also acts like a cooling layer of insulation that traps the cold.
iv) Changes to glaciers are related to weather that happened several years ago.
v) Glaciers have always gone through periods of retreat (shrinking) and advance (growing).

24. Types of questions students asked
Questions to find information or explanations
Why have some glaciers got rubble on top?
Do glaciers change in size relative to there [sic] distance from the sun?
Investigative questions
By how much does a rubble layer slow down melting?
Adversarial (debate style) questions
Does appearance not matter? Rubble can be the preserver of our glaciers.
Pattern-seeking questions
Are other glaciers exposed to same/similar weather conditions increasing as well?
We found out we cannot assume…
students will know how to ask questions that relate general concepts to a specific context.
Or that they will be willing to do this.
Source: a new ARB item. More of these sorts of insights are on the way

25. Adding a “values” question:
Why is it important to think about more than one piece of evidence before you make up your mind about something?
"Both sides of the story“: taking all relevant aspects into account and possibly changing your mind (41% of students):  
Because one piece of evidence could have been influenced by other factors.
"More is better“: a general sense that more pieces of evidence are a good thing without saying why (14% of students) :  
Because you need all the facts before you judge.
Use of terms related to scientific ways of investigating (e.g., fair test; getting proof, looking for outliers, greater accuracy, developing new theories, seeking alternative explanations etc (7% of students).
Opinion-based decisions: emphasis on persuading others to your point of view rather than deciding with reference to all the evidence to hand (17%).
More evidence means there is a more likely chance for both sides to win.
No response, or an answer that did not address question (21% of students).
Because if you wanted to move a glacier to a cooler place to keep it colder it might say in another fact that they can't survive when they are moved!!
ARB item PE9536.

26. What does all this mean for assessment?
Students need to be active participants in assessment – this may not feel comfortable to them at first
Looking at how reasonable students' responses are, how well they are justified and the strategies students used in arriving at the answers all become more important than how closely their answers conform to the expected responses. The focus is on how students use knowledge rather than on how much knowledge they have.
Students need opportunities to show how they are willing and able to use their learning in increasingly diverse and complex situations.
Could the context-concept interplay be used to add a more participatory dimension to web-based inquiry tasks?

27. A simple self-assessment strategy
Which of the following changes could happen if the greenhouse effect gets bigger? Only select "yes" or "no" if you are certain you are right.  If you are not sure, choose the "don't know" option, so the class can discuss this idea later.
New Zealand will be sunnier overall (45% agreed)   
New Zealand will have less rain overall   
More people will get skin cancer (77% said yes – a common misconception)  
The air will get dirtier (46 percent said yes – a common conflation of issues)   
Sea levels will rise around our coast   
Violent storms will happen more often   
Winters will get colder in New Zealand   
Our glaciers will shrink
Source: PE9537
‘Don’t know” is potentially ambiguous here.
Consider an “it depends” category to encourage systems thinking and further inquiry

28. More new challenges for transforming assessment
Many traditional assessments measure knowledge gains (and sometimes the ability to apply this), not the appropriate use of knowledge in meaningful tasks
Assessment often focuses on individual aspects of learning, even within learning areas. Tasks that assess integrated knowledge are less common
Dispositions can only be demonstrated in action – assessment is of the moment and needs to be set in a meaningful context
The overall shape of dispositions is “fuzzy” so it is difficult to predetermine in advance what could be observed and documented

29. Traditional curriculum outcome
Socialisation – fitting in, responding appropriately in different contexts and to relevant authorities (both knowledge and people), being a “good citizen”
Enriched outcomes when a key competency focus is added
Key competencies focus on ongoing development of identity as a “person who …”
Thinking and acting autonomously includes a focus on why it is appropriate to act in certain ways in diverse contexts, and on rights, roles and responsibilities
Group learning is valued and practised, and is seen as key for development as a lifelong learner
Student voice is valued – students have a more active role in making decisions about their learning

30. Student involvement in assessment as authentic participation
Students as:
informants on assessment contexts
collectors of evidence
assessment task designers
evaluators of evidence
Assessment as inquiry...
What does it mean to know?
Delandshere, 2002

31. Advantages of greater student involvement in assessment
Consistent with development of dispositions for lifelong learning.
Consistent with focus on metacognition and self regulated learning.
Consistent with intent of all five key competencies (potentially allows for integration).
Potentially makes a space for better assessment of collective work.
ICT opens up horizons for the new types of tasks needed!