1. Assessment 2.0 Assessment in the age of Web 2.0 Bobby Elliott
“Modernising assessment” and no-one was interested… adding “2.0” made it attractive!
But the presentation is about modernising assessment.
Based on a paper I wrote called “Assessment 2.0”.
This is not the SQA policy!

2. Summary of presentation
Traditional assessment has served us well
But it’s time for change
We’re going though a cultural revolution
Education has resisted change
The contemporary classroom is detached from reality
We need to modernise education
Including assessment
E-assessment systems are not the answer
We should use the tools that are natural to today’s learners
Takes around minutes... depending on how much I remember to say!

3. Evolution of assessment
Traditional assessment (Assessment 1.0)
Computer-based assessment (Assessment 1.5)
Tool-assisted assessment (Assessment 2.0)
Three main stages in the evolution of assessment.
I am going to argue that we are at a transition between stages 2-3.

4. Assessment 1.0 Assessment from 618AD to today Characteristics
Paper-based
Classroom based
Formalised
Synchronised
Controlled
Industrialised
Enjoys public and political confidence
Changed little since early 20th Century
It has served us well.
Egalitarian roots: Chinese Civil Service promotion based on merit (knowledge) rather than family ties.
Churchill story (Eton entrance exam... blotting answer book... “I am a Churchill”).
Remains one of the few times when a working class child is on a level playing field with the other classes (the A Level is the same for every pupil).

5. Spot the difference
Picture on left taken in One on left in 2007.
The school master on the left would not be too uncomfortable being on the right...
Unlike doctors or most other professions.

6. Assessment 1.5 Computer-based assessment Types Embedded in most VLEs
E-testing
E-portfolios
Embedded in most VLEs
Stand-alone systems
Familiar to students and teachers
Computer assisted assessment became popular about 10 years ago.
VLE embedded is the quiz feature in Moodle... becoming more powerful (more question types).
VLE/CAA systems are fairly comfortable for teachers e.g. “virtual staff room”, familiar question types (MCQs).

7. Problems with 1.0… Expensive to run Inflexible
Doesn’t scale well
Inflexible
Arranged around diets
One size fits all (not personalised)
Not delivering contemporary skills
Collaboration, problem solving, flexibility
Drives teaching and learning
“Teaching to the test”
Memorisation not understanding
Employers complain that education system is still not producing the right sort of people (collaborative, etc.).
Teachers complain that it dominates their teaching due to targets and league tables.
My story about marking Higher Grade Computing in early 1990’s (seemed a deep paper – but pupils rote learned answers).

8. “Problem solving is really done through memorisation”
So most exam papers are actually answered by rote learning.
And most setting teams know this and refuse to change the wording of questions (“But that would confuse them – they won’t have seen a question like that in a past paper”).
There was a time when memory was important – when information was scarce.
But not now. “Information at your fingertips” (Bill Gates).
“Memorisation is valueless when students are one click away from Google and Wikipedia”

9. The irony is lost on the writer of the article.
And proof that this exam is about memorisation!
Is there any value in an exam that can be answered by Wikipedia?

10. “Five minute university”.
Apologies for bad Spanish accent!
Apologies if you have seen it before. I will not show it all (couple of minutes).

11. And problems with 1.5… Imitates traditional assessment
“Reproduces the paper experience”
“Completely locks-down the computer”
Limited question types
Crude simulations
E-portfolios: little more than online storage?
Simply automates Assessment 1.0?
Still assessing memorisation
Not really modernising assessment
Constrain innovation in assessment?
E-assessment simply automates traditional assessment.
It does not really change assessment in any fundamental way.
In fact, it can constrain innovation by only offering traditional and e-assessment.
By doing so, it still ignores the world that you people actually inhabit (Web 2.0 as we will see).
And it still appears unnatural and a hurdle to be jumped to learners...

12. Student perceptions Artificial and contrived
Something that is done to them
Doesn’t measure anything important
Hurdle to be jumped
Not part of their learning
Sole purpose of their learning
Don’t under-estimate the number of students who think that the only purpose of education is to pass exams.

13. Cultural revolution OK – so that the past – good and bad.
We’re currently going through a cultural revolution.
Not sure when it started ’s? 1980’s?
But the world is changing...
Go through images from top left to bottom right... “different morals, rapidly spreading technology (even to Third World), reduced respect for authority”.
Botswana story... “no shoes on their feet but ipods and laptops”.

14. Web 2.0 User-generated content Architecture of participation
Network effects
Openness
Data on an epic scale
Power of the crowd
Part of that “cultural revolution” is technological change.
One of the main technological changes going on right now is “Web 2.0” (“oh” not “zero”).
An excellent JISC paper defined six characteristics of Web 2.0 (Anderson).

16. But this is an easier way to describe Web 2.0...
Flickr is simple to use, easy to contribute, massive amount of data, and free.
And Web 2.0 services omit final vowel!

17. New types of learner? “Millennials” “Google Generation” “Generation X”
Lots of commentators have commented on new types of learner.
e.g. “Google generation” is anyone born after 1984 (now 24).
Marc Prensky coined the most famous description... “digital native”...
“Net Geners”
“Digital natives”

18. Digital natives Use books Passive learning Contrived tasks
IMMIGRANT
NATIVE
Use books
Passive learning
Contrived tasks
Process oriented
Memorise
Library
Compete
Use Web
Active learning
Authentic tasks
Goal oriented
Search
Google
Collaborate
He contrasted the “digital native” (young learners) with “digital immigrants” (you and me).

19. Kids don’t use libraries.
“Pupils borrow on average one library book a term”... and search Google 20 times a day.

20. Old types of rules… New learners... old rules.
Mr Chumley-Warner said last bit!

21. This is the environment that young people inhabit.
If you don’t use these services, you’re kids or grandchildren will.

22. Hidden curriculum Assignment
Google
Wiki-pedia
Skype
Assignment
The way young people do assessments is not the way we imagine.
They don’t do the literature reviews that we give them and use libraries.
They use Wikipedia as a starting point, the Google some, then ask their friends... and produce an essay with a couple of references to your reading list!
This is the “hidden curriculum” – the real resources that students use.

23. Assessment 2.0 Authentic Natural Personalised Negotiated Problem-based
Deep
Collaborative
Peer and self-assessed
Tool supported
I am proposing an update to assessment.
Not a technological update – an update to the characteristics of a “modern assessment”.
A modern assessment should exhibit some or all of these characteristics.
Tool support is not *permitted* - it’s encouraged!

24. Evidence Naturally occurring Digital Multimedia Distributed
Student
Facebook
Skype
Google mail
MSN
Wikipedia
del.icio.us
Naturally occurring
Digital
Multimedia
Distributed
A modern assessment will produce digital evidence in a range of formats.
Such as , chat logs, Wikipedia pages (written by the student), etc.
It will embrace accreditation of prior learning by accepting their previous work (such as an old dialogue or video they made a few years ago).

25. Assessment 1.0 v Assessment 2.0
Given
Done alone
Descriptive
Text
Closed book
Done in class
Teacher assessed
Negotiated
Done collaboratively
Researched/Deep
Text/audio/video
Open web
Done anywhere
Self- and peer-assessed
Here is a summary of the characteristics of “traditional” and “modern” assessment.
They are almost opposites.

26. Assessment 1.0 v Assessment 2.0
Write an essay describing the rise of Fascism in Germany during the period
You may not confer nor refer to notes or other reference material.
Working with other students, choose an aspect of the rise of Fascism in Germany during the 1930’s and research this.
Create a team blog to record your findings.
Here is an example of a traditional assessment task and a modern assessment task.
Note that the old one involves an essay that must be done along, in class.
The new approach has an element of choice, group working and collaboration. The output is a group blog.

27. Assessment 1.0 v Assessment 2.0
The essay
The blog
In February 1933 the Reichstag was burnt down. In March the Nazis won 44% of the popular vote making it the largest party in Germany. The Nazis were bad. The Allies were good.
Here is the sort of answers you might expect.
It’s been argued that essays are really “academic papers written by children”. That they encourage a simplistic approach to complex subjects. War is complex. This period was complex e.g. US industrialists complaining about peace to FDR because they had retooled their factories.
The blog would have a theme chosen by the students e.g. the role of US corporations in the rise of Nazism.
Modern assessment might facilitate a more research-based approach to assessment. More factual (e.g. source material) and less simplistic.
Every student in each group would contribute to the blog (each could research a specific aspect of US support for the Nazis e.g. IBM’s support or Ford Motor Company or US media...)

28. This front page might feature in the students’ blog.
The front page of Time Magazine declaring “Man of the Year for 1939”. Guess who?
(Let’s hope it was mid-1939 since we WWII started in September of that year)

29. Web 2.0 services
Web service
Example
Cycle
Use(s)
Personal portal
Netvibes
Evidence organisation
Combining Web services on single page
Google Mail
Evidence storage
Storing evidence and searching archive for evidence
Blog
Wordpress
Recording activities; e-portfolio; log-book/diary
RSS
Bloglines
Evidence discovery
Subscribing to evidence sources
Social bookmarking
Del.icio.us
Evidence capture
Capturing URLs of potential evidence sources
Instant messaging
MSN
Discussion; group work; collaboration
VOIP
Skype
Capturing audio evidence; candidate authentication
Wiki
Wikispaces
Evidence creation
Collaborative writing; projects; research findings; group work
Search engine
Live Search
Locating evidence
Online storage
Box.net
Saving and storing evidence
Video upload
YouTube
Creating and storing video evidence
Social network
Facebook
Collaborating and publishing evidence
These are the tools that young people use for socialising and learning.
We should use them for assessment.
We don’t need to worry about obsolescence since if Facebook disappears another service will take over from it.
Assessment 2.0 is not dependent on specific Web 2.0 services. It is not a tool-set.

30. The case for abandoning your VLE/CAA…
“Because you’re pouring money into a black hole that students don’t like, which is unnatural to them, which can’t possibly keep up with developments on the Web, and which is little more than a comfort blanket to teachers who can’t, or won’t, embrace the 21st Century.”
If this sounds damning, there are advantages…

31. The case for retaining your VLE/CAA…
It’s an important evolutionary step
Not every student is a digital native
Not every teacher can use Web 2.0
“I can’t get my staff to use the quiz in Moodle so what chance is there that they’ll embrace Web 2.0?”
It solves immediate problems
reduces cost of assessment
supports assessment on demand (life long learning)

32. Challenges posed by Assessment 2.0
Plagiarism
Authentication
Authentic assessment
Up-skilling assessors
Rubrics for collaboration
How to assess a group blog?
Peer and self-assessment
But “Assessment 2.0” is not a panacea. The are problems with it too.
e.g. both awarding bodies and universities support peer assessment in theory – but not in practice (rarely used and when it is it is for a tiny number of marks). Yet research shows that it is reliable.
A problem not listed is that students might not want to mix their social lives with the educational lives e.g. using Facebook or their for assessment.

33. The future Education Technology
Education as differentiator in global economy
Growth of life-long learning
Growth of e-learning
especially mobile learning
Personalised learning/assessment
Recognition of informal learning
Technology
Web 3.0
Ubiquitous computing

34. Summary Traditional assessment is past its sell-by date
E-assessment imitates traditional assessment
Ubiquitous computing will digitise everything
Education is becoming detached from reality
We should embrace ICT and the Internet
We should use the same tools that students use
Assessment 2.0 is half-baked
But we need to modernise assessment
…urgently