Assessment 2.0 Assessment in the age of Web 2.0 Bobby Elliott bobby.elliott@sqa.org.uk “Modernising assessment” and no-one was interested… adding “2.0” made it attractive! But the presentation is about modernising assessment. Based on a paper I wrote called “Assessment 2.0”. This is not the SQA policy!

Summary of presentation Traditional assessment has served us well But it’s time for change We’re going though a cultural revolution Education has resisted change The contemporary classroom is detached from reality We need to modernise education Including assessment E-assessment systems are not the answer We should use the tools that are natural to today’s learners Takes around 30-45 minutes... depending on how much I remember to say!

Evolution of assessment Traditional assessment (Assessment 1.0) Computer-based assessment (Assessment 1.5) Tool-assisted assessment (Assessment 2.0) Three main stages in the evolution of assessment. I am going to argue that we are at a transition between stages 2-3.

Assessment 1.0 Assessment from 618AD to today Characteristics Paper-based Classroom based Formalised Synchronised Controlled Industrialised Enjoys public and political confidence Changed little since early 20th Century It has served us well. Egalitarian roots: Chinese Civil Service promotion based on merit (knowledge) rather than family ties. Churchill story (Eton entrance exam... blotting answer book... “I am a Churchill”). Remains one of the few times when a working class child is on a level playing field with the other classes (the A Level is the same for every pupil).

Spot the difference Picture on left taken in 1940. One on left in 2007. The school master on the left would not be too uncomfortable being on the right... Unlike doctors or most other professions.

Assessment 1.5 Computer-based assessment Types Embedded in most VLEs E-testing E-portfolios Embedded in most VLEs Stand-alone systems Familiar to students and teachers Computer assisted assessment became popular about 10 years ago. VLE embedded is the quiz feature in Moodle... becoming more powerful (more question types). VLE/CAA systems are fairly comfortable for teachers e.g. “virtual staff room”, familiar question types (MCQs).

Problems with 1.0… Expensive to run Inflexible Doesn’t scale well Inflexible Arranged around diets One size fits all (not personalised) Not delivering contemporary skills Collaboration, problem solving, flexibility Drives teaching and learning “Teaching to the test” Memorisation not understanding Employers complain that education system is still not producing the right sort of people (collaborative, etc.). Teachers complain that it dominates their teaching due to targets and league tables. My story about marking Higher Grade Computing in early 1990’s (seemed a deep paper – but pupils rote learned answers).

“Problem solving is really done through memorisation” So most exam papers are actually answered by rote learning. And most setting teams know this and refuse to change the wording of questions (“But that would confuse them – they won’t have seen a question like that in a past paper”). There was a time when memory was important – when information was scarce. But not now. “Information at your fingertips” (Bill Gates). “Memorisation is valueless when students are one click away from Google and Wikipedia”

The irony is lost on the writer of the article. And proof that this exam is about memorisation! Is there any value in an exam that can be answered by Wikipedia?

“Five minute university”. Apologies for bad Spanish accent! Apologies if you have seen it before. I will not show it all (couple of minutes).

And problems with 1.5… Imitates traditional assessment “Reproduces the paper experience” “Completely locks-down the computer” Limited question types Crude simulations E-portfolios: little more than online storage? Simply automates Assessment 1.0? Still assessing memorisation Not really modernising assessment Constrain innovation in assessment? E-assessment simply automates traditional assessment. It does not really change assessment in any fundamental way. In fact, it can constrain innovation by only offering traditional and e-assessment. By doing so, it still ignores the world that you people actually inhabit (Web 2.0 as we will see). And it still appears unnatural and a hurdle to be jumped to learners...

Student perceptions Artificial and contrived Something that is done to them Doesn’t measure anything important Hurdle to be jumped Not part of their learning Sole purpose of their learning Don’t under-estimate the number of students who think that the only purpose of education is to pass exams.

Cultural revolution OK – so that the past – good and bad. We’re currently going through a cultural revolution. Not sure when it started... 1960’s? 1980’s? But the world is changing... Go through images from top left to bottom right... “different morals, rapidly spreading technology (even to Third World), reduced respect for authority”. Botswana story... “no shoes on their feet but ipods and laptops”.

Web 2.0 User-generated content Architecture of participation Network effects Openness Data on an epic scale Power of the crowd Part of that “cultural revolution” is technological change. One of the main technological changes going on right now is “Web 2.0” (“oh” not “zero”). An excellent JISC paper defined six characteristics of Web 2.0 (Anderson).

But this is an easier way to describe Web 2.0... Flickr is simple to use, easy to contribute, massive amount of data, and free. And Web 2.0 services omit final vowel!

New types of learner? “Millennials” “Google Generation” “Generation X” Lots of commentators have commented on new types of learner. e.g. “Google generation” is anyone born after 1984 (now 24). Marc Prensky coined the most famous description... “digital native”... “Net Geners” “Digital natives”

Digital natives Use books Passive learning Contrived tasks IMMIGRANT NATIVE Use books Passive learning Contrived tasks Process oriented Memorise Library Compete Use Web Active learning Authentic tasks Goal oriented Search Google Collaborate He contrasted the “digital native” (young learners) with “digital immigrants” (you and me).

Kids don’t use libraries. “Pupils borrow on average one library book a term”... and search Google 20 times a day.

Old types of rules… New learners... old rules. Mr Chumley-Warner said last bit!

This is the environment that young people inhabit. If you don’t use these services, you’re kids or grandchildren will.

Hidden curriculum Assignment Google Wiki-pedia Skype Assignment The way young people do assessments is not the way we imagine. They don’t do the literature reviews that we give them and use libraries. They use Wikipedia as a starting point, the Google some, then ask their friends... and produce an essay with a couple of references to your reading list! This is the “hidden curriculum” – the real resources that students use.

Assessment 2.0 Authentic Natural Personalised Negotiated Problem-based Deep Collaborative Peer and self-assessed Tool supported I am proposing an update to assessment. Not a technological update – an update to the characteristics of a “modern assessment”. A modern assessment should exhibit some or all of these characteristics. Tool support is not *permitted* - it’s encouraged!

Evidence Naturally occurring Digital Multimedia Distributed Student Facebook Skype Google mail MSN Wikipedia del.icio.us Naturally occurring Digital Multimedia Distributed A modern assessment will produce digital evidence in a range of formats. Such as e-mail, chat logs, Wikipedia pages (written by the student), etc. It will embrace accreditation of prior learning by accepting their previous work (such as an old e-mail dialogue or video they made a few years ago).

Assessment 1.0 v Assessment 2.0 Given Done alone Descriptive Text Closed book Done in class Teacher assessed Negotiated Done collaboratively Researched/Deep Text/audio/video Open web Done anywhere Self- and peer-assessed Here is a summary of the characteristics of “traditional” and “modern” assessment. They are almost opposites.

Assessment 1.0 v Assessment 2.0 Write an essay describing the rise of Fascism in Germany during the period 1932-39. You may not confer nor refer to notes or other reference material. Working with other students, choose an aspect of the rise of Fascism in Germany during the 1930’s and research this. Create a team blog to record your findings. Here is an example of a traditional assessment task and a modern assessment task. Note that the old one involves an essay that must be done along, in class. The new approach has an element of choice, group working and collaboration. The output is a group blog.

Assessment 1.0 v Assessment 2.0 The essay The blog In February 1933 the Reichstag was burnt down. In March 1933 the Nazis won 44% of the popular vote making it the largest party in Germany. The Nazis were bad. The Allies were good. Here is the sort of answers you might expect. It’s been argued that essays are really “academic papers written by children”. That they encourage a simplistic approach to complex subjects. War is complex. This period was complex e.g. US industrialists complaining about peace to FDR because they had retooled their factories. The blog would have a theme chosen by the students e.g. the role of US corporations in the rise of Nazism. Modern assessment might facilitate a more research-based approach to assessment. More factual (e.g. source material) and less simplistic. Every student in each group would contribute to the blog (each could research a specific aspect of US support for the Nazis e.g. IBM’s support or Ford Motor Company or US media...)

This front page might feature in the students’ blog. The front page of Time Magazine declaring “Man of the Year for 1939”. Guess who? (Let’s hope it was mid-1939 since we WWII started in September of that year)

Web 2.0 services Web service Example Cycle Use(s) Personal portal Netvibes Evidence organisation Combining Web services on single page E-mail Google Mail Evidence storage Storing evidence and searching archive for evidence Blog Wordpress Recording activities; e-portfolio; log-book/diary RSS Bloglines Evidence discovery Subscribing to evidence sources Social bookmarking Del.icio.us Evidence capture Capturing URLs of potential evidence sources Instant messaging MSN Discussion; group work; collaboration VOIP Skype Capturing audio evidence; candidate authentication Wiki Wikispaces Evidence creation Collaborative writing; projects; research findings; group work Search engine Live Search Locating evidence Online storage Box.net Saving and storing evidence Video upload YouTube Creating and storing video evidence Social network Facebook Collaborating and publishing evidence These are the tools that young people use for socialising and learning. We should use them for assessment. We don’t need to worry about obsolescence since if Facebook disappears another service will take over from it. Assessment 2.0 is not dependent on specific Web 2.0 services. It is not a tool-set.

The case for abandoning your VLE/CAA… “Because you’re pouring money into a black hole that students don’t like, which is unnatural to them, which can’t possibly keep up with developments on the Web, and which is little more than a comfort blanket to teachers who can’t, or won’t, embrace the 21st Century.” If this sounds damning, there are advantages…

The case for retaining your VLE/CAA… It’s an important evolutionary step Not every student is a digital native Not every teacher can use Web 2.0 “I can’t get my staff to use the quiz in Moodle so what chance is there that they’ll embrace Web 2.0?” It solves immediate problems reduces cost of assessment supports assessment on demand (life long learning)

Challenges posed by Assessment 2.0 Plagiarism Authentication Authentic assessment Up-skilling assessors Rubrics for collaboration How to assess a group blog? Peer and self-assessment But “Assessment 2.0” is not a panacea. The are problems with it too. e.g. both awarding bodies and universities support peer assessment in theory – but not in practice (rarely used and when it is it is for a tiny number of marks). Yet research shows that it is reliable. A problem not listed is that students might not want to mix their social lives with the educational lives e.g. using Facebook or their e-mail for assessment.

The future Education Technology Education as differentiator in global economy Growth of life-long learning Growth of e-learning especially mobile learning Personalised learning/assessment Recognition of informal learning Technology Web 3.0 Ubiquitous computing

Summary Traditional assessment is past its sell-by date E-assessment imitates traditional assessment Ubiquitous computing will digitise everything Education is becoming detached from reality We should embrace ICT and the Internet We should use the same tools that students use Assessment 2.0 is half-baked But we need to modernise assessment …urgently