VLEs for fieldwork preparation, assessment and evaluation: the Dark Peak website, WebCT quizzes and surveys Julia McMorrow School of Geography University of Manchester GEES LTSN, Virtual Learning Environments, Birmingham, 18 Nov 2003

Structure _____________________________ 1.Dark Peak website Preparing for fieldwork 2.WebCT quiz Formative assessment 3.WebCT surveys Pre- and post-fieldwork course evaluation 4.Conclusion Advantages and disadvantages of VLEs for fieldwork preparation, formative assessment and course evaluation

1. Dark Peak website ______________________________________________________ Purpose: To prepare first year students for a one day physical Geography field course in the south Pennines: - Give students a feel for the physical landscape at various spatial scales - Introduce or reinforce knowledge of the topics and techniques to be encountered in the field - Provide a resource for reflective follow-up work University resource: - Graphics-rich resource for interdisciplinary learning and teaching Public resource: - Student recruitment - Showcasing work of Upland Environment Research Unit (UpERU)

The Dark Peak web site ________________________________________

The Virtual Tour Linear structure + map with hotlinks for direct access to the 7 stops

VT page

Click to enlarge and annotate image

Theme page

Using the website with WebCT ______________________________________________________ Pre-fieldwork lab class 1. Dark Peak website Virtual Tour (VT) and Theme Pages studied alongside… 2. WebCT Quiz for formative assessment of learning from the website 3. WebCT Survey to evaluate website before going into the field Post-fieldwork individual use WebCT Survey to evaluate the field course and retrospective views on website Reflective learning; re-take Quiz, use website for follow-up work

2. WebCT quiz ______________________________ Aims To encourage students to prepare for fieldwork by engaging with the Dark Peak website To allow students to test their understanding of topics covered in the website; formative assessment Logistics Compulsory: must pass to gain course credits No time limit or limit on the number of attempts Accessible within specified dates

WebCT quiz instructions

Fieldwork Quiz: question formats 1 Multiple choice, least correct statement Visual and text prompt to the relevant web page

Fieldwork Quiz: question formats 2 Multiple choice, correct combination Multiple choice, match letters to labels Answer log updated as answers are saved SENDA compliance issues

WebCT quiz: Providing online feedback Students receive score and feedback for each answer as soon as the quiz is submitted Links to relevant web page in General Feedback at end of each question Reflective: able to view earlier submissions

Automated marking: Submission and Report to track progress and compare results For each student: number of attempts, score, time taken Individual % marks and global mean for Quiz

Summary to compare results by question For each question Quartiles Mean score Standard deviation Allows discriminatory power of questions to be evaluated

Detail gives responses by student and by question Rows: individual student responses to each question Columns: response summary for each multiple choice question

3. WebCT Surveys ________________________________ Survey 1: Pre-fieldwork evaluation To evaluate the website prior to fieldwork Survey 2: Post-fieldwork evaluation To re-evaluate the website after fieldwork To evaluate the field course Format Likert scale and free text Visual prompts Not marked, but, like the quiz, both must be submitted online to gain the course credits

WebCT surveys: response format Multiple choice Likert scale Free text question: Post-fieldwork survey, What was the most useful feature of the web resource in preparing you for the field visit?

4. Conclusion: on the minus side Disadvantages of WebCT quizzes and surveys for assessment and evaluation ____________________________________________________ Time required to for tutor to learn WebCT and prepare content Registration issues Scores merged for duplicated surnames No analysis of free text responses Lower submission rates for Survey than for paper questionnaires collected on bus or in a lecture SENDA compliance; designing for accessibility

On the plus side: Advantages of WebCT quizzes and surveys for assessment and evaluation _____________________________________________________ For the tutor Graphics easily incorporated Ability to incorporate feedback Instant, automated marking of multiple choice questions Tracking submissions Evaluating questions: detailed breakdown of results by question For the student Distributed: access from any internet connection Self-paced, reflective learning: no time limit, access to earlier attempts

Acknowledgements Curriculum Innovation Fund, University of Manchester and UMIST for grant to develop Dark Peak website Cassandra Worman, co-author of Dark Peak website Colleagues at Manchester, especially Martin Evans, Kate Richardson, Chris Perkins and Joe McQuaker for contributions to website content Graham Bowden, School of Geography Cartography Unit for website maintenance Anita Gill, Roger Hewitt, Faculty of Arts, for WebCT support