Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson Lecturers’ use of computer-aided assessment How history and contradictions help shape lecturers’

Slides:

Advertisements

Similar presentations

ProfessionalValues and Practice Look for differentiation. Monitor lesson plans (planning links). IEPs used Short evaluations should be encouraged (and.

Advertisements

Demanding Questions and Difficult Answers Alan Maddocks Carol Newbold Loughborough University.

QAA Enhancement Themes Conference Heriot Watt University Wednesday 5 th March 2008 Poster Presentation by Mhairi Freeman (lecturer), Sally Michie, Stephanie.

School of Medicine FACULTY OF MEDICINE AND HEALTH Does interprofessional education and working have any impact on perceptions of professional identity.

Religion and Culture in the Classroom and the Curriculum Developing an Inclusive Curriculum Friday 7 September 2007 Dr Emma Tomalin, Subject Centre for.

Innovation in Assessment? Why? Poor student feedback regarding feedback timeliness and usefulness Staff workloads Student lack of awareness as to what.

HELM Computer Aided Assessment Presented by David Pidcock Project Manager.

QAA-HEA Education for Sustainable Development Guidance Document Consultation 5 November 2013, Birmingham Professor James Longhurst Assistant Vice Chancellor.

Making Assessment Count (e-Reflect) Mark Clements and Gunter Saunders University of Westminster Mark Kerrigan University of Greenwich.

OECD/INFE High-level Principles for the evaluation of financial education programmes Adele Atkinson, PhD OECD With the support of the Russian/World Bank/OECD.

Module 4 Social Determinants of Financial Reporting

The HELM Project: Helping Engineers Learn Mathematics

Integrating e-learning into your practice Stuart McGugan Educational Developer Centre for Lifelong Learning

Presenter: Han, Yi-Ti Adviser: Chen, Ming-Puu Date: Jan 19, 2009 Sitthiworachart, J. & Joy, M.(2008). Computer support of effective peer assessment in.

Program Evaluation. Lecture Overview  Program evaluation and program development  Logic of program evaluation (Program theory)  Four-Step Model  Comprehensive.

Responding to the Assessment Challenges of Large Classes.

Bridging the Gap: Teaching Across Abilities Mandy Atkinson Business School University of Greenwich.

“Learning to be lawyerly”

Exploration of Students & Mentors Experiences of Grading Student Competence in Practice. Janet ScammellVanessa Heaslip Senior Academic.

IVETTE:Implementation of virtual environments in training and in education Targeted Socio-Economic Research Programme Mario Barajas University of Barcelona.

Miguel Martins, Senior Lecturer – Marketing Department eLearning Celebration 22 July 2007 Improving performance on a Marketing module through the use of.

Learning to teach Secondary science: An Activity Theory analysis of issues concerning the use of constructivist approaches Tanvir Ahmed The Open University.

Measuring Learning Outcomes Evaluation

Standards and Guidelines for Quality Assurance in the European

Continuous Assessment: Challenges and Opportunities for In-service Training Nizar Ibrahim 27/3/2011.

Comparing models of first year mathematics transition and support Findings from the First Year in Maths (FYiMaths) project.

Critical Partnerships: Using Peer Support to Develop Skills in Writing at Masters Level Sue Forsythe Maarten Tas School of Education

The Basics of Budgeting ELPA Fiscal Management  Decision maker  Follower of institutional policy 2.Listening Post  First line of defense for.

Introducing small-group workshops as formative assessment in large first year psychology modules Suzanne Guerin School of Psychology, University College.

Using formative assessment. Aims of the session This session is intended to help us to consider: the reasons for assessment; the differences between formative.

Assessment Careers: Enhancing Learning Pathways through Assessment Dr Gwyneth Hughes Dr Holly Smith Tim Neumann.

Evaluation of Respondus assessment tool as a means of delivering summative assessments A KU SADRAS Project Student Academic Development Research Associate.

OECD/INFE Tools for evaluating financial education programmes Adele Atkinson, PhD Policy Analyst OECD With the support of the Russian/World Bank/OECD Trust.

The University for business and the professions Neal Sumner Instructional Designer E Learning Unit City University WebCT Regional Users Group meeting

Re-Engineering Assessment Practices in Scottish Higher Education Dr David Nicol, Project Director Ms Catherine Owen, Project Manager Centre.

Universal Design in Third Level Design Teaching in Ireland Marie Callanan, Antoinette Fennell, Anthony Owens, Mark R. Dyer, James Hubbard, Ger Craddock.

E-Learning in the Disciplines| slide 1 e-Learning in the Disciplines John Cook Centre Manager Reusable Learning Objects CETL Helen Beetham Research Consultant.

Designing a Work Integrated Assessment Collaborate Project Bringing together staff, students and employers to create employability focused assessments.

Feb using assessment to motivate learning. QAA Assessment Driven Learning Jean Cook

Curriculum development and curriculum assessment TEMPUS: Second consortium meeting, Koblanz Landau, Germany, March 2013 Dr. Roxana Reichman Working team.

CPLA Video Case Studies Making Media Nursing. Making Media - Background First year, semester long 20 credit core module Involved 200 students with 4 staff.

Toward improving the academic performance in National Diploma Mechanical Engineering Thermodynamics 2 students Louw, C Mammen, K.J.

1 Ideas of Problem-based Learning As a learner-centred process, problem- based learning meets the learners' interests and as such gives room for developing.

1 of 27 How to invest in Information for Development An Introduction Introduction This question is the focus of our examination of the information management.

20081 E-learning Lecture-10: EVALUATING THE IMPACTS OF E-LEARNING week 12- Semester-4/ 2009 Dr. Anwar Mousa University of Palestine Faculty of Information.

USING CHAT AS A FRAMEWORK FOR USING FACEBOOK GROUPS: A CASE STUDY OF WSU WRITING CENTRE Khanyisile Ngodwana Walter Sisulu University Writing Centre.

Developing online activities for postgraduate students in computing Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT)

OPEN AND DISTANCE LEARNING TUTORING SKILLS Paper Presented By DR (MRS.) ADAEZE A 5-DAY WORKSHOP ON OPEN DISTANCE & e-LEARNING Organized By Centre.

Tuning of Higher Education Programmes on the Basis of Profiles, Competences and Learning Outcomes WHYHOWWHAT.

Creating an Equality Duty Toolkit for the VCS Equality and Diversity Forum and Citizens Advice October

Strategies for engaging students in computer based assessment – Stage 1, Taking stock Joanne Smailes Northumbria University.

Module 4 Challenges in measuring corruption What challenges are involved in measuring corruption? Take 3 minutes to write your own list.

The Maths, Stats & OR Network Current Perspective Michael Grove Network Manager April 2005.

Towards the Implementation of an Undergraduate Package for Self-Assessment to compliment the PASS Initiative Melanie Giles, School of Psychology Amanda.

Exploration of the Academic Experience of International Students Studying Project Management *Dr Reda M Lebcir, Hany Wells and Angela Bond The Business.

Alessio Peluso 1 Critical evaluation of the module ‘Introduction to Engineering Thermo Fluid Dynamics’ First Steps in Learning and Teaching in Higher Education.

Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson Lecturers’ beliefs and practices on the use of computer-aided assessment to enhance learning.

Communities of Practice Stephen Merry & Paul Orsmond Staffordshire University Faculty of Sciences.

Researching Technology in South Dakota Classrooms Dr. Debra Schwietert TIE Presentation April 2010 Research Findings.

Exploring the experiences of arts practitioners engaging with Educational Development programmes Dr. Karen Treasure Plymouth University VC Teaching and.

The Further Mathematics network

IPSP Outcomes Reporting Framework What you need to know and what you need to do.

Seeking the views of the Mathematical Sciences Community Michael Grove 1 & Bill Cox 2 1 Maths, Stats & OR Network, School of Mathematics, University of.

Induction Videos for Students with Asperger’s Syndrome (and a resource pack to help staff in HE support students) Videos: Mark Hughes (University of Manchester)

School of Life Sciences What works? Evaluating two different approaches to supporting first year students Susan K Robbins BSc PGCE MPhil PhD FHEA Principal.

Engaging Students in Technical Modules: The Quest to Promote Student Identification of Problematic Knowledge. Dr William Lyons, School of Engineering,

A transactional distance perspective on the management of tutorial support in distance learning Myrina Coetzee Registrar: Academic, Enrolments and Administration.

JET Education Services: Innovations in Teacher Support and Curriculum Development Presentation to the Care and Support for Teaching and Learning Regional.

Parent-Teacher Partnerships for Student Success

Dr. Subhashinie Wijesundera,

Presentation transcript:

Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson Lecturers’ use of computer-aided assessment How history and contradictions help shape lecturers’ practice

Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson Introduction Background to CAA at the target institution Background to this project Activity theory as the theoretical framework Methodology Our findings at this stage Conclusions and future work

Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson Background to CAA at the target institution

Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson The HELM project Helping Engineers Learn Mathematics Started in 2002 by a consortium of universities Includes workbooks and a CAA system Question bank with thousands of maths questions Concurrently A separate bank of questions was developed for maths students Harrison et al. (2007), HELM (2006).

Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson Example CAA question

Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson Example CAA solution

Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson Ten years on How do lecturers and students use CAA now? What sort of student improvement do lecturers and students desire from CAA? Does CAA practice affect whether these outcomes are achieved?

Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson Background to this project

Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson Previous work Student focus groups How students use CAA The nature of the feedback given The definition of student improvement The role of the lecturer Broughton, Hernandez-Martinez and Robinson (2011, 2012a, 2012b).

Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson Activity theory as the theoretical framework

Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson Aspects of interest Students’ practices as influenced by their aims Students’ identities and its impact on the way they act Formative assessment with respect to desired results or “improvement” The role of community and the development of tools Lecturers’ practices and the influences on it Dealing with contradictions between practice and ideals Engeström (2000); Roth and Lee (2006).

Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson Lecturer activity SUBJECT Lecturer OBJECT Assessing students TOOLS CAA, coursework, exams COMMUNITY School of Mathematics Wider academic community DIVISION OF LABOUR e-Learning staff create and upload tests RULES University regulations Plagiarism rules Departmental codes

Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson … or perhaps… SUBJECT Lecturer OBJECT Helping students improve TOOLS CAA, coursework, tutorials, support centres COMMUNITY School of Mathematics Wider academic community DIVISION OF LABOUR e-Learning staff create and upload tests Support centre staff direct students to sources of help RULES University regulations Plagiarism rules Departmental codes

Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson … or even… SUBJECT Lecturer OBJECT Saving time (and money) TOOLS CAA COMMUNITY School of Mathematics Wider academic community DIVISION OF LABOUR e-Learning staff create and upload tests Postgraduate markers mark coursework RULES University regulations Plagiarism rules Departmental codes

Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson Lecturers’ CAA practice For this part of the project, we ask: 1. Why are lecturers using CAA? 2. How do lecturers implement CAA in their maths modules? 3. What are lecturers’ perceptions of issues arising? 4. How do lecturers deal with these issues?

Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson Methodology

Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson Getting answers to questions Approached 13 lecturers of mathematics-based modules to complete a questionnaire Received nine questionnaires Six took part in a follow-up interview

Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson Findings

Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson Why are lecturers using CAA? What are the aims of the activity? Save time by continuing inherited practice Support student learning by encouraging them to practise Assess students more frequently by offering immediate feedback Conform to department influence by following advice from colleagues

Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson Why are lecturers using CAA? What are the aims of the activity? Save time by continuing inherited practice Support student learning by encouraging them to practise Assess students more frequently by offering immediate feedback Conform to department influence by following advice from colleagues

Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson Why are lecturers using CAA? What are the aims of the activity? Save time by continuing inherited practice Support student learning by encouraging them to practise Assess students more frequently by offering immediate feedback Conform to department influence by following advice from colleagues

Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson What actions have lecturers made towards their aims? How do lecturers use CAA in their modules? CAA coursework test Paper-based coursework test No test InvigilatedNon-invigilatedInvigilated Access to practice test granted more than one week before a test Access to practice tests granted after the test 0211 Access to practice test granted up to one week before a test Access to practice tests granted after the test 0100 Access to practice tests not granted after the test 3100

Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson What issues have lecturers faced? What contradictions have lecturers experienced while using CAA? The questions are formulaic and not challenging enough. The questions do not test conceptual knowledge and understanding. The feedback is not beneficial to every student. Lecturers do not know how students approach the tests.

Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson What issues have lecturers faced? What contradictions have lecturers experienced while using CAA? The questions are formulaic and not challenging enough. The questions do not test conceptual knowledge and understanding. The feedback is not beneficial to every student. Lecturers do not know how students approach the tests.

Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson What issues have lecturers faced? What contradictions have lecturers experienced while using CAA? The questions are formulaic and not challenging enough. The questions do not test conceptual knowledge and understanding. The feedback is not beneficial to every student. Lecturers do not know how students approach the tests.

Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson What issues have lecturers faced? What contradictions have lecturers experienced while using CAA? The questions are formulaic and not challenging enough. The questions do not test conceptual knowledge and understanding. The feedback is not beneficial to every student. Lecturers do not know how students approach the tests.

Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson How have lecturers responded to these issues? How has the activity changed? Authoring new CAA questions that challenge conceptual knowledge Replacing the online test with a paper version Introducing other assessments and restructuring the assessment profile Reducing the weighting of CAA towards the overall module mark

Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson How have lecturers responded to these issues? How has the activity changed? Authoring new CAA questions that challenge conceptual knowledge Replacing the online test with a paper version Introducing other assessments and restructuring the assessment profile Reducing the weighting of CAA towards the overall module mark

Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson What changes do lecturers anticipate making? How might the activity change in future? If lecturers are happy with the current setup, we might not expect further change. If they are unhappy, they might tweak CAA practice further. They might also seek a different CAA system.

Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson Conclusions and future work

Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson In ten years of CAA practice Lecturers practices have become more diverse and continue to change. Lecturers inherit CAA practice and are keen to continue using CAA. They retain CAA as a tool for assessing students, but look to other assessment tools to address contradictions. There is a growing desire for a new system that addresses these contradictions.

Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson Implications The lecturers are likely to continue to use CAA for the foreseeable future. They will not rely heavily on CAA for summative assessment and grading. They do not believe that CAA is the most appropriate assessment tool for assessing conceptual knowledge. There are concerns that students do not use CAA in the manner expected or desired.

Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson Future work Student study Questionnaires and individual interviews with students that use CAA Aims of students Student activity Lecturer study Questionnaires and individual interviews with lecturers that do not use CAA Compare aims and activity to lecturers that use CAA

Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson Further information Short links References sbpr.es/rm12a This presentation sbpr.es/rm12b