mlearning Assessment From A Continuous Quality Improvement Perspective Gary Marrer | March 4rd 2010

Agenda Goal: Discussion of mLearning Assessment Share questions, practices, experiences My Definitions and Assumptions One Approach from the Research Other Research Let’s have some discussion…

Scope of Discussion Focus is on Course Learning Objectives and how assessment of mLearning can provide continuous quality improvement. Assumptions/My Background At my Campus, we use mobile assisted learning. I am an advocate of continuous quality improvement. I am technology faculty and an early adopter This presentation shares my experience on my campus. Please answer the two survey questions as I present…Thanks! Survey: What is your experience with mobile learning? (1) Not using yet (2) Using as part of traditional classroom delivered class (3) Using with on-line delivered class (4) Have classes that are delivered entirely on mobile device

An Assessment Definition (a teaching perspective) What does it mean to assess the effectiveness of a pedagogy or technology? “For many instructors, assessment just means grading, but it actually entails much more than this. When designed thoughtfully and implemented carefully, assessments can provide important evidence about what and how students are learning and whether course goals have been met.”1 1Strategies for Assessing Learning Effectiveness : by Constance A. Steinkuehler & Sharon J. Derry University of Wisconsin–Madison Copyright© NJIT & Constance Steinkuehler 2001 . http://www.alnresearch.org/HTML/AssessmentTutorial/ Survey: For mobile learning assessment we (1) do not do assessment (2) we use traditional classroom assessment techniques (3) we use on-line classroom assessment techniques (4) we have specially designed assessment instruments designed for mobile learning.

mLearning Pedagogy Mobile Devices Support: mLearning can support: Anytime anywhere learning Small portable computers Support multi-tasked lifestyles mLearning can support: Exploration Observation Social Learning Evidenced based learning Experiential learning But mLearning supports traditional assessment also: Testing Surveys Interviews

The Application of Mobile Learning Mobile Learning can be practiced in any of this environments to varying degrees.

Some research I like… Vavoula, G & M. Sharples. (2008). Challenges in Evaluating Mobile Learning. REQUIREMENTS FOR MOBILE LEARNING EVALUATION 2 Capture and analyse learning in context with consideration of learner privacy Assess the usability of the technology and how it affects the learning experience Look beyond measurable cognitive gains into changes in the learning process and practice Consider organisational issues in the adoption of mobile learning practice and its integration with existing practices and understand how this integration affects attributes of in/formality Span the lifecycle of the mobile learning innovation that is evaluated, from conception to full deployment and beyond 2 http://www.scit.wlv.ac.uk/brendan/mLearn2008.pdf (page 298) or http://www2.le.ac.uk/Members/gv18/downloads/publicationpreprints/conference-proceedings/VavoulaSharples-mlearn2008.pdf/at_download/file

A Three Level Framework (Vavuola and sharples) (systematic) Vavuola and Sharples – A three level framework for systematically evaluating mobile learning. “Micro level: the micro level examines the individual activities of the technology users and assesses usability and utility… Meso level: the meso level examines the learning experience as a whole, to identify learning breakthroughs and breakdowns; it also examines how well the learning experience integrates with other related learning … Macro level: the macro level examines the longer term impact of the new technology on established educational and learning practice…”3 ________________________________________________ 3 Vavoula, G & M. Sharples. (2008). Challenges in Evaluating Mobile Learning. Proceedings of the mLearn Conference Ironbridge Gorge, Shropshire, UK. http://www.scit.wlv.ac.uk/brendan/mLearn2008.pdf (page 298) or http://www2.le.ac.uk/Members/gv18/downloads/publicationpreprints/conference-proceedings/VavoulaSharples-mlearn2008.pdf/at_download/file

Research Sample - Empirical Pragmatic/Empirical – Comparison along similar variables / Data Driven Mobile learning in higher education: an empirical assessment of a new educational tool http://www.scribd.com/doc/4494956/MOBILE-LEARNING-IN-HIGHER-EDUCATION-AN-EMPIRICAL-ASSESSMENT-OF-A-NEW-EDUCATIONAL-TOOL MLearning: pedagogical, technical and organizational hypes and realities, Pul B. Muyinda, Department of Distance Education, Institute of Adult and Continuing Education, Makerere University, Kampala, Uganda, and Department of Information Systems, Faculty of Computing and IT, Makerere University, Kampala, Uganda http://www.emeraldinsight.com/Insight/viewPDF.jsp?Filename=html/Output/Published/EmeraldFullTextArticle/Pdf/1650240202.pdf

Research Sample – Institutional Commitment Mobile learning work at Abilene Christian University College with “focused” commitment to Mobile Learning Over 1000 iPhone/iTouch devices for Students, Faculty and Staff. Customized ACU content ACU's iPhone initiative: a year later by Brett Terpstra http://www.tuaw.com/2009/07/24/acus-iphone-initiative-a-year-later/

mLearning Resources …continued Excellent mobile learning articles Are You Ready for Mobile Learning? By Joseph Rene Corbeil and Maria Elena Valdes-Corbeil , http://www.educause.edu/EDUCAUSE+Quarterly/EDUCAUSEQuarterlyMagazineVolum/AreYouReadyforMobileLearning/157455 REPORT 11: FUTURELAB SERIES - Laura Naismith, Peter Lonsdale, Giasemi Vavoula, Mike Sharples University of Birmingham, http://www.futurelab.org.uk/resources/documents/lit_reviews/Mobile_Review.pdf Defining, Discussing and Evaluating Mobile Learning: the moving finger writes and having writ . . . . John Traxler University of Wolverhampton, UK http://www.irrodl.org/index.php/irrodl/article/view/346/875

mLearning Resources …continued Pedagogy: JISC TechDis http://www.techdis.ac.uk/index.php?p=9_5_32_4 The Pedagogical Perspectives of Mobile Learning, http://www.google.com/url?sa=t&source=web&ct=res&cd=1&ved=0CA4QFjAA&url=http%3A%2F%2Fwww2.le.ac.uk%2Fprojects%2Fimpala%2Fpresentations%2FBerlin%2FThe%2520Pedagogical%2520Perspectives%2520of%2520Mobile%2520Learning&ei=NBZ7S7ONAY6otgPHuKS8Cw&usg=AFQjCNH0-w_oWn-W5JjGSlB2EKlW7hQ7hQ&sig2=fzpMyJN0XtLbc-1er5m3KQ Chapter 6 - Pedagogical forms for mobile learning: framing research questions Diana Laurillard, London Knowledge Lab, Institute of Education, London http://eprints.ioe.ac.uk/627/1/Mobile_C6_Laurillard.pdf

Let’s talk… Is mlearning assessment just another example of technology being assessed in the learning process? or Is it different? Is mobile assisted learning have a different assessment process than distance mobile learning? Can traditional techniques work? Who is doing something with mobile learning assessment? What are the successes and failures you can talk about. Lessons learned…

Glendale Community College THANK YOU Gary Marrer Glendale Community College Glendale, Arizona gary.marrer@gcmail.maricopa.edu