What is assessment, anyway? Assessment is a confusing concept. To some, it means assessment of what students know and what they are able to do, for others, it means assessment of programs for the purpose of improvement, and for others, it means analysis of resources for effective implementation of technology throughout the university. At a high level, assessment means consideration of accreditation standards. In still other instances, assessment can mean comparisons of technology based courses and traditional courses. And finally, it can mean counting resources, users, and access as justification of expenditures. All of these are frequently lumped together as “assessment” which makes the concept even more vague. (Spurlin, 2006) For the purposes of this presentation, we will consider assessment of technology in order to discover how it influences learning and how technology can be used to improve student learning. How does your university define assessment? The Problems One “problem” with assessment of technology and its effect on teaching and learning is the research question itself, “does insert the name of the technology here enhance learning?” Technology by itself does not enhance learning and there is an issue with how we define “learning” The Clark-Kosma Debate: Richard E. Clark said "studies clearly suggest that media do not influence learning under any conditions". Robert B. Kozma “media do influence learning and, therefore, media selection is significant.” Kozma, however, focused on the attributes of media. Learning Are grades and reliable and valid measure of learning? Shouldn’t we be considering what students know and what are they able to do? What is knowing? Why can’t we consider transfer of what students know and what they are able to do to new and novel situations? Methods vs. Tools The methods that we use to meet the learning objectives are the most important part of the learning paradigm. If the method is wrong, it doesn’t matter what technology is used to support it. Social Science/Psychology/Education Research It is difficult, but not impossible to conduct randomized experiments in education. The most frequently used research tools are surveys, but many do not consider reliability or validity. Qualitative methods, but now especially mixed methods, are becoming the tools of choice. We base some of what we do and say on myths or unsubstantiated claims Are ideas such as the Cone of Learning, Learning Styles, Multitasking, and others true? If they are true, how important are they? Assessment of Technology in Teaching and Learning Mace Mentch, Ph.D. Instructional Technology and Academic Computing Case Western Reserve University CONCLUSIONS We must determine what our dependent variables are and how we will measure them. Satisfaction is important but… Is self report data valid (surprisingly, yes, except when students are reporting their GPA) Some things to think about: In lecture capture, the highest frequency users are those who do not miss class and are very high functioning students The reason collaborative technologies fail is because students don’t know how to collaborate in the first place Students may forget their clickers, but they won’t forget their cell phones Students tell us that if the technology is not easy to use, they won’t use it Students expect us to provide them with the latest and greatest technologies, even if they don’t use them Make sure that you use the results of your assessments to improve student learning The most important aspect of the instructional process is the method, not the technology When used appropriately, technology can and will enhance learning Student motivation is more important than learning style Is this true or false? Kolb Learning Style Inventory Dunn and Dunn Learning Styles Vark Learning Styles 4mat Learning Styles Myers Briggs Keirsey Learning Styles Felder-Silverman Learning Styles Are these true or false? If true, how important are they? The newer generation of students are multitaskers. Is this true or false? Basing our assessments on theories Some important ones Face to Face Telepresence Videoconferencing Cell phone Written documents More Effective Less Effective Richer Mediums Leaner Mediums Media Richness Theory Communication The degree of awareness of the other person in a communication interaction On a continuum of social presence, the face-to-face medium is considered to have the most social presence, whereas written, text-based communication, the least Ability to communicate body language, inflection, voice tone, personal focus, natural language, immediate feedback Social Presence Theory Rich media Lean media better for tasks that require the quick transmission of facts and information better at ambiguous tasks that require resolution of different views and opinions