21 Perspectives on Assessment of Educational Technologies for Informal Learning Wilhelmina C. Savenye

Introduction complex area of practice and research many factors operating in varied ways, numerous complex issues, a diversity of views and approaches, few easy answers, and even some possibly political and controversial concerns. More freedom This freedom does come with considerable professional responsibility.

Introduction two critical fields 1）museum/free-choice learning 2）informal science education Informal learning = nonschool learning? museums, after-school clubs, botanical gardens, zoos, science centers community centers ....

What Is Learning? “… learning is common but definitely not straight forward…” ——Falk and Dierking ( 2000 ) school learning “achievement” Workplace learning ”competence”and ”performance” “a change in an individual that is persistent and that results, not just from growth, but from that person’s interaction with his or her environment” (Gagne, 1984). expanded our view of learning “master”

Definition and Dimensions of informal learning Two dimensions related to informal learning are the degree of formality/informality of the “organization” of learning compares with “setting” in which the learning occurs. —— Sefton-Green (2004) learning activities may be “intentional” or “incidental”. Formal or informal learning rests in who is directing the development of the learning materials or programs —— Clark (2010)

Definition and Dimensions of informal learning typically take place out of school university, or formal training settings. —— National Science Teachers Association,2012 Can take place in formal/school settings and informal settings —— Hofstein and Rosenfeld (1996) There is little general agreement about the definitions

Dimensions of IL Location Timing Structure Control Pacing Regulation Content Not tied to space

A Framework for Informal Learning The US National Research Council (NRC) report ( 2009 ) on learning science in informal environments presents an “ecological framework for understanding learning across places and pursuits”.adds place and culture to the mix. informal learning hard to measure experiences: short and spontaneous. goal: Pleasure and fun outcomes:“unexpected” and “emergent”

Developing Outcomes, Goals and Objectives What is success? broader view of indicators a learner’s increased interest and excitement about a topic, desire to learn more, change in conceptions, surprise about new learning, and even change in an individual’s per- ceptions of identity with regard to a topic or field. Example: The NRC report oultlines six “Strands of Science Learning”

Why Assess for Informal Learning? What is assessment? Assessment is typically defined as measuring learning. How does assessment relate to evaluation? Two aspects to be examined: learner’s performance (assessment) learning environment (evaluation) usually focus on a program , course or organizational level Formative evalutation / summative evaluation Critical appraisal / remedial evaluation

Purposes of Assessment for Informal Learning provide formative feedback to participants provide evaluation data to improve the organization’s learning goals. ——Falk and Dierking ( 2000 )

Methods for assessment traditional tests and measures surveys group and individual interviews Observations artifact and product analysis ethnographic or case studies ...

A Few Examples of Methodologies for Assessing IL and ET 1.Traditional and Informal Learning Measures Tests and quizzes Concept mapping Classroom assessment techniques（CATs） self and peer reviews Embedded assessment performance assessment

A Few Examples of Methodologies for Assessing IL and ET 2.Reflective Writing, Discussion and Media Creation (consrtuct /share/examine) Blogs, vlogs, micrologs, and other types of journaling and reflective writing . Wikis. Discussions and online discussions. Learner-created media for formative assessment. Use of rubrics (criteria) (subject-based rubrics, rubrics for web 2.0 tools...)

A Few Examples of Methodologies for Assessing IL and ET 3.Survey (quetionnaires)—— affective variables closed-ended questions/open-ended questions (data analyze) / ( broader or unanticipated view) design and test increased "motivation" in IL Elements of motivation: Attention, Relevance, Confidence and Satisfaction (ARCS) —— John Keller，2006

A Few Examples of Methodologies for Assessing IL and ET 4. Interviews Interviews not necessarily tied to questions with rating scales be conducted individually or in groups may be structured or unstructured Read, think-aloud protoxcols .

A Few Examples of Methodologies for Assessing IL and ET 5. Obervations Learners who participate in informal experiences for learning are particularly “free.” What we learn about learners in these free-choice settings may in the future be applied in Web-based open learning communities and with free-choice technologies, such as Web 2.0 tools. Participation, for instance, can be tallied.... When conducting formative evaluations in settings such as these, it is often important to observe learners to determine: how many use what, where they go, and what they do when experiencing learning programs or materials...

Conclusions examined the issues and approaches involved in effectively assessing for informal learning. introduced a few methodologies that have proven useful in free-choice and informal learning settings. briefly toured a few ways that new educationaltechnologies may impact informal learning and approaches to measuring learning.

Recommendations educators use an informal learning, free-choice perspective to develop their own goals and outcomes for the learner’s experience. assessment should consider goals of both learners and organizations. Develop assessment plan —— Select and build assessment approaches / instruments creatively —— Use them efficiently —— Report using strategies —— engage in a continuous iterative cycle of improvement

Thanks for listening,waiting for suggestions! 21 Thanks for listening,waiting for suggestions! Xie Xun Sep. 21