The Secrets to Measuring & Enhancing Learning Results Eric Shepherd Good day and welcome! I’m James Lower with Questionmark, and today I would like role and benefits assessments throughout the learning process.

Types of “Assessments” Tests & Exams Job Task Analysis Quizzes Surveys First, let’s clarify what mean by “assessment. ” We’re talking about Tests and exams that measure knowledge, skills and abilities Quizzes during learning And surveys gauge opinions and attitudes

Learning and Assessments Have Evolved Then… Now… As learning professionals I’m sure you can all appreciate just how much the learning and assessment landscape has changed dramatically over the past 5-10 years. There is a real opportunity to use technology in making assessments more widely available.

Learning Without Questions zzzzzzzz zzzzzzzz Content zzzzzzzz zzzzzzzzz What happens in a learning experience without questions? Well, one risk is that learners might disengage … and then they might miss some important points!  

Learning Benefits with Questions Content Questions Using questions as part of learning helps keep our brains engaged! They help learners focus on key points. They help learners “practice” retrieving information they just learned – and that “practice” helps them remember more!

Assessment Thru Learning Process Instructor Collaborative & E-learning Work Place Mentoring Prepare For Learning In/As Learning Assessments Of-Learning Assessments Questions and assessments play a vital role BEFORE learning, DURING learning and AFTER learning. They guide people to powerful learning experiences. They can increase learning and decrease forgetting. And they can confirm – and document – knowledge and skills. Evidence Accumulated

Pre-Learning Assessments Purpose: Create intrigue Route to course Inform instructors Establish benchmarks Learning Instructor Collaborative & E-learning Work Place Mentoring Prepare For Learning In/As Learning Assessments Of Learning Assessments Assessments before learning help create intrigue, inform instructors and establish benchmarks. For example, a job-task analysis surveys help us document the content most relevant to a certain job role. And diagnostic assessments help us spot knowledge gaps. Evidence Accumulated

In-Learning Assessments Purpose: Encourages focus on learning Provide instant feedback to instructor Stimulate and strengthen memory Correct misconceptions Learning Instructor Collaborative & E-learning Work Place Mentoring Prepare For Learning In/As Learning Assessments Of Learning Assessments Quizzes used during learning help learners focus on key points, correct misconceptions, and stimulate and strengthen memory. They also help instructors see if their students are understanding key concepts. Evidence Accumulated

Post-Learning Assessments Purpose: Was learning environment okay? Knowledge transfer occurred? Reduce Forgetting Curve Certifications Learning Instructor Collaborative & E-learning Work Place Mentoring Prepare For Learning In/As Learning Assessments Of Learning Assessments Post learning assessments such as post-course tests and certification exams help us measure and document the results of the learning experience. Course evaluations help us see if the learning environment was ok. And follow-up quizzes can help reduce forgetting. Evidence Accumulated

In a Perfect World Well in a perfect world, everything that was taught during the learning event, would make it into the learners brain and be retained…

In the Actual World They Don’t Get it ALL! But in the real world, sometimes information gets dropped and other times it goes over the learners head so only SOME of the information makes it into the learners brain. We need to figure out what made it and what didn’t.

In the Actual World - They Forget It! In the actual world, we forget. It’s imperative that pre-learning assessments are administered to determine what needs to be learned and questions and quizzes be used during the learning process to reinforce learning.

Learning Curve Ouch! The Forgetting Curve Knowledge/Skills Time Certification Experience Questions Ouch! The Forgetting Curve Learning Experience Questions Knowledge/Skills Learning Experience Research shows that questions that can "simulate" the learners' performance situation will help them climb the “learning curve” more quickly. But then comes the forgetting curve… if we don’t practice retrieving knowledge, we can eventually start forgetting. Time

Slow the Forgetting Curve Questions Certification Experience Questions Questions Learning Experience Questions Knowledge/Skills Learning Experience We need to slow the forgetting curve. Asking questions at regular intervals, after the learning experience, allows the learner to further exercise their memory muscle. Retrieving information from memory strengthens memory recall and reduces the forgetting curve. Time

Questions Thru the Learning Process Certification Questions Certification Experience Questions Questions Reduce Forgetting Create Intrigue Questions Questions Strengthen Memory Knowledge/Skills Questions are essential throughout the learning process to stimulate sustained knowledge and skills. This graph shows how knowledge is significantly increased and after a series of questions are asked over time reducing the forgetting curve. Time

Feedback to Focus and Enhance Learning Focus on important topics Correct misconceptions Understand difficult questions Question Corrective feedback Unsurprisingly the largest gain in learning takes place when people received information feedback. Feedback corrects misconceptions and improves learning while supporting retention and improving future assessment performance.

Job Task Analysis Report The JTA demographic report allows users to make comparisons of JTA ratings between different groups of participants. In this example, the user is comparing how doctors and nurses rated different tasks. Another example might be a comparison between task ratings for managers, employees, and clients.

Correlation - Discrimination Difficulty/ P-Value Questionmark’s item analysis report provides flexibility and ease of use to: Quickly spot potentially problem items flagged with visual cues in: Graphical representations – like at the top of the screen Tables… such as a the bottom of this screen You can also define the parameters for “acceptable” difficulty and correlation-discrimination

Questionmark’s Assessment Technologies Questionmark’s products are used worldwide to create, deliver and report on tests, exams, quizzes and surveys to measure knowledge, skills and attitudes – and improved the learning process for all. Multilingual Integration

Learn more… Come and visit us at our booth! Our clients trust us to deliver valid and reliable assessments. Interested in learning more about Questionmark? Come and visit us at our booth We can offer a demo and answer any specific questions you may have