Lesson 17: Discerning Good Data from Bad Data Design principles: Less is More: Limited capacity of working memory Colored highlighting: Direction selection of important information

“After taking our e-learning course, sales people sold 3.5 more units per day” Do you buy it? Design principles: Less is More: Limited capacity of working memory Highlighting: Directing selection of important information Use of images and words E-Learning ?

Learning Objective: Develop a statistical tool to tell what data is believable and what data should be ignored. Design principles: Less is More: Limited capacity of working memory Highlighting: Directing selection of important information Providing learning objective: Directing selection of important info How it will help you: In future assignments you will be able to read reports about e-learning studies and see if the data is statistically significant.

Back to the knives: Look at the data? Design principles: Highlighting: Directing selection of important information Text & Pictures close together: Contiguity principle E-Learning Without training Average sales of 100 salespeople = 87.5 knives With training Average sales of 100 salespeople = 91 knives

Design principles: Highlighting: Directing selection of important information Text & Pictures close together: Contiguity principle Practice exercises: Help integration E-Learning Without training Average sales of 100 salespeople = 87.5 knives With training Average sales of 100 salespeople = 91 knives What do the averages tell us?  Training increases sales of knives  Training decreases sales of knives  Training has no impact on sales of knives

Have you seen data that uses averages to convince you of a point? Design principles: Give context to previous work to provide hooks for retrieval and transfer. Use this space to describe some point in your job that you have seen data that includes averages.

Key Take-Away 1: Averages are not the whole story. Design principles: Less is more Highlighting Provide learning objective Learning Objective 2: We need to learn about Standard Deviation (SD). = SD

Key Take-Away 1: Averages are not the whole story. Design principles: Less is more Highlighting Provide learning objective Learning Objective 2: We need to learn about Standard Deviation (SD). = SD

Design principles: Less is more Highlighting Practice and involvement for integration Two histograms of 100 numbers: Both have an average of 80? What do you see as the difference?

Design principles: Less is more Highlighting Standard Deviation = 2 Standard Deviation = 6 A lower standard deviation means the results are CLOSER to the average A high standard deviation means the results are further from the average

Design principles: Less is more Practice exercise for integration Which has the LOWER Standard Deviation? A OrB  A  B  A  B

Design principles: Highlighting: Directing selection of important information Text & Pictures close together: Contiguity principle E-Learning Without training Average sales of 100 salespeople = 86.9 knives SD=20 With training Average sales of 100 salespeople = 90.6 knives SD=20 Back to the knives! How does SD fit in?

Design principles: Keep it simple Highlighting: Directing selection of important information Practice for integration Without training Average sales of 100 salespeople = 87.5 knives SD=20 What will the histogram look like? or

Design principles: Keep it simple Highlighting: Directing selection of important information Practice for integration With training Average sales of 100 salespeople = 91 knives SD=20 What will the histogram look like? or

Design principles: Keep it simple Highlighting: Directing selection of important information With training Average sales of 100 salespeople = 91 knives SD=20 Without training Average sales of 100 salespeople = 87.5 knives SD=20 Can you tell them apart?

Design principles: Keep it simple Highlighting: Directing selection of important information How can we be sure? With such a big SD (20), the difference between an average of 87.5 and 91 seems insignificant.

Key Take-Away 2: We need a tool to tell if a difference in averages is significant. Design principles: Less is more Highlighting Provide learning objective Learning Objective 3: We need to learn about p-values. = P-Value

Design principles: Highlighting: Directing selection of important information Text & Pictures close together: Contiguity principle E-Learning Without training Average sales of 100 salespeople = 86.9 knives SD=20 With training Average sales of 100 salespeople = 90.6 knives SD=20 Back to the knives! What is p-value? The P-Value of this data = 0.087

Design principles: Highlighting: Directing selection of important information Text & Pictures close together: Contiguity principle The P-Value of this data = This means that there is a 8.7% chance that the difference between 87.5 and 91 knives was due to chance, not the e-learning program. It is “statistically insignificant and should not be trusted. P<.05 “Statistically significant” Believable P>.05 “Statistically insignificant” Do not believe!

Design principles: Practice for integration Text & Pictures close together: Contiguity principle Try it out! P-value= P-value=.25 P-value=.049

Design principles: Exercise for retrieval and transfer to the job (student) context Bringing it Home Next class, a classmate says that by using video games, the average student improves her reasoning skills by 4% What should you ask first? What kind of video game was it? How many students did you test? What is the p-value of that data?

Design principles: Practice for integration Text & Pictures close together: Contiguity principle Bringing it Home What is the p-value of that data? They respond that P-value = What should you think? Hmmm. This data does not sound so good. Wow, there is about a 1% chance this finding is due to change. That is great data!

Design principles: Practice for integration Text & Pictures close together: Contiguity principle Bringing it Home They respond that P-value = What should you think? Hmmm. This data does not sound so good. Wow, there is about a 1% chance this finding is due to change. That is great data!

Design principles: Encourage metacognitive monitoring by self-checking Self-Check: Click how you are feeling Having a hard time with basics Want to practice Got it Averages Standard Deviation P-Values