SLCI Scores- ten brave Utah Valley Conference Participants

* Evaluated through rubrics Metacognitive Tools For Expert Learners

Unifying Concepts - Metadiscipline of Science 1.Science explains physical phenomena based upon testable information about the physical world. 2.In modern life, science literacy is important to both personal and collective decisions that involve science content and reasoning. 3.Doubt plays necessary roles in advancing science. 4.Scientists use evidence-based reasoning to select which among several competing working hypotheses best explains a physical phenomenon. 5.A theory in science is a unifying explanation for observations that result from testing several hypotheses. 6.Peer review generally leads to better understanding of physical phenomena than can the unquestioned conclusions of involved investigators. 7.Science can test certain kinds of hypotheses through controlled experiments. 8.All science rests on fundamental assumptions about the physical world. 9.Science differs from technology. 10.Scientific knowledge is discovered, and some discoveries require an important history. 11.Science employs modeling as a method for understanding the physical world. 12.Scientific knowledge imparts power that must be used ethically.

Metadisciplinary Concepts Converted to Outcomes STUDENTS WILL BE ABLE TO… 1.Define the domain of science and determine whether a statement constitutes a hypothesis that can be resolved within that domain. 2.D escribe through example how science literacy is important in everyday life to an educated person. 3.E xplain why the attribute of doubt has value in science. 4.Explain how scientists select which among several competing working hypotheses best explains a physical phenomenon. 5.E xplain how "theory" as used and understood in science differs from "theory "as commonly used and understood by the general public. 6.E xplain why peer review generally improves our quality of knowing within science. 7.E xplain how science employs the method of reproducible experiments to understand and explain the physical world. 8.Articulate how science’s way of knowing rests on some assumptions. 9.Distinguish between science and technology by examples of how these are different frameworks of reasoning. 10.C ite a single major theory from one of the science disciplines and explain its historical development. 11.E xplain and provide an example of how modeling i s used in science. 12.E xplain why ethical decision-making becomes increasingly important to a society as it becomes increasingly advanced in science.

DRAFT: Metadisciplinary Outcomes for the Arts Students should be able to… 1.Explain the significance of creative expression and art to the human experience. 2.Discern objective vs. subjective scholarship, criticism and analysis of the arts. 3.Articulate in his/her own words a definition for what constitutes the arts. 4.Communicate ideas and emotions through the practice and study of the arts. 5.Recognize and value creative expression from various cultural and historical perspectives. 6.Explain in his/her own words reasons why critical thinking and problem solving have value in the arts. 7.Describe, using at least two specific examples, how art literacy is important in everyday life.

A realization that should arise from becoming educated: every metadiscipline offers a valuable way of knowing Articulate concepts Rewrite concepts as outcomes Develop assessment instruments Teach the outcomes Use assessment results to try improve learning

OK…Can we assess this stuff?

The Instrument For each outcome – Construct several concept inventory items. – Use established methods for drafting items that have been developed in other concept inventories. In addition… – Test reasoning, not factual knowledge. – Administer the inventory under the conditions in which a citizen will use common information.

…initial instrument constructed through the collegial efforts of Edward Nuhfer, Faculty Development & Geology, Channel Islands Jerry Clifford, Physics, Channel Islands Christopher Cogan, Environmental Sciences & Resource Management, Channel Islands Anya Goodman, Biochemistry, San Luis Obispo Carl Kloock, Biology, Bakersfield Beth Stoeckly, Physics, Channel Islands Christopher Wheeler, Geology, Channel Islands Gregory Wood, Physics, Channel Islands Natalie Zayas, Science Education & Environmental Sciences, Monterey Bay

Science Literacy Concept Inventory Incorporates 25 validated items that map to the twelve concepts Reliability of.85 Tested on over 8000 students in about 30 institutions

Outcome. Student can Define the domain of science and determine whether a statement constitutes a hypothesis that can be resolved within that domain. Science explains physical phenomena based upon testable information about the physical world. Science is on a mission to refute religion; scientists study the paranormal; untestable statements are like scientific hypotheses. Concept Some Misconceptions

Which of the following statements presents a hypothesis that science can now easily resolve? A. Warts can be cured by holding quartz crystals on them daily for a week. B. A classmate sitting in the room can see the auras of other students. C. Radio City Music Hall in New York is haunted by several spirits. D. People with chronic illnesses have them as punishment for past misdeeds. 0, 1, or 2?

Which of the following statements presents a hypothesis that science can now easily resolve? A. Warts can be cured by holding quartz crystals on them daily for a week. B. A classmate sitting in the room can see the auras of other students. C. Radio City Music Hall in New York is haunted by several spirits. D. People with chronic illnesses have them as punishment for past misdeeds.

What did we learn that made this so interesting? Starting with…Do experts (professors) outscore novices (students)?

YES! They do! Also, students on average do come to us with some science literacy: zero literacy = about 25% (random guessing).

Does knowing “stuff” advantage a person?

Professors in Metadisciplines by Mean Score %

We can learn about our students’ science literacy

Our current GE science courses don’t produce science literacy…

Development seems more apparent long term through ranks…

We can learn much about our students

Gender-equality (n = 12,120)

Science Major or Not: 73% vs 67%

Freshmen Sophomores Juniors Seniors Science Major or Not: difference generally increases through undergraduate ranks

First Generation Students: 67% vs 72%

English as a First Language: 63% vs 71%

Cumulative first-generation and ESL: 9 point disadvantage 64% versus 73% for those who have English as a first language AND are not first-generation students. That may be another way of showing that economic status and struggles of students and their families infuse differences.

SLCI by Institutions with Other Measures As Eric Gaze and the Quantitative Literacy test researchers have noted, these little literacy tests may be powerful predictors of student success.

Institutions

Given that our GE science courses don’t produce much increase in reasoning of science literacy…

What are some ways that we can convey citizen competency in science literacy to our students?

Metacurriculum for Metacognition ActivityKnowledge or Skills Knowledge SurveysGoal-setting, Monitor. & Eval. Reading ReflectionsReflection & Monitoring How I Earned an “A”Goal-setting & Monitoring Learning ReflectionsRefl., Monitoring & Evaluation Exam WrappersEvaluation & Goal-setting Critical ThinkingStrategies for Thinking

* Evaluated through rubrics Metacognitive Tools For Expert Learners

Example Survey Item I can outline Piaget’s four main stages of cognitive development, and comment on how children’s thinking changes during these four stages. Knowledge Surveys Response Options 2 = I have current ability to address this challenge very well. 1 = I can now only partially address the challenge. 0 = I currently have insufficient skill/knowledge to address the challenge.

Knowledge Surveys Metacognition – If I think I can do this, how well can I do this? – Can I look at the items and start to distinguish which emphasize knowledge, skills or reasoning? – Am I able to do this at the targeted level of understanding? For construction, consult the tutorials at

Correlations between performance and several metacognitive self- assessments (n = 1011) First Impression Global Self- assessment Knowledge Survey item- by-item Self- Assessment Post- Knowledge Survey Overall Self- Assessment Post-SLCI Overall Self- assessment (*n=590) SLCI Actual Score r = 0.29r = 0.58r = 0.45r = 0.62* Average 72 % Average 75% Average 79% Average 76% Average* 76%

Stages of Intellectual Development Level 1 & 2 thinkers believe that all problems have right and wrong answers, that all answers can be furnished by authority (usually the teacher), and that ambiguity is a needless nuisance that obstructs getting at right answers. Level 3 thinkers realize that authority is fallible and doesn't have all answers. They respond by concluding that all opinions are equally valid, that arguments are just about proponents thinking differently. Evidence doesn't change this. Level 4 thinkers recognize that not all challenges have right or wrong answers, but they do not yet recognize frameworks through which to resolve how evidence best supports one among several competing arguments. Level 5 thinkers can use evidence and begin to accept that evaluations that lead to best solutions can be relative to the context in which a problem occurs. Level 6 thinkers appreciate ambiguity as a legitimate quality of many issues, can use evidence well to explore alternative viewpoints. They recognize that the most reasonable answers often depend upon context and value systems. Levels 7, 8 and 9 thinkers incorporate metacognitive reflection in their reasoning, and they increasingly perceive how their personal values act alongside context and evidence to influence chosen decisions and actions. William J. Perry Jr. (1968) Forms of Intellectual and Ethical Development in the College Years

Traditions of Critical Thinking* Metadisciplines Where Concentrated Some realms of Inquiry** Logic and philosophy HumanitiesWhat is truthful/ethical? ScienceScience and quantitative reasoning What is testable/probable? PragmatismAll metadisciplines including technology What is consequential? PsychoanalysisSocial science, artsWhat is authentic/valued? Critical theoryHumanitiesWhat is privileged? * From Brookfield, 2012 **Modified from Carole Huston, San Diego University “Critical Thinking” and Metadisciplinary Reasoning

What might a rubric look like… To guide development of a sophisticated written teaching philosophy for a faculty member To guide development of a sophisticated written learning philosophy… For anyone?

Reflective Exercises & Learning Journals….

Strategic Reading Expert Readers: read with a purpose and in “extensive mode” accomplished in use of prior knowledge utilize a wide variety of strategies for monitoring and comprehension (e.g., prediction, integration, self- questioning, reflecting) Novice Readers: focus on decoding single words/phrases fail to adjust for different texts/purposes seldom use text-processing strategies Paris et al. (1996)

Reading and Reflecting Reading Reflections: Completed after each reading assignment Short responses to a few questions Submitted online before class Credit awarded for “reflective” submissions What is the main point of this reading? What did you find surprising? Why? What did you find confusing? Why?

Metacognitive Goals of Reflections What is the main point of this reading? Summarizing (Anderson & Thiede, 2008) Keywords (Thiede et al., 2005) What did you find surprising? Why? Misconceptions (Bransford et al., 2000) Affect (Winne & Hadwin, 1998; Pintrich and Zusho, 2007) What did you find confusing? Why? Monitoring, Evaluation, and Reflection (Ertmer and Newby, 1996; Zimmerman, 2002)

Early Results Reading Reflections vs. Course Grades MACALESTER GEOLOGY Pearson = p-value = <0.001 HAMLINE ECONOMICS Pearson = p-value = <0.001

Classroom Activities Effect Size = 1.35 (Large) Clicker Questions Effect Size = 1.08 (Large)

Exam “Wrapper” Achacoso (2004) Lovett (2008) Self Evaluation Preparation Strategies Performance Analysis Planning

Backwards Design Worth being familiar with Important to know and do Highest Priority Wiggins and McTighe (1998)

We Must Act Beyond the Scale of Courses

A Guide for Learning to Learn… Purpose of Education Levels of Thinking Affective Domain Significant Learning Meanings of Learning Research on The Brain Intellectual Development Critical Thinking & Reasoning Metacognition Learning Styles Behavioral Dimensions of Grades