Gerald Kruse and David Drews Juniata College Huntingdon, PA

MA 103, Quantitative Methods, aka “QM” “Mathematics 103 prepares students to be quantitatively literate citizens in today's world. By learning to think critically about quantitative issues, students will be able to make responsible decisions in their daily lives. …as well as to present quantitative output and verbal arguments. ”

MA 103, Quantitative Methods, aka “QM” “Mathematics 103 prepares students to be quantitatively literate citizens in today's world. By learning to think critically about quantitative issues, students will be able to make responsible decisions in their daily lives. …as well as to present quantitative output and verbal arguments. ” Three Projects during the semester - began using CLA performance tasks Spring authentic and open-ended

MA 103, Quantitative Methods, aka “QM” “Mathematics 103 prepares students to be quantitatively literate citizens in today's world. By learning to think critically about quantitative issues, students will be able to make responsible decisions in their daily lives. …as well as to present quantitative output and verbal arguments. ” Three Projects during the semester - began using CLA performance tasks Spring authentic and open-ended Pre and Post Assessment (Skills and Attitudes) - 55 min exam given on the first and last class of semester - Fall 2009 transition from math skills to CLA performance task

“The Collegiate Learning Assessment (CLA) provides one possible example of an assessment that fits a situated notion of QR.” Richard Shavelson, “Reflections on Quantitative Reasoning, an Assessment Perspective,” In B.L. Madison & L.A. Steen (Eds.), Calculation vs. context: Quantitative literacy and its implications for teacher education. MAA.

Being quantitatively literate is being “able to think and reason quantitatively when the situation so demands”

Evaluating Evidence Higher-Order Skills Assessed

Evaluating Evidence Analysis / Synthesis / Conclusion Higher-Order Skills Assessed

Evaluating Evidence Analysis / Synthesis / Conclusion Presenting / “creating” evidence Higher-Order Skills Assessed

Evaluating Evidence Analysis / Synthesis / Conclusion Presenting / “creating” evidence Acknowledging alternatives to THEIR conclusion Higher-Order Skills Assessed

Evaluating Evidence Analysis / Synthesis / Conclusion Presenting / “creating” evidence Acknowledging alternatives to THEIR conclusion Completeness Higher-Order Skills Assessed

Experimental Design

Explicit scoring guidelines (based on rubric) established. Fall 2009, Version 1.o

Explicit scoring guidelines (based on rubric) established. Scoring guidelines gave “good” reliability. Fall 2009, Version 1.o

Explicit scoring guidelines (based on rubric) established. Scoring guidelines gave “good” reliability. Encouraging, but not statistically significant, results indicated that students in the section with performance task based projects showed more improvement in critical thinking skills. Fall 2009, Version 1.o

Explicit scoring guidelines (based on rubric) established. Scoring guidelines gave “good” reliability. Encouraging, but not statistically significant, results indicated that students in the section with performance task based projects showed more improvement in critical thinking skills. Use results to prepare for next round of assessment in Spring Fall 2009, Version 1.o

The content of the course Fall 2009 vs. Spring 2011 remained the same, as did 95% of the “classroom experience,” but the course was reframed with a focus on quantitative reasoning (q. r.) and critical thinking (c. t.) Modifications for V2.0 “How was the student experience different in Spring 2011 vs. Fall 2009?”

The content of the course Fall 2009 vs. Spring 2011 remained the same, as did 95% of the “classroom experience,” but the course was reframed with a focus on quantitative reasoning (q. r.) and critical thinking (c. t.) - syllabus - assignments - opportunities during lecture - “salt and pepper” Modifications for V2.0 “How was the student experience different in Spring 2011 vs. Fall 2009?”

The content of the course Fall 2009 vs. Spring 2011 remained the same, as did 95% of the “classroom experience,” but the course was reframed with a focus on quantitative reasoning (q. r.) and critical thinking (c. t.) - syllabus - assignments - opportunities during lecture - “salt and pepper” The pre/post assessment was modified: - better linkage with specific learning outcomes - more open-ended scenario - names - one prompt Modifications for V2.0 “How was the student experience different in Spring 2011 vs. Fall 2009?”

Present the idea of a rubric: - objective assessment, “trust” - familiarize with elements - create one for “chips” Modifications for V2.0 continued “How was the student experience different in Spring 2011 vs. Fall 2009?”

Scoring Guidelines

Present the idea of a rubric: - objective assessment, “trust” - familiarize with elements - create one for “chips” Improved feedback for projects completed during the semester: - students used guideline to score their work - compared this to my scoring - general trends discussed with entire class - scheduled time to meet for specific feedback Each of the three projects emphasized different quantitative content (what we were doing at the time) as well as different categories in the rubric Modifications for V2.0 continued “How was the student experience different in Spring 2011 vs. Fall 2009?”

Results for V2.0 Total Score PrePost% change % of possible improvement my section other section Evaluating Evidence PrePost% change % of possible improvement my section other section

Results for V2.0 Analysis, Synthesis, Conclusion PrePost% change % of possible improvement my section other section Presenting, Creating Evidence PrePost% change % of possible improvement my section other section

Results for V2.0 Acknowledging Alternatives to Their Conclusion PrePost% change % of possible improvement my section other section Completeness PrePost% change % of possible improve- ment my section other section

Rubric Reliability Dimension% spot on% +/- 1 Evaluation Anal/Synth/Concl Create Alternatives Completeness Total Score

Rubric Reliability Correlation of Total Scores Pearson correlation of D and J = P-Value = 0.000

Critical thinking involves evaluating/making good arguments Making good arguments involves... Clearly stating a conclusion Evaluating and selecting evidence Creating links between evidence and conclusion We consider quantitative reasoning as critical thinking involving numbers/data…

MA 103, Quantitative Methods at Juniata College Juniata has a Quantitative Skills Requirement, Q = QM + QS MA 103, Quantitative Methods, is offered for students who don’t fulfill the “Q” in their POE Three Projects during the semester - began using CLA performance tasks Spring authentic and open-ended

Making good arguments involves... Clearly stating a conclusion Evaluating and selecting evidence Creating links between evidence and conclusion We can then consider quantitative reasoning as critical thinking involving numbers/data… Critical thinking involves evaluating/making good arguments

Identifying relevant evidence, evaluating evidence credibility, reliability, relevance Not Attempted (0) Emerging (1,2) Mentions one or two documents, with: - No or wrong evaluation on both (1) - cursory-to-OK eval on document C, flawed on other (2) Developing (3,4) Mentions two documents (one must be C), with: - cursory-to-OK evaluation on both (3) - good evaluation on both (4) Mastering (5,6) Evaluation of C is good, and evaluates two other doc with: - acceptable evaluations (5) - good evaluations (6) Evaluating Evidence Category on Rubric

Identifying relevant evidence, evaluating evidence credibility, reliability, relevance Not Attempted (0) Emerging (1,2) Incorrectly implies, or states directly, agrees that “banning aspartame would improve the health of the state’s citizens” with: - no evidence (1) - evidence (2) Developing (3,4) Implies, or directly disagrees, with Sauer, noting inconsistency of claim with data in doc C but reason is: - inaccurate/unclear or incomplete(3) - good (4) Mastering (5,6) - Says C doesn’t support claim and is clear about reason and uses F reasonably well (5) - Satisfactorily uses conditional probability when discussing relationship between headaches and aspartame usage (6) Analysis/Synthesis/Conclusion Category on Rubric

Sen. Nathan Dulce is running for re-election vs. Pat Sauer Proposed bill to ban aspartame, an artificial sweetener, from being added to any soft drink or food product, Dulce opposes, Sauer approves. Pat Sauer made two arguments during a recent TV interview: (1) Strong correlation between the number of people who consume aspartame and headaches, so,“banning aspartame would improve the health of the state’s citizens.” (2)“Aspartame should be banned and replaced with sucralose.” Pat Sauer supported this argument by referring to a news release. Performance Task Scenario for Pre and Post-Assessment