Taking it to the Source: Disciplinary Faculty Contribute to, Interpret, and Act upon Local Assessment WPA 2012: 7 20 2012 Pamela Flash University of Minnesota.

Slides:



Advertisements
Similar presentations
SCIENCE & TESTING. CMT-SCIENCE Given for the first time in Spring 2008 in grades 5 & 8 Consists of multiple choice and open ended questions Based on student.
Advertisements

ACADEMIC DEGREE ASSESSMENT & GENERAL EDUCATION ASSESSMENT Nathan Lindsay Arts & Sciences Faculty Meeting March 12,
What Behaviors Indicate a Student is Meeting Course Goals and Objectives? “Indicators”
Assessment Report Department of Psychology School of Science & Mathematics D. Abwender, Chair J. Witnauer, Assessment Coordinator Spring, 2013.
Academy for Student-Centered Learning – Workshop Two Melia Fritch, Shawna Jordan, & Shannon Washburn October 28, 2013 CREATING STUDENT-CENTERED LEARNING.
General Education (GE) Assessment College of Arts and Sciences.
Sheila Roberts Department of Geology Bowling Green State University.
CRITICAL APPRAISAL OF SCIENTIFIC LITERATURE
Comp 20 - Training & Instructional Design Unit 6 - Assessment This material was developed by Columbia University, funded by the Department of Health and.
Welcome to IS299 Associate’s Capstone Rebecca Burton, Instructor AIM: rqbmom (by appt)
LeMoyne-Owen College December 15, 2009 Mimi Czarnik, Professor of English and Dean of Humanities Becky Burton, Associate Professor of Biology Alverno College,
Assessment of Student Learning in General Education AAHE/NCA 2003 Assessment Workshop Omaha, Nebraska ● June 2003.
AP English Language & Composition. STRENGTHEN THE EFFECTIVENESS OF YOUR WRITING THROUGH CLOSE READING AND FREQUENT PRACTICE AT APPLYING RHETORICAL STRATEGIES,
Next Generation Science Standards “Taking the Steps to Implement NGSS” March 29, 2013 TEEAM Conference.
Greenbush. An informed citizen possesses the knowledge needed to understand contemporary political, economic, and social issues. A thoughtful citizen.
MUS Outcomes Assessment Workshop University-wide Program-level Writing Assessment at The University of Montana Beverly Ann Chin Chair, Writing Committee.
DIRECTED ADMINISTRATIVE PORTFOLIO MSA 698. DIRECTED ADMINISTRATIVE PORTFOLIO CAPSTONE ALTERNATIVE Credits: 3 16 weeks The course is centered on the development.
Learning Goals, Objectives, & Curriculum Mapping Linking curriculum & pedagogy to demonstrations of knowledge, skills, and abilities Dr. Marjorie Dorimé-Williams.
Advanced Writing Requirement Proposal
Module 1: Overview of the Framework for K–12 Science Education
Conceptual Change Theory
DIRECTED ADMINISTRATIVE PORTFOLIO
Making assessment in PhD programs more useful for faculty and students
LOGIC MODEL A visual depiction of what a project does and what changes it is expected to bring about. Learn more: Readings, template, examples:
Assessment Planning and Learning Outcome Design Dr
ODU Faculty Development March 18, 2011
Introduction to Curriculum Mapping
CRITICAL CORE: Straight Talk.
D2L Refresher Upload content into the Content section in a D2L course
How to Research Lynn W Zimmerman, PhD.
Consider Your Audience
Teaching Writing to Science Undergraduates
Fullerton College SLOA Workshop:
Closing the Assessment Loop
Computational Reasoning in High School Science and Math
Scholarship of Teaching and Learning
IB Assessments CRITERION!!!.
MUHC Innovation Model.
Introduction to Program Learning Assessment
TRCC TAP Course Vetting: What we have learned so far
Welcome.
Tutorial Welcome to Module 13
General Education Assessment Subcommittee Report
A Case at Their Own Pace: Staging Case-Based Scenarios
Institutional Learning Outcomes Assessment
Introduction to the NSU Write from the Start QEP
Critical Response Strategies
The General Education CLAS Core
Parent Involvement Committee EQAO Presentation
Institutional Effectiveness USF System Office of Decision Support
Student Learning Outcomes Assessment
Assessment: Measuring the Impact of our Programs on Students
Teaching and Learning Commons West Virginia University
Advanced Program Learning Assessment
Creating Assessable Student Learning Outcomes
Reading Research Papers
Assessment Day 2018 New Student Experience
Assessment and Accreditation
SCIENCE AND ENGINEERING PRACTICES
General Education Redesign Task Force
Presented by: Skyline College SLOAC Committee Fall 2007
Writing Learning Outcomes
Assessing Academic Programs at IPFW
Introduction to the AP Capstone™ Program
Using the 7 Step Lesson Plan to Enhance Student Learning
Eloise Forster, Ed.D. Foundation for Educational Administration (FEA)
Creating-1: Generate and conceptualize artistic ideas and work.
Career Technical Education Minimum Qualifications Tool Kit
2017: SLOs & Assessment Reboot
AP U.S. History Exam Details
Presentation transcript:

Taking it to the Source: Disciplinary Faculty Contribute to, Interpret, and Act upon Local Assessment WPA 2012: 7 20 2012 Pamela Flash University of Minnesota flash@umn.edu / http://wec.umn.edu

Faculty conceptions of writing and writing instruction WEC Faculty conceptions of writing and writing instruction Writing instruction (and assessment) within courses Student conceptions of writing and writing instruction Curricular transformation Improved student writing Writing-Enriched degrees

create implement assess WRITING-ENRICHED CURRICULUM SECTION I: CHARACTERISTICS OF WRITING? SECTION II: WRITING ABILITIES? SECTION III: CURRICULAR SEQUENCING? SECTION IV: ASSESSMENT? SECTION V: SUPPORT? SECTION VI: PROCESS? create implement assess Meeting #1 Meeting #2 Meeting #3 Meeting #4

WEC PROCESS AY0 F1 S1 F2 S2 F3 S3 F4 S4 F5 S5+ Survey & Sample Create Writing Plan Implement Writing Plan Implement Writing Plan Implement Writing Program Assessment 1st ed. Plan (“start- up”) Assessment 2nd ed. Plan (“two- year”) Assessment Writing Program Assessment: Rating #1 Assessment: Rating #2

3 sources of productively “disappointing” assessment data WEC Survey (3 versions) @ start of Creation phase). Questions: Baseline values, expectations, and evaluation? Curricular mapping/research (Implementation phase). Questions: Who is teaching with writing in this unit? Where? How? Relevance of outcomes? Writing Assessment (every three years). Question: What effect is WEC/Writing Plan having on student writing?

How important is writing to the scholarly and professional work done in this major’s discipline? (Computer Science and Engineering Fall 2011)

Which three writing abilities do feel it most critical to strengthen in students in the major? (EEB) Instructor Question 9, Student Question 9

 Department of Ecology, Evolution, and Behavior: M1 SH: One thing we haven’t talked about is objectivity—the opposite of expressing feelings, which we didn’t select as one of the abilities we value, but I think that is meaningful because we want the opposite of that; that’s what we strive for. SL: And it is one of the things I certainly respond most vitriolicly to when I get something to read and it’s all about “I did this…” and “I thought this…” SH… ugh…it makes your skin crawl. [1:41:50] PF: So how would you describe this? Because if you say “objective,” no one really knows what that means, do they? SL: They don’t? Why don’t they? SH: Remove the writer out of it…They often say “Shaw et al. said this” and the emphasis is on Shaw et al. rather than the result or the finding... PF: So, you want them to deemphasize the scientists and focus on the science, but... do students in your courses run the danger of accepting the science they report as Truth? SH: The difference is that if someone else came along and did the exact same study that so and so described that they did they should have gotten the exact same results. So I see scientific writing as not so egotistical…it’s contributive. JC: That’s a bunch of crap. You’re mixing undergraduate and graduate writing. Replicable yes, contributive no.

3 sources of productively “disappointing” assessment data WEC Survey (3 versions) @ start of Creation phase). Questions: baseline values, expectations, and evaluation? Curricular mapping/research (Implementation phase). Questions: who is teaching with writing in this unit? Any development? Writing Assessment (every three years). Question: What effect is WEC/Writing Plan having on student writing? Disappointing to whom?

Computer Science undergraduate curriculum

SDS= synthesizing disparate sources IS=interrogating sources Abilities communicated implicitly vs. explicitly SDS= synthesizing disparate sources IS=interrogating sources RCW = results centered writing GRM = grammatically accurate writing ACE = analyze for cause and effect RIV = recognize the importance of variability

Interrogating Sources Tools

Synthesis Tools

3 sources of productively “disappointing” assessment data WEC Survey (3 versions) @ start of Creation phase). Questions: baseline values, expectations, and evaluation? Curricular mapping/research (Implementation phase). Questions: who is teaching with writing in this unit? Any development? Writing Assessment (every three years). Question: What effect is WEC/Writing Plan having on student writing?

ASSESSING WEC: RATING # College of Biological Sciences (2011) Sufficient (N=16) 1 Directly communicates a scientific narrative using an overt logical structure: Moves from problem, to procedure, data, conclusions, and back to target problem. .76 2 Describes significant gaps in scientific knowledge by articulating a target question or problem and describing its significance. .71 3 Demonstrates critical analysis of published work. .35 4 Makes appropriate choices about which data to represent visually. .48 5 Uses precise terminology. .83 6 Presents a synthesis of multiple sources, relying predominantly on primary, peer-reviewed sources. .60 7 Presents conclusions that are based on synthesis of evidence. .52 8 Uses all components of a scientific paper (Abstract and/or Introduction, Methods and Materials, Results, Discussion). .73 Expected Writing Abilities are translated into ratable criteria Iterative rating sessions are held in which 3-4 raters (2-3 from outside the target unit) rate capstone-level papers/projects against faculty-generated criteria. I W D D 3. Rating results (and debrief comments) are presented to faculty, who are asked for their reactions/observations/next moves

History 2010 2012 Demonstrates an understanding of the importance of historical context 0.67 0.81 3. Demonstrates awareness of the particular nature, value, limitations, and incompleteness of historical sources 0.29 0.53 4. Formulates and expresses viable historical research questions and hypotheses 0.71 0.58 9. Explains the broader significance of historical events the topic 0.57 0.49 11. Communicates ideas in lucid, compelling, accessible prose 0.80 0.75

interpreting “low” rating scores Possibilities offered Questions asked Criteria was not addressed by capstone assignment. Should it have been? Where, in the curriculum, are these specific criteria addressed? What sorts of instructional moves might be useful here? How can we help? Raters did not understand criteria. Should criteria be revised? Students require more experience and or instruction in this aspect of writing. Where in the curriculum should this occur? What form(s) should instruction take? Uncontrollable variables tied to student/instructor populations. Maybe…but what makes you think this is the case?

Why is disappointing data yielding positive results? WEC Phase Instrument Why is disappointing data yielding positive results? Writing Plan Creation online surveys (3) Function: investigative: discussion fodder Data source: local stakeholder populations Criteria: generated by Campus Writing Board Audience: local faculty Writing Plan Implementation curricular maps/research Function: investigative: baseline status Data source: local curriculum, course artifacts Criteria: generated by local faculty Writing Plan Assessment longitudinal rating of capstone-level writing Function: primarily formative: informing future implementation action/requests Data source: independent faculty panel Audience: local faculty…..> Provost’s Office?