Austin Community College Achieving Excellence Committee November 30, 2007 Zen and the Art of Assessment: A Framework for Effective Practice Based on: Assessment in Context: A Systems Approach to Educational Effectiveness Ronald L. Baker

Austin Community College Achieving Excellence Committee November 30, 2007 Educational Practice Teaching Learning Assessing Knowledge

Austin Community College Achieving Excellence Committee November 30, 2007 Historical Paradigm Knowledge Maze of disparate often isolated information Teaching Dispensing discipline-specific information Learning Recording and assembling information Assessing Testing retention of information

Appearance Most of us do not look as handsome to others as we do to ourselves. Assiniboine Tribal Proverb

Austin Community College Achieving Excellence Committee November 30, 2007Change Outcomes Implied  Explicit What does that mean? Achievements Assumed  Assessed How do you know? Evidence Anecdotal  Verifiable Show me the data!

Austin Community College Achieving Excellence Committee November 30, 2007 Strategic Plan We need elucidation of the obvious more than investigation of the obscure. Oliver Wendell Holmes

Austin Community College Achieving Excellence Committee November 30, 2007Perspective The challenge is for us to see beyond the innumerable fragments to the whole, stepping back far enough to appreciate how things move and change as a coherent entity. Margaret Wheatley

Austin Community College Achieving Excellence Committee November 30, 2007Approach Analysis Synthesis Method consists of two processes, correlative and complementary to each other: Analysis of complex totalities into their parts; and Synthesis of parts into their totality. Joseph L. Esposito

Austin Community College Achieving Excellence Committee November 30, 2007Amplification Analysis knowledge Analysis - Reductionistic examination of parts to acquire knowledge of their properties and functions. Synthesis understanding Synthesis - Holistic study of relationships among parts to gain understanding of their roles and the purpose of the whole in which they exist.

Austin Community College Achieving Excellence Committee November 30, 2007CharacteristicsAnalysis Reductionistic Take Apart Isolate Answer Questions Synthesis Holistic Put Together Correlate Ask Questions

Isolated Practices

Interlocking Practices

Analysis Fit

Synthesis Fit

Austin Community College Achieving Excellence Committee November 30, 2007Nuances Improvement (Achieve Outcomes) Student Learning Curriculum Pedagogy Performance (Evaluate Outcomes) Student Achievement Teaching Effectiveness Program Success Accountability (Defend Outcomes) Cost Efficiency Viability

Austin Community College Achieving Excellence Committee November 30, 2007Definition Assessment is a system of analysis and synthesis to appraise and document quality, achievements, and effectiveness consisting of: Collection Analysis Evaluation Evidence Assimilation

Austin Community College Achieving Excellence Committee November 30, 2007Elements Collection - systematic acquisition of data linked to indicators of achievement. Analysis - critical examination of data. Evaluation - meaning derived from analysis. Evidence - documentation to support judgments, conclusions, and actions. Assimilation - use of evidence-based findings to inform and improve practice.

Austin Community College Achieving Excellence Committee November 30, 2007Interests Student Learning Teaching Effectiveness Program Effectiveness Support Unit Effectiveness Institutional Effectiveness Personal Fulfillment

Austin Community College Achieving Excellence Committee November 30, 2007Frameworks Student Learning Focus: Learning Outcomes Assess: Progress/Achievements Teaching Effectiveness Focus: Promotion of Learning Assess: Student Engagement Program Effectiveness Focus: Stakeholder Expectations Assess: Post-completion Success

Austin Community College Achieving Excellence Committee November 30, 2007Frameworks Unit Effectiveness Focus: Role and Objectives Assess: Processes and Practices Institutional Effectiveness Focus: Mission and Goals Assess: Fulfillment of Intentions Personal Fulfillment Focus: Individual Expectations Assess: Satisfaction and “Fit”

Austin Community College Achieving Excellence Committee November 30, 2007Forms Formative - ongoing formal and informal evaluations that provide feedback on progress to date and gives direction to achievement of intended outcomes. Summative - formal evaluations at specific points in time to judge performance in relation to the intended outcomes of the course or unit.

Austin Community College Achieving Excellence Committee November 30, 2007Fulcrum How much assessment is formative? How much assessment is summative? How commensurate is that distribution with your mission and values?

Austin Community College Achieving Excellence Committee November 30, 2007 Student Learning Assessment Primary Purposes: Assessment of Student Learning Evaluate/certify student achievements Assessment for Student Learning Assist students in fulfilling their fullest potential as learners

Austin Community College Achieving Excellence Committee November 30, 2007Differentiation Assessment of learning is the process of measuring, evaluating, and documenting students’ knowledge, skills, and abilities.

Austin Community College Achieving Excellence Committee November 30, 2007Differentiation Assessment for learning is the process of seeking and interpreting evidence for use by learners and their teachers to decide where the learners are in their learning, where they need to go, and how best to get there. Assessment for Learning: 10 Principles Assessment Reform Group (ARG)

Austin Community College Achieving Excellence Committee November 30, 2007Characteristics Assessment of Learning Measures Past Learning Reports Results Accountability Yields Feedback Summative Evaluative Formal Assessment for Learning Fosters Future Learning Influences Results Improvement Yields Feedforward Formative Diagnostic Informal

Austin Community College Achieving Excellence Committee November 30, 2007Balance Assessment of learning, balanced with assessment for learning, interconnects assessments with assignments in a recursive cycle so that summative feedback on one outcome yields formative feedforward that enhances achievement of a subsequent outcome. Integrative Assessment (Quality Assurance Agency for Higher Education)

Austin Community College Achieving Excellence Committee November 30, 2007 AExcellent C-Fine C/C-Interesting COpening terrible - main part OK - give references next time C-Your writing is appalling DTerrible! Didn’t you learn anything at school about writing essays?!Ambiguity

Purposeful Practice

Austin Community College Achieving Excellence Committee November 30, 2007 Purposeful Practice Matrix Teaching Methods Intended Outcomes 1) 2) 3) 4) 5) Student Engagement Student Demonstration Assessment Considerations Assessment of Learning Assessment for Learning Corroborating Evidence Learning Theory Achievement Indicators

Austin Community College Achieving Excellence Committee November 30, 2007 Questions and Discussion

Austin Community College Achieving Excellence Committee November 30, 2007Values What is your foremost institutional value?

Austin Community College Achieving Excellence Committee November 30, 2007Application Does assessment practice reflect that value?

Austin Community College Achieving Excellence Committee November 30, 2007Authenticity How well do assessments promote outcomes achievement or enable judgments of authentic achievement of your outcomes?

Austin Community College Achieving Excellence Committee November 30, 2007Assimilation How are assessment results used to enhance achievements/improve effectiveness of practice?

Austin Community College Achieving Excellence Committee November 30, 2007 Good Luck! Ron Baker