Common Definitions Activities undertaken by teachers – and by their students in assessing themselves— that provide information to be used as feedback to modify the teaching and learning activities in which they are engaged. Black, P.J. & Wiliam, D. (1998) Inside the Black Box: Raising standards through classroom assessment. King’s College, London. (see http://en.wikipedia.org/wiki/Assessment_for_Learning)

Types of Assessment Assessment for learning Assessment as learning Assessment of learning Evaluation Pre-assessment Formative assessment Summative assessment

Assessment for Learning Comprises two phases. Pre-assessment. Formative assessment. Is based on a variety of information sources. Portfolios. Works in progress. Teacher observation. Conversation. Provides feedback (verbal or written) to the student that is descriptive and emphasizes strengths, identifies challenges, and points to next steps. Involves instructor adjustment of instruction to keep students on track. Does not provide grades or scores; record-keeping is anecdotal and descriptive. Occurs throughout the learning process, from the outset of the course of study to the time of summative assessment.

Assessment as Learning Requires student awareness of goals of instruction and criteria for performance. Involves goal-setting, monitoring progress, and reflecting on results. Implies student ownership and responsibility for moving his or her thinking forward (metacognition). Occurs throughout the learning process.

Assessment of Learning Is accompanied by a grade (summative) Compares one student’s achievement with standards Produces results that can be communicated to the student Occurs at the end of the learning unit

A judgment made on the basis of a student’s performance. Evaluation A judgment made on the basis of a student’s performance.

Pre-Assessment Determines what a student does and does not know about a topic Determine a student's learning style or preferences used to determine how well a student can perform a certain set of skills related to a particular subject or group of subjects Occurs at the beginning of a unit of study Is used to inform instruction Makes up the initial phase of assessment for learning

Formative and Summative Assessment Formative assessment Determines a student’s knowledge and skills, including learning gaps as they progress through a unit of study. Is used to inform instruction and guide learning. Occurs during the course of a unit of study. Makes up the subsequent phase of assessment for learning. Summative assessment Is done at the end of a unit of study to determine the level of understanding the student has achieved. Grades against an expected standard.

Levels of Analysis Assessment can be conducted at various levels: Student Course Major Degree Program

Association of American Colleges and Universities: AAC&U VALUE (Valid Assessment of Learning in Undergraduate Education) assumes that: To achieve a high-quality education for all students, valid assessment data are needed to guide planning, teaching, and improvement; Colleges and universities seek to foster and assess numerous essential learning outcomes beyond those addressed by currently available standardized tests; Learning develops over time and should become more complex and sophisticated as students move through their curricular and cocurricular educational pathways toward a degree; Good practice in assessment requires multiple assessments, over time; Well-planned electronic portfolios provide opportunities to collect data from multiple assessments across a broad range of learning outcomes while guiding student learning and building self-assessment capabilities; e-portfolios and assessment of work in them can inform programs and institutions on progress in achieving expected goals. http://www.aacu.org/value/index.cfm

AACSB Assurances of Learning Example: AACSB Assurances of Learning

Fundamentals Introduced outcomes assessment in 1991 standards Changed in 2003 to reflect need for improved accountability measures. Emphasize direct assessments of student learning. Requires formulation of specific learning goals. Support two principles: accountability and continuous improvement.

Outcome Assessment The systematic collection, review, and use of information about educational programs undertaken for the purpose of improving student learning and development. Assessment essentials: planning, implementing, and improving assessment in higher education, Palomba and Banta, 1999.

Process Define student learning goals and objectives. Align curricula with the adopted goals. Identify instruments and measures to assess learning. Collect, analyze, and disseminate assessment information. Use assessment information for continuous improvement.

Process: Another Form What will our students learn in our program? What are our expectations? How will they learn it? How will we know they have learned it or not? What will we do if they have not learned it?

Program Focus The Focus is the Program Course-level assessments are not the focus Learning goals should reflect: Broad educational expectations for each degree program, regardless of major. Major intellectual and behavioral competencies a program intends to instill in its students due to the total educational experience across a given program.

Program Focus “Learning goals should express expectations that reflect the expected depth and breadth of student knowledge and skills that are the sustainable foundations for life-long learning in support of their professional and personal development.”

Learning Goal Examples Our graduates will understand the importance of behaving ethically in their professional lives (i.e., have an ethical perspective).

Learning Goal Examples Corresponding Objectives: Our students will know the professional code of conduct within their discipline. Our students will identify the activities/issues in their chosen profession that may present ethical challenges, and will articulate the consequences associated with unethical behavior. Our students will identify an ethical dilemma in a scenario case and apply an ethics model or framework to propose and defend a resolution.

Learning Goal Examples Our graduates will demonstrate problem solving skills, supported by appropriate analytical and quantitative techniques.

Learning Goal Examples Corresponding Objectives: In a case setting, students will use appropriate analytical techniques to identify a business problem, generate and compare alternatives, and develop a solution. In a case setting, students will recognize and analyze ethical problems, choose, and defend a solution.

Learning Goal Examples Our graduates will be effective communicators. Corresponding Objectives: Our students will produce professional quality business documents. Our students will deliver a professional quality presentation accompanied by appropriate technology. Our students will demonstrate effective interpersonal communication skills in a team setting.

Assessment Measures Examples Learning Objective: Students will demonstrate competency in a second language to qualify for admission. Assessment Measure: Acceptable score on standardized examination of a second language is required for admission to a business program.

Assessment Measures Examples (selection) Learning Objective: Students will demonstrate competency in statistical analysis and reasoning skills. Assessment measure: A placement exam is given to measure competency as part of the admissions process.

Assessment Measures Examples (course embedded) Learning Objective: Our students will produce professional quality business documents. Assessment measure: Students’ work from a case analysis assigned in a senior capstone course is assessed for writing effectiveness.

Assessment Measures Examples (course embedded) Learning Objective: Our students will demonstrate appropriate abilities to integrate knowledge across business disciplines in a case solution. Assessment measure: A case analysis presentation in a business strategy class is assessed for developing a strategic recommendation that spans functional areas.

Indirect Measures Examples: alumni surveys, graduating student surveys, and/or employer surveys. Information from indirect measures can complement the direct assessment processes, but they are not acceptable substitutes for direct assessment of learning.

Course Grades Don’t Cut It Course grades are aggregate measures on examinations, research papers, other projects, homework, presentations, class participation, etc. and, therefore, cannot measure students’ performance or achievement of a specific skill or knowledge set. Professors set the criteria and standards for their courses and these may not be completely aligned with program- level learning goals because of the focus on learning related to the specific business discipline for which the faculty member is responsible.

Generalizability? Course level: instructors want to know whether their students are getting the “key points” of the course. Major Some majors have “disciplinary” expectations. Department chairs/deans want to know whether their students are achieving the learning goals for the major. Program Most programs have accreditation requirements. Provosts want to know whether students are achieving the learning goals for the program

Misconceptions About Program Assessment The results of assessment will be used to evaluate faculty performance. No. Faculty awareness, participation, and ownership are essential, but assessment should not be used to evaluate or judge individual faculty performance. Program assessment findings are used to improve programs. Our program is working well; we don’t need to bother with assessment. Program assessment findings are used to improve programs – and there’s always room for improvement. One person can do this; too many faculty will only complicate things. Having all faculty involved means bringing in different perspectives and ideas for improving the academic program – and helps with buy-in. From University of Central Florida: http://oeas.ucf.edu/doc/acad_assess_handbook.pdf

Misconceptions About Program Assessment Program assessment might be used to eliminate some programs. Program assessment is generally intended to be formative evaluation. Assessment is a waste of time and does not benefit the students. Assessment “forces” articulation of objectives and learning outcomes in order to improve student learning. Is this a waste of time? Not so bad; we’ll come up with a good plan and keep using it. Program assessment must be an ongoing and continuous process. Okay – but it’s time-consuming and complex. Effective program assessment will take some of your time and effort, but there are ways to make it manageable.

AACSB FAQs Must all students be assessed? Sampling is okay as long as there is a high degree of confidence that the data are representative, valid, and reliable.

AACSB FAQs What is the minimum performance standard? The goal for overall student performance on any given learning goal should be determined by each school consistent with its mission, degree programs, and student profile.

AACSB FAQs Must all graduates meet the expected standard on all learning goals? No, but if students are not achieving the learning goals at acceptable levels, action must be taken to strengthen the curriculum for future students.

AACSB FAQs Can a faculty member be responsible for the assessment of a learning goal within his/her class? Yes, but the learning goal, operational objective, and supporting rubrics for evaluating performance must be used consistently and follow the collective decisions of the faculty as a whole regarding what constitutes acceptable student performance.

AACSB FAQs Must learning goals be measured separately? Each learning goal should have its own performance standard, but a common method or activity can be used to gather data on more than one learning goal. For example, a case analysis may be useful for assessing analytical thinking as well as writing skills or a presentation may be used for evaluating oral communication skills and business disciplinary competence.

AACSB FAQs Is benchmarking against other institutions required? No. Should student learning be assessed at multiple points in the curriculum? This is not required – but it’s a good idea!

AACSB FAQs If specific students do not perform well on assessments, is remediation required? No, but again, if many students have problems, the curriculum should be addressed and evidence of that should be available.

AACSB FAQs Can the collective work of student teams be used for assessment? Not for individual performance, but the collective work of the team may provide a basis for assessing performance as a team member. How many learning goals are needed? Four to 10 learning goals for each degree program. There is no limit, but this is the guidance in order to keep the assessment program manageable.

AACSB FAQs Must all learning goals be assessed each year? No, but a systematic process is needed to insure all goals are assessed to support meaningful curricula change and development. Normally, each goal should be evaluated at least twice over a five-year AACSB review cycle.

AACSB FAQs What documentation must be maintained? Schools should maintain copies of instruments, course- embedded assignments, scoring grids or rubrics, summary of data and analyses, samples of student products used, documentation that the data was used, and documentation of the curricula actions that were taken based on assessment results.

AACSB FAQs What are the most popular AACSB learning goals? Effective communication. Skills. Ethics. Knowledge of all business disciplines Critical thinking. Effective decision making.

AACSB FAQs What are the most popular AACSB learning goals (continued)? Problem solving ability. Ability to integrate across business disciplines. Global perspective. Team skills. Competency in the major.

Questions/Discussion?