Assessing Course Outcomes Module (4) Assessing Course Outcomes

How Assessment Works ASSESSMENT IS AN ITERATIVE FEEDBACK PROCESS FOR CONTINUAL PROGRAM IMPROVEMENT, BASED ON THE MODEL SHOWN BELOW. STEP ONE is to define intended program learning objectives: specifically, what do we want our graduates to know and actually to be able to do? STEP TWO is to define measurable outcomes that will serve as evidence of how well each objective has been met, and then actually to measure them. Because this step requires explicit articulation of program success criteria, it often has the added benefit of clarifying faulty assumptions. STEP THREE is to compare actual observed outcomes to intended program objectives: how well did we meet our objectives in general, and our student learning objectives in particular? STEP FOUR Finally, in, based on how well or how poorly achieved outcomes compare to intended outcomes, elements of the program (including assessment elements) are redesigned as appropriate, and a new assessment cycle begins.

COGNITIVE, PSYCHOMOTOR Classification of Knowledge Domains: COGNITIVE, PSYCHOMOTOR AND AFFECTIVE

JSU, BLOOM’S TAXONOMY & RUBRICS

Different types of Bloom’s Taxonomy domains

Bloom’s Taxonomy Within the taxonomy learning at the higher levels is dependent on having attained prerequisite knowledge and skills at lower levels (Orlich, et al. 2004). A goal of Bloom's Taxonomy is to motivate educators to focus on all three domains, creating a more holistic form of education.

2 1 3 6 4 5

Cognitive Domain Skills in the cognitive domain revolve around knowledge, comprehension, and critical thinking of a particular topic. Traditional education tends to emphasize the skills in this domain..

Cognitive Domain

Cognitive Domain

Teaching & Learning Activities by Outcomes and Bloom’s Taxonomy Student Learning Outcomes Bloom's Taxonomy of Cognitive Categories Knowledge Comprehension Application Analysis Synthesis Evaluation 1.a.1 Define the three tenets of the Central Limit Theorem Class lecture and students will read assigned chapter. Class lecture and students will read assigned chapter. 1.a.2. Describe three key distributions 1.a.3 Combine to explain the relationship between the three distributions In class, students will calculate sample means and construct a sampling distribution. Homework will reinforce lesson. 1.b.1 Outcome 1.c.1 Outcome 2.a.1 Outcome 2.b.1 Outcome

Assessments by Outcomes and Bloom’s Taxonomy Student Learning Outcomes Bloom's Taxonomy of Cognitive Categories Knowledge Comprehension Application Analysis Synthesis Evaluation 1.a.1 Define the three tenets of the Central Limit Theorem Homework and examination. 1.a.2. Describe three key distributions Homework and examination. 1.a.3 Combine to explain the relationship between the three distributions Homework and examination. 1.b.1 Outcome 1.c.1 Outcome 2.a.1 Outcome 2.b.1 Outcome

Psychomotor Domain Skills in the psychomotor domain describe the ability to physically manipulate a tool or instrument like a hand or a hammer. Psychomotor objectives usually focus on change and/or development in behavior and/or skills. Bloom and his colleagues never created subcategories for skills in the psychomotor domain, but since then other educators have created their own psychomotor taxonomies.

Psychomotor Domain

Psychomotor Domain

Affective Domain Skills in the affective domain describe the way people react emotionally and their ability to feel another living thing's like pain or joy. Affective objectives typically target the awareness and growth in attitudes, emotion, and feelings.

Affective Domain

Affective Domain

Alignment Between Course Outcomes and Institutional Outcomes Intended Learning Outcomes of the Institution Design Backward Intended Learning Outcomes of the Academic Program Intended Learning Outcomes of the Course Intended Learning Outcomes of the Unit Intended Learning Outcomes of the Lesson Deliver Forward

INTRODUCTION TO ENTRANCE SURVEY

INTRODUCTION TO EXIT SURVEY

TYPES OF CONTINUOUS ASSESSMENT Test Quiz Assignment Project Papers Case Study Presentation Experiments Seminars Community Project Final Exam Lab report Physical model Practical Project Laboratory work Observation Peer assessment Soft skill Coursework Survey report Site report

DIRECT MEASURES OF LEARNING

ASSESSMENT MATTERS!!! “ Every assessment is also based on a set of beliefs about the kinds of tasks or situations that will prompt students to say, do, or create something that demonstrates important knowledge and skills. The task to which students are asked to respond on an assessment are not arbitrary.” National Research Council. Knowing what students know: The science and design of educational assessment. Washington, D.C.: National Academy Press, 2001, p.47

STUDENTS AND ASSESSMENTS: “ What and how students learn depends to a major extent on how they think they will be assessed.” John Biggs, Teaching for Quality Learning at University: What The Student Does. Society for Research into Higher Education & Open University Press, 1999, p.141.

Students Lecturers Institutions (i.e. IKN) Community at Large WHO NEEDS TO KNOW ABOUT ASSESSMENT AND WHY? Students Lecturers Institutions (i.e. IKN) Community at Large

Students need to know: how they are progressing with their studies in relation to last semester’s performance. how are they progressing in relation to other learners. are they achieving the required standard.

Students need to know: what are their strengths and weaknesses and further development needs. what they should do if they are not achieving. that they have gained certification of a level of achievement.

Lecturers need to know: all students are attaining the intended learning outcomes. course materials and teaching activities are effective. the learning support strategy is effective.

Lecturers need to know: remedial action is required. they are able to certify that students have achieved standards or met requirements. any adjustments need to be made to accommodate the needs of disabled students.

Institutions (i.e. IKN) need: to provide evidence of achievement of institutional aims. to know whether programs and teaching staff are effective in their stated aims. to make claims to employers and professional bodies regarding graduate qualities.

Institutions (i.e. IKN) need: to certify that learners can practice in specific vocational areas. to make judgments about access and admission to programs. to provide evidence of compliance with legislation such as MQA.

The Community needs to know whether: institutions (i.e. IKN) and lecturers are effective students are adequately prepared for their careers education is being geared to meet the broad, longer-term needs of society. all students are supported.

A Quote: “Best practice in assessment includes both direct and indirect measures of student performance and the use of more than one measure to begin to triangulate the data.”  Northern New Mexico College, 2009

ASSESSMENT OF STUDENT LEARNING: THE BIG PICTURE OUTCOMES INPUTS DIRECT AND INDIRECT MEASURES

Direct Measures Direct measures provide for the direct examination or observation of student knowledge or skills against measurable learning objectives

Indirect Measures Indirect measures of student learning ensure the value of learning experiences

ASSESSMENT DIRECT MEASURES: THE BIG PICTURE Identify outcomes Align pedagogy with outcomes Align assessments with outcome statements

DIRECT MEASURES Direct Measures: directly assess the skills and abilities described in the learning outcomes.

Direct Measures: DIRECT MEASURES To see whether students are attaining the intended learning outcomes The effectiveness of course materials and teaching activities The effectiveness of the learning support strategy Take necessary remedial action to assist student learning Certify that students have achieved standards or met requirements?

Direct Measures: DIRECT MEASURES Define Outcomes Then, followed by assessment methods For example:

TYPES OF DIRECT MEASURES Examples of direct measures of student performance i. Review of senior work by faculty teaching seniors ii. Direct observation of student performance in practicum/internship in work place (with reliable, valid rubric) iii. If students take a certification exam, this can be an additional, but not only direct measure iv. Other direct measure(s)

TYPES OF INDIRECT MEASURES Examples of indirect measures of student performance i. Senior student surveys ii. Alumni surveys iii. Employer surveys and/or focus groups iv. Job placement rates v. Advisory board input vi. Other indirect measure(s)

Locally-developed exams Performance appraisals Behavioral observations ASSESSMENT METHODS Standardized exams Exit and other surveys Locally-developed exams Focus groups Oral exams External examiners Performance appraisals Behavioral observations Simulations Archival records Written surveys Portfolios Questionnaires

ALIGNING ASSESSMENT METHODS WITH OUTCOME STATEMENTS

EDU 703: CURRICULAR ISSUES IN EDUCATION EXAMPLE 1 EDU 703: CURRICULAR ISSUES IN EDUCATION

EDU 703: CURRICULAR ISSUES IN EDUCATION LEARNING ACTIVITIES & MEASURABLE OUTCOMES ASSESSMENT METHODS Discuss issues relating to curriculum development based on information gained from article review. Make an Oral Presentation on the article review and relate the main ideas to curricular issues in education. Students will also submit the written review of the article. 2. Examine the relationship between educational policies and curriculum development. Write a Reaction Paper on the topics presented in week 1 to week 5 to display their ability to use information and knowledge and give their opinion on the meaning of curriculum, the issues raised and the relationship between the issues and the curriculum used in Malaysian schools or institutions of higher learning.

EDU 703: CURRICULAR ISSUES IN EDUCATION LEARNING ACTIVITIES & MEASURABLE OUTCOMES ASSESSMENT METHODS Analyze changes in curriculum and curriculum evaluation. Through Leading a Discussion , students will demonstrate a critical awareness of the content, critical and creative analysis of relevant issues and display their conceptual understanding of the topic presented. 4. Discuss issues and innovations in curriculum development By writing a Term Paper, students will display their ability to use information and knowledge effectively in order to form new understanding and disseminate knowledge.

EDU 703: CURRICULAR ISSUES IN EDUCATION LEARNING ACTIVITIES & MEASURABLE OUTCOMES ASSESSMENT METHODS Analyze changes in curriculum and curriculum evaluation. Through Leading a Discussion , students will demonstrate a critical awareness of the content, critical and creative analysis of relevant issues and display their conceptual understanding of the topic presented. Evaluate the role of the teacher as a curriculum developer. Through Designing a Course Project in groups, students display their ability to use the information and knowledge effectively to carry our a project. The students will also need to integrate theory into practice.

EDU 752: TRENDS IN TESTING, ASSESSMENT AND EVALUATION MEASURABLE OUTCOMES ASSESSMENT METHODS 1. Recall and apply the main ideas in assessment of learning. During a Test, students will demonstrate knowledge of the types, purposes and principles of assessment of learning. 2. Discuss issues relating to assessment of learning based on information gained from article review. Students will make an Oral Presentation on the article they have reviewed and relate the main ideas to assessment of learning. Submit a Written Review of the article. 3. Identify and write a case study in a chosen assessment practice in a school or institution of higher learning. Submit a Written Case Study where they display their ability to use information and knowledge effectively to form new understanding and disseminate knowledge.

EDU 752: TRENDS IN TESTING, ASSESSMENT AND EVALUATION LEARNING ACTIVITIES & MEASURABLE OUTCOMES ASSESSMENT METHODS 4. Work in a group to plan, construct, administer, analyze and evaluate a project related to classroom testing. In their Written Project, students will display their ability to use the information and knowledge effectively to carry out a project. They will also integrate theory into practice and communicate effectively in speaking and writing. 5. Compile information related to the course in the form of a portfolio. Create a Portfolio containing information on the course, notes, handouts, assignments and additional resources related to the course. 6. Participate in class discussion, group activities and presentations Demonstrate Participation in class by making 5 impromptu responses.

A PUBLIC SPEAKING COURSE EXAMPLE 2 A PUBLIC SPEAKING COURSE

“Students will demonstrate effective oral communication skills” Program Learning Outcome: “Students will demonstrate effective oral communication skills”

Performance Criteria: Acceptable Standard of Performance Effective oral communication Personal appearance is appropriate Speaks clearly and with sufficient volume Achieves rapport with audience Uses engaging vocalization Responds effectively to questions and comments Uses audience-appropriate vocabulary, content, and style

Public Speaking Evaluation Sheet Student: __________________________________ Date: ________ Title of Presentation: _____________________________________ Evaluation Scale: Yes, a lot () 4 3 2 1 0 () No, not at all Criteria Score Totals Presentation Style: 1. Personal appearance is appropriate _____ 2. Speaks clearly and with effective volume _____ 3. Achieves rapport with audience _____ 4. Uses engaging vocalization _____ 5. Responds effectively to questions and comments _____ 6. Uses audience-appropriate vocabulary, content, and style _____ Presentation style total: ______ Content: 7. Uses the grammar of standard English _____ 8. Presentation includes introduction, body, and conclusion _____ 9. Organizes content logically and sequentially _____ 10. Presents ideas and arguments clearly and logically _____ 11. Uses appropriate audiovisual materials _____ 12. Cites sources appropriately _____

Example Rubric Presentation PRESENTATION FORMAT Excellent Good Satisfactory Moderate Poor VERBAL PRESENTATION Very clear and easily heard voice projection. Easily heard voice projection. Moderately heard voice projection. Hardly heard voice projection & instruction not in order. No voice projection at all. WRITTEN REPORT PRESENTATION Cover available, clean binding and very informative contents. Cover available, clean binding but contain moderate informative contents. Cover available, but no binding with less informative contents. Cover available, but no binding without any information as required. No cover available for assignment and weak contents. APPEARANCE Presentable appearance with appropriate dress Pleasant appearance with less appropriate dress Unpleasant appearance with inappropriate dress Untidy appearance for presentation DESIGN PORTFOLIO PRESENTATION Tidy design and follow exact specifications as outlined. Tidy design but not fully following outlined specifications. Less tidy design but follow exact specifications as outlined. Less tidy design and is not parallel to specifications as outlined. Weak presentation and does not follow the outlined specifications. Example