Assessment of Student Learning At OSU-OKC

Opening Remarks Bill Pink, PhD Vice President Academic Affairs

Workshop Overview What skills, abilities, knowledge and/or attitudes do we expect students to demonstrate? What learning opportunities support student accomplishment of the desired learning outcomes? How do we know students are accomplishing the desired learning outcomes?

Workshop Overview What does the evidence tell us and how do we respond? How do we record and report assessment of student learning to meet accountability requirements?

Learning Outcomes Participants will: describe the assessment of student learning process at OSU-OKC. Implement outcomes-based assessment of student learning at the program and course level.

Assessment of Student Learning Handbook

What skills, abilities, knowledge and/or attitudes do we expect students to demonstrate?

Writing Learning Outcomes “Students will _ __ _” “Students will produce an estimate of materials and cost for a building.”

Using Bloom’s Taxonomy

A good learning outcome is… Specific Measurable Aligned to the curriculum “Students will produce an estimate of materials and cost for a building.”

Levels of Learning Outcomes Campus- wide learning outcomes Program Learning Outcomes

Campus-Wide Learning Outcomes Upon completion of an OSU-Oklahoma City degree: Students solve problems by evaluating arguments or propositions and making judgments that guide the development of their beliefs and actions. Students communicate effectively using organized and coherent written and oral presentations appropriate for the audience and situation. Students use computer and network technologies to gather, analyze and communicate information. Students engage in the community through activities effecting positive change in society and the environment. Students display sensitivity to cultures across local, national and global communities.

Program Student Learning Outcomes “Upon completion of the Public Service courses the student will discuss various dispute resolution systems.” “Upon completion of the Horticulture AAS, a graduate will explain fundamentals of plant science in relation to practical horticultural applications and the environment.” [Students will] apply computer principles and software applications to provide business solutions commonly encountered in entry level computer and management careers. “

Program Student Learning Outcome Examples [Students will] examine the evidence that underlies clinical nursing practice to challenge the status quo, question underlying assumptions and offer new insights to improve the quality of care for patients, families, and communities. Students will analyze dysfunctional family systems and the impact of family dynamics on addiction and recovery processes. Students frame a three phase high voltage transmission line per industry standards.

Course Student Learning Outcomes Students will identify different cultural attitudes towards crime and victimization. Students will identify and discuss that human beings grieve on account of variety of losses. Upon successful completion of this course, the student will describe the evolution of the U.S. telecommunications industry and regulatory structure. The student will install and configure the Windows client operating system.

What learning opportunities support student accomplishment of the desired learning outcomes?

Curriculum Mapping The curriculum map answers two questions: What are the primary student learning outcomes for this program? What courses in the program support each of the program student learning outcomes?

What is curriculum mapping? Matrix created by program faculty to ensure that students are given the opportunity to develop competence in program level SLO’s in core courses SLO’s can be: –Not covered in a course –Introduced in a course –Developed in a course –Assessed in a course

How is curriculum mapping used for assessment? Allows faculty to evaluate and improve curricula by focusing on where SLO’s are Introduced, Developed or Assessed Ensures that SLO’s are introduced early and that they are reinforced and developed throughout the curriculum Ensures that SLO’s are aligned with core courses in the academic program Provides a guideline for choosing where to collect assessment data

Curriculum Mapping – Program Student Learning Outcomes Program Courses Program Student Learning Outcomes

Curriculum Mapping – Campus Wide Outcomes Program and other required courses Campus-wide Student Learning Outcomes

Curriculum Map Example I = Introduced D = Developed A = Assessed

Assessment Methods How will we know if students are accomplishing the desired learning outcomes?

Direct Assessment Methods require students to demonstrate knowledge and skills learned within their educational experience and whether these achievements meet the expected outcomes set for the program.

Direct Measures Objective Tests Capstone Courses and projects Portfolios Authentic Assessment Internships/Service Projects

Defining Assessment Methods

Assessment Methods Example

Indirect Assessment Methods includes surveys and interviews which ask students to reflect on their learning.

Indirect Measures Surveys Focus Groups Interviews Records

What does the evidence tell us and what will we do with it?

Indirect Measures Administered at OSU-OKC James Anderson Coordinator of Assessment, Accreditation & Grant Compliance

Planning for Assessment James Saunders Department Head Engineering Technologies

Question What are we assessing? A.The student B.The instructor C.The course D.Student learning outcomes E.All of the above

Planning for Assessment Components of an Assessment Plan Selecting Assessment Strategies Establishing a Context for Assessment Reliability and Validity Ethical Assessment Before You Begin

Components of An Assessment Plan Start with a matrix listing: –The Student Learning Objectives (SLOs) –Alignment of the SLOs with the curriculum –How will each SLO be assessed –When and how often –Who will be involved

Assessment Strategies Direct Assessments –Published tests –Locally developed tests –Embedded assignments & activities –Portfolios

Assessment Strategies Indirect assessments –Surveys –Interviews –Focus groups –Alumni –Employers

Establish a Context for Assessment In-course: Assignments and exam questions Capstone courses – final projects or portfolios Community service learning Exit interviews or exit surveys Piggyback on other campus assessment efforts

Reliability and Validity Reliability refers to measurement precision and stability or the correlation coefficient. The closer the reliability coefficient is to one the better the reliability. Objectively scored procedures are more reliable than subjectively scored procedures.

Validity Validity refers to how well a procedure assesses what it is supposed to be assessing. Types of Validity –Construct, criterion-related, face, formative, and sampling

Ethical Assessment We must behave ethically when collecting information about people. We must have honest cooperation of respondents and treat them with respect.

Ethical Concepts –Anonymity –Autonomy –Confidentiality –Data ownership –Data security –Deception –Disclosure of rights –Harm –Informed consent

Before You Begin Determine assessment focus Who will be affected by the results Who will get the results How will you collect the data Link the focus, strategy, data analyses, and the decisions of the findings

Data in Action Ann Lowrance Department Head Social Services

….but where do I get data? Formal Assessment –Course Level –Program Level –Capstones –Institutional Level

….but where do I get data? Surveys and Questionnaires Focus Groups Informal Assessment Student Feedback Feedback from Peers (Academic and Professional)

….but where do I get data? Other Information Sources: –SCT –Institutional Grants and Research –Profession/Community Advisory Boards Part Time Faculty Accreditation Groups

OK. Then what? Elder Abuse Current Trends Police Science CLEET Compliance Victimology Prerequisites Oral Presentation Skills Retention Recording and Trending Data

Documenting Assessment How will we record and report assessment of student learning to meet accountability requirements?

OSU-OKC Assessment Reporting Program Assessment of Student Learning Plan Program Assessment of Student Learning Report Course-Level Assessment of Student Learning Report

Timeline May 1, 2013 : Program Assessment of Student Learning plan for –August - December -- Data collection as indicated by the plan Dec. 20, 2013?: Initial report from plan. Late December/Early January -- Institutional Assessment report due to OSRHE Feb 1, 2014: Assessment Day Report and revised assessment plan. –January - May -- Data collection as indicated by the plan July 1, 2014: Final Program Assessment of Student Learning report for academic year and updated assessment plan for