1. The Mission Hierarchy: A Model for Integrating and Assessing Institutional Outcomes
Presented To: SLO Symposium
By: Joachin Arias, Rhea Estoya and Edward Pai
Los Angeles Harbor College
9 February 2018

2. Introductions: Joachin Arias, Ph.D. Rhea Estoya Edward Pai, Ph.D.
Chemistry Faculty member
SLO Coordinator
Assessment Committee Chair
HAPS co-designer
Rhea Estoya
Research Analyst, Office of Institutional Effectiveness
Programmer/Operations Manager for HAPS
Edward Pai, Ph.D.
Dean, Institutional Effectiveness

3. Creating a Model: The “Systems” Approach
Von Bertalanffy defined a system as "elements in standing relationship“
Harbor’s Assessment-based Planning System (HAPS) makes the following assumptions:
Alignment of Mission, Institutional Outcomes and Assessment activities
Identifying the “Elements” and “Relationships” in HAPS
Establishes the “Mission Hierarchy”
Roll-up assessment of all outcomes
Assessment-based Planning Model
Assessments result in Improvement Actions that make up a Unit’s Plan
Closing the Assessment Loop
Macro vs. Micro Assessment processes to improve quality
Macro (Strategic) processes assure Institutional Outcomes are achieved
Micro (Operational) processes assure assessment results are delivered

4. “Elements” of the System
Student Success / College Mission
Pyramid of Student Success (with thanks to John Wooden)
Institutional Outcomes:
ISLOs: Communication, Cognition, Information Competency, Social Responsibility
SEMP Goals: Access & Preparation, Teaching & Learning, Organizational Effectiveness, Resources & Collaboration
Program-level Outcomes:
PLOs: Courses in Program or Pathways that lead to a degree or certificate
GELOs: Courses in a discipline that lead to a GE degree/requirement
Service Areas: Student Support Administration
SEMP Objectives / Strategies
Course/Service-level Outcomes:
SLOs: All courses that have assigned grades and services with a student learning component
SEMP Measures: Student Achievement Outcomes metrics and standards (IEPI, SSSP, Strong Workforce), Federal, State or Grant requirements,
SAOs: Services Area Outcomes for Student Support and Administrative Services

5. The “Mission Hierarchy"
Outcomes Assessment-based Planning → Improvement Action
Student Achievement and Course / Service-level Learning Outcomes
Program / Service Area Outcomes
Institution-level Outcomes
Institution
Mission
SEMP/Goals
Objectives
Measures
Improvement Actions
ISLOs
PLOs
SLO/SAOs
Macro (Strategic)
Micro
(Operational)

6. Assessment-based Planning
All plans are based on an assessment of outcomes
Two types of outcomes:
Student Achievement (Strategic Goals)
State scorecard
Assessed in Program Review
Student Learning (ISLOs)
Workforce skills/soft skills
Assessed in the SLO Process

7. Assessment-based Planning
SEMP
Learning Outcomes

8. SLO Assessment Outcomes

9. Conceptual/Data Roll-up Model Assessing Mission Achievement
Screen shot of Institutional Outcomes Report

10. Assessing Mission Achievement: The Mission Hierarchy
Connects the Classroom to the College Mission
The Mission Hierarchy establishes the relationships (Parent-Child) between:
The College Mission
Institutional Outcomes
Program Outcomes
Classroom-based Student Achievement and Student Learning (SLOs)
The Hierarchical Structure provides “roll-up capabilities”
Children roll-up into Parents
Direct assessments from the classroom are aggregated to assess program and institution-level outcomes

11. Macro/Strategic/College Level Assessment: ISLO Achievement

12. Program Level Assessment: PLO Achievement

13. HAPS Demonstration HAPS DEMO (Recording an Assessment)
Institutional Outcomes Summary
ISLO Assessment Summary
Discipline Level Assessment Summary

14. Mission/SEMP Revision Framework
Particpation
Coverage
Outcome achievement
CPC meeting 1/22 and Retreat 1/29
STEP 1: Review & Evaluation of SEMP
District SWOT
District SMP (1/22)
Other external initiatives/requirements
CPC Retreat 1/29
STEP 2: Assess External Environment
Existing SEMP +
External Scan/District SMP =
New/revised SEMP
Proposed 1/22, discussed 1/29, approved 2/12
STEP 3: Create new/revised SEMP

15. CPC Responsibilities CPC: College Planning Council Annually:
Apex governance committee in charge of strategic planning
Annually:
Oversee and maintain SEMP
Oversee planning processes
Integrate cluster and unit priorities to develop annual priorities for the college based on achievement of the SEMP
Use Annual priorities to guide resource allocation processes
SEMP Cycle ( ):
Evaluate achievement of the college mission
Renew SEMP based on college progress (annual reviews, external priorities)
Identify goals, objectives and measures to add or remove

16. Mission/SEMP Achievement Evaluation Framework
Participation
Are all Operational Units of the college participating?
Are Operational Units participating appropriately?
Coverage
Are all Goals, Objectives and Measures addressed?
Is the amount of coverage appropriate?
Outcome Achievement
Are the activities to address SEMP goals being completed?

17. Mission/SEMP Achievement Measurement Methodology
HAPS: Harbor’s Assessment-based Planning System
Implemented using SharePoint forms and list services
HAPS Records All Outcome Assessment Activities and Associated Improvement Actions (assessment-based planning)
Outcomes (student achievement and student learning) Assessment Process results provide the data
All results are self-reported
Roll-up Model Allows Evaluation of Institutional and Program Level Outcomes

18. SEMP Participation Results
100% Complete
Advance to next year
Discontinued
Abandoned
Incomplete
Total
Cluster
Count %
Academic Affairs
154
41.8%
135
36.7% 38
10.3% 11
3.0% 30
8.2%
368
100.0%
Administrative Services 20
76.9% 2
7.7%
0.0% 26
Student Services
119
82.6% 3
2.1% 4
2.8% 15
10.4%
144
Grand Total
293
54.5%
140
26.0% 42
7.8% 16 47
8.7%
538

19. SEMP Evaluation Survey Results: Participation

20. SEMP Goal Coverage Results
Academic Affairs
Admin Services
Student Services
Total
SEMP Goal
Count
Percent
Goal 1: Access and Preparation for Success 17
21% 0% 63
79% 80
100%
Goal 2: Teaching and Learning for Success
294
83% 61
17%
355
Goal 3: Organizational Effectiveness 34
64% 10
19% 9 53
Goal 4: Resources and Collaboration 1 4% 14
61% 8
35% 23
Other Requirement 20
80% 2 8% 3
12% 25
(blank)
Grand Total
368
68% 26 5%
144
27%
538

21. SEMP Evaluation Survey Results: Coverage – Focus - Priorities

22. SEMP Outcome Achievement
100% Complete
Advance to next year
Discontinued
Abandoned
Incomplete
Total
SEMP Goal
Count
Percent
Goal 1: Access and Preparation for Success 66
82.5% 5
6.3% 3
3.8% 1
1.3% 80
100.0%
Goal 2: Teaching and Learning for Success
162
45.6%
118
33.2% 26
7.3% 11
3.1% 38
10.7%
355
Goal 3: Organizational Effectiveness 29
54.7% 8
15.1%
20.8%
5.7% 2 53
Goal 4: Resources and Collaboration 21
91.3%
0.0%
4.3% 23
Other Requirement 15
60.0% 7
28.0%
4.0% 25
(blank)
Grand Total
293
54.5%
140
26.0% 42
7.8% 16
3.0% 47
8.7%
538

23. SEMP Evaluation Survey Results: Outcomes Achievement

24. Evaluating ISLOs Roll-up methodology (like SEMP) Three dimensions:
Participation
Coverage
Outcome Achievement
Group Exercise:
Evaluate ISLOs
Handouts
Complete Survey (we will use these results for validating our process)

25. ISLO Participation Results
Academic Affairs
Admin Services
Student Services
Total Count
Total Percent
INSTITUTIONAL OUTCOME
Count %
Goal 1: Access and Preparation for Success 1
14% 0% 6
86% 7
100%
Goal 2: Teaching and Learning for Success 23
70% 2 6% 8
24% 33
Goal 3: Organizational Effectiveness
33% 3
Goal 4: Resources and Collaboration
11%
78% 9
ISLO 1 - Communication
501
ISLO 2 - Cognition
981 4
985
ISLO 3 - Information and Technological Competency
265
98% 5 2%
270
ISLO 4 - Social Responsibility and Ethics
143
Other Requirement
57%
29%
(blank) 30
94% 32
Grand Total
1947 14 1% 29
1990

26. ISLO Participation Evaluation
Did all CLUSTERS participate in ISLO/SAO assessment?
Did CLUSTERS meet their assessment responsibilities?

27. ISLO Coverage, Part I: Results
Count
Percent
ISLO 1 - Communication
501
26%
ISLO 2 - Cognition
985
52%
ISLO 3 - Information and Technological Competency
270
14%
ISLO 4 - Social Responsibility and Ethics
143 8%
Grand Total
1899
100%

28. ISLO Coverage, Part 1: Evaluation
Were all ISLOs assessed?
Was the distribution of assessment efforts appropriate?

29. ISLO Coverage, Part II: Academic Divisions
ISLO 1 - Communication
ISLO 2 - Cognition
ISLO 3 - Info and Tech Competency
ISLO 4 - Social Resp. and Ethics
Grand Total
Academic Affairs Office 33
Business 59
115 78 32
284
Communications 81 28 1
110
Counseling 4 2 9
Health Sciences/Nursing 3 23 60
Honors Transfer Program
Humanities and Fine Arts
221
386 72 73
752
Kinesiology, Health and Wellness 47 64 13
133
Math/Technology
117
120
Science/FCS 11
141 14
169
Social Science 94 25 19
216
501
981
265
143
1890

30. ISLO Coverage, Part 2: Evaluation
Was the distribution of assessment efforts by ACADEMIC DIVISION appr0priate?
Did each DIVISION assure the delivery of all college ISLOs?
Were you able to identify priority areas based on this report?

31. ISLO Outcome Achievement
Yes No
N/A
Blank
Grand Total
ISLO 1 - Communication
76%
11% 2%
100%
ISLO 2 - Cognition
70%
13% 3%
14%
ISLO 3 - Information and Technological Competency
77%
10% 1%
12%
ISLO 4 - Social Responsibility and Ethics
73% 6%
21%

32. ISLO Outcome Achievement:
Did the college achieve its ISLOs?
Did improvement occur as a result of the SLO assessment process?

33. Please Complete the Evaluation Survey
The college will use these results to validate our evaluation rubric
We will present these results at the RP Group Conference in April
THANK YOU!
QUESTIONS & DISCUSSION