1. Using CAS Standards and Review Process for Improving Student Learning and Program Outcomes
Patricia Carretta
George Mason University
CAS Officer-at-Large
NACE Representative to CAS
NASPA 2008 International Assessment & Retention Conference
Introduce self and related experiences

2. Purpose CAS Standards and Review Process
Integrating CAS into assessment of programs and services and student learning outcomes
The purpose of this session is to (1) introduce the CAS Standards and its review process and (2) to increase participants’ understanding of how to integrate CAS into their department, division or institutional assessment of educational programs and services and assessment of student learning outcomes. ?

3. Role and Use of CAS Standards:
Student Affairs / Student Services at the division level?
Student Affairs / Student Services at the departmental or program level?
Other?
Have looked at the CAS Standards for at least one functional area?
Have participated in a department/program review process using CAS standards?
Have used CAS standards for other purposes?
Assess participants’ knowledge of and experience with CAS and self study

4. Participant Outcomes
Define the role of CAS self-assessment / review process within a comprehensive assessment plan
List the seven step process of conducting a CAS review
Appreciate the value of engaging a variety of institutional stakeholders in the review process
Strategize how to conduct a comprehensive assessment of programs, services and student learning using CAS Standards
Formulate an assessment plan
Understand the importance of assessing and reporting on input data as well as student learning and development outcomes as a strategies for improving learning and program quality

5. About CAS:
The mission of the Council for the Advancement of Standards in Higher Education (CAS) is to promote the improvement of programs and services to enhance the quality of student learning and development.
CAS is a consortium of professional associations who work collaboratively to develop and promulgate standards and guidelines and to encourage self-assessment.
Founded in 1979 for the purpose of promoting generally accepted standards of practice in student affairs, student services, and student development programs in order to enhance the quality of programs and services for students as well as student learning and development
CAS encourages self-assessment through deployment of tools and processes that produce informed perspectives on the strengths and deficiencies of programs and services and a plan for improvement.
A consortium of 36 professional organizations comprising a constituency of over 100,000
CAS Board of Directors comprised of representatives of these member associations who work collaboratively to develop and promulgate standards and guidelines and to encourage self assessment

6. CAS Resources CAS Professional Standards, 2006 edition
Self-Assessment Guides
CAS Statement of Shared Ethical Principles
CAS Characteristics of Individual Excellence
Frameworks for Assessing Learning and Development Outcomes
CAS resources include:
2006 Edition of CAS Professional Standards (35 standards included); Next edition 2009
Self Assessment Guides for 38 functional areas: Translates CAS standards into an effective workbook format is used for self-assessment of a program’s strengths and weaknesses and the development of an action plan. The individual guides can be downloaded; or you can purchase the 2007 version of the Self-Assessment Guides CD which contains 37 sets of functional area self-assessment guides, functional area contextual statements, PowerPoint presentation, and E-learning course for conducting assessments
The Statement of Shared Ethical Principles highlights the intersections of the ethical statements that guide the various areas our profession represents.
The CAS Characteristics of Individual Excellence lists those characteristics we believe to describe exemplary practice. Professionals could use this as an individual self assessment tool or use it in conjunction with a supervisors or peers to create a professional development plan.
(Both of these statements may be downloaded from the CAS web site)
FALDOs – provides a theoretical description and definition of the 16 learning outcome domains referenced in each CAS Standard (e. g., leadership development, social responsibility, career choices), reviews relevant research; provides practitioners with examples of research questions and possible instruments for conducting assessment activities, and additional resources

7. CAS Guiding Principles
Students and Their Institutions
Diversity and Multiculturalism
Organization, Leadership, and Human Resources
Health Engendering Environments
Ethical Considerations
Using CAS Standards, the Self Assessment Guides and process, the FALDOS will help you express to students, faculty, and administrators alike the complex and vital nature of student support programs and services, and their relationship to student learning and development. Embedded in the CAS standards and the work of the organization are the following key principles:
Students and Their Institutions:
Student is considered as a unique, whole person; Institutional environments shape learning; Responsibility for learning rests with the student; Institutions provide opportunities for learning; Institutions reflect society and its diversity
Diversity and Multiculturalism
Institutions embrace diversity and eliminate barriers that impede student learning; Justice and respect for differences bond individuals to community
Organization, Leadership, and Human Resources
Leadership is essential for institutional success; Institutional success is related to clarity of mission; Qualifications of staff members is tied directly to quality of educational programs and services; Leaders possess sound educational preparation and experience
Health Engendering Environments
Educational programs and services prosper in benevolent environments that provide students with appropriate levels of challenge and support.
Ethical Considerations
Educational service providers provide impeccable ethical behavior in their professional and personal lives.

8. Organization of CAS Standards
Mission
Program
Leadership
Human Resources
Ethics
Legal Responsibilities
Equity and Access
Diversity
Organization and Management
Campus and External Relations
Financial Resources
Technology
Facilities and Equipment
Assessment and Evaluation
As I mentioned previously, the CAS guiding principles are embedded in the standards and guidelines, which are organized into these 14 components. BTW- at the April 2008 CAS board meeting, the components were reordered and a separate component was created for technology
Every set of standards is structured essentially the same.
Advantage to this when conducting self assessments at division level.
Similarly, there is a set of General Standards, revised in 2008, that forms the skeleton of every other set of standards. We refer to it as “boilerplate.” If you use more than one set of standards, you’ll notice areas where they are the same. There are principles so basic that they have been determined to be essential to every functional area.

9. Student Learning & Development
Must promote student learning and development outcomes
Must identify relevant and desirable student learning & development outcomes
Must assess outcomes
Must provide evidence of their impact on student learning and development
Must articulate how they contribute to or support student learning and development
CAS identified learning and development outcomes to address the whole student and includes those outcome domains in the Standards.
Each functional area standard must either directly influence the outcomes, contribute to the outcomes or support student learning and development. And each functional area must assess and provide evidence of impact on student learning and development.

10. CAS: Student Learning & Development Outcomes
Knowledge acquisition, construction, integration and application
Cognitive complexity
Intrapersonal development
Interpersonal competence
Humanitarianism & civic engagement
Practical competence
This past year, CAS formed a study group to reexamine student learning and development outcomes and how they should be incorporated into the CAS standards in ways that would encourage their assessment. The group looked at CAS’s 16 learning outcomes and compared them to those in Learning Reconsidered. What emerged from the work of the study group are 6 learning outcome domains listed in this slide as well as dimensions for each domain, which further define or clarify the learning outcomes. At the April 2008 CAS meeting, members discussed the proposed learning outcome domains and dimensions and how they should be incorporated into the CAS standards. CAS recognized that some of the outcomes may be more or less salient to the work of a functional area, but that all of them must be on our radar screen

11. Student Learning & Development Outcomes
Consistent with the institutional mission, programs and services must identify relevant and desirable student learning and development outcomes from among the six domains. When creating opportunities for student learning and development, programs and services must explore possibilities for collaboration with faculty members and other colleagues.
Programs and services must assess relevant and desirable student learning and development outcomes and provide evidence of their impact on student learning and development. Programs and services must articulate how they contribute to or support student learning and development in the domains not specifically assessed.
As a result of that discussion, the CAS general standards were revised in a way that we believe more reasonably encourages assessment of student learning and development outcomes. The new general standards read…
So…what CAS provides us and our programs and services are sets of Standards and Guidelines, Student Outcomes, including learning outcomes, and a process or cycle for addressing issues or concerns for quality through self assessment, feedback and action implementation.

12. Essential Elements in the CAS Approach
Culture that values involvement of all its members
Quality indicators
Use of standards and guidelines
Willingness/capacity:
To examine itself and assemble results
To report and use the results
The CAS Approach to improvement of programs and services and the enhancement of the quality of student learning and development works best in an environment or culture:
That values involvement of all its members and where this value is reflected in the planning and decision-making
That has identified or determined indicators of quality for its programs and services and expectations for student learning and development
That values and supports the use of standards and guidelines in quality assurance initiatives
That values and encourages self-regulation and self-assessment, i.e., the willingness and capacity of the organization to examine itself meticulously, faithfully, and reliably; and then to assemble the pertinent results of that examination into coherent reports that constituents can comprehend and use.
There are some assumptions in the CAS approach:
Collaboration is good
There are shared values across the profession. (students are viewed holistically; the total environment is educational; institutions should reflect diversity; responsibility for student development rests with the student, challenge & support, etc.)
Some organizational pieces are critical, like mission statements.
Every good program has many elements in common, although they may be conducted differently on different campuses.
The individual campus will decide how in-depth to go with the process, and how to use the results.
Providing standards is a way to improve higher education and services to students

13. Uses of CAS Standards Credibility, accountability, improvement:
Program & service improvement; measures of quality and effectiveness; measures of impact on learning
Design of new programs & services
Restructuring / reorganization
Change in leadership
Institutional self-studies
Preparation for accreditation
Staff development
Academic preparation
ASK PARTICIPANTS HOW CAS STANDARDS ARE USED AT THEIR INSTITUTIONS
The use of CAS Standards and the self assessment process is motivated primarily by external demands for accountability and internal commitments to improvement – the two forces identified by the authors of Assessment Reconsidered: Institutional Effectiveness for Student Success as forces driving assessment in higher education. The authors argue that the “more pressing” of these two motives for assessment is the internal - “a commitment among educators and administrators within colleges and universities to do good work, promote student success, use resources effectively, provide a sound student experience, and serve the public good.” (p. 3) AND they argue that in order to do their best work, student affairs professionals must be assessing learning (p.4).
Additionally, CAS standards have been used in support of staff development and in the academic preparation of graduate students pursuing degrees in higher education and student affairs.
Are you aware of other ways in which CAS standards have been used?

14. CAS: Standards and Guidelines
STANDARD (BOLD TYPE; AUXILARY VERB MUST)
To support the institution’s mission and goals, Career services must provide faculty and staff and administrative units with information, guidance, and support on career development and employment issues and linkages with the broader community.
GUIDELINE (LIGHT-FACED TYPE; VERB SHOULD)
Career Services should support faculty and staff and administrative units by:
Identifying and disseminating information on employment trends, top employing organizations, and co-op and internship sponsors.
CAS believes that quality is foundationally grounded in standards.
In CAS, standards represent a level or degree of quality or achievement for our programs and services; Standards set the minimums (we must all do them);
A standard is “designed to provide practitioners with criteria against which to judge the quality of the programs and services offered.
A standard reflects an essential level of practice that, when met, represents quality performance.
CAS Standards use auxiliary verbs “must” and “shall” presented in bold print.”
CAS guidelines are suggestions for practice and serve to clarify, elaborate or amplify standards. Guidelines are often employed to enhance the program practice beyond essential functions. Guidelines can be tailored to our individual circumstances.
Guidelines appear in light-faced type and use auxiliary verbs Should and May.

15. Comprehensive Assessment Plan
Includes:
Assessment of programs and services
Assessment of student outcomes, including student learning and development

16. Program Evaluation
Is the program or service functioning effectively to achieve its mission?
What evidence is available to support the determination?
How is evidence used to make program decisions?

17. Student Outcomes Assessment
What is the effect of our work on students?
How are they different as a result of interacting with our programs and services?
How do we demonstrate their learning?
How do we know? Measure?

18. The CAS SAG: A Self-Assessment Guide
Translates CAS standards into an effective workbook / evaluative format
Promotes program self-assessment and development
Informs on program strengths and weaknesses
Supports professional staff development
Leads to enhanced programs and services that benefit student learning and development
The CAS process for conducting a comprehensive self-assessment of programs and services and student outcomes begins with the CAS SAG or Self Assessment Guide.
The SAG serves several functions:
Translates…
Promotes … by providing a means for functional area and institutional reviews
Users of the SAG will gain informed perspectives on strengths and weakness or deficiencies of the program and services
Staff members benefit from having a common language and purpose for reflection
Leads to … and encourages planning for future improvement

19. CAS Self-Assessment Process
Establish and prepare the Self-Assessment Team
Initiate the Self Study
Compile and review Documentary and Evaluative Evidence
Identify Discrepancies
Determine appropriate Corrective Action
Recommend Steps for Program Enhancement
Prepare an Action Plan
Before conducting CAS self assessment, determine how you want to use the CAS standards. As a check-off for your own evaluation of your program?
As a staff development tool, or for annual goal setting?
As an internal self-study, within your office or division only?
Who will be involved?
How will the results be shared & used?
The process or steps noted here are fairly straightforward, particularly if you take time for training up front will result in a better process.
Preparing the team to do all the tasks in the self study will result in a better process

20. The Self-Assessment Team
Appoint and Train the Team:
Size of team
Composition: internal, external
Coordinator/Leader
Establish team ground rules
Discuss meaning of each standard
Establish team’s inter-rater reliability
Encourage team discussion; expect disagreements; commit to consensual resolution
Size of team depends on focus of self-assessment. Team of 8-10 would make a division-wide assessment; whereas only 3-5 would be needed for a single functional area.
Composition: professional staff members, faculty, students, and others, at least some of whom are from outside the area(s) undergoing review. Include knowledgeable individuals who have no vested interest in the program
Ground Rules: Clarity about purpose, charge of the team; about the self assessment process; confidentiality as it applies to team deliberations, sharing of information, data reviewed; how to request data and information; staff who will support the team; expectations about the timelines, final report, etc.
Standards and guidelines – team should be well trained/informed about the standards and guidelines - discuss meaning of each standard and guideline as a team and reach agreement on its meaning; Team, in collaboration with staff members from functional area, may want to discuss each standard to determine the level of consensus about its meaning. In collaboration with functional area, should decide if any of the guidelines will be included in the assessment. And if standards, in addition to those included in the SAG, should be added.
Team should be trained in use of the SAG.
Team members must be able to judge, both individually and as part of the team, how well the areas under review meet standard criterion measures. Team members should talk deliberately about how to employ common rating strategies in order to establish rater consistency and inter-rater reliability. Assessment team reviews standards BEFORE making judgments – come to agreement about what a “fully met” would look like.
Recognizing that differences of opinion or judgment will naturally arise, the team should discuss how they will reach consensual resolution of differences.
Team’s Toolbox
CAS STANDARDS and GUIDELINES
CAS assessment criteria
CAS criterion measure rating scale
Documentary evidence

21. Team Actions
Decide whether to include guidelines or other measures that go beyond the standards
Gather and analyze quantitative and qualitative data
Individuals rate each and every criterion measure
Obtain additional documentary evidence if required to make an informed team decision
Complete the assessment, ratings & action plan worksheets
Select guidelines to be judged as standards; Summarize evaluative evidence
5. Judge program against standards . Describe discrepancies between standards and practice
7. Identify corrective actions needed
8. Recommend enhancement actions
9. Prepare an overall action plan

22. Evaluative Evidence STUDENT RECRUITMENT MATERIALS:
Brochures & other program information
Participation policies & procedures
PROGRAM DOCUMENTS:
Mission statements; program purpose & philosophy statements
Catalogs & related materials
Staff & student manuals; policies & procedures statements
ADMINISTRATIVE DOCUMENTS:
Organization charts; student & staff profiles
Financial resource statements & budgets
Annual reports
STAFF ACTIVITY REPORTS:
Curriculum vitae & resumes; professional activity
Service to other programs, departments, or community
STUDENT ACTIVITY REPORTS:
Portfolios, developmental transcripts, resumes
Reports of student service
RESEARCH & EVALUATION DATA:
Needs assessments & self-studies
Program evaluation; graduate & follow-up studies
The self assessment is not complete until relevant data and related documentation are in place to support the raters’ judgments.
Team must identify and summarize documentary evidence to make valid decisions about either ratings or recommendations.
Fortunately, evidence is everywhere. There is a wide range of documentary and evaluative evidence
For a CAS self study, team will need to compile and review many types of documentary evidence. If you think about the 14 components of the CAS functional area standard, you can begin to imagine the scope of evidence needed for informed judgments.
One of the keys is to get creative about what kind of documentary evidence you use. – think outside the box! How do you know??
Brainstorm – where can you find information??
Append relevant documents to final self-assessment rep
Keep focused on quality assurance as the goal – how do you know it’s being done well?

23. Quantitative Data
Summary of objective responses on a questionnaire or program evaluation
Statistics about use by students and various population subgroups
Needs assessments, follow-up studies, and self-study reports
Institutional research reports and fact books
Comparative or benchmark data at local, regional or national level
It is important to recognize that both quantitative and qualitative data will have utility.
Quantitative data involve numbers. Team members may examine:

24. Qualitative Data Focus group information
Written summaries of responses to open-ended questions in interviews and on evaluations
Client satisfaction surveys, self-reports, and written comments, both solicited and unsolicited
Team members must also examine equally valuable qualitative data such as:
Focus group info – which can elicit reason why student are not using an available program or service
A written summary of responses to open ended questions on evaluations or in an interview – which can show changing patterns of student needs
Comments on satisfaction surveys, etc.

25. Using the SAGs SAGs offer a ready format for evaluation
In most instances, there are multiple criterion statements for each standard
Each criterion measure focuses on a particular aspect of the standard, allowing raters to express more detailed and specific judgments
Show/demo the SAG CD – scroll through, quickly, one of the functional areas

26. Criterion Measure Rating Scale
ND NR
Not Done Not met Minimally Well Fully Not Rated
Met Met Met
ND NR
Not Done Not met Minimally Well Fully Not Rated
Met Met Met
Using this scale, consider each criterion statement and decide…
The extent to which each criterion measure has been met by the program or service
CMRS is a tool designed to decide the extent to which each criterion is being met.
Team member will use the scale – with its6 possible rating points – to express judgments about program quality.
In addition to the 4 point rating scale, the SAG provides two non-numerical ratings options: Not Rated (NR) and Not Done (ND)
Under rare circumstances a criterion statement/measure may be excluded when deemed not applicable to the program (e.g., a circumstance directed or mandated by institutional policy or by state law; e.g., single sex or other unique institution that cannot meet a criterion measure for that reason) In such a case, ND rating can be used along with the rationale for exclusion
The NR response can be used when relevant data are unavailable to support a judgment. Again, an explanatorynote should be entered. NR items should generate careful team consideration and follow-up action as appropriate.
Ratings of below a 3 indicate that appropriate change strategies are needed to remedy shortcoming. These should be included/addressed in the action plan.

27. Assessment Criteria Example
Part 6: FINANCIAL RESOURCES
Counseling Services (CS) must have adequate funding to accomplish its mission and goals. Funding priorities must be determined within the context of the stated mission, goals, objectives, and comprehensive analysis of the needs and capabilities of students and the availability of internal or external resources.
CS must demonstrate fiscal responsibility and cost effectiveness consistent with institutional protocols.
ND NR
Not Done Not Met Minimally Met Well Met Fully Met Not Rated
PART 6. FINANCIAL RESOURCES (Criterion Measures) Rating Scale NOTES
6.1 The program has adequate funding to accomplish its mission and goals. ND NR
6.2 Funding priorities are determined within the context of program mission,
student needs, and available fiscal resources. ND NR
6.3 The program demonstrates fiscal responsibility and cost effectiveness
consistent with institutional protocols. ND NR
Part 6: Financial Resources Overview Questions
A. What is the funding strategy for the program?
B. What evidence exists to confirm fiscal responsibility and cost-effectiveness?
Let’s look at an example…
In deciding on your rating, you will consider the evidence provided to you.
The overview questions

28. CAS: Work Form A Assessment, Ratings, and Significant Items
INSTRUCTIONS:
This work form should be completed following individual ratings of the participants. For each of the 13 Parts, identify (circle) the criterion measure item number(s) in the column labeled for which there is a substantial rating discrepancy (two or more ratings apart). Items not circled should reflect consensus among judges that practice in that area is satisfactory. Items where judgment variance occurs need to be discussed thoroughly by team members. Follow this action by determining which practices (criterion measures) can be designated as “excellent” or “unsatisfactory” and record them in the Step One column. In Step Two, list the items requiring follow-up action including any criterion measure rated as being unsatisfactory by any reviewer.
Step One
Part
Items
Excellent
Unsatisfactory
1. Mission
1.1a
1.4
1.1b
1.5
1.1c
1.6
1.2
1.3
2. Program
2.6
2.8d
2.11
2.7
2.8e
2.12
2.3
2.4.5
2.4.10
2.4.15
2.8a
2.8f
2.13
2.4.1
2.4.6
2.4.11
2.4.16
2.8b
2.9
2.4.2
2.4.7
2.4.12
2.5
2.8c
2.10
When the individual ratings have been reviewed and translated into a collective rating, the team is ready to move to the interpretation phase of the assessment process. WORK FORM A
For each of the 14 Parts, identify the criterion measure item number(s) for which there is substantial rating discrepancy.
Items not circled should reflect consensus among raters that practice in that area is satisfactory.
Items where judgment variance occurs need to be discussed thoroughly by team members.
Follow this action by determining which practices can be designated as “excellent” or “unsatisfactory.”
Ratings will begin to converge, show themes & trends. This will lead to the focus of the action plan.
(Show example of spreadsheets developed by Texas Tech to support individual and team ratings)

29. CAS: Work Form A continued
Step Two: List item number(s) for each Part determined to merit follow-up and describe the practice weaknesses that require attention 1. 2. 3. 4. 5.
List the items requiring follow-up action, including any criterion measure rated as unsatisfactory by any reviewer.
Complete the Overview Questions at the end of each Part.

30. CAS Work Form B: Follow Up Actions
INSTRUCTIONS:
The purpose of this work form is to begin the planning for action to be taken on practices judged to merit follow-up (See Step 1, Work Form A). In Step Three, transfer short descriptions of the practices requiring follow-up and detail these items using the table format provided.
Step Three: Describe the current practice that requires change and actions to initiate the change
Practice Description
Corrective Action Sought
Task Assigned To
Timeline Due Dates
Used to summarize actions required for the Program to Meet Standards.
List specific actions identified in the self study that require implementation
Prioritize the list by importance, need and achievability of the desired change

31. CAS Work Form C: Summary Action Plan
Step Four:
This form concludes the self-assessment process and calls for action to be taken as a consequence of study results. Write a brief action plan statement in the spaces below for each Part in which action is required.
Part 1: Mission
Part 2: Program
Part 3: Leadership
Part 4: Human Resources
Part 5: Ethics
List each specific action identified in the self study that would enhance and strengthen services
Establish priorities for the action plan
Identify resources that are essential to program enhancement
Set dates by which specific actions are to be completed
Identify responsible parties to complete the action steps
Set tentative start up date for initiating subsequent self study.
(show Longwood College Student Union CAS Action Plan –

32. Closing the loop Purpose & scope of study is defined
Self-study team is trained
Self-study is conducted
Evaluative evidence is collected
Discrepancies are identified
Appropriate actions are determined
Special actions for program enhancement are recommended
Action plan is developed and communicated

33. Student Outcomes Assessment
What is the effect of our work on students?
How are they different as a result of interacting with our programs and services?
What have students learned?
How have they developed?
Mentioned previously that a comprehensive assessment plan includes assessment of programs and services and of student outcomes. Frye (1999) distinguishes between student outcomes and student learning outcomes as follows:
“Student outcomes generally refer to aggregate statistics on groups of students [for example, in career services, that might include employment and graduate education rates, salary levels, etc.] These are often output only data used more as a measure for comparison with other institutions or as a measure of program performance rather than the changes that have occurred in students themselves due to their collegiate experience.”
“On the other hand, student learning outcomes encompass a wide range of student attributes and cognitive/affective abilities which are a measurement of how their collegiate experiences have supported their development as individuals.”

34. Why Focus on Student Learning?
Accountability
Alignment with institution’s mission
Consistent with language of the academy
Quality improvement
More effective learning
Better defined and more effective planning and strategy development
Richard Keeling in Learning Reconsidered 2, where he asserts that we should “Account for satisfaction and effectiveness, but emphasize outcomes. He goes on to say that … if outcomes are achieved, students (and parents and other constituents) will have abundant reasons to be satisfied. But if there are no clear student outcomes … or if those outcomes are not produced, ultimately no one will be satisfied.” [from Rethinking Learning – p. 58]
Therefore, We should be focusing on student learning for the following reasons :
Accountability–Regional accrediting organizations have moved toward requiring all components of the institution to identify their outcomes and provide evidence that they are accomplishing those outcomes for students. Accrediting agencies are asking questions such as: “What are students learning? What difference are you making in their lives? What evidence do you have that you’re worth the investment” (The Council of Regional Accrediting Commissions)
Another reason to focus on learning outcomes is to better align your department/division with institution mission: Our services exist to support learning and enhance the student experience either by directly promoting learning or by removing or reducing barriers to academic achievement.
The use of learning outcomes is also consistent with the language used throughout the academy and lends more credibility to the effort.
Focusing on learning outcomes, will lead to improved quality of our programs or activities. Why? Because when assessing learning we have to precisely identify what a particular program or activity is designed to achieve; We must be more intentional about the techniques or strategies we employ in the program/activity; and as a result, we will obtain much more relevant data to modify and improve the programs. Instead of relying solely on satisfaction indicators, we can plan and improve based on what students have actually learned.
When your programs/services make learning outcomes clear to students and the programs/services are planned to engage students in learning, students will learn more effectively. Learning outcomes make it clear what students can hope to gain from participating in a particular program or activity. Clarity of learning outcomes helps us select the most appropriate techniques and activities to help students learn.

35. Defining Learning Outcomes
“Outcomes usually identify growth in some dimension of knowing, being or doing.”
Susan Komives & Sarah Schoper, “Developing Learning Outcomes,”
Learning Reconsidered 2 (2006)
Knowledge (cognitive)
Attitude (affective)
Skill (psychomotor)
So how should you go about identifying learning outcomes? Let’s first be clear about what we mean by learning outcomes.
Quote from Susan…
A learning outcome is a statement of what is expected that a student will demonstratively know (knowledge) or be able to do (skill/behavior). It also is about “How the student’s way of thinking will be changed” – his/her attitude or beliefs - as a result of a learning activity or intervention.
A well-written learning outcome statement should contain an active verb, an object, and a qualifying clause or phrase that provides a context or condition, for example, Students will present career goals consistent with the job and organization for which they are interviewing. Or Students will convey skills necessary to be judged competent for the position for which they are interviewing.

36. CAS: Student Learning & Development Outcomes
Knowledge acquisition, construction, integration and application
Cognitive complexity
Intrapersonal development
Interpersonal competence
Humanitarianism & civic engagement
Practical competence
CAS Student Learning and Development Outcomes. The CAS general standard states the following:
Consistent with the institutional mission, programs and services must identify relevant and desirable student learning and development outcomes from among the six domains.
When creating opportunities for student learning and development, programs and services must explore possibilities for collaboration with faculty members and other colleagues.
Programs and services must assess relevant and desirable student learning and development outcomes and provide evidence of their impact on student learning and development.
Programs and services must articulate how they contribute to or support student learning and development in the domains not specifically assessed.

37. Plan for Assessing Student Learning Outcomes
Review/revise mission
Identify major services, programs, activities
Specify desired student outcomes
Identify information needed to show student learning
Identify existing and new data needed
Develop process for measuring outcomes
Identify ways to use findings to change/improve
If you haven’t developed a plan for assessing student learning outcomes, let me recommend the following. These are the steps we took in our division of University Life. Each department:
We did not attempt to measure all the outcomes we identified all at once. We started with a few key programs/activities and each year we added to the plan. Based on what we learned, we made changes and improvements to programs. We also made changes/improvements to the process and tools used to measure.

38. Writing Learning Outcomes
Identify words/phrases that describe what an ideal program participant or college graduate would be able to:
Know – Be – Do … as a result of engaging with your program, activity or service.
What would he/she learn from the experience?
There are a variety of methods you may use to identify and write learning outcomes statements.
One approach is to Assemble standards statements (NACE or CAS). Do a content analysis and look for themes.
Recommend a group process involving your major stakeholders. You might begin by:
Using “sticky notes” and asking them to “write a word or phrase to identify something they would hope an ideal program participant or college graduate would be able to know, be or do as a result of engaging with your program, activity, or service. Brainstorm. Then cluster the sticky notes into categories. Build your own learning outcome goal clusters through these themes.
Option1: take one specific program/activity/service OR
Option 2: take several programs/activities/services
BRAINSTORM
CLUSTER INTO SLO THEMES
An alternative approach is to give a list of outcomes to the group and ask them to rank the outcomes as “essential” or “desirable” or “optional” and then discuss.
After identifying appropriate outcomes, design a survey asking stakeholder groups (students, alumni, employers, faculty, student affairs colleagues) to rank each outcome based on its importance to learning at your college/university.

39. Student Outcomes Matrix
Mission: The Multicultural Research and Resource Center (MRRC) consolidates and disseminates research on intercultural and cross-cultural inclusion, and collaborates with academicians to develop curriculum based in related theory. Additionally, the Center assists students with research and practice regarding multiculturalism in higher education; and administers an international model of leadership development training university-wide and in partnership with other entities in the regional, national and international community.
CAS Student Outcome Domain: Humanitarianism and Civic Engagement
Major Services, Programs, Activities
Dimensions of Outcome Domains:
Understanding and appreciation of cultural and human differences
Social responsibility
Global Perspective
Sense of civic responsibility
Classroom presentations
Curriculum/faculty development
Campus speakers
Applied research on issues affecting multiculturalism, community and handling controversial issues
Building Bridges partnership
NCBI training
Desired outcomes:
Understands one’s own identity and culture; seeks involvement with people different from oneself; articulates the advantages and impact of a diverse society; identifies systematic barriers to equality and inclusiveness, then advocates and justifies means for dismantling them; in interactions with others, exhibits respect and preserves the dignity of others
Recognizes social systems and their influence on people; Appropriately challenges the unfair or uncivil behavior of individuals or groups; Participates in service/volunteer activities that achieve reciprocity; Articulates the values and principles involved in personal decision making; Affirms and values the worth of individuals and communities
Understands and analyzes the interconnectedness of societies worldwide; Demonstrates effective stewardship of human, economic, and environmental resources
Demonstrates consideration of the welfare of others in decision making; Critiques decisions and learns from mistakes; Engages in principled dissent; understands and participates in relevant governance systems
Mapping Outcomes
Adopt a set of desired outcomes
Each specific department, program plays a role in contributing to the overall achievement
A chart can be developed to map where each of the desired outcomes will be targeted and assessed
The goal is not that everyone individually does everything; the ogal is that everyone collectively addresses everything
Another approach is to develop a sequence of outcomes across several years so that all are addressed with that span of time

40. Writing Learning Outcomes
Describe the learner – characteristics of users/participants of programs and services. What knowledge, skills, experiences do they start with?
Determine acceptable levels of performance
Consider how you would provide evidence of achievement
As you are developing your learning outcomes, try to keep in mind who your students are – their characteristics and the knowledge, skills and experiences they start with. For example, if your students are primarily graduate students, the learning outcomes may be quite different than those you’d write for undergraduates. Even among undergraduates, there could be significant differences, for example, between first year students and seniors.
Use data you have collected on the users of your programs and services to help you determine “levels of performance” to expect or require.
Then consider how you would provide evidence of achievement.
(Show Longwood College Learning Outcomes/Assessment Plan)

41. Ways to measure learning outcomes
Surveys, questionnaires, self-report (indirect)
Observations
Interviews
Focus groups
Reflective journals
Performance evaluations
Pre-test/post-test
Portfolios
Capstone projects
Review of project/performance by peers and internal/external reviewers
Recommend using multiple assessment methods to measure student learning outcomes.
Assessment evidence can be direct or indirect.
Palomba and Banta (1999) define “direct evidence” as methods of collecting information that require the students to display their knowledge and skills. “Indirect evidence” as methods that ask students or someone else to reflect on the student learning.
Choose measurement tools intentionally – what do you want to measure? How can you gather the BEST evidence?
FALDOS as resource
For accountability purposes, direct strategies provide better evidence

42. Tips, suggestions Identify all your program outcomes
Rank outcomes based on importance
Activity outcome
Program outcome
Institutional or division outcome
Identify where learning opportunities are occurring (mapping)
Design and implement assessment cycle – one outcome at a time
Identify all…Use something like the Student Outcomes Matrix or a rubric
Rank… don’t try to assess every program outcome every year. Develop a plan and timeline based on what seems logical and important; perhaps beginning with the easiest and progressing strategically to the others. Perhaps start with activity outcomes and build program outcomes from those. Consider institutional, division or unit outcomes and see how your program/activity ties to those.
These types of information help allocate resources responsibly; also help identify factors that influence learning so staff can explore what makes these experiences so powerful.

43. Remember: Start small Start doable Start narrow Start targeted Start!
One program, one workshop, one desired outcome, one measured tool

44. Additional CAS Resources
Standards book
Prologue & Context with “How to” information
FALDOs – book and CD formats as package
SAGs – CD or individually