1. Writing Effective Assessment Plans Eva Bagg Associate Dean of Institutional Effectiveness Kim Anderson Course Evaluation Subcommittee Chair Summer 2009

2. Why Assessment Plans? "the systematic collection, analysis, interpretation, and use of information to understand and improve teaching and learning" (Tom Angelo). Facilitates periodic, not episodic assessment of student learning and program outcomes Serves as a planning management guideline for program review process Leads to improvement in student learning, retention and program completion Enables faculty to play central role in managing student learning Facilitates collaboration within and among departments Required by accreditation

3. Who Is Involved in Organizing Assessment Plans? Faculty and co-curricular professionals Department Heads Deans and Directors Associate Dean of Institutional Effectiveness Who is Involved with Assessment Plans? All Colleagues contribute and should participate

4. Assessment Plan Development Things to Consider Time How consistent is the evaluation? How robust is the final data set? How will our process meet the standards set by the college’s planning process? How will our process meet the standards set by ACCJC? How effectively does our process provide guidance to our associates? How well can our data be used to improve student learning?

5. What Are The Components of Assessment Plans? First, clearly state outcomes Second, identify an assessment tool Third, establish the criteria and level of expectation for success Fourth, gather and analyze the results Fifth, identify the actions taken for improvement

6. Assessment Plan Format Intended OutcomeAssessment TaskCriteria/Expected Level of Achievement Results of Assessment Actions Taken

7. 1. Identify intended outcomes SLOs = Core, overarching, broad statements of knowledge and skills learned in a course or in an outside of class learning opportunity SUOs = statements of key service effectiveness, efficiency, accuracy, volume, compliance, and experience that is distinct to the unit; process and satisfaction – Avoid complex statements—if outcome contains more than one measurable item, then separate into different outcomes – Use an active verb indicative of expected learning or experience – State what want at end of experience—the outcomes reflect what you want to achieve – The outcomes achievable, reasonable, and measurable – If accredited by a national organization, the outcomes address the expected student learning or service standard of such organizations – If it is part of a sequence so is well-reasoned in relationship

8. Outcome Statements Format SLOSUO Statements about what a student will know, do, or feel upon completion of the course/learning opportunity Statements about what a client will experience, receive, or understand as a result of a given service Client = anyone receiving a service (e.g. student, faculty member, staff member, community member) Service = any established function central to the unit (e.g. front desk interactions;, processing of requests; management, repair, or maintenance of systems) Incorporate the course, learning opportunity, or program’s core knowledge, skills, and/or attitude; must be specific to course/learning opportunity Process = what unit intends to accomplish Volume of unit activity Efficiency of processes as conducted Accuracy of work produced Compliance with regulations & “good practices” Effectiveness of service in accomplishing outcome Use an active verb and is primarily critical thinking (Bloom’s Taxonomy) Satisfaction = what client experienced; with entire process received or with individual services

9. 2. Choose Assessment Task Assessment Tools Direct reasonable replications of real world tasks; capstone exam/project, pre- and post-test, rubric, computerized tracking counts Indirect performance proxies; surveys, student interviews, employee questionnaire Quantitative numerical or statistical values; use actual numbers (scores, rates, etc.) to express quantities of a variable Qualitative descriptive information from questions or statements that provides feedback and possible suggestions; surveys, or summary reports Formative ongoing; to improve learning or process steps Summative final; to gauge quality end product or process Process service received; effective, efficient, correct, accurate, compliance Satisfaction quality of the service interaction; overall or specific service Common agree so can aggregate results; time efficient & ease of use Who students, alumni, employers, clients When time frame & cycle; rotate so not every year but all every cycle How embed in course assignment or service experience

10. Suggested Assessments Examples for SLOs Examples for SUOs Capstone project or Final exam = direct assessment methods integrate knowledge, concepts, and skills associated with an entire sequence of study in a course Surveys (exit, satisfaction, perception) = ask questions providing respondents an opportunity to answer (open-ended questions) or choose a response from finite set of answers (closed-ended questions); completed orally or in writing; administered by phone, mail, e-mail, via the web Rubric = flexible direct assessment to score any product or performance (e.g. essays, portfolios, skill performance, oral speech, project/product creation, etc.) Internal & External Reporting Systems = established or required software or database opportunities  External Audit Reports  PeopleSoft custom queries  Chancellor’s Office Reports (Data Mart, MIS, ARCC)  Reports from business intelligence tools Pre-Test/Post-Test = direct assessment tool administered at the beginning and at the end of a course to evaluate the progress students make Computerized Tracking System (log books) = compilation of numbers of clients who use a service and when can generate data for process functioning Focus Groups = guided discussion whose intent is to gather open-ended comments about a specific issue; provides feedback from the clients who use the services as to the performance and process; requires a facilitator to ask questions and collect comments

11. Checklist for Assessment Tools Embedded in everyday activities. Authentically measures student skills, knowledge, or ability/service process. Based upon clearly defined material. Evaluated by explicit criteria. Collects valid data concerning a specific outcome. Provides reliable results or reproducible evaluative results. Assessment criteria and expectations have been validated through discussion with colleagues. Systematically administered to students or clients. Conforms to institutional mission and outcomes. Reflects outcomes consistent with department or program outcomes. Cost effective concerning resources, supplies and administration.

12. 3. Establish criteria/expected level of achievement 1.Minimum expectation for success (%, fraction, actual number); 1 st time determine a majority then base expectations on previous assessment information when available 2.What is the achievement standard (satisfactory, a passing grade, numeric score, narrative indicator) 3.Which students/clients count in data (successful completion of course only, all queried, those identified from this to that date, random sample)

13. 4. Gather and analyze results of assessment Gather, organize, aggregate, analyze, and report Process protocol and who’s involved; periodicity; deadlines; regular cycle Discussions at various meetings – need time Aggregate – Within the course/department/program – Office of Institutional Effectiveness Analysis and evaluation – Through discussion with vested individuals, which can mean within and outside of the department/program – Was the data valid and helpful; what does it say that is working well; what could be improved – Discuss the discipline issues – Talk with other departments and community members that are part of the holistic pathway through the program/unit

14. 5. Actions Taken Based on results ascertain a step or steps to improve student learning, service, or process Establish definite actions and who is responsible Implement the actions and document them on the assessment plan Identify follow-up process and timeframe On-going assessment cycle

15. Sample Document Layout Intended OutcomeAssessment TaskCriteria /Expected Level of Achievement Results of Assessment Actions Taken Write the intended outcome in this column. Student Learning Outcomes (SLOs) are identified in the course outline of record. Program Level Outcomes (instructional SLOs/SUOs) are identified statements for a program/unit that will be assessed to measure the effectiveness of the program/unit. Explain the assessment task for each outcome in this column. Make sure that a balance of direct and indirect assessment methods are used program- wide. Describe what the assessment is, who will be responsible for the task, how the assessment task will be implemented, when the assessment will be conducted, and where activities will be carried out. The expectations for success should be explained in this column. Establish a minimum expected score for achieving the outcome (e.g. %, fraction, actual number), determine the acceptable success standard (e.g. satisfactory, a C grade, narrative indicator), as well as the students/clients to be included (e.g. successful completers, all queried, random sample). Internal and external conditions should be considered. Once the assessment data is collected analyze the information through small group discussion and in relationship to previous assessment data (if available). Report the actual results and compare with the original expectations in this column. Highlight key findings from the data. Develop supportable conclusions from this information. Determinations will then be made on what action should be taken. Report here what the members have done as a result of the findings. Identify any changes made, who has made the modifications and indicate the time frame for for re- assessment of the outcome. The actions should generally be precise and detailed and written in the past tense; although there are times when no action is necessary and this can be stated as well.

16. Outcomes Assessment Loop Intended Outcome (SLO or SUO) Assessment Task (What/How/When/Who/Where) Criteria/Expected Level of Achievement (Criteria/Standards/Who is Counted) Results of Assessment (Results/Highlights/Conclusion) Actions Taken (Modifications/Time Frame/Who Implements)