1. SUSAN BECK-FRAZIER SUSAN MORRISSEY KRISTEN SPRINGER DREYFUS ALEX SENIOR 2013 Assessment Review Committees Training Sessions 1

2. Why is the assessment review process important? ECU is committed to documenting the quality and effectiveness of all its programs and services.  Responsibility for assessment SACS Comprehensive Standard 3.4.10 – The institution places primary responsibility for the content, quality, and effectiveness of the curriculum with its faculty SACS Comprehensive Standard 3.3.1.2-5 - The institution identifies expected outcomes, assesses the extent to which it achieves these outcomes, and provides evidence of improvement based on analysis of the results in: administrative and academic support services, research and public service  ECU Accountability-statewide push Our Time, Our Future 2013: Strengthening Academic Quality 2

3. Faculty and Staff Unit Assessment Coordinator Assessment Review Committees Institutional Assessment Advisory Council Institutional Effectiveness Council ECU Assessment Report Review Model 3

4. Why am I here? 4 All educational programs and support units on campus have developed assessment plans and annually report results, actions taken, and follow up to the previous year’s actions taken. March 2013 the Academic Council approved the Assessment Review Committee (ARC) structure in order to facilitate the institutional effectiveness assessment process at ECU. ARCs are chaired by representatives on the Institutional Assessment Advisory Council (IAAC). You have been asked to be an ARC member.

5. Roles and Responsibilities The duties of the Assessment Review Committee (ARC) include:  Guiding faculty and staff in adhering to university and internal review criteria and deadlines  Reviewing and evaluating the quality of assessment reports using the approved university’s rubric  Working with assigned ARC chair to prepare the summary report due in the spring 5

6. Units/ARC Chairs/OIA Resource Person Academic Units/Divisions ARC Chair/IAAC MemberOIA Resource Person BSOM Rich Franklin Kristen Dreyfus COE Diana Lys Kristen Dreyfus CAHS Kevin O’Brien Kristen Dreyfus CON Mary Holland Kristen Dreyfus CHHP Susan McGhee Kristen Dreyfus TECS Leslie Pagliari Kristen Dreyfus CFAC Jeff Ward Susan Morrissey CHE Ginger Woodard Susan Morrissey SODM Geri Crain Susan Morrissey COB Shanan Gibson Susan Beck-Frazier HCAS  Undergraduate Don Bradley Susan Beck-Frazier  Graduate Heather Ries Susan Beck-Frazier  Foundations/Support George Bailey Susan Beck-Frazier Administrative Units/Divisions Academic Affairs Karen Summey Susan Beck-Frazier Admin. & Finance Steve Duncan Susan Morrissey Chancellor’s Division/ Erin Eatman Susan Morrissey Division of Univ. Adv. Kay Murphy Division of HS Patricia Fazzone Susan Morrissey Division of RGS Ron Mitchelson Susan Morrissey Division of SA Kathy Hill Susan Morrissey 6

7. What am I reviewing? Educational Program Assessment Reports  3 Program Learning Outcomes  2 Institutional Learning Outcomes (global & leadership)  Means of Assessment  Criterion for Success  2012-13 Results  2012-13 Actions Taken  2011-12 Follow-up to Actions Taken Support Unit Assessment Reports  3 Outcomes  Means of Assessment  Criterion for Success  2012-13 Results  2012-13 Actions Taken  2011-12 Follow-up to Actions Taken 7

8. What should I see in the assessment report? Educational units  Program Learning Outcomes  Directly related to student learning  Description of what students should know, think or do by the time they complete a degree program (undergrad, grad, certificate, stand-alone minor, foundations)  Institutional Learning Outcomes  Reminder: No rating needed for the component – Outcome Statement – for the Global and Leadership Outcomes Support units  Outcomes  Directly related to the functions of the unit  Description of what the office/center does, why it is part of ECU 8

9. What should I see in the assessment report? Means of Assessment  Description of how data is collected  Clearly measure the outcome  Direct measure must be used for educational units  Multiple means of assessment must be used for at least one outcome (direct and indirect, as appropriate)  Assessment instruments attached, where appropriate Criterion for Success  Description of how well students/unit should perform  Clearly matches the means of assessment 9

10. What should I see in the assessment report? Results  Summary of the data collected using the Means of Assessment  Clearly state whether or not the Criterion for Success was met Actions Taken (Use of Results)  Summary of what has been done, based on analysis of Results, to improve the program/unit Follow-up to Actions Taken  Description of whether or not the Actions Taken from the previous year worked to improve the program/unit  Directly linked to the Results collected in the current reporting year 10

11. What am I looking at in the ECU Assessment Report (4- column report)? Educational Program 11

12. What am I looking at in the ECU Assessment Report (4- column report)? Support Unit 12

13. Demonstration of running 4 column report in TracDat Log in to TracDat with your ECU username and passphrase From the drop down menu, select the unit you have been assigned to review Click Reports  Choose “run” next to ECU ASSESSMENT REPORT  Choose the following:  Currently Being Assessed  Institutional Learning Outcome & Program Learning Outcome (for educational programs only)  Reporting Years 2011-12 and 2012-13  Open Report 13

14. Demonstration of Completing Rubric in SharePoint Go to IPAR’s homepage – www.ecu.edu/ipar www.ecu.edu/ipar  Under Highlights (middle column) click Assessment Rubric  Click New Rubric  Follow instructions 14

15. IPARASSESSMENT: 2012-2013 Assessment Report Review Feedback (unit name) 15 ECU Collaborate Workflow Notification East Carolina University is committed to documenting the quality and effectiveness of all its programs and services. The Office of Institutional Assessment, the Institutional Assessment Advisory Council (IAAC), Assessment Review Committees and Unit Assessment Coordinators work together to ensure that an effective and timely review process is completed. This formal review is done with the use of a rubric (one for educational programs and one for support units – both attached). Review feedback for your unit can be viewed by clicking here If you are receiving this email as the Unit Assessment Coordinator (previously called the Responsible Assessment Person or RAP), please share the information with your faculty or office personnel. Department chairs and directors, please review the feedback information with faculty at an upcoming departmental meeting. Department chairs, directors and IAAC representatives have been copied as well.here If any part of your assessment report was rated “developing” or “acceptable”, discuss the areas of concern with faculty and office personnel and make the necessary changes in TracDat. It is recommended that the attached rubrics be used for guidance. For specific questions or points of clarification, contact the member of your Assessment Review Committee who reviewed your report. The reviewer’s name can be found at the top of the rubric.

16. Next Steps Attend meetings of your ARC in order to contribute to discussions that facilitate authentic assessment Contact assigned UACs before the reporting due date to remind of deadlines (August 1 for support units, October 15 for educational programs) Review 2012-13 assessment report for assigned units and complete the rubric (by October 15 for support units, December 15 for educational programs) Follow up with assigned UACs to verify they have received feedback and to ensure that changes are made in Tracdat Work with ARC chair to prepare presentation for IAAC (spring 2014) 16

17. Frequently Asked Questions Do I rate the Global and Leadership Outcome Statements?  No because the Global and Leadership outcomes are currently established by the institution. How do I rate outcomes with multiple means of assessment (MOA), criterion for success (CFS), results (R) and actions taken (AT)?  The lowest rating among the multiple means of assessment, criterion, results and actions taken is the reported rating. Comments are provided for each of the multiple entries.  For example, MOA1: assessment instrument needs to be attached. MOA2:proficient. MOA3: Assessment instrument vaguely described – I do not understand what it is. What do I do if a Follow Up is not appropriate because this is the first year of results?  Select the “Acceptable” rating and provide comments that although none is available, it is appropriate due to the fact that this is the first year of results. 17

18. Frequently Asked Questions 18 What do I do if the 4 column assessment report is blank?  Contact your OIA resource person. It is possible that the unit has turned off their current outcomes and created new ones for the upcoming year. What do I do if I have questions regarding the assessment review of my units?  Contact your ARC chair What do I do if I have questions regarding the SharePoint technology?  Contact Alex Senior (seniora@ecu.edu)seniora@ecu.edu

19. Helpful Hints Review a report together for inter-rater reliability Read through the report thoroughly before selecting outcomes to review and starting the rubric Complete in Word for spellcheck Complete an entire assessment report rubric at one time 19

20. Additional Notes Saved rubrics can be accessed using the link that SharePoint sends once the “save” button at the bottom has been selected Email your ARC chair with questions as soon as you have them If you would like to practice completing a rubric, make sure you select Test College/Test Unit 20

21. 21