1. INtopFORM Assessment Plan Please sit with other faculty members from your program of study

2. Goals for today’s session By the end of today’s session, you should: – Understand basic assessment of student learning; – Tentatively identify when and where assessment of the six ITF LOs will take place in your program; – Develop a plan for assessment measures to be used (ITF rubric, major field test, standardized final, etc.); – Develop a plan for collecting baseline data (preferably during Spring 2014); – Decide when you and your colleagues will discuss assessment results; – Know what your responsibilities are for reporting; and – Be familiar with the ITF assessment calendar.

3. Assessment Basics

4. What is assessment? 1.Collecting data so we can report it to SACS (or THEC, TBR, AACSB, etc.) 2.Something the folks in Dossett Hall came up with to keep faculty busy 3.A useful teaching and learning tool

5. Assessment is Deciding what we want our students to learn Making sure they learn it From Linda Suskie, courtesy of Jane Wolfson, Towson University http://img4.wikia.nocookie.net/__cb20100313135124/simpsons/images/4/47/Dabf20.gif

6. Assessment Cycle Excerpted from Assessing Student Learning by Linda Suskie, Jossey-Bass, 2009 1) 2) 3) 4)

7. Program-Level Assessment for INtopFORM

8. Assessment Template Outcomes: Your six discipline-specific ITF LOs Opportunities: Where will students practice ITF skills? Measurement Sites: Which courses and assignments? Measures: Projects, papers, exam questions, clinicals Sampling (if used) Baseline Data: When, where, and how collected Sharing: Who can act on results to improve student learning?

9. Step 1: Establish Learning Goals Six INtopFORM Learning Outcomes What do they mean in YOUR discipline? – See Parts 1 and 2 of your planning worksheets Record draft ITF learning outcomes for your program in Column A of the Assessment Plan Template

10. Step 2: Provide Learning Opportunities Must allow students to practice and develop skill in each of the six INtopFORM Learning Outcomes Where will this take place in your curriculum? – See results of Curricular Mapping exercise What methods might you use? – See Teaching Ideas (Part 3 of planning worksheets) for each ITF LO Record possible learning opportunities for your students in Column B of the Assessment Plan Template; please be specific (course + assignment or activity)

11. Step 3: Assess Student Learning Where will you measure student learning? Must be measurable or observable Goal – Collect “reasonably accurate, truthful evidence” Doesn’t have to be worthy of publication – Good enough to use with confidence Possibilities include single assignments, capstone courses, projects, final exam items

12. Assessment Measures Two options: – Modify the ITF assessment rubric to evaluate papers or projects (preferred); – Study students’ performance on designated items on major field tests, standardized final exams, or locally designed tests of information fluency Record where assessment might take place (course, assignment) in your curriculum in Column C of the Assessment Plan Template. Include possible measures (exam items, rubrics) for your program in Column D.

15. INtopFORM Assessment Guidelines Must evaluate all six ITF LOs Must occur annually Must provide data on the culminating skills and abilities of students in a program of study – Ordinarily in the senior year / 4000-level courses Must yield findings applicable to all majors – Can evaluate all majors’ work or that of a sufficient representative sample

16. Sampling More evidence collected means higher confidence in conclusions, but be practical If you use a sample, it should adequately represent your student population – Online vs. on ground – Adult vs. traditional Record from whom you will collect assessment information in Column E of the Assessment Plan Template; describe sampling (if applicable)

17. Baseline Data Must be collected during Year 1 – Spring 2015 preferred Use standard reporting template (provided) to include: – Overall assessment for each of the six outcomes Mathematical average (can be weighted) or separate rating – Number and % of students rated as: 3 = Excellent 2 = Satisfactory 1 = Unsatisfactory – Whether each student was rated satisfactory on all outcomes Record ideas of how your program might collect baseline data in Column F of the Assessment Plan Template.

19. Step 4: Use the Results Assessment is worthless if it isn’t useful – How will results be analyzed? By whom? – How and when will results be shared? – Who can act on your assessment data to make improvements? Discuss how your program might use the assessment information you collect and record in Column G of the Assessment Plan Template.

20. You are not alone INtopFORM provides support during Faculty Learning Communities INtopFORM staff will hold meetings with each program to discuss assessment and implementation plans The Director of Assessment (with assistance from the Director of the QEP) will – Assist faculty in designing assessments – Approve each programs’ assessment protocols

21. Assessment Goals of INtopFORM

22. Institutional Assessment Goals For each program: Improvement in Years 2-5 compared to baseline data in Year 1 >30% of the rubric ratings or test scores on each INtopFORM learning outcome are “excellent” >90% of the rubric ratings or test scores on each INtopFORM learning outcome are at least “satisfactory” >90% of program majors achieve rubric ratings or test scores demonstrating at least “satisfactory” performance on all INtopFORM learning outcomes

23. California Critical Thinking Skills Test ETSU’s Senior Exit Exam, administered since 2005 for general education assessment 34 questions, 45 minutes in length Five areas: Analysis and Interpretation, Inference, Evaluation and Explanation, Deductive Reasoning, and Inductive Reasoning CCTST maps well to EVALUATING and USING INtopFORM learning outcomes Office of Assessment will collect CCTST data annually and provide reports to programs of study

24. CCTST Score Interpretation CCTST AreaDescription Score Interpretation WeakModerateStrong Analysis (maps to EVALUATING) used when pulling apart arguments and points of view 0 – 23 – 4 5 or more Inference (maps to USING) used whenever drawing conclusions based on reasons and evidence 0 – 56 – 11 12 or more Evaluation (maps to EVALUATING) used when deciding believability of a given statement 0 – 34 – 7 8 or more Induction used when deciding that evidence supports truth in given conclusion 0 – 56 – 11 12 or more Deduction used when deciding a false conclusion is impossible considering all arguments are true 0 – 56 – 11 12 or more CCTST Total Score 8-12 (Weak) 13-18 (Moderate) 19-23 (Strong) ≥24 (Superior)

25. CCTST Trend, 2005-06 to present YearNMean scorePercentile Rank 2005-06113517.561 st – 68 th 2006-07267717.454 th – 61 st 2007-08181517.754 th – 61 st 2008-09159317.255 th 2009-10151217.356 th 2010-11167217.559 th 2011-12185917.657 th 2012-13172316.549 th 2013-14201617.051 st

26. Goals for CCTST Performance For each program: Mean CCTST Total Scores and subtest * scores should exceed baseline in Years 2-5 >30% of scores on each CCTST subtest * at or above “strong” >90% of scores on each CCTST subtest * at or above “moderate” >30% of CCTST Total Scores are at least 19 (“strong”) >90% of CCTST Total Scores are at least 15 (“moderate”) *mapped to an INtopFORM learning outcome

27. Annual Assessment and Improvement Meetings Each fall in Years 3-5 Includes Director of Assessment, QEP Director, program leaders, and other interested faculty Purpose – Review previous year’s assessment data – Use results to refine plan for enhancing information fluency – As needed, define improvement actions, responsible parties, and calendars Annual assessment and improvement report drafted by Directors and approved by program faculty

28. Overview of Assessment Timeline Programs of Study, Cohort 2 Year 1: 2014-15Year 2: 2015-16 SuFaSpSuFaSp Develop program assessment planXXX (refine) Collect baseline data X Collect and analyze results, refine plan, make improvements XXX Year 3: 2016-17Year 4: 2017-18 SuFaSpSuFaSp Develop program assessment plan Collect baseline data Collect and analyze results, refine plan, make improvements XXXXXX

29. Year 1 Assessment Calendar, Programs of Study First Year of Participation WhenWhatWho FallCreate a plan to help all students in the program attain the six INtopFORM learning outcomes Program of Study Faculty Design assessment of ITF learning outcomes Program of Study Faculty SpringCollect and report baseline assessment dataProgram of Study Faculty Fall, Spring, and Summer Collect and report baseline CCTST dataOffice of Assessment SummerSend participating programs of study analysis of baseline data, including CCTST Office of Assessment

30. Year 2 Assessment Calendar, Programs of Study Second Year of Participation WhenWhatWho FallMeet to review baseline data and discuss implications for planned improvements in program Director, QEP; Director, Assessment; Program of Study Faculty Fall and SpringImplement plans to help students attain learning outcomes Program of Study Faculty Collect and report data on ITF learning outcomes Program of Study Faculty Fall, Spring, and Summer Collect and report CCTST dataOffice of Assessment SummerSend participating programs of study analysis of data, including CCTST Office of Assessment

31. Years 3-5 Assessment Calendar, Programs of Study Third, Fourth, and Fifth Year of Participation WhenWhatWho Early FallMeet to evaluate prior year assessment data and refine program’s INtopFORM plan Program of Study Faculty; Director, Assessment; Director, QEP Fall and SpringCollect and report data on ITF learning outcomes Program of Study Faculty Fall, Spring, and Summer Collect and report CCTST dataOffice of Assessment SummerSend participating programs of study analysis of data, including CCTST Office of Assessment

33. Decide what you want your students to learn Make sure they learn it

34. Contact Information Cheri Clavier Director of Assessment clavier@etsu.edu 439-7483 136 Sherrod