1. Analyzing Assessment Data

2. A process to consider... Student Learning Outcomes identified for program. Courses identified as to where the outcomes will be introduced, practiced, and assessed. (curriculum mapping) Assignments developed to create an artifact to assess. Assessment methods determined. Assessment data collected. Data analyzed. Decisions made and recorded regarding course or program changes that should be made.

3. See additional handouts Vanderbilt assessment plan: This will show you a way to document the entire process. My suggestion is to add this to the annual report each year, one plan for each program in the department. Assessment Summary Document Methods of Assessment Stiggins book: Table 4.1 for matrix for determining assessment methods

4. Analyzing Assessment Data What does the assessment data “mean?” The assessment results need to be analyzed to learn whether or not the criteria on the student learning outcomes were met. To give meaning to the information that has been collected, it needs to be analyzed for context, understanding, and to draw conclusions. This step gives the information meaning; it is essential to effectively communicate and utilize the assessment results. How is assessment data analyzed? Analyzing data includes determining how to organize, synthesize, interrelate, compare, and present the assessment results. These decisions are guided by what assessment questions are asked, the types of data that are available, as well as the needs and wants of the audience/stakeholders. Since information may be able to be interpreted in various ways, it may be insightful to involve others in reviewing the results. Discussing the data in groups will result in greater understanding often through different perspectives. What can data be compared to? Data can be compared to findings from previous assessments, baseline data, existing criteria/standards, etc. The example below shows the various methods of comparing data:

5. Looking at data: Look at the following chart. How much information can you actually draw from it? Yes, not much. Median data would be better, i.e., the score at which half are above and half below, in this case that would be 27.5. Mode score, 30 makes the class look much better, but median gives you more realistic information.

6. http://www.sheboygan.k12.wi.us/intech/plc/do cuments/SASDDataAnalysisPPT.pdf Student Name# Correct% Correct Student 1 30100% Student 2 2996% Student 3 1343% Student 4 30100% Student 5 1963% Student 6 30100% Student 7 2790% Student 8 2893% Student 9 2583% Student 10 2583% Average26.287%

7. Assessment Plan example from Vanderbuilt Assessment\VUAdminAssessmentPlanTemplateArchitecture.doc

8. Aggregating Data, i.e., combining multiple data points When you have multiple assessments contributing to the same student learning outcome, you might want to aggregate the data to get an overall picture. Determine level for Exceeding Standard, Meeting Standard, Not Meeting Standard Could be Exceeds = A, Meets = B-C, Does Not Meet = D-F Could be Exceeds = 90%, Meets = 80%, Does Not Meet = Less than 80% Could be Exceeds = 3.5-4, Meets = 2.0-3.49, Does Not Meet = Less than 2.0

9. Inter-rater reliability When using rubrics for program level assessment, assuring that different raters are consistent requires a measure of inter-rater reliability. Make multiple copies of the same artifact. Work with a small group of raters to “grade” the artifact. Check what are the points of disagreement and discuss why different raters gave different score. Have the group “grade” another artifact—again, look at points of difference and seek to define more explicitly the factors that contribute to the “grade” decision. Eventually you want very close agreement among the raters.