1. Introduction to Indirect Student-Learning Assessment (Part V)
Summer DeProw
Topeka Small
Wayne Wilkinson
November 9, 2016
ITTC Faculty Center
Arkansas State University

2. Indirect Assessments Require the inference of student learning:
No direct evidence or demonstration
Common topics of indirect assessments:
Perceptions of successfully meeting program outcomes
Satisfaction/attitudes toward program
Utility of program

3. Common Forms of Indirect Assessment
Interviews
Focus groups
Classroom assessment techniques
Curriculum and syllabus analysis
Surveys

4. Analyzing Interviews and Focus Groups
Dr. Topeka Small

5. Coding, Simply
The first step to analyzing interview or focus group information is coding the data.
Coding, loosely defined, is the process by which information is categorized to facilitate analysis. Codes are assigned to information that is similar in nature or in the same category. For example, comments from a survey may fall under two different categories: positive and negative. Perhaps you could outline all positive information in green and negative information in red. In this case, color “coding” is used. However, other codes may be used. Plus and minus signs could be used as codes.
From here, information can be categorized into more narrow categories. For example, positive comments in green could be further coded as positive comments about the faculty, positive comments about the facilities, positive comments about the curriculum, etc. These comments could be delineated using another code like happy faces and sad faces. Keep in mind the “code” you use is not nearly as important as what the codes mean. Create a legend if your coding system is very intricate.
After coding the information as much as necessary, you can begin to identify the themes that emerge from the data. If you used color “codes”, a simple glance reveals if there are more positive comments than negative. Then, the next level of color “coding” could easily reveal if most of the positive comments are program-related, faculty-related, curriculum-related, etc.

6. Analyzing Curriculum Maps and Syllabi
Dr. Summer DeProw

7. Curriculum Maps: Horizontal Analysis

8. Curriculum Maps: Horizontal Analysis
Questions to ask if you find that a course/s does not contribute to any of the program- level outcomes:
Does this class need to be a part of the curriculum?
If the class is important to the program, should you reconsider your outcomes?
If the class is important to the program and the outcomes are accurate, does the class need to be updated or rethought?

9. Curriculum Maps: Vertical Analysis

10. Curriculum Maps: Vertical Analysis
Questions to ask if you find that a program-level outcome is not supported by any courses:
Is the outcome/s important enough to be a program-level outcome?
If the outcome/s is important, should the curriculum be restructured to support the outcome/s?

11. Syllabi Analysis Course Descriptions Program-Level Outcomes
Coordination with Curriculum Maps

12. Analyzing Survey Data
Dr. Wayne Wilkinson

13. Prerequisites for Data Analysis
Important qualities of survey questions:
Reliability: responses are due to attribute rather than error
Can be estimated and quantified
Validity: The questions measure what you think they are measuring
Associations with other measures (e.g., direct assessments)

14. Types of Data
The appropriate analysis depends on scale of measurement:
Nominal (numbers as categories)
Ordinal (rough categorical estimates)
Interval (continuous variables)
Ratio (continuous with absolute zero) 1 2 3 4
Freshman
Sophomore
Junior
Senior

15. Types of Data Analysis Data analysis can have multiple purposes:
Descriptive: Describing the phenomenon
Relational: Looking for relationships among variables
Inferential: Looking for differences between groups

16. Bringing It All Together
Scale of Measurement
Analysis Type
Common Procedures
Nominal/Ordinal
Descriptive
Counts/Frequencies
Relational
Phi (nominal) or Tau (ordinal) correlation coefficients
Inferential
Chi-square Tests
Interval/Ratio
Central tendency (mean, median, & mode)
Pearson correlation coefficient
t-tests (2 groups) and ANOVA (3+ groups)
Note: Other correlation coefficients exist for when the two variables being compared have different scales of measurement (e.g., point-biserial for nominal/interval data)