3/12/20031 RUSH TO ASSESSMENT: Designing Assessment to Produce Failure Roberta Madison, Gloria Melara and Robert Lingard

3/12/20032 Session Overview An example of a recent assessment project at CSUN A group activity designed to gain insight into effective approaches to assessment Summary and review of group results

3/12/20033 Session Objectives At the end of the session participants should be able to: Identify potential problems in designing outcomes assessment Select/create appropriate assessment measures and tools Identify the steps in designing a productive assessment project

3/12/20034 Common Problems in Assessment Assessment results are often needed quickly Insufficient resources (people, tools, etc.) are usually applied Assessment is sometimes viewed as an end in itself (e.g., an accreditation team must be convinced that assessment is being done)

3/12/20035 An Assessment Example Examine an approach taken toward assessing Computer Literacy at CSUN Look at the difficulties and problems encountered Suggest some recommendations for improving the assessment process

3/12/20036 Why Was Computer Literacy Chosen for Assessment? Ensuring the Computer Literacy of students was viewed as an important University goal The effectiveness of CSUN’s general education course in Computer Literacy was unknown

3/12/20037 The Approach Taken To get results quickly, an existing instrument, TekXam, was selected to measure student’s computer skills Students taking the General Education course in Computer Literacy and a control group not taking this course were given pre tests and post tests The results were analyzed

3/12/20038 Study Results

3/12/20039 TekXam Passing Rates

3/12/ Completion Rates

3/12/ Problems Encountered The TekXam instrument did not test what our Computer Literacy course covered TekXam was difficult for students to use Administration of TekXam was tedious and rigid Student participation rate was low

3/12/ Conclusions and Recommendations A clear definition of Computer Literacy is needed before it can be assessed Clear (measurable) objectives need to be established for the assessment Incentives are needed to increase student participation Better assessment tools are needed

3/12/ A Sample Problem Assume you are given the task of assessing the effectiveness of online instruction for a particular program What needs to be done?

3/12/ First Group Task (Risk Analysis) Identify potential problems or obstacles that might be encountered in trying to do this assessment List them in order of importance (high probability & high impact)

3/12/ Problems/Obstacles Identified

3/12/ Second Group Task For one of the problems identified suggest possible solutions List the advantages and disadvantages of each solution

3/12/ How Can We Ensure a Successful Assessment? Make sure the goal of the assessment is clearly understood Establish measurable objectives toward achieving the goal Determine how to measure each objective (in multiple ways)

3/12/ GOAL OBJECTIVE MEASURE

3/12/ GOAL OBJECTIVE DATA ANALYZE CHANGES? MEASURE Obstacles

3/12/ Final Group Task Suppose your task was to assess the effectiveness of online instruction in teaching written composition Specify one measurable objective and identify two or more ways to measure it.

3/12/ Measurable Objective Example Students will be able to write a grammatically correct English sentence with a subject and a verb.

3/12/ Conclusions A successful assessment project must be carefully planned with sufficient resources allocated (we must avoid rushing to assessment) The goal or goals of the assessment must be clearly understood Specific measurable objectives must be established Appropriate tools must be found/created to measure each objective in multiple ways

3/12/ References A short class feedback/assessment. It assesses students’ perception of the classroom environment This assessment activity clarifies expectations and mutual responsibilities for student learning and success. An extensive list of many college assessment sites from North Carolina State University Nine Principles of Good Practice for Assessing Student Learning from AAHE, Alexander W. Astin, et.al A good resource with information from classroom to institutional assessment

3/12/ Contact Information Roberta Madison - Gloria Melara - Robert Lingard -