1. 3/12/20031 RUSH TO ASSESSMENT: Designing Assessment to Produce Failure Roberta Madison, Gloria Melara and Robert Lingard

2. 3/12/20032 Session Overview An example of a recent assessment project at CSUN A group activity designed to gain insight into effective approaches to assessment Summary and review of group results

3. 3/12/20033 Session Objectives At the end of the session participants should be able to: Identify potential problems in designing outcomes assessment Select/create appropriate assessment measures and tools Identify the steps in designing a productive assessment project

4. 3/12/20034 Common Problems in Assessment Assessment results are often needed quickly Insufficient resources (people, tools, etc.) are usually applied Assessment is sometimes viewed as an end in itself (e.g., an accreditation team must be convinced that assessment is being done)

5. 3/12/20035 An Assessment Example Examine an approach taken toward assessing Computer Literacy at CSUN Look at the difficulties and problems encountered Suggest some recommendations for improving the assessment process

6. 3/12/20036 Why Was Computer Literacy Chosen for Assessment? Ensuring the Computer Literacy of students was viewed as an important University goal The effectiveness of CSUN’s general education course in Computer Literacy was unknown

7. 3/12/20037 The Approach Taken To get results quickly, an existing instrument, TekXam, was selected to measure student’s computer skills Students taking the General Education course in Computer Literacy and a control group not taking this course were given pre tests and post tests The results were analyzed

8. 3/12/20038 Study Results

9. 3/12/20039 TekXam Passing Rates

10. 3/12/200310 Completion Rates

11. 3/12/200311 Problems Encountered The TekXam instrument did not test what our Computer Literacy course covered TekXam was difficult for students to use Administration of TekXam was tedious and rigid Student participation rate was low

12. 3/12/200312 Conclusions and Recommendations A clear definition of Computer Literacy is needed before it can be assessed Clear (measurable) objectives need to be established for the assessment Incentives are needed to increase student participation Better assessment tools are needed

13. 3/12/200313 A Sample Problem Assume you are given the task of assessing the effectiveness of online instruction for a particular program What needs to be done?

14. 3/12/200314 First Group Task (Risk Analysis) Identify potential problems or obstacles that might be encountered in trying to do this assessment List them in order of importance (high probability & high impact)

15. 3/12/200315 Problems/Obstacles Identified

16. 3/12/200316 Second Group Task For one of the problems identified suggest possible solutions List the advantages and disadvantages of each solution

17. 3/12/200317 How Can We Ensure a Successful Assessment? Make sure the goal of the assessment is clearly understood Establish measurable objectives toward achieving the goal Determine how to measure each objective (in multiple ways)

18. 3/12/200318 GOAL OBJECTIVE MEASURE

19. 3/12/200319 GOAL OBJECTIVE DATA ANALYZE CHANGES? MEASURE Obstacles

20. 3/12/200320 Final Group Task Suppose your task was to assess the effectiveness of online instruction in teaching written composition Specify one measurable objective and identify two or more ways to measure it.

21. 3/12/200321 Measurable Objective Example Students will be able to write a grammatically correct English sentence with a subject and a verb.

22. 3/12/200322 Conclusions A successful assessment project must be carefully planned with sufficient resources allocated (we must avoid rushing to assessment) The goal or goals of the assessment must be clearly understood Specific measurable objectives must be established Appropriate tools must be found/created to measure each objective in multiple ways

23. 3/12/200323 References A short class feedback/assessment. It assesses students’ perception of the classroom environment. http://www.cod.edu/outcomes/SoaClsDs/StopStartCont_Assessment.PDF http://www.cod.edu/outcomes/SoaClsDs/StopStartCont_Assessment.PDF http://www.cod.edu/outcomes/SoaClsDs/feedback.PDF This assessment activity clarifies expectations and mutual responsibilities for student learning and success. http://home.att.net/~p.klassen/SuccessAttribution.htm An extensive list of many college assessment sites from North Carolina State University http://www2.acs.ncsu.edu/UPA/assmt/resource.htmhttp://www2.acs.ncsu.edu/UPA/assmt/resource.htm Nine Principles of Good Practice for Assessing Student Learning from AAHE, Alexander W. Astin, et.al http://www.aahe.org/assessment/principl.htm http://www.aahe.org/assessment/principl.htm A good resource with information from classroom to institutional assessment http://www.cod.edu/Dept/Outcomes/AssessmentBook.pdf http://www.cod.edu/Outcomes/

24. 3/12/200324 Contact Information Roberta Madison - roberta.madison@csun.edu Gloria Melara - gem@ecs.csun.edu Robert Lingard - rlingard@ecs.csun.edu