1. 1 ICTCM Pre session, Oct 2004 Assessment - overview G. Donald Allen Department of Mathematics Texas A&M University ICTCM – October 28, 2004

2. ICTCM Pre session, Oct 20042 Speakers Don Allen Mike Speed Dave Thomas Mike Pilant Diem Nguyen Greg Tobin

3. ICTCM Pre session, Oct 20043 What is a “Student Learning Outcome? Student learning outcomes are properly defined in terms of the knowledge,skills, and abilities that a student has attained at the end (or as a result) of his or her engagement in a particular set of education experiences. Council for Higher Education Accreditation

4. ICTCM Pre session, Oct 20044 The Purpose of an Assessment Is: to provide information about skills in a content area to guide instruction for the individual and for groups to provide feedback or information to plan further instruction to improve instructional practices to focus on what the student can do to guide decisions about instruction (Formative)

5. ICTCM Pre session, Oct 20045 The Purpose of a Test/Exam Is: to provide feedback about skills learned or obtained via instruction to determine the areas in which an individual needs re-teaching to provide grades to focus on what the student cannot do to indicate attainment of skills (Summative)

6. ICTCM Pre session, Oct 20046 Characteristics of Educational Measures Educational measures are –Indirect –Incomplete –Relative –Used as classification Educational measures have errors

7. ICTCM Pre session, Oct 20047 Desirable test attributes Reliability Validity Objectivity Balance Fairness Practicality Test length should be 1.Long enough to be reliable 2.Short enough to be administered

8. ICTCM Pre session, Oct 20048 Goals for making tests Be concise Be singular – focus on one aspect Be realistic Use definite terms Delineate expected behavior

9. ICTCM Pre session, Oct 20049 Blooms taxonomy 1.Knowledge (basic) 2.Comprehension 3.Applications 4.Analysis 5.Synthesis 6.Evaluation (advanced)

10. ICTCM Pre session, Oct 200410 Simplified Bloom’s for testing I.Knowledge (basic) II.Understanding – ability to restate, interpret III.Applications – beyond restatement to include analysis, synthesis, and evaluation

11. ICTCM Pre session, Oct 200411 Function of a test Advancement (criterion referenced) Achievement of objectives (norm referenced, items should discriminate well)

12. ICTCM Pre session, Oct 200412 Cognitive testing – another view Recall Recognition Differentiation Constructed response Problem solving

13. ICTCM Pre session, Oct 200413 Capacity to assess Ability Achievement Personality Skill Opinion Attitude

14. ICTCM Pre session, Oct 200414 Assessment What can it do? Provide –to students a chance to reflect on what they have learned and need to learn –to teachers a chance to reflect on what they have taught well and what needs further attention

15. ICTCM Pre session, Oct 200415 Assessment What can it do? Provide feedback –to the teacher on the clarity of given assessments –to teachers about the progress of learning

16. ICTCM Pre session, Oct 200416 Assessment What can it do? Providing –clarity to students on the type of mathematics knowledge valued –clarity to students on the type of mathematics proficiency valued

17. ICTCM Pre session, Oct 200417 Assessment What can it do? Become a routine part of ongoing classroom activity Promote regular class attendance Promote multiple solutions and approaches, giving a well rounded picture that allows each student to show his/her best strengths

18. ICTCM Pre session, Oct 200418 What Counts as Evidence for Student Learning Outcomes? “Evidence” refers to the kinds of information about student learning outcomes that is most appropriate to accreditation settings. Council for Higher Education Accreditation

19. ICTCM Pre session, Oct 200419 Evidence vs. Anecdotes However… the plural of anecdote is not evidence! “Evidence” can simultaneously embrace the results of quantitative and qualitative approaches to gathering information

20. ICTCM Pre session, Oct 200420 Types of evidence Comprehensive examinations and assignments. Performance on licensing examinations. Professionally judged performances. Portfolios of student work. Samples of representative student work.

21. ICTCM Pre session, Oct 200421 What is a “Standard” of Student Learning? “Standard" refers to a specific expectation or level of performance that an institution or program establishes for student learning. Standards are the point of comparison against which to judge the actual evidence of student learning once it is collected.

22. ICTCM Pre session, Oct 200422 Assessment Capacity to Assess Achievement Personality Skill Opinion Attitude

23. ICTCM Pre session, Oct 200423 Testing for what? PREDICTION EVALUATION DIAGNOSIS SCREENING

24. ICTCM Pre session, Oct 200424 Assessment Weaknessess Lack of content validity Lack of scoring economy Scorer unreliability (leniency, severity) Halo effect Informed judgement Extraneous factors (quality of hand-writing, pen vs. pencil, spelling errors)

25. ICTCM Pre session, Oct 200425 Essay questions Item types –Short answer –Extended answer Item Construction –Focus the direction of the response

26. ICTCM Pre session, Oct 200426 Guidelines for essay questions Allow adequate time Should have a problem to be solved –Problem should be defined explicitly –Problem should be limited Directions should be explicit Don’t use optional questions Construct detailed key for each question

27. ICTCM Pre session, Oct 200427 Assessment – It is… Broadly defined and used Complex Detailed Wrought with unfairness Essential Different in different contexts

28. ICTCM Pre session, Oct 200428 Engines ALTEC – online tools for teachers http://altec.org/index.php –QuizStar –Rubistar –Profiler Pro Survey Monkey - http://www.surveymonkey.com/ Blackboard, WebCT

29. ICTCM Pre session, Oct 200429 Forms of Assessment Collaborative/group projects Direct observation Essays Examinations (unseen), Examinations (seen/open book) Multiple-choice tests, Oral questioning after observation Performance projects, Portfolios Practical projects, Presentations Problem sheets, Projects Self-assessment, Case-studies Short-answer questions

30. ICTCM Pre session, Oct 200430 Two simple assessments

31. ICTCM Pre session, Oct 200431 The Muddiest Point What point was least clear during this session? What point needs further clarification? What point prevented me from learning fully the contents of this session?

32. ICTCM Pre session, Oct 200432 The one minute paper 1.What was the most useful or meaningful thing you learned during this session? 2.What questions remain uppermost in your mind as we end this session? Answer each question in one or two sentences.