Gen Ed Assessment Critical Thinking Outcome Multiple Choice Question (MCQ) Development Project in the Social Sciences BASED ON SLIDES FROM DEC. LAURA BLASI, PH.D., DIRECTOR, INSTITUTIONAL ASSESSMENT

Critical Thinking – The Gen Ed Outcome  When testing Critical Thinking in the General Education program at Valencia we are focused on the three indicators addressing – (1) bias, (2) use of evidence, and (3) context. A pattern in faculty work focused on Critical Thinking since 2002.

Accomplishment 1 – Materials and Website – Valencia College Gen Ed Faculty Resources Specific to Critical Thinking  planning/institutional-assessment/loa/ResourcesCriticalThinking.cfm planning/institutional-assessment/loa/ResourcesCriticalThinking.cfm

Purpose of the Multiple Choice Question (MCQ) Project The current MCQ project : (1) invests the money in our faculty and a homegrown test item bank that is emerging from our experience with the students in Gen Ed; (2) increases the college’s capacity for reliability testing (in our IR office) moving away from reliance on consultants; (3) assures that faculty concerns about external reliability using pilot data are addressed by recognized experts in the field; (4) provides an option after Social Science faculty discovered that nationally normed example can be $6 per student or higher.

Standards for reviewing the questions specific to the outcome

Different forms of questions are possible…. Examples can be taken from the standardized tests used across the country – for example –  excerpt of a study

excerpt dialogue, speech, or a current event

 Provide a premise and test student assumptions about the idea

Applying the Standards Imagine you are a 2 nd year student… Notice the question begins with reference to Political Science but it is broad enough to be accessible to students who have not taken Political Science.

Bias (analyze others and one’s own)

Context (beyond knowing it is important – examine its relevance when presenting ideas)

Bias (beyond recognizing it, analyzing it)

Evidence

Next steps needed  Questions by Jan 31  Pilot February – March  Expert Analysis April  Discussion of Results Assessment Day  50 questions to develop (we can include review of those we have)  Work is distributed and faculty-led  Questions stand up to peer review applying external standards  So our “Self Test” questions hold up when applied to the items (internal standards.)  The MCQ creation strategies – not discipline specific - from the Steve Downing workshop are followed and adhered to (external standards.)  Timeline for the pilot takes into account student administration and validation study by USF

Dr. Steve Downing Tips for Developing Multiple Choice Questions Across Disciplines (examples)  Write a clear “testing point” or objective for item [context, bias, evidence]  Pose a clear question - review, edit, rewrite  Focus on important/essential information  Assure that question can be answered without reading options  Write clear, concise items; avoid superfluous information  Include most information in stem, avoiding lengthy options  Don’t use trick questions  Test higher-order cognitive knowledge (he refers to Bloom’s Taxonomy) Application, problem solving, judgment, synthesis

 Questions?