1. Developing MCQ test items for the Competence Based Curriculum
Grade 6 Summative Assessment Workshop
Developing MCQ test items for the Competence Based Curriculum
October

2. To enable summative assessment writers to:
Course Objectives
To develop an understanding of summative assessment and the need for assessments to exemplify good technical qualities.
To enable summative assessment writers to:
know and follow appropriate principles for developing and reviewing summative assessments
be able to use these assessment methods in teaching,
identify and accommodate the limitations of different informal and formal assessment methods.
develop improved Grade 6 assessments for end of semester assessments.
namely:
cognitive complexity,
content quality,
meaningfulness,
language appropriateness,
ability to make general statements about student achievement from the results,
fairness, and reliability.

3. Workshop Agenda
Types of assessments in the competence based curriculum
Reviewing Test Items
Spot Testing
Categories of test item Types
Data entry
General Tips about Testing
Data Analysis
Establishing the Technical Quality of a Test
Report Writing
Multiple Choice item Types
Producing a Reading Test

4. Types of assessments Task
Think
Pair
Share
Discuss
Task
Types of assessments
Q1: Name types of assessments used in the competence based curriculum
Q2: What are the purposes of each type?

5. Types of assessments in schools
Diagnostic
Before Learning
Formative
While Learning
Summative
After Learning

6. Task 2 Think-Pair-Share These questions might help you:
How would you articulate your own view of summative assessment?
These questions might help you:
What is Summative assessment?
What is it used for?
What results from summative assessment?
Who can benefit from this information?
Were these some of the answers?
Formal examinations, finding out what students know; whether students could do things. Giving students a test or about checking their work. Recording the marks they have achieved in an assignment or about how the marks would be entered into a mark book. Conversation with a student or how they felt about something, or what they thought about their learning

7. Performance Test Items
Types of questions
Objective
True & False
Matching
Completion
MCQs
Subjective
CRQs
Problem Solving
Performance Test Items
Open End questions
Extended responses
Short Answers

8. Reflection
“What makes a test good is that good tests serve the purpose they are deigned for”
2 key purposes for assessment in a competence based curriculum are”
What do we measure?
It should be noted that it is always tempting to emphasize the parts of the curriculum that are easiest to test, rather than the parts that are important to test.
Improve Teaching
Improve Learning

9. Task 4
Look at the test items in the test for grade 6 and decide which competence is tested and the subject skills according to level of difficulty for each item.
*You will need to refer to the Kuwait Assessment Guide and the competences document
After this task teachers will reflect on the tests

10. Is it repeated in the assessment?
Task 4
Review the test , identify the specific competences in lights of the curriculum standards and
Specific Competences
Types used
Is it repeated in the assessment?
Why?

11. When preparing a summative test
* Kuwait National Curriculum and Standards document – Grade 6
*Exam framework
* Test Specification table
*Synopsis
*Intermediate Stage Curriculum Document
* Curriculum standards descriptors
You should consult
Summative assessment test should be written according to the competences standards not just the text book content.

12. General Tips About Testing
Length of Test
Clear, Concise Instructions
Mixing Different Item Types
Check for Accuracy
Length of Test
In theory, the more items a test has, the more reliable it is. On a short test a few wrong answers can have a great effect on the overall results. On a long test, a few wrong answers will not influence the results as much. A long test does have drawbacks. If a test is too long, and particularly if students are doing the same kind of item over and over, they may get tired and not respond accurately or seriously. If a test needs to be lengthy, it should be divided it into sections with different kinds of tasks, to maintain the student's interest.
Clear, Concise Instructions
It is necessary to give clear, concise instructions. It is useful to provide an example of a worked problem, which helps the students understand exactly what is necessary. What seems to be clear to the item writer may be unclear to someone else.
Mixing Different Item Types
It is often advantageous to mix types of items (multiple choice, true-false, CRQ) on a written examination or to mix types of examinations (a performance component with a written component). Weaknesses connected with one kind of item or component or in students’ test taking skills will be minimized.
Check For Accuracy
There is a need to be cautious about using tests written by others. Items developed should be checked for accuracy and appropriateness.

13. Criteria for Establishing the Technical Quality of a Test
Cognitive Complexity – the levels of learning to be tested – recall of information, basic reasoning, complex reasoning, extended reasoning.
Content Quality – the test specification – skills to be tested, number of questions, number of sections, testing format.
Meaningfulness - students should not need to “guess” what is to be included in the test.
Language Appropriateness – should reflect classroom language
Results enable Generalisations to be made
Fairness
Reliability so as to encourage learning

14. Criteria for Establishing the Technical Quality of a test.
1. Cognitive Complexity.
Recall information : know vocabulary ,recall , recognize , define , translate , know basic concepts
Basic reasoning: make decisions, identify and organize, extend patterns, identify/describe cause and effect, make inferences , interpret, compare, contrast
Complex reasoning: solve problems ,create a pattern, generalize from given facts, make choices using reasoning, draw conclusions, make predictions, verify the value of evidence , information
ExtendedReasoning:analyze,synthesize,examine,explain,apply, design, ,combine and synthesize ideas into new concepts.
In the Kuwait Assessment Guide, four levels of intellectual understanding are listed

15. Multiple Choice Questions
(MCQs)

16. Write- Pair - Share
Using Grade 6 textbook, Write 5 MCQs based on the different levels of difficulty

17. Multiple Choice Items
Read the question below, identify the possible answers and select the correct ONES.
Which ability(ies) may Multiple Choice Questions be used to test?
Recalling memorized information;
Applying theory to known things;
Applying theory to new situations;
Using judgment in analyzing and evaluating.
The multiple choice item consists of the stem, which identifies the question or problem and the response alternatives or choices. Usually, students are asked to select the one alternative that best completes a statement or answers a question.

19. Tips for developing good MCQs;
State the stem as a direct question, rather than an incomplete statement.
The stem should be meaningful (present a problem – clear instruction).
MCQs do not have blanks.
A well-constructed MCQ does not have negative stem and it does not include words such as the least/the worst.
MCQs should not be tricky.
Distractors and answer should roughly have the same length.
Choices should not include none of the above /all of the above.
Each question should have only one answer.

20. Write- Pair - Share
Using Grade 6 textbook, Write 3 MCQs based on the different levels of difficulty

21. Sample of MCQs Choose the correct answer:
1- What is a suitable title for the text?
Water: How to Save
Water: the Source of Life
Keeping Water Safe
Building Desalination Plants
2- How long can a person live without food?
For about seven days
For nearly ninety days
For more than thirty days
For less than seven days
Teachers will review the questions and discuss

22. Choose the correct answer:
3- There is one similar thing between Kuwait Towers and Water Towers which is
a. They are both famous
b. They both hold water
c. They are both beautiful
d. They both have cafes
4- Why is water important to us?
---- Swim
---- Survive
---- Play
---- Eat
Are these meant to be bad examples of MCQs that you will use for explaining how to develop a good MCQ using the criteria provided in the Test development guidelines?

23. Reading Comprehension Characters, setting, events
Conclusions
Generalizations
Main Idea/ details
Cause/ Effect
Facts/ Opinions
Character Traits
Summarize
Sequence of events
verify/ predict
Characters, setting, events
Predicting
Developing MCQs
for a reading text

24. Choosing Reading Texts
It is essential that texts are of value in their own right. A good test cannot be written using a poor text. The text chosen should not be from material which is widely used for teaching purposes. The text should be:
Intrinsically interesting and relevant to the Stage being assessed;
Linguistically accessible to students at a particular grade;
Complete in itself;
Well-written;
Well-structured.

25. Different Types of Reading Texts which can be used for Testing
Examples of Text Type
Descriptive
Report
Recount
Story
Leaflets
Letters
Informative
Explanatory Text
Reports
Recounts
Newspaper accounts
Types of Reading Texts
Examples of Text Type
Instructive
Story Planner
Instructions
Explanatory Texts
Persuasive
Advertisements
Argumentative Texts
Leaflets
Letters

26. What is a Flesch-Kincaid score?
Readability Test
What is a Flesch-Kincaid score?
 The Flesch-Kincaid readability score uses the sentence length (number of words per sentence) and the number of syllables per word in an equation to calculate the reading ease. Texts with a very high Flesch-Kincaid reading ease score (about 100) are very easy to read. They have short sentences and no words of more than two syllables.

27. How do I check Felsch Kincaid on Microsoft Word?
Click the Microsoft Office Button, and then click Word Options.
Click Proofing.
Make sure Check grammar with spelling is selected.
Under When correcting grammar in Word, select the Show readability statistics check box.

28. Scores can be interpreted as shown below:
notes
School level
Score
Very easy to read. Easily understand by an average 11 years old
5th grade
90-100
Easy to read.
6th grade
80-90
Fairly easy to read
7th grade
70-80
Plain English. Easily understood by yrs
8th & 9th grade
60-70
Fairly difficult to read
10th -12th grade
50-60
Difficult to read
College
30-50
Very difficult to read
College graduate
0-30

29. Group Activity
After choosing a suitable reading text for grade 6, develop the text to a test and write 10 MCQs
Revise & Edit

30. Review test items developed
Curriculum standard
Stem
Distractors
Model answer
Self review
Peer Review
Group
Open discussion
Whole group review
What are your common findings of reviewing MCQ items developed?

32. Spot Test Development Process
Review SC and SC
Review framework
Design
Develop
Review
Pilot
Analyze
Decide
Review test specifications
Reject
Edit
Item Bank
self
peer
Distractors
Difficulty
group
Discrimination
Administer
Mark
Entry

33. Practicum Choosing a sample: Why?
40 students from 5 different classes taught by 5 different teachers (preferable),
Boys and girls schools should be represented.
8 students chosen from each class ranked as:
2. Below Average Average
2. Above average Excellent
Explain to the students, the purpose of the short test – to check how good the questions are.
Tell them that their names will not be on the test.
Answer students’ procedural questions only
Tell the students that if they do not understand a question they should tell the administrator about what they find difficult. Make notes of any questions related to the test questions themselves. These notes will help you improve the items.
Next to each individual student identifier write the starting time of the spot test.
Next to each individual student identifier write the finishing time of the spot test.
Make a note of the students’ difficulties by identifying the question number and also the student identifier.
If you observe that some of the students are not answering the questions but are not asking for support, ask them why and make a note of their verbal answers by using the Question number and the student identifier.
Why?

34. Test Analysis
check the Item Discrimination, Facility levels and Performance of Distractors.

36. Discussing spot test analysis

37. Analysing spot test Difficulty Index:
Keep questions between ( p p 0.30)
Discrimination Index:
Keep questions with a positive discrimination value (above ”0” to +1)
Quality of Distractors:
It is recommended to change the distractors that were not chosen by students “0” as they do not represent common mistakes

38. Discussing the results and analysis of Spot testing
Group Activity
Discussing the results and analysis of Spot testing
What conclusions could be derived form the students’ results?
Writing a report including conclusions and recommendation

39. Retaining the good items in the test bank
Group Activity
Decision Making:
Retaining the good items in the test bank
Amending some items and spot testing again.
Discarding the items that do not qualify.

40. Submitting good question items in Test Bank

41. Thank You