1. Οξούζογλου Λεωνίδας Επιβλέπων Καθηγητής: Οικονομίδης Αναστάσιος Εξεταστής 1: Σατρατζέμη Μαρία Εξεταστής 2: Ξυνόγαλος Στυλιανός ΔΙΑΤΜΗΜΑΤΙΚΟ ΠΡΟΓΡΑΜΜΑ ΜΕΤΑΠΤΥΧΙΑΚΩΝ ΣΠΟΥΔΩΝ ΣΤΑ ΠΛΗΡΟΦΟΡΙΑΚΑ ΣΥΣΤΗΜΑΤΑ STUDENTS’ ATTITUDES TOWARDS ALTERNATIVE MULTIPLE CHOICE QUESTIONS

2.  objective  easy to score  cost-effective (especially in large scale assessment)  sampling of content can be broader  items can be stored in item-banks and reused  the use of large number of items is possible  individual test items can be subjected to statistical analysis

3.  more difficult to construct  when used for assessing higher levels, the reliability is questioned  the examinee might be exposed to acquiring false knowledge  the score might be affected by the use of guessing  lack of assessment of partial knowledge

4. Developed to address the problem of guessing and to assess partial knowledge

5. the scoring of a test just by counting the number of correct answers if a wrong option is chosen, the score of the item is 0

6. where k, the number of choices in an item Scoring Rule

7. the examinee, in case of an incorrect choice, is permitted to choose one of the remaining options. this procedure is repeated until the correct option is selected.

8. S(k,a) = k – a where k is the number of choices in an item and a is the number of attempts Scoring Rule

9. (Coombs et al., 1956) the student has to indicate the wrong options of an item

10. Scoring Rule for MCQs with 4 options

11. the examinee is permitted to choose more than one options

12. General Scoring Rule: the larger the number of options selected the lesser the score credited to the item

13. Liberal Tests (Bush,2001) where k is the number of options in a MCQ and a is the number of options selected by the examinee

14. Zeidner (1990) the majority of students believe that essay type exams are: more reflective of students' true achievements, more suitable for assessing course objectives, generally fairer than MCQs students find MCQ type exams : easier and less time consuming to prepare for feel less time-pressure when taking the exams

15. Birenbaum & Feldman (1998) students with good learning skills who have high confidence in their academic ability, tend to prefer the constructed-response type of assessment over the MCQ type students with high test-anxiety tend to prefer the MCQ testing format

16. Birenbaum (1997); Furnham et al. (2008); Gijbels & Dochy (2006) students’ approaches to learning have a significant effect on assessment preference

17. what are the students’ preferences and opinions regarding assessment using MCQs ? how do factors like gender, grade of education, school performance and course preference affect students’ preference in a certain method ? what is the relation, if any, of student’s preference in an assessment method and his/her approach to learning ?

18. Participants: 178 students from the area of Thessaloniki 96 students of secondary education 82 university students

19. Participants: 178 students from the area of Thessaloniki 100 females 78 males

20. Questionnaires : Revised two-factor learning process questionnaire (R-LPQ-2F, Kember et al., 2004) Revised two-factor study process questionnaire (R-SPQ-2F, Biggs et al., 2001)

21. Questionnaires : Preferred Assessment Method Methods of Assessment Essay type questions. Number of Right (NR). Negative Marking (NM) Answer Until Correct (AUC). Elimination Testing (ET) Subset Selection Testing (SST)

22. Questionnaires : Preferred Assessment Method Indicate how fair you think is this method. Indicate how accurate you think is this method in measuring each person’s level of knowledge. Indicate how stressful for the student you think this method is. Indicate the level of your personal preference in this method. (9-point Likert – type scale)

23. Procedure: The students were given the questionnaires in class, under test conditions and given adequate time to think and respond. The assessment methods had been thoroughly explained to them in a previous meeting. A written explanation of the methods was given to them along with the questionnaires.

24. Correlations between perceived fairness and level of preference were positive and significant at the 0.01 level Correlations between perceived accuracy and level of preference were also positive and significant at the 0.01 level Correlations between perceived student anxiety and level of preference were negative and not all of them significant Correlations between perceived fairness and perceived accuracy were computed and found to be positive and significant at the 0.01 level

25. Method of Assessment Level of Preference Perceived Fairness Perceived Accuracy Perceived Student Anxiety Essay Type Mean5.41 5.836.786.55 SD2.596 2.3422.2072.456 Number of Right Mean6.21 5.975.804.98 SD2.115 2.1052.1082.052 Negative Marking Mean3.20 3.504.476.73 SD2.195 2.3912.4102.095 Answer Until Correct Mean4.85 4.683.864.06 SD2.382 2.4922.2912.161 Elimination Testing Mean5.03 5.344.914.47 SD2.231 2.0862.0432.174 Subset Selection Testing Mean5.01 4.764.574.90 SD2.669 2.6922.5062.370

26. Essay Type Level of Preference NR Level of Preference NM Level of Preference AUC Level of Preference ET Level of Preference SST Level of Preference Essay Type Level of Preference Pearson Correlation 1 Sig. (2-tailed) NR Level of Preference Pearson Correlation,0361 Sig. (2-tailed),631 NM Level of Preference Pearson Correlation -,067,0271 Sig. (2-tailed),373,721 AUC Level of Preference Pearson Correlation -,310 ** -,025,150 * 1 Sig. (2-tailed),000,740,047 ET Level of Preference Pearson Correlation -,275 ** -,104,093,451 ** 1 Sig. (2-tailed),000,168,215,000 SST Level of Preference Pearson Correlation -,158 * -,143,171 *,200 **,290 ** 1 Sig. (2-tailed),036,057,022,008,000 Correlations between Level of Preference in different methods

27. Grade SecondaryUniversity Mean Standard DeviationMean Standard Deviation Essay Type Level of Preference 5,222,6535,632,526 NR Level of Preference * 5,862,0196,622,165 NM Level of Preference 3,192,1683,222,239 AUC Level of Preference 4,922,1994,772,595 ET Level of Preference 5,142,2084,902,264 SST Level of Preference 5,322,6184,632,697 * the difference of mean values is significant at the 0.05 level Level of Preference categorized by grade of education

28. Essay Type Level of Preference NR Level of Preference NM Level of Preference AUC Level of Preference ET Level of Preference SST Level of Preference Deep Aproach Scale Score (%) Pearson Correlation,185 *,190 *,026-,033,048,122 Sig. (2-tailed),013,011,730,661,524,106 Surface Aproach Scale Score (%) Pearson Correlation -,071-,212 ** -,074,094,004,186 * Sig. (2-tailed),343,005,327,214,961,013 Correlations between Level of Preference in each method and learning approach scale scores

29. Learning Approach and Gender factors in Level of Preference in Essay Type (significant at the 0.01 level) and SST (significant at the 0.05 level). ANOVA analysis significant interactions between:

30. ANOVA analysis Tests of Between-Subjects Effects Dependent Variable: Essay Type Level of Preference Source Type III Sum of SquaresdfMean SquareFSig. Corrected Model62,678 a 320,8933,216,024 Intercept4727,7201 727,738,000 LearnApp23,3341 3,592,060 Gender,0241,004,952 LearnApp * Gender44,0671 6,783,010 Error1130,3841746,496 Total6403,000178 Corrected Total1193,062177 a. R Squared =,053 (Adjusted R Squared =,036) Learning Approach * Gender Dependent Variable: Essay Type Level of Preference Learning ApproachGenderMean Std. Error 95% Confidence Interval Lower Bound Upper Bound DAFemale5,172,3354,5125,833 Male6,224,3645,5066,943 SAFemale5,452,3934,6766,229 Male4,448,4733,5145,382 Level of Preference in Essay Type

31. ANOVA analysis Tests of Between-Subjects Effects Dependent Variable: SST Level of Preference Source Type III Sum of SquaresdfMean SquareFSig. Corrected Model58,686 a 319,5622,831,040 Intercept4382,7151 634,273,000 LearnApp30,7481 4,450,036 Gender6,1031,883,349 LearnApp * Gender35,2531 5,102,025 Error1202,3091746,910 Total5721,000178 Corrected Total1260,994177 a. R Squared =,047 (Adjusted R Squared =,030) Learning Approach * Gender Dependent Variable: SST Level of Preference Learning ApproachGenderMean Std. Error 95% Confidence Interval Lower Bound Upper Bound DAFemale4,966,3454,2845,647 Male4,429,3763,6875,170 SAFemale4,905,4064,1045,705 Male6,207,4885,2437,170 Level of Preference in SST

32. Learning Approach and Grade factors in Level of Preference in SST (significant at the 0.05 level). ANOVA analysis significant interactions between:

33. ANOVA analysis Tests of Between-Subjects Effects Dependent Variable: SST Level of Preference Source Type III Sum of SquaresDfDfMean SquareFSig. Corrected Model57,943 a 319,3142,793,042 Intercept2987,3531 432,067,000 LearnApp1,6931,245,621 Grade21,2371 3,072,081 LearnApp * Grade27,4451 3,969,048 Error1203,0521746,914 Total5721,000178 Corrected Total1260,994177 a. R Squared =,046 (Adjusted R Squared =,030) Learning Approach * Grad e Dependent Variable: SST Level of Preference Learning ApproachGradeMean Std. Error 95% Confidence Interval Lower Bound Upper Bound DASecondary4,650,4163,8295,471 University4,761,3214,1275,395 SASecondary5,804,3515,1106,497 University4,067,6792,7275,407 Level of Preference in SST

34. Learning Approach, Gender and Grade factors in Level of Preference in NR (significant at the 0.05 level). ANOVA analysis significant interactions between:

35. ANOVA analysis Level of Preference in NR Tests of Between-Subjects Effects Dependent Variable: NR Level of Preference Source Type III Sum of Squaresdf Mean SquareFSig. Corrected Model71,435 a 710,2052,409,022 Intercept3922,0771 925,960,000 LearnApp22,6741 5,353,022 Gender4,6931 1,108,294 Grade,1461,035,853 LearnApp * Gender,0051,001,973 LearnApp * Grade18,5051 4,369,038 Gender * Grade8,1141 1,916,168 LearnApp * Gender * Grade17,0281 4,020,047 Error715,8311694,236 Total7611,000177 Corrected Total787,266176 a. R Squared =,091 (Adjusted R Squared =,053) Learning Approach * Gender * Grade Dependent Variable:NR Level of Preference Learning ApproachGenderGradeMean Std. Error 95% Confidence Interval Lower Bound Upper Bound DAFemaleSecondary6,278,4855,3207,235 University6,925,3256,2837,567 MaleSecondary5,636,4394,7706,503 University6,769,4045,9727,566 SAFemaleSecondary5,419,3704,6906,149 University6,000,6214,7757,225 MaleSecondary6,320,4125,5077,133 University4,2501,0292,2196,281

36. Correlations between Level of Preference in each method and school performance Essay Type Level of Preference NR Level of Preference NM Level of Preference AUC Level of Preference ET Level of Preference SST Level of Preference Last Year’s Final Grade Pearson Correlation,269 **,124-,042-,186-,064-,228 * Sig. (2-tailed),008,227,681,069,535,025

37. Level of Preference categorized by Preference in Mathematics Preference in Maths NoYes MeanSDSD SD Essay Type Level of Preference4.792.7105.562.582 NR Level of Preference5.482.1446.171.881 NM Level of Preference3.022.2473.312.118 AUC Level of Preference4.572.1205.192.241 ET Level of Preference5.382.3474.942.096 SST Level of Preference5.762.5554.982.639 Preference in Science NoYes MeanSDMeanSD Essay Type Level of Preference6.003.0004.912.460 NR Level of Preference5.591.9865.972.036 NM Level of Preference3.042.0473.252.226 AUC Level of Preference4.521.9685.072.277 ET Level of Preference5.072.4335.162.133 SST Level of Preference5.112.2075.412.772 Level of Preference categorized by Preference in Science

38. Level of Preference categorized by Preference in Language Preference in Language NoYes MeanSDMeanSD Essay Type Level of Preference4.932.6005.632.705 NR Level of Preference*6.301.9905.251.918 NM Level of Preference3.382.2452.932.055 AUC Level of Preference*5.302.3044.381.944 ET Level of Preference4.982.1535.352.293 SST Level of Preference5.522.6565.052.572 * the difference of mean values is significant at the 0.05 level

39. Fairness and accuracy seem to be significant variables in shaping students’ level of preference. Less significant role seems to play the anxiety that a method brings to a student. MCQs with NR scoring scheme is the students’ most preferred method The least preferred method of assessment is MCQs with a NM scoring scheme

40. Students who have a higher level of preference in one of the alternative methods (AUC, ET and SST) tend to have high level of preference in the remaining two. Students that prefer Essay Type questions have a lower level of preference in alternative methods.

41. the deeper the approach a student adopts the more he/she prefers conventional method of assessment as Essay Type and NR MCQs

42. factors like Gender, Grade of Education or Learning Approach have an impact on students’ preference in assessment method students with a high school performance tend to favor Essay Type Questions while those with a lower performance tend to favor the SST method preference in certain courses (Mathematics, Science and Language) is related with a preference in some methods of assessment

43. Future research the study of students’ attitudes before and after testing using each method

44. The Goal PROVIDING EDUCATORS WITH METHODS OF ASSESSMENT USING MCQs, that: Reduce the effect of guessing Assess partial knowledge Have the acceptance of students

45. ΕΥΧΑΡΙΣΤΩ