1. TCU-ACU 2 nd Joint Workshop ‘e-Learning Class Evaluation' Bangkok, Thailand, November 18 th ~ 19 th Jong Sun Park, Ph.D. Director General of ACU Secretariat Professor of Seoul Cyber University Director of Center for Teaching & Learning

2. At the end of this session the participant should be able to: Outline e-learning class evaluation Discuss the possible ways to solve online assessment problem Learning Objectives

3. Achievement evaluation criteria: norm-referenced test Exception: use criterion-referenced test - practicum class - subjects which class size is less than 30 students All evaluations are performed on-line Each online assessment data should be stored in LMS storage SCU’s Academic achievement assessment

4. Regular exam: intermediate and final examinations in course Occasional exam: a simple test or quiz conducted by occasional lectures Assignment: the evaluation report, essay, group work, etc. Discussion: student’s opinion on the subject material of the lesson Team Projects: students taking the assignment imposed by organized groups with multiple teams Engagement: student participation ratio on bulletin board in class Attendance: lecture attendance progress ratio Academic assessment items

5. Attendance will be used as a requirement evaluation criteria for the e-Learning course Attendance ratio: 10% of the total amount of e- Learning course evaluation criteria Attendance recognition time: 2 weeks within open the class of the week Late attendance: 50% attendance recognition when students participate 2 weeks after class open of the week Attendance Management

6. Discussion activity frame - Discussion topic - Writer - Day - The number of view Discussion Participation

7. Discussion evaluation list - Identification number - Name - Department - The number of discussion - The number of apply - The number of recommend - Total score - Evaluation day Discussion Evaluation

8. Assignment Evaluation Assignment evaluation list - Identification number - Name - Department - The day of submit - Evaluation score - The day of evaluation - The copy ratio

9. Example of Report Copy Protection

10. Participation Management Participation management list - Identification number - Name - Department - The number of free board participation - The number of learning resources room participation - The number of Q&A participation - The number of survey participation - Total score

11. Assessment itemsReflected proportion Regular exam 0 ~ 50% (Middle, final exams, each within 30%) Attendance0 ~ 20% Other engagement0 ~ 10% Assignment, Team Project Each 0 ～ 50% (However, less than 20% of per assignment) Discussion, Occasional exam Each 0 ～ 30% Each itemized evaluation grade ratios shall be reflected in professor's discretion within the scope of the table below. Typically, professors utilize 20% of middle exam, 30% of final exam, 10% of attendance ratio, 10% of engagement ratio, 10% of Discussion, and 20% of assignment.

12. Performance evaluation criteria must be utilized at least five items - Attendance: 10% - Exam: within 50%(mid term 20%, final term 30%) - Assignment: 10%(within 20%) - Discussion: 10% - Learning activity participation: 10% - Survey participation: 10% Online Syllabus Assignment Midterm Final term Assignment Discussion Participation

13. The test is carried out regular exam (midterm, final) for the each course of each semester Students who did not take the regular exam can take an additional exam participation opportunity(midterm, final) The additional exam is performed within 7 days after the end of regular exam Principles of exam

14. Online real-time exam(1) Online real-time exam is so called "concurrent exam". Concurrent exam is to take the test within the given available time by connecting to the Internet connection. Once tested, the exam time is going, and after the end of the exam time, the exam is terminated regardless of the degree of problem solving. Concurrent exam time of the test is based on the one hour (60 minutes), which is generally carried out for 60 minutes.

15. Online real-time exam(2) The exam time of subjects will be released to the students prior when the initial enrollment in semester. Because it is to prevent exam overlap to other subjects Concurrent exam subjects ratio of 2014 is accounted for approximately 50%, and current trend is going to increase the proportion of each semester. The main thing at concurrent exam is to decide when students can take the exam: Currently proceeding is mainly from weekday to weekend, time is often the case that proceeds mainly in the evening hours.

16. Advantages of online real-time exam The reaction for the concurrent exam is varied in accordance with the student. Students who study hard tend to favor the simultaneous exam, but students who can not study hard prefer a non-concurrent exam. It is difficult to do cheating behavior because students will need to take the exam on another student It is almost impossible to share the problem with other students, so they can not negotiating to resolve their exam. Assessing students' exam grade through the simultaneous exam is more easy than non-concurrent aspects. Sufficient students are not prepared in advance, because it can have difficulty in solving exam problems.

17. Non-concurrent exam is connected to the Internet at a convenient time within a given connection time to taking the test. Non-concurrent exam access question bank at which exam time shall be 24 hours. Concurrent exam shall be involved in the evaluation, so that at any given time to the students, if somewhat uncomfortable. The majority of the students that have the job is not easy to take the test at a specific time. Disadvantages of online real-time exam

18. Recognize D grade or higher grades in acquired credit If found any credit mistakes or misconduct, and cancelled Difference between scores in the same ratings is limited to not more than 30 points If the obtained score is less than 10 points, grade F Grade and rating RatingsRatio A+0 ~ 15% A16% ~ 30% B ⁺, B 31% ~ 70% C ⁺, C, D ⁺, D, F71% ~ 100%

19. Student performance evaluation grade: average of 1.0 or less 4th or subsequent three times take academic warning: expelled Academic warning RatingsScoreMarksRatingsScoreMarks A+ 100 ∼ 95 4.5C 74 ∼ 70 2.0 A 94 ∼ 90 4.0D+ 69 ∼ 65 1.5 B+ 89 ∼ 85 3.5D 64 ∼ 60 1.0 B 84 ∼ 80 3.0F 59 ∼ 0 0 C+79 ∼ 752.5 P Absolute evaluation scores table:

20. Types of test items multiple-choice type of exam questions: the OX-type multiple-choice, four-choice paper, and 5 multiple choice paper, short answer, etc. Open ended short answer test: requires an answer is a word or phrase. Essay type short answer: enter an answer to the problem of logical essay type directly on the Internet. Practical problem is the problem of the form to submit answers by sending a separate file or in the form of a report created by the separate processing programs (Word, Excel, or other applications) to the server.

21. Among these types, professors are mostly like multiple choice and the open-ended short-answer. Because multiple choice and open ended short answer is automatically scoring in the learning management system. The majority of professors use the multiple choice eluent and the open ended short answer test but they has the issue of grades discrimination for the students. - in the case of multiple-choice, some professors use the slightly modified form of multiple choice tends to confuse students: picking everything wrong, or choose one to suit everyone. - professors do not want to explain or reveal the answer to the problem after exam. In recent years, it emphasized disclosure to explanation on the student's exam questions and answers from the university. Professors often does not use essay - type eluent because it takes a lot of time and effort to mark the answers.

22. First, the dominant reaction is difficult to study than in the past due to the use of a variety evaluation criteria (tests, attendance, participation, discussion, assignments, etc.). Second, students often complain of discomfort caused by simultaneous examination test participation. The last third is that students' test scores are calculated prevents fulfill the normal distribution. Many professors utilize multiple choice and open ended short answer test. So students achievement score is tend to come out a little higher scores, which are difficult to discriminate student's skills. Because of this problem is that students' scores do fulfill a normal distribution is calculated as biased tendency to the left. To solve this problem it is necessary to introduce a qualitative test problems utilizing the essay type. Because of burden on the eluent and scoring, a lot of professors are still utilized in some subjects. it is necessary to consider a realistic approach to helping students of grades discrimination. Students' general reaction for the academic evaluation

23. feedback about the test (middle, final) should reveal the correct answer and score after scoring in our college. But, some professors does not provide explanation by systematically at the university level. Feedback on assignments have to improved. Assignments are practical in content because that situation does not nearly done. This is also shown by the results of the monitoring of the operations of the university class. Some professors utilize just assignment scoring. They can not provide content level feedback to the students. So feedback about the actual content need to revised and improved. There is a need to improve the student's learning activities with feedback, and it just about assignments. Provide Feedback on academic assessment(assignments, tests)

24. A similar situation is also a discussion. At least once a semester to discuss are utilized the evaluation criteria in the majority of subjects. Mainly quantitative discussion and have a quantitative evaluation criteria and scoring is done automatically. So the professor must obtain a plan that gives feedback or management plan on the discussion process and outcome. Provide Feedback on academic assessment(discussion)

25. Have you got any question?

26. 100700

27. Please send 1 page report to (very short) to eunae.kim@aseancu.org