Educational Communication & E-learning Cappelletti Ingrid 724606 Matteo Barbero 724740 Barbara Bragatto 724741 Szymon Janikowski 735636 Paweł Gilewski 735648

Summary Sustainability Evaluation Cost & Benefit of Forum Evaluation definition Measure RACE Type of test Objective test Subjective test Donald Kirkpatrick’s learning evaluation theory

Sustainability We will be using an example of FORUM Number of posts – wrong criterion of measuring effort of a forum user. Rule „The more, the better” doesn’t work here. For learning purposes it’s necessary to introduce rules time constraints teacher’s and students’ effort. 3

Sustainability – example 1 Topic: Distance Education in IT Universities (50 students) Rule Time constraint Student’s time consum. Teacher’s time consumption Post one very thoughtful message By date x 1 hour to write 50 x 2 min = 100 min Read 3 messages of your colleagues and give them feedback Date x + 1 => date y 30 min to read 15 30 min to write replies 150 x 0.5 min = 75 min Read the comments you have got and reply to them in a single wrap-up message Date y + 1 => date z 30 min to prepare wrap 50 x 1 min = 50 min SUM 2.5 h 3h 45 min

Sustainability – example 2 Collaborative reading of book with 8 chapters (153 pages) as a part of 84 h course ~ 15 hours - too much! Solution: Everyone should choose 1 chapter and summarize it. Reading: 2 hours Writing: 40 minutes Everyone reads ALL summaries, then focus on two of them and comment them Reading: 30 minutes Comment: 10 minutes Remember about time constraints – deadline with time-zone Not enough time to demand that every student reads th whole book.

Evaluation definition learning process ability level L Measuring profit is testing 20 profit L 5 t1 t2 time What you can tell about the results of the test is called EVALUATION CMS Communi- cation Class manag. Evaluation & Testing

Measure styles In/Out or Pass/Fail Cardinal measure E.g. driving license tests Cardinal measure E.g. chess and tennis players ranks E.g. eligible & not eligible lists Shows ranks, not distances between examinees

Measure styles Scale points, grades Most common school grading systems Not all areas of scale covered A mark is often not linearly dependant on one’s knowledge Hidden psychological issues: 5++ or 6-- Paolo Francesca min max points

RACE An acronym for criteria a test has to meet. It must be: Relevant and measure what was intended Acceptable by the examinees, fair and appropriate Comparable - test results may be used to examine and note the similarities or differences among the tested Economic as correct strategies and amounts of resources are assigned for examining

Evaluation: Type of test Objective test Subjective test Definition: Test where you can not assign a score to advance require the presence of an expert Example: short essay oral text Advantages: You can assess additional aspects in addition to knowledge as the ability to present arguments in a clear and decisive way. Definition: Test where you can assign in advance a score to every possible output Example: questions with answers true false multiple choice questions Advantages: The objective test can offer a total feedback You can check the correctness of each answer. 10

Evaluation: Type of test Simulation test Definition: Test where you realize a certain activity, simulation, in which the success or failure can be detected automatically by an application. This is a type of an objective test. Example: Flight simulation Driving Simulation Advantages:  Try to play a real business 11

Donald Kirkpatrick’s learning evaluation theory The effect on the business or environment by the trainee The extent of applied learning back on the job implementation ROI The measurement of the increase in knowledge - before and after Transfer Knowledge How the students felt about the training experience Reaction Layer 12

Examples of evaluation tools and methods Donald Kirkpatrick’s learning evaluation theory Evaluation type Examples of evaluation tools and methods Relevance and practicability Reaction E.g.., 'happy sheets', feedback forms also verbal reaction, post-training surveys or questionnaires Quick and very easy to obtain Not expensive to gather or to analyze Knowledge Typically assessments or tests before and after the training interview or observation can also be used Relatively simple to set up; clear-cut for quantifiable skills Less easy for complex learning Transfer Observation and interview over time are required to assess change, relevance of change, and sustainability of change Measurement of behavior change typically requires cooperation and skill of line-managers Impact Measures are already in place via normal management systems and reporting - the challenge is to relate to the trainee Individually not difficult; unlike whole organization Process must attribute clear accountabilities 13

Thank you for your attention!