1. Kypros-net Web Site: Learn Greek! On-line Course SI 622 Evaluation of Systems and Services M. Elledge P. Zaphiris 4/15/00

2. The Project Evaluate Learn Greek! language course –Usability –Functionality –Quality Client: Kypros-net Originator/Administrator

3. The Product Kypros-net description –Formation –Purpose

4. Kypros-net Web Site

5. Learn Greek! Course Course Description –Independent study course to learn Greek 105 audio lessons, transcriptions, vocabulary, spell checker, dictionary –Objectives Assist people in learning Greek language Develop site using User Participation philosophy Minimize expense (tapes from Cyprus Broadcasting Corporation (CyBC) radio; volunteers; MIT server)

6. Learn Greek! Home Page

7. Methods Used Interviews with site creator, content contributors (2), director and administrator of U-M Forty-three user surveys, 371 discussion board postings Literature search of PsychInfo and Wilson databases Comparative Evaluation with 16 web language sites User testingUser testing of 8 tasks with 5 subjects GTN for first two levels Metaphor, Vocabulary analyses Checklist/Cognitive walk-through on course GOMS on 2 tasks

8. Key Client Questions How useful is the course? How functional is the course? How well does it meet on-line course standards? What can be done to improve the course?

9. Question #1 How useful is the course? –Does it meet client objectives? –Does it meet user expectations? Findings –Client objectives exceeded –User expectations met –Still, there are complaints

10. Findings Client Objectives –Participatory Design: 105 lessons transcribed by 8 users; dictionary, spell checker installed by users; on-going help provided by users –Steady increase in site visits over time

11. Findings User Expectations –High user satisfaction

12. Findings –Positive Comments: EASY TO UNDERSTAND Fun It helped refresh my Greek Finally after surfing for a long time I found that you could learn Greek from the internet It is a very progressive course, builds up confidence as it goes Very good method of learning It falls into place after listening a couple of times. Greek is the most kick-butt language on earth!!!!

13. Findings Some User Complaints –Can’t download lessons –No pronunciation guide –Can’t compare text with lessons –Few instructions or information about course content

14. Discussion Board Postings Category/Subject#%Total % Technical Problems with needing/installing keyboard fonts5615.1% Problems with/wanting to download lessons4913.2% Problems with installing/using RealPlayer4010.8% 39.1% Content Postings of on-line notes7319.7% Questions about availability of text, CDs or including links to web sites3710.0% 29.7% Comprehension Questions about vocabulary287.5% Questions about grammar/spelling174.6% Questions about dialect25.4% 17.5% Miscellaneous6918.6% 18.6%

15. Question #2 How functional is the course? –Do the operations work? –Is the architecture successful? Findings –Site is subject to Internet vagaries –Learn Greek! architecture can be improved

16. Findings Operations –ON the site generally work Translations, lessons, etc. function Waiting can become tedious –Web operations less dependable Server downtime Slow lesson downloads Intermittent transmission

17. Findings Architecture –Layout Violates requirement for visual integrity Inconsistent formatting –Labeling Confusing (Greek to English in Greek?) Incorrect (Vocabulary/notes = transcripts?) Out-of-order (Instructions after the task?)

18. Findings Architecture (cont.) –Content Few instructions (RealPlayer, Greek fonts) No “Help” or FAQ Multiple translations –Navigation Hard to find things (transcripts) Search engine can’t target course

19. Question #3 Does it meet language course standards? –Is the content appropriate? –Is the methodology appropriate? Findings –Fairly unusual: hybrid of independent study + web-based learning –Pedagogy okay, course content can be enhanced

20. Findings Lacks some features

21. Findings Meets basic teaching methodologies

22. Conclusions Yin/Yang: Strengths are also weaknesses –Accessibility vs. dependability –Participation vs. consistency –Frugality vs. thoroughness

23. Recommendations Technical Architecture Content

24. Questions?

25. Other Findings Findings Recommendations

26. Conclusions

27. Next Steps

28. Learn Greek! Site Visits