1. Monotrans: Human-Computer Collaborative Translation Chang Hu, Ben Bederson, Philip Resnik Human-Computer Interaction Lab Computational Linguistics and Information Processing Lab University of Maryland Crowdsourcing Translation with People Who Speak Only One Language

2. Why translation by monolingual people? How Monotrans works Research prototype Preliminary evaluation Outline

3. Source: Global Reach, Internet World Stats Languages on Internet by Population

4. Source: Global Reach, Internet World Stats Languages on Internet by Population

5. Source: Global Reach, Internet World Stats Languages on Internet by Population

6. A real-world problem: International Childrens Digital Library www.childrenslibrary.org

7. Machine Translation (MT) Large volume, cheap, fastUnreliable quality ( = restaurant, dining hall)

8. Professional Translators High quality, but slow and expensive (even for common language pairs)

9. Translation with the Crowd Bottle neck: bilingual people

10. Translation with the Crowd vs. 75,000 contributors Wikipedia: 800 translators Translation with the Monolingual Crowd

11. Quality Affordability Machine Translation Machine Translation Professional Bilingual Human Participation Amateur Bilingual Human Participation Monolingual Human Participation Monolingual Human Participation

12. Why translation by monolingual people? How Monotrans works Research prototype Preliminary evaluation Outline

13. Basic Idea Original source sentence Fluent translation MT Inaccurate back translation Fluent, accurate source sentence MT Et cetera… Source language speaker MT Inaccurate translation Target language speaker

14. An (Richer) Example

15. Pierre Says: En général, on s'entend bien, tous les deux. (lit. In general, we get along together, the two of us.) Mary

16. Pierre Says: En général, on s'entend bien, tous les deux. (lit. In general, we get along together, the two of us.) Mary Sees: In general, it means well, both. MT

17. Pierre Says: En général, on s'entend bien, tous les deux. (lit. In general, we get along together, the two of us.) Mary Sees: In general, it means well, both. Edits into: In general, it is about both of us. MT

18. Pierre Says: En général, on s'entend bien, tous les deux. (lit. In general, we get along together, the two of us.) Sees: En général, Il est à la fois de nous.(*) Mary Sees: In general, it means well, both. Edits into: In general, it is about both of us. MT

19. Pierre Says: En général, on s'entend bien, tous les deux. (lit. In general, we get along together, the two of us.) Sees: En général, Il est à la fois de nous.(*) Edits into: En général, nous nous entendons bien. (lit. In general, we get along well.) Mary Sees: In general, it means well, both. Edits into: In general, it is about both of us. MT

20. Pierre Says: En général, on s'entend bien, tous les deux. (lit. In general, we get along together, the two of us.) Sees: En général, Il est à la fois de nous.(*) Edits into: En général, nous nous entendons bien. (lit. In general, we get along well.) Mary Sees: In general, it means well, both. Edits into: In general, it is about both of us. Sees: In general, we get along fine. MT enrichment In generalEn général Get alongNous entendons

21. Pierre Says: En général, on s'entend bien, tous les deux. (lit. In general, we get along together, the two of us.) Sees: En général, Il est à la fois de nous.(*) Edits into: En général, nous nous entendons bien. (lit. In general, we get along well.) Mary Sees: In general, it means well, both. Edits into: In general, it is about both of us. Sees: In general, we get along fine. MT enrichment

22. Pierre Says: En général, on s'entend bien, tous les deux. (lit. In general, we get along together, the two of us.) Sees: En général, Il est à la fois de nous.(*) Edits into: En général, nous nous entendons bien. (lit. In general, we get along well.) Mary Sees: In general, it means well, both. Edits into: In general, it is about both of us. Sees: In general, we get along fine. Edits into: In general, we get along well. MT enrichment

23. Pierre Says: En général, on s'entend bien, tous les deux. (lit. In general, we get along together, the two of us.) Sees: En général, Il est à la fois de nous.(*) Edits into: En général, nous nous entendons bien. (lit. In general, we get along well.) Sees: En général, nous nous entendons bien. (lit. In general, we get along well.) Mary Sees: In general, it means well, both. Edits into: In general, it is about both of us. Sees: In general, we get along fine. Edits into: In general, we get along well. MT enrichment

24. Pierre Says: En général, on s'entend bien, tous les deux. (lit. In general, we get along together, the two of us.) Sees: En général, Il est à la fois de nous.(*) Edits into: En général, nous nous entendons bien. (lit. In general, we get along well.) Sees: En général, nous nous entendons bien. (lit. In general, we get along well.) Proposes to stop with current translation Mary Sees: In general, it means well, both. Edits into: In general, it is about both of us. Sees: In general, we get along fine. Edits into: In general, we get along well. Agrees to stop with current translation MT enrichment

25. Monotrans Protocol

26. Why translation by monolingual people? How Monotrans works Research prototype Preliminary evaluation Outline

28. Web link Image Mark OK Mark unclear

29. Why translation by monolingual people? How Monotrans works Research prototype Preliminary evaluation Outline

30. Preliminary Evaluation Older version of the UI (same protocol) Childrens book, Russian to Chinese 2 Russian speakers and 4 Chinese speakers formed 4 Pairs* 1 hour per pair

31. Results 44 sentences (6 pages) worked on 28 sentences finished ( 4 pages) Overall translation speed: 50 words per hour professional translator speed: 250 words per hour

32. Evaluation

33. Google Translate …

34. … Monotrans

35. Where to from here? Larger and more formal validation of the protocol Richer annotations Images Web links Marking correct spans Marking incorrect spans Paraphrase Word clouds …?? Large-scale crowd support (CrowdFlow talk @1:20PM)

36. Monolingual translation can help large-scale translation Translation with monolingual people is actually feasible Take-Away Message

37. Sponsors

38. Thank You Q&A

39. Backup slides

40. Projected annotation Project information from one language to another using word alignments as a bridge Illustration of how this has been done for natural language annotation [Kolak 2005]

41. Projected annotation Tout le monde doit entendre l'histoire de Cendrillon Everybody has heard the business by Cinderella MT Tout le monde doit entendre l'histoire de Cendrillon Everybody has heard the business by Cinderella MT Everybody has heard the story about Cinderella Tout le monde doit entendre l'histoire de Cendrillon Everybody has heard the business by Cinderella MT => Pilot experiment results: Projected annotations helped improve translation

42. One of my examples involves rmvng ll th vwls frm th wrds nd shwng tht th rdr cn stll ndrstnd th sntnc.

43. Tout le monde doit entendre l'histoire de Cendrillon. MT Pilot experiment results: Post-editing machine translation output by monolingual people improves translation quality Everybody has hear story about Cinderella Everybody has heard the story about Cinderella Three Types of Errors I. Detectable and Correctable Error

44. Everybody has heard the story about Cinderella Tout le monde doit entendre l'histoire de Cendrillon. Everybody has hear story about Cinderella MT Everybody has heard the business by Cinderella II. Detectable but not Correctable Error Communication needed Three Types of Errors

45. Everybody has heard the story about Cinderella Tout le monde doit entendre l'histoire de Cendrillon. Everybody has hear story about Cinderella MT Everybody has heard the business by Cinderella II. Detectable but not Correctable Error Pilot experiment results: Communication through enrichment channel can improve translation Three Types of Errors

46. Everybody has heard the story about Cinderella Tout le monde doit entendre l'histoire de Cendrillon. Everybody has hear story about Cinderella MT Everybody has heard the business by Cinderella Everybody loves the story about Cinderella Need more redundancy III. Undetectable Error Add more redundancy, reduce it to type I or type II Three Types of Errors

47. Prototype Evaluation System seems promising (1=unintelligible, 4=very intelligible)(1=not translated, 5=full meaning)