Post-editing: how to future-proof your career in translation Paulo Camargo, PhD. Owner, Terminologist BLC - Brazilian Localization Company Web:

Purpose of this presentation Promote the adoption of Machine Translation (MT) and Post-editing (PE) –How we can work faster, better, and make more money Target-audience: –Novice, experienced, and advanced free- lance translators –Small LSPs and in-house translators

Perspectives: novice translator Introduce PE as a new profession –Background information –Current adoption of PE –PE productivity/compensation Explore availability of PE training –Why a translator need PE training –What are the required skills –PE certifications available: TAUS, and SDL

Perspectives: experienced/advanced Use MT output as translation aid -Research shows MT increases productivity -Translators prefer MT instead of unaided -GT, SDL Cloud, MS Hub, Systran Advanced: combine MT / term. manag. -Term extraction/customization of MT -Generation/PE of MT output ↑ productivity -Replace combined TM/on-line TM servers?

Perspectives: small LSP Large/medium LSPs use MT (> decade) –Small LSPs: need to catch-up How to get started with low budget -MT developments: ↓ need of specialized IT -Key-resource: in-house translator -Terminology management -Customizable MT -Preliminary analysis/PE Guidelines

Definition of post-editing (TAUS) Post-editing: “the correction of machine-generated translation output to ensure it meets a level of quality negotiated in advance between client and post-editor”. “Post-editing seeks the minimum steps required for an acceptable text”

Background information PE reality: driven by advances in MT –Hybrid MT: rule-based / statistic-based –Rule-based: dictionaries, rules; e.g Systran –Statistic-based: training data (TM); e.g GT Pre-editing –Customization: glossary, training data –Preliminary analysis: language rules, client rules, example card

Preliminary analysis (Rico, 2011) After engine customization –Select MT samples –Check term consistency/accuracy –Check for recurrent MT errors Draw guidelines (quality acceptance) –Quality/errors to expect / how to proceed –Language independent/dependent rules –Feedback (glossary update, errors report)

Guidelines for PE (Rico, 2011) Language independent rules –Fix terminology, syntactic, morphology –Fix misspelling, punctuation, omissions –Edit offensive/inappropriate text Language dependent rules –Language specific examples: –Example card: expected errors/how to fix

Custom MT: what to expect (O’Brien, 2002) Custom MT: high-level MT output –Most segments = 85% TM fuzzy –Some better than 100% TM match (review) –Some bad translations: retranslate Translator: critical to MT success Need human assessment: always! “Not only will MTPE not replace the translator but it also will not happen without the translator”

Full MT post-editing (Dillinger, 2004) Goal: human-quality output Most frequent: higher visibility texts Quality expectations: high (TEP) Grammar, syntactic, semantic correct Stylistically appropriate Productivity expected: 4K – 10 K w/day

Current adoption of post-editing Common Sense Advisory report (2012) –Freelance: 21.7% (15.4% plan) –Small LSP: 32.5% (22.6 % plan) –Large LSP: 72.0% (28.0% plan) ** ALC report (2015) –Small LSP: 20.0% (USA), 25.0% (Europe) Lionbridge (Marciano, 2015) –Apply 30% projects (goal 50%), 60M, 2014

Post-editing productivity data Post-editing productivity (O’Brien, 2006) –Equal/higher than editing TM High Fuzzy –Typical: 4K to 10K words/day –Proficiency: 100K w (1 month full-time PE) Other productivity data – Full PE: 5K–8K w/day (DePalma, 2011)

PE compensation: follow TM fuzzy TM fuzzy matches (Guerberof, 2013) – 60-66% of full TR rate for 75%-94% match MT full post-editing –70-50% of rate (Guerberof, 2013) –65-68% of rate (Marciano, 2015) –Smaller companies: prefer to pay per hour

Proposal for PE training (O’Brien, 2002) PE: what do TRs think about? –Dislike for correcting repetitive errors –Fear of losing proficiency (poor MT output) –Dislike for limited freedom of expression Why do TRs need PE training? –≠ skills: 2 source texts –Quality requirements, different error types –Qualified translator ≠ successful post-editor

What skills does a post-editor need? Same as the translator (O’Brien, 2002) -Expert in subject area and target language -Excellent knowledge of source language -Word-processing (WP) skills, tolerance Skills for post-editor only (Rico, 2011) -Adv WP: RegEx, S&R, term. management -Positive attitude towards MT

Proposal for PE course (O’Brien, 2002) Theoretical component –Intro to PE/MT tech / controlled language –Adv. term. Management / text linguistics –Basic programming skills Required background –TRA skills; basic linguistics/term manag –IT skills; intro lang tech; source/target skills

Sources for PE certification TAUS (Transl. Automation User Society) –English >23 lang (European, Arabic, Asian) –Also Spanish > English –Cost: 60 Euro (member), 80 Euro (non) SDL MT PE Certification –Free with SDL Language Cloud MT

Perspectives: experienced/advanced What possibilities can MT offer other than post-editing? Is it worth using MT output as an aid to increase TR productivity? Can MT replace with advantages the use of combined TM/on-line TM servers?

Efficiency of PE for language translation Rigorous, controlled analysis (Spence, 2013) -Hypothesis 1: PE reduces translation time -Hypothesis 2: PE increases quality -Hypothesis 3: MT primes the translator Compared PE vs. unaided translation –Blind experiment: TR did not GT was used –Pre-interview: TR showed strong MT dislike –16 PRO TRA/pair: EN-AR, EN-FR, EN-GE

Results clarify value of post-editing Which one is faster? 69% PE Useful? Yes 56%, No 29%, Unsure 15% Suggestions improved quality (all) MT output primes the translator –PE text (closer MT) ≠ Unaided ≠ Raw MT –Lower the TR experience → closer to MT

Does MT output increase productivity? Example 1: Google Translate –Now a paid service: $20/M characters –Plug-in to SDL Trados/other CAT tools –General statistical MT engine –Not customizable –Confidentiality issues –See app for complete setup procedure

Does MT output increase productivity? Example 2: SDL Cloud MT –Price range: $5 – $75 /month (Expert) –Plug-in to SDL Trados/other CAT tools –Complete confidentiality (nothing is stored) –Pre-trained engines: Travel, IT, Life Sciences, Automotive, Consumer Electronics –Customizable MT: can add own glossaries –Comprehensive analytics (quality analysis)

Does MT output increase productivity? Example 3: Microsoft Translator Hub –Plug-in to SDL Trados/others, secure –First 2M char free; 4M/mon $40 –Fully customizable MT engine Previous translations (> 20K words) Add glossaries Request training / evaluate results Option to “Use Microsoft Models”

How about confidentiality? Consider e.g. Microsoft and Google –Among the largest providers of MT –Among the largest buyers of translation –Control information flow around the globe Confidentiality should not be problem –Google not option → MS Hub/SDL Cloud –Uncomfortable sending data to MS/SDL –Use desktop/server solution: Systran

Changes in MT offer to TRs Common scenario for TR (4 years ago) –One affordable desktop product (Systran) –Macros, RegEx, format conversion –No plug-in for CAT tools (high-end) Current scenario –Software as a service (GT, SDL, MS Hub) –Plug-in for CAT tools is standard –Much lower IT requirements

Can MT replace combined/online TMs? Experienced/advanced translators –Use combined/on-line TM for productivity –Proud users: ↑ 50% prod, see TM as asset –TM is error-prone (consistency, mistranslation) –Need to check term consistency MT improved a lot in last 5 years –TRs trust TM fuzzy > raw MT (Guerberof, 2008) –Mistake MT output for TM output (human?)

MTPE can provide a better result Avoid problems in combined/on-line TM: –Terminology inconsistencies –Mistranslations –Waste time correcting TUs that will never use New approach using MTPE –Extract terms (Systran, rule-based) –Customizable MT (SDL Cloud or MS Hub) –Post-edit fresh MT for ↑ productivity/quality

How small LSPs can get started Scenario: 40% TRs use MT (TAUS) Actual post-editing offer (2015) –PE of our MT engine output (GT?) –Payment: 900 words/hour –Instruction: as readable as possible –No pre-editing: cal, gloss or guidelines Translators were really upset

Need more than just Google Translate Pre-editing: custom., prelim. analysis –Key: in-house translator (O’Brien, 2002) Allocate translator for MTPE activities –Use secure on-line customizable engines –Define suitable projects –Invest in terminology management –Develop PE guidelines –No rate discount initially (learning curve)

MT implementation at BLC (4 years) Smaller projects: K words –Terminology extraction (Systran, rule-based) –Normal TR + ED procedure –Semi-customized: SDL Cloud + Multiterm Larger projects > 50K words –Extract bigger glossary (higher coverage) –Raw MT (Systran, SDL Cloud, MS Hub) –PE + ED (no pre-editing, no discount)

MT implementation at BLC What does MT do for BLC? –Leverage my knowledge: engineering/science –Increase productivity (more/larger projects) –Increase quality (terminology/TM updates) Future developments –MS Hub, SDL Cloud; Systran –Hire new translator (2016) –Develop a PE team/service

Conclusion The combination of machine translation (MT) and post-editing (PE) is a disruptive innovation that can improve translator’s productivity and translation quality, no matter how you plan to use it. Can you afford to ignore it?

References Guerberof, Ana (2008). Productivity and Quality in the Post-editing of Outputs from Translation Memories and Machine Translation. Masters Dissertation. Universitat Rovira i Virgili. O’Brien, Sharon (2006). Eye-tracking and Translation Memory matches. Perspectives: Studies in Translatology 14 (3), Spence, Green et al (2013), The Efficacy of Human Post-Editing for Language Translation, ACM Human Factors in Computing Systems (CHI), Computer Science Department, Stanford University Rico, Celia et al (2011), EDI-TA: Post-editing Methodology for Machine Translation, MultilingualWeb-LT. O’Brien, Sharon (2002), Teaching Post-editing: A Proposal for Course Content. Proceedings of the 6th EAMT Workshop on Teaching Machine Translation. EAMT/BCS, UMIST, Manchester, UK DePalma, Donald (2011), Common Sense Advisory, Trends in Machine Translation. Dillinger, Mike et al (2004), Implementing Machine Translation, LISA Best Practice Guides. TAUS (2014), MT Post-editing Guidelines Marciano, Jay (2015), Personal communication.