Download presentation
Presentation is loading. Please wait.
Published byBaldric McCormick Modified over 6 years ago
1
Language Technologies Institute Carnegie Mellon University
Machine Translation Post-Editing Study Project Kent State Project Meeting Alon Lavie Language Technologies Institute Carnegie Mellon University 8 June 2011
2
Meeting Goals Work out details of a summer pilot project on MT post-editing involving CMU and Kent State Discuss long-term research goals and possible funding opportunities Identify concrete target program for a research grant proposal
3
Longterm Project Goals
Technology Goal: design MT systems that are most useful and productive for use by human translators as a CAT tool Project Goals: Develop an in-depth understanding of the characteristics of MT post-editing within commercially- relevant settings Develop measures for quantifying the suitability of MT systems to the task of MT post-editing Explore advanced methods for optimizing MT for post- editing, and for integrating MT into CAT environments
4
Research Questions What types of MT errors are easy for human translators to correct, and what types are difficult? Can we create a taxonomy of such errors? How do these error characteristics of MT systems vary across different MT approaches and technologies (i.e. "rule-based" systems vs. "statistical" systems)? How do these error characteristics vary for different target-languages and language-pairs? How do these error characteristics differ between "generic" MT systems (such as Google) vs. MT systems that are directly adapted to domain and client data? How should translations produced by MT be presented and displayed to translators most effectively for post-editing? Should poor MT translations be filtered out as to not confuse translators? Can we design measures that better capture the post-editing "difficulty" of MT output? If so, can we use these measures to produce MT output that is easier for translators to post-edit?
5
Pilot Project Goals Collect preliminary data that supports developing a solid scientific research agenda for a long-term research project Become familiar with the task and challenges involved Develop an effective working relationship between MT research team at CMU and translation studies research team at Kent State
6
Commercially-Relevant Setting
Research should be framed in a commercially-relevant setting, where MT has been shown to produce significant gains in translator productivity, so that outcomes bear immediate impact on translation industry Main characteristics of such settings: Commercially-relevant domain and data MT and TMs integrated within a common CAT editing environment for human translators (i.e. TRADOS) Domain and/or client-adapted MT as opposed to “generic” MT engines (i.e. Google) Probably too complex and difficult to create a complete commercial setup for the summer pilot project, so simplify to the minimum required in order to collect meaningful data
7
Proposed Setting for Pilot
Domain: Computer Hardware and Software documentation and software localization Language-Pair: English-Spanish In what direction? English-to-Spanish? Spanish-to- English? Both? No Translation Memories or integration of MT with TMs Simple GUI for MT error classification and MT post-editing
8
Proposed MT Systems Domain-specific statistical MT system can be developed by Safaba Translation Solutions Data: About 4 million TUs (60 million words) of domain-specific training data that Safaba has acquired from the TAUS Data Association (TDA) System can be trained and ready for use within a couple of weeks Will be made available online for remote access and connection Use two other MT systems as comparisons for the study: Google: “generic” (unadapted) high-quality SMT system BabelFish/SYSTRAN: “generic” (unadapted) rule-based MT system Is this too much?
9
Proposed Pilot Study Task-1: collect data on high-level classification of MT utility for post-editing: Translators classify MT-translated segments into one of three categories: MT translation does not require any post-editing (perfect) MT translation requires post-editing and can be post-edited MT translation is unintelligible and cannot be effectively post-edited Task-2: analysis of the data collected: Inter and Intra coder agreement levels Distributional analysis Variation across type of MT and other controlled variables Task-3: Perform a more detailed classification of the data from category-2 into types of error and their difficulty Task-4: Perform actual post-editing of data from category-2, with time and end-quality measurements
10
Preliminary Tasks Selection of documents for the pilot study
Domain relevant data from online resources Preferably with target human translations Controlling for document and segment difficulty (and length)? Who does this, and how soon? Creation of the required user interfaces Design and develop simple online interfaces Testing Identifying and selecting translator subjects Do you have students and are they available? IRB
11
Discussion…
12
Grant Opportunities NSF:
NSF Information and Intelligent Systems (IIS) Core Programs: rg=IIS&from=fund Medium-size Projects: Proposals due 9/15/2011 Cyber-Enabled Discovery and Innovation (CDI) program: _key=nsf11502 Next deadline is unclear Highly competitive Grant Opportunities for Academic Liaison with Industry (GOALI) program: rg=IIS&from=fund This program accepts proposals anytime, but the funding level is unclear. Other US Government Funding Sources, such as NSA, NVTC
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.