Coping with Babel How to Localize XML

Designing for Localization Document design can seriously impact the costs of translation and localization. Remember that you are designing for all languages, not just English. There are clear do’s and don’ts. Overriding principle is good XML practice. Always consider the target language implications.

Entity references Do not use entity references for word substitution: Use a &tool; to release the catch. Cause problems for inflected languages Cause problems for parsing/translation tools Use boiler plate text instead

Translatable attributes Avoid using translatable attributes: Use a to release the CPU retention catch. Cause problems for inflected languages Cause extra burden for translators More to go wrong

CDATA sections Avoid using CDATA sections that may contain translatable text: Please refer to the index page page for further information ]]> Lose syntactical control How are translation tools to cope?

Processing instructions Avoid Processing Instructions in translatable text: Use a to release the CPU retention catch. Syntactically week Confuse translation memory operations

Infinite Naming Schemes Avoid the use of infinite naming schemes: Cannot open file $1. Hint: does file $1 exist. Incorrect value. Hint: Must be between $1 and 2. Connection timeout. No clear element definitions

Typographical elements Avoid the use of "typographical" elements: Do not use type elements. Bad XML practice. Causes problems for translators. Target language text may be in the opposite order.

Do not break sentences Never break a linguistically complete text unit over more than one non-inline element: This text should not be broken this way – the translated text may well be in a different order.

XML Translation Standards LISA - Localization Industry Standards Association: OASIS - Organization for the Advancement of Structured Information Standards: W3C - World Wide Web Consortium: OLIF Consortium:

LISA Standards TMX - Translation Memory Exchange format: TBX - Termbase Exchange format: SRX - Segmentation Rules Exchange format: GMX - GILT Metrics Exchange format:

OASIS L10n Standards XLIFF - XML Localization Interchange File Format: open.org/committees/tc_home.php?wg_abb rev=xliff TransWS - Translation Web Services: open.org/committees/tc_home.php?wg_abb rev=trans-ws

W3C and OLIF W3C to start on Localization Directives standard. OLIF - Open Lexicon Interchange Format:

xml:tm XML Text Memory A radical new approach to translating XML documents

Machine Translation Translation Memory Hybrid Linguistic Inferencing Engines Terminology Computational Linguistic Methodologies

Translation memory Advent in early 1980’s Intermediate format Alignment Storage Leveraged memory Fuzzy matching – statistical Advantages: cost reduction, consistency Drawbacks: proofreading, managing memories No significant advances in technology

XML namespace Major new feature of XML compared to SGML Allows the mapping of different ontological entities onto the same representation Allows different ways to look at the same data Namespaces can be made transparent

xml:tm namespace Text Memory namespace Can be mapped onto any XML document Vertical view of document in terms of ‘text segments’ Can be totally transparent xml:tm

xml:tm namespace xml:tm Example of the use of namespace in an XML document: Namespace is very flexible. It is very easy to use.

xml:tm namespace doc title section para text tm te sentence tu te sentence tu te sentence tu tm namespace view original document view te text tu text te sentence tu para text para text para text para text para text te sentence tu te sentence tu

xml:tm namespace text te sentence tu original document view tm namespace view

xml:tm namespace Namespace is very simple. It is easy to use. te sentence tu original document view tm namespace view Namespace is very simple. It is easy to use. text

xml:tm Text Memory Author memory Maintain memory of source text Authoring statistics Authoring tool input Translation memory Automatic alignment Maintain perfect link of source and target text Reduce translation costs xml:tm

Updated Source Document tu id=”1” tu id=”3” tu id=”4” tu id=”7” tu id=”6” d eleted tu id=”8” new Source Document tu id=”1” tu id=”2” tu id=”3” tu id=”4” tu id=”5” tu id=”6” xml:tm DOM differencing origid=”5” modified

xml:tm Author Memory Namespace aware differencing Identify changes from the previous version Unique text unit identifiers are maintained Modification history Text units can be loaded into a database Authoring environment integration xml:tm

xml:tm Translation Memory The tm namespace can be used to create XLIFF files Automatic alignment of source and target languages Allows for more focused translation matching –Perfect matching –Leveraged matching from document - identical text –Leveraged matching from database –Modified text unit matching –Linguistically enhanced fuzzy matching –Non translatable text unit identification xml:tm

xml:tm translation Source Document tu id=”1” tu id=”2” tu id=”3” tu id=”4” tu id=”5” tu id=”6” Translated Document tu id=”1” tu id=”2” tu id=”3” tu id=”4” tu id=”5” tu id=”6” XLIFF Document trans-unit id=”1” trans-unit id=”2” trans-unit id=”3” trans-unit id=”4” trans-unit id=”5” trans-unit id=”6”

doc title section para tekst tm te zdanie tu te zdanie tu te zdanie tu translated tm namespace view translated document view te tekst tu tekst te zdanie tu para tekst para tekst para tekst para tekst para tekst te zdanie tu te zdanie tu xml:tm translated document

Source Document tu id=”1” tu id=”2” tu id=”3” tu id=”4” tu id=”5” tu id=”6” Translated Document tu id=”1” tu id=”2” tu id=”3” tu id=”4” tu id=”5” tu id=”6” Perfect alignment xml:tm perfect alignment

Updated Source Document tu id=”1” tu id=”2” tu id=”3” tu id=”4” tu id=”7” tu id=”6” d eleted tu id=”8” modified new Matched Target Document tu id=”1” tu id=”3” tu id=”4” tu id=”7” tu id=”6” tu id=”8” Perfect Matching requires translation xml:tm perfect matching

xml:tm contextual memory Source Document tu id=”1” tu id=”2” tu id=”3” tu id=”4” tu id=”5” tu id=”6” Translated Document tu id=”1” tu id=”2” tu id=”3” tu id=”4” tu id=”5” tu id=”6” Perfect alignment

Source Document tu id=”1” tu id=”2” tu id=”3” tu id=”4” tu id=”5” tu id=”6” Translated Document tu id=”1” tu id=”2” tu id=”3” tu id=”4” tu id=”5” tu id=”6” Perfect alignment DB xml:tm leveraged DB memory

xml:tm in-document leveraged matching Updated Source Document tu id=”1” tu id=”2” tu id=”3” tu id=”4” tu id=”7” tu id=”6” d eleted tu id=”8” modified new:same id=”3” Matched Target Document tu id=”1” tu id=”3” tu id=”4” tu id=”7” tu id=”6” tu id=”8” Perfect Matching requires translation requires proofing leveraged match

xml:tm in-document fuzzy matching Updated Source Document tu id=”1” tu id=”2” tu id=”3” tu id=”4” tu id=”7” tu id=”6” d eleted tu id=”8” mod:origid=”5” New:same Matched Target Document tu id=”1” tu id=”3” tu id=”4” tu id=”7” tu id=”6” tu id=”8” Perfect Matching requires translation requires proofing fuzzy match leveraged match

Updated Source Document tu id=”1” tu id=”2” tu id=”3” tu id=”4” tu id=”7” tu id=”6” d eleted tu id=”8” mod:origid=”5” new:same Matched Target Document tu id=”1” tu id=”3” tu id=”4” tu id=”7” tu id=”6” tu id=”8” Perfect Matching requires translation requires proofing fuzzy match doc leveraged match tu id=”9” xml:tm db leveraged matching DB requires proofing DB leveraged match

Updated Source Document tu id=”1” tu id=”2” tu id=”3” tu id=”4” tu id=”7” tu id=”6” non trans tu id=”8” new:same Matched Target Document tu id=”1” tu id=”3” tu id=”4” tu id=”7” tu id=”6” tu id=”8” Perfect Matching requires translation requires proofing fuzzy match doc leveraged match tu id=”9” DB requires proofing DB leveraged match tu id=”2” requires no translation non translatable xml:tm non translatable text

Traditional Translation Scenario xml:tm source text PublishingTranslation source text extract Extracted text tm process Prepared text Translate Translated text target text merge target text QA

xml:tm xml source text Publishing Translator extract Extracted text tm process Prepared text Translate xml target text merge Web perfect matching leveraged matching Automatic Process web interface QA Automatic Process xml:tm Translation Scenario

xml:tm matching Perfect Matching driven by Author Memory Leveraged Matching: 100% same text In document Leveraged Matching Database Leveraged Matching Fuzzy Matching Modified Matching Linguistically aware Fuzzy Matching Non translatable element identification Alphanumeric Numeric Measurements xml:tm

xml:tm benefits Enterprise level scalability Totally integrated within the XML framework Source text is automatically extracted and matched Word counts are controlled by the customer Text can be presented for translation via the web Online composition The most up to date translation is held by the customer Data is merged automatically at end of translation cycle All memory operations are totally automated Can be used transparently for relay translations Much cheaper to implement and run More accurate – better matching xml:tm

xml:tm summary Can be used to build consistent authoring systems Can be used to produce automatic authoring statistics Translation Memory generation and alignment is totally automatic Memory is held within the documents themselves Extraction and merging for translation are automatic The system provides much more efficient matching mechanisms Structure of the XML document is protected during translation xml:tm

Fully specified XML based standard xml-tm.html Maintained by xml-intl.com Detailed article on Offered for consideration as a Lisa standard xml:tm

Any questions? xml:tm