Weighted Tree Transducers A Short Introduction Cătălin Ionuţ Tîrnăucă Research Group on Mathematical Linguistics, Rovira i Virgili University Pl. Imperial Tarraco 1, 43005, Tarragona, Spain Pl. Imperial Tarraco 1, 43005, Tarragona, Spain 9 th of January 2006 Seminar I

Outline  History  What is a WTT?  Why WTT’s?  Where can we apply WTT’s?  Work plan  References

History 1960s: tree automata and tree languages emerged quite naturally from the view of finite automata as unary algebras (J.R. Büchi and J.B. Wright). 1970: tree transducers were introduced independently by Rounds and Thatcher as a generalization of finite state transducers: - top-down tree transducer (root-to-frontier) - bottom-up tree transducer (frontier-to-root). 1970’s: weighted transducers and rational power series were developed by M. P. Schützenberger, S. Eilenberg, A. Salomaa, W. Kuich, J. Berstel, M. Soittola, C. Reutanauer, M. Mohri.

History (II) 1999, 2001: tree series transducers were generalized from tree transducers by allowing tree series as output rather than trees, where a tree series is a mapping from output trees to some semiring (W. Kuich, J. Engelfriet, Z. Fülöp, H. Vogler ): - the semantics is defined in an algebraic style. 2004: weighted tree transducers were introduced as an alternative approach of tree series transducers by Z. Fülöp and H. Vogler: - the semantics is defined in an operational style. MAIN RESULT: Tree series transducers and weighted tree transducers are semantically equivalent for both the top-down and the bottom-up case.

What is a WTT? A tree transducer is a finite state machine which computes a tree transformation. In other words, given an input tree over the input ranked alphabet, the tree transducer computes a set of output trees over the output ranked alphabet. Informally, a weighted tree transducer is a tree transducer each (term rewriting) rule of which is associated with a weight taken from a semiring. Along a successful transformation the weights of the involved rules are multiplied and, for every pair of input tree and output tree, the weights of its successful transformations are summed up.

What is a WTT? (II) So, we can say that a WTT combines two extremely powerful tools: tree transducers weighted transducers Weighted Tree Transducer Input tree (Output tree, Weight) A scheme of a weighted tree transducer (WTT) can be visualised in the following picture:

What is a WTT? (III) Formally, a weighted tree transducer is a tuple where: - is a tree transducer with R being the set of its term rewriting rules; - is a semiring; - is a function which associates with each rule a weight in the semiring A. There are two types (approaches): - top-down; - bottom-up.

Why WTT’s? -MOTIVATION:

Why WTT’s? -MOTIVATION: NLP (machine translation, speech recognition)

Why WTT’s? -MOTIVATION: NLP (machine translation, speech recognition) -TOOLS:

Why WTT’s? -MOTIVATION: NLP (machine translation, speech recognition) -TOOLS: finite-state automata and transducers

Why WTT’s? -MOTIVATION: NLP (machine translation, speech recognition) -TOOLS: finite-state automata and transducers -IDEA:

Why WTT’s? -MOTIVATION: NLP (machine translation, speech recognition) -TOOLS: finite-state automata and transducers -IDEA: the machines probabilistically transform input strings into output strings, and they can be quickly assembled to tackle new jobs via generic mathematical operations like composition and forward application.

Why WTT’s? -MOTIVATION: NLP (machine translation, speech recognition) -TOOLS: finite-state automata and transducers -IDEA: the machines probabilistically transform input strings into output strings, and they can be quickly assembled to tackle new jobs via generic mathematical operations like composition and forward application. -PROBLEM:

Why WTT’s? -MOTIVATION: NLP (machine translation, speech recognition) -TOOLS: finite-state automata and transducers -IDEA: the machines probabilistically transform input strings into output strings, and they can be quickly assembled to tackle new jobs via generic mathematical operations like composition and forward application. -PROBLEM: these machines are a bad fit for many important problems that require syntax-sensitive transformations and large- scale re-ordering.

Why WTT’s? -MOTIVATION: NLP (machine translation, speech recognition) -TOOLS: finite-state automata and transducers -IDEA: the machines probabilistically transform input strings into output strings, and they can be quickly assembled to tackle new jobs via generic mathematical operations like composition and forward application. -PROBLEM: these machines are a bad fit for many important problems that require syntax-sensitive transformations and large- scale re-ordering. -SOLUTION:

Why WTT’s? -MOTIVATION: NLP (machine translation, speech recognition) -TOOLS: finite-state automata and transducers -IDEA: the machines probabilistically transform input strings into output strings, and they can be quickly assembled to tackle new jobs via generic mathematical operations like composition and forward application. -PROBLEM: these machines are a bad fit for many important problems that require syntax-sensitive transformations and large- scale re-ordering -SOLUTION: finite automata have to be replaced by more powerful tools like weighted (tree) automata and that trees should take the place of the strings.

Where can we apply WTT’s? translation systems (the project TREEWORLD at ISI’s Natural Language Group): - more accurate language processing system; - better understanding of how to model language translation more deeply and accurately; - syntactic and lexical translation knowledge can still be acquired fully automatically by the machine; computational biology text recognition (compression, indexing, pattern matching) image processing (filters, image compression); speech recognition (speech synthesis, large-vocabulary); others…?

Work plan MAIN GOAL: BACKGROUND THEORETICAL FOUNDATIONS APPLICATIONS Applications of WTT’s in machine translations weighted transducerstree transducers and tree transformations tree series transducers weighted tree transducers analyse the algorithms developed so far with the above formal models

Work plan (II) MAIN GOAL: BACKGROUND THEORETICAL FOUNDATIONS APPLICATIONS Applications of WTT’s in machine translations various types of weighted bottom-up and top-down tree transducers compare weighted tree transformations defined by different types of such transducers consider compositions of WTTs and closure properties of the various classes w.r.t. composition consider decompositions of WTTs of a given type into compositions of WTTs of simpler types

Work plan (III) MAIN GOAL: BACKGROUND THEORETICAL FOUNDATIONS APPLICATIONS Applications of WTT’s in machine translations design efficient computer science algorithms for generic tree operations design efficient machine learning algorithms for inducing tree automata, tree transducers and probabilities from linguistic data use weighted tree automata and tree transducers to accurately model problems in automatic language

References  Z. Fülöp. A Short Introduction To Tree Transducers. XIX Tarragona Seminar on Formal Syntax and Semantics, FS&S. (2004)  Z. Fülöp, H. Volger. Weighted Tree Transducers. Journal of Automata, Languages and Combinatorics, 9. (2004)  M. Mohri, F. C. N. Pereira, M. Riley. Weighted Finite-State Transducers in Speech Recognition. Computer Speech and Language, 16(1): (2002)  J. Engelfriet, Z. Fülöp, H. Volger. Bottom-up and Top-down Tree Series Transformations. Journal of Automata, Languages and Combinatorics, 7: (2002)  W Kuich. The transducers and formal tree series. Acta Cybernetica, 14(1): (1999)