30 March – 8 April 2005 Dipartimento di Informatica, Universita di Pisa ML for NLP With Special Focus on Tagging and Parsing Kiril Ribarov

Lecture structure Machine learning in general T.M. Mitchell, Machine Learning (1997, McGraw- Hill): hypothesis, decision trees, ANN, computational learning theory, instance-based learning, genetic algorithms, (Bayesian learning), [case-based, analytical learning] Natural Language – linguistically Natural Language – computationally and stochastically

Lecture structure – cont. The Prague Dependency Treebank – data and tools Morphological Tagging  Bayesian Learning (HMM, smoothing, EM, Maximum Entropy, related issues as Viterbi search, Lagrange multipliers)  Rule-Based Approach  Perceptron-Based Approach  Tagset structure, tagset size  Morphological contexts

Lecture structure – cont. Parsing  The problem of parsing, dependency parsing  Statistical parsing  Rule-Based parsing  Language graphs and sentence graphs  Naïve parsing  Rule-Based revisited  Perceptron-Based parsing

Lecture structure – cont. Parsing by tagging and tagging by parsing  Morphological contexts, tree contexts  G-tags, tagging by g-tags  Alignment of g-tags and m-tags Some problem definitions

We will include as well Problems of evaluation and its measurement for tagging and for parsing Specialties of dependency trees, surface and deep syntax, projectivity and non-projectivity Current trials on high-quality MT Ongoing research on valencies

Our aim To present general ML techniques To present the Prague Dependency Treebank To present NLP specific approaches, their modifications, applications (medium: PDT) To present mistakes and successes To present the newest ideas developed for automatic dependency acquisition To raise questions and thus indicate new directions for research in NLP

(Restriction) to Tagging and Parsing Tagging and parsing as the two most important NLP modules for various application domains. Tagging and parsing undoubtedly improve: grammar checking, speech processing, information retrieval, machine translation, … Each of the applications does not necessarily use the same tagging/parsing outputs; modifications are introduced to serve best the specific application Each of the applications has its specific core modules different than tagging/parsing Many technicalities in these are approaches are nevertheless similar

Machine Learning in General T.M. Mitchell, Machine Learning (1997, McGraw- Hill): hypothesis, decision trees, ANN, computational learning theory, instance-based learning, genetic algorithms