FreeLing Uncovered Lluís Padró Centre de Recerca TALP Universitat Politècnica de Catalunya

Seminari TALP-GPLN 29/09/2016 Have you got a FreeLing ? Introduction What is FreeLing ? A configurable and extensible linguistic analysis library, developer-oriented. What is FreeLing not? A user-oriented off-the-shelf linguistic analyzer. What do people use it for? analyze -f es.cfg < mydocs.txt

Seminari TALP-GPLN 29/09/2016 Have you got a FreeLing ? Processing Classes

Seminari TALP-GPLN 29/09/2016 Have you got a FreeLing ? Linguistic Data Classes

Seminari TALP-GPLN 29/09/2016 Have you got a FreeLing ? Processing sequence Sample program Initialization: Create required modules with desired configuration files. Read and process text: send each input line through processing chain

Seminari TALP-GPLN 29/09/2016 Have you got a FreeLing ? Processing Sequence: Tokenizer Tokenizer: Initialization: tokenizer.dat tokenizer.dat Input: TEXT (class: string) El gato come pescado. El Sr. García pasea al perro a las Output: TOKENS (class: vector )

Seminari TALP-GPLN 29/09/2016 Have you got a FreeLing ? Processing Sequence: Splitter Splitter: Initialization: splitter.dat splitter.dat Input: TOKENS (class: vector ) Output: SENTENCES (class: list )

Seminari TALP-GPLN 29/09/2016 Have you got a FreeLing ? Processing Sequence: Maco Maco: Submodules: numbers, punctuation, dates, dictionary, multiwords, NER, quantities, probabilities. Initialization: punct.dat, maco.db, sufixos.dat, locucions.dat, np.dat, quantities.dat, probabilitats.dat punct.dat maco.db sufixos.dat locucions.dat np.dat quantities.dat probabilitats.dat Input: SENTENCES (class: list ) Output: SENTENCES (class: list )

Seminari TALP-GPLN 29/09/2016 Have you got a FreeLing ? Processing Sequence: Senses Senses: Initialization: senses16.db senses16.db Input: SENTENCES (class: list ) Output: SENTENCES (class: list )

Seminari TALP-GPLN 29/09/2016 Have you got a FreeLing ? Processing Sequence: Tagger Tagger: Initialization: tagger.dat / constr_grammar.dat tagger.dat constr_grammar.dat Input: SENTENCES (class: list ) Output: SENTENCES (class: list )

Seminari TALP-GPLN 29/09/2016 Have you got a FreeLing ? Processing Sequence: NEC NEC: Initialization: nec.rgf, nec.lex, nec.abm nec.rgf Input: SENTENCES (class: list ) Output: SENTENCES (class: list )

Seminari TALP-GPLN 29/09/2016 Have you got a FreeLing ? Processing Sequence: Chart parser Chart parser: Initialization: grammar-dep.dat grammar-dep.dat Input: SENTENCES (class: list ) Output: SENTENCES (class: list )

Seminari TALP-GPLN 29/09/2016 Have you got a FreeLing ? Processing Sequence: Dependencies Dependency parser: Initialization: dependences.dat dependences.dat Input: SENTENCES (class: list ) Output: SENTENCES (class: list )

Seminari TALP-GPLN 29/09/2016 Have you got a FreeLing ? Other modules Semantic DB: Browse EuroWordNet by lemma or by synset. Initialization: senses16.db, wn16.db senses16.db wn16.db

Seminari TALP-GPLN 29/09/2016 Have you got a FreeLing ? Including new languages (1) Tokenizer & Splitter: Adapt config files. Maco: Index form dictionary Adapt suffixation rules Provide (if any) multiwords file Develop (if needed) date, number, and quantities modules

Seminari TALP-GPLN 29/09/2016 Have you got a FreeLing ? Including new languages (2) Tagger (and probabilities module) Use a tagged corpus to train taggers and compute lexical probabilities. Scripts are provided with FreeLing Chart parsers and Dependency parsers Develop appropriate grammars (or adapt some of the existing ones to the new language)

Seminari TALP-GPLN 29/09/2016 Have you got a FreeLing ? Some NLP applications using FreeLing (1) OpenTrad (PROFIT, ) Spanish & English analysis for es-ba and en-ba syntactic transfer machine translation. Adaptations Improve/develop chunking grammars and dependency parser rules Produce appropriate XML output

Seminari TALP-GPLN 29/09/2016 Have you got a FreeLing ? Some NLP applications using FreeLing (2) ASOMO (Judo Socialware, ) ML-based NER development environment for opinion mining on highly unstructured documents (blogs, forums, etc.) Adaptations: Extend/adapt JAVA API Develop ad-hoc modules to use Omlet&Fries to train NER modules.

Seminari TALP-GPLN 29/09/2016 Have you got a FreeLing ? Some NLP applications using FreeLing (3) VKM (Cromosoma, essukaia.lsi.upc.edu/~vkm ) essukaia.lsi.upc.edu/~vkm CIDEM project to evaluate the viability of using NLP techniques in interactive Videogames. Closed- domain dialogue and QA system. Adaptations: Use semantic dictionary with basic logical forms instead of WN synsets. FreeLing output is processed by a DCG.

Seminari TALP-GPLN 29/09/2016 Have you got a FreeLing ? Some NLP applications using FreeLing (4) T-Incluye (Fundación CTIC, ) Exclusive language detector Adaptations: Adapt the form dictionary lemma criteria for some words (e.g. Príncipe-princesa) Develop an ad-hoc grammar for noun phrases, to pre-filter correct/irrelevant/incorrect phrases. Improve JAVA API for Semantic DB access.

FreeLing Uncovered Lluís Padró Centre de Recerca TALP Universitat Politècnica de Catalunya