1. Corpora Linguistics 23.08.041 The Corpógrafo Belinda Maia & Luís Sarmento PoloFLUP LINGUATECA

2. Corpora Linguistics 23.08.042 A bit of history PALC ’97 – 'Do-it- yourself corpora... with a little bit of help from your friends!' CULT 1998 - ‘Making corpora – a learning process’  Contrastive linguistics  Corpora linguistics  Translation teaching  General > specific language

3. Corpora Linguistics 23.08.043 A bit of history 2000 – 1 st Master’s in Terminology and Translation at FLUP PALC 2001 - ‘Training Translators in Terminology and Information Retrieval using Comparable and Parallel Corpora’  Specialized translation and terminology  Contact with domain experts  Importance of IT  Need for technical help for more ambitious students!

4. Corpora Linguistics 23.08.044 A bit of history LREC 2002 - ‘Corpora for terminology extraction – the differing perspectives and objectives of researchers, teachers and language services providers’ 2002 – 2 nd Master’s in Terminology and Translation at FLUP  Plea for help to Diana Santos  October 2002  LINGUATECA - Polo FLUP

5. Corpora Linguistics 23.08.045 LINGUATECA See http://www.linguateca.pthttp://www.linguateca.pt Leader > Diana Santos (SINTEF – Oslo) Objective - to create resources and tools for the computational processing of Portuguese Nodes at Oslo, Lisbon, Braga and Porto Porto - Polo CLUP/FLUP

6. Corpora Linguistics 23.08.046 Polo CLUP/FLUP General focus See http://www.linguateca.pt/poloclup/http://www.linguateca.pt/poloclup/ On constructing resources specific to the needs of FLUP/CLUP –For researchers, teachers and students –For teaching methodology at FLUP  BNC & Reuter’s corpora on intranet  A small ‘chat’ corpus  Comparable corpora

7. Corpora Linguistics 23.08.047 More history 2003 – Poster of the GC – at CL2003 2003 – ‘What are comparable corpora?’ CL2003 2003 – Experimentation with evaluation of Machine Translation 2003 – Experimentation with GC 2003 – 3 rd Master’s in Terminology and Translation at FLUP

8. Corpora Linguistics 23.08.048 Polo CLUP/FLUP Research focus See http://www.linguateca.pt/poloclup/http://www.linguateca.pt/poloclup/ On-line suite of corpora tools to work with comparable corpora with emphasis on bilingual research –Focus on special domains –Construction of terminology databases, ontologies and domain models  Corpógrafo

9. Corpora Linguistics 23.08.049 And... Evaluation of Machine Translation –Experimentation with evaluation –Teaching + research focus –Tools for collecting empirical data Results: –TrAva – MT evaluation tool –CorTA – Corpus of 1 EN input + 4 MT output sentences

10. Corpora Linguistics 23.08.0410 The Corpógrafo results from: Terminology, translation and language study and research (Belinda) Computational linguistics research and production of resources (Diana) Information retrieval and artificial intelligence (Luís) Terminology data (Domain experts) = Discussions on priorities!

11. Corpora Linguistics 23.08.0411 GC – Integrated Web Environment for Corpora Linguistics Motivation Lack of Comprehensive, wide-scope Corpora Tools Commercial Packages are usually difficult to Integrate/Customize Tools are not prepared to support cooperative work. Linguistic knowledge is not usually integrated in tools. What is GC? GC is a Web tool being developed at Linguateca/CLUP that aims to provide a comprehensive work environment for Corpora-Based Linguistic Research. GC allows users to: access several Corpora tools from a single entry point using a regular web browser access and query generic Corpora (BNC, Reuter’s, COMPARA, CETEMPúblico) build personal simple, parallel and comparable Corpora from text files (PDF, PS, Word, HTML, TXT) use several (on-line/off-line) tools with their personal Corpora (statistics, POS-taggers, Filters, etc.) communicate and exchange results with other users Internet Integration GC provides seamless integration with the World Wide Web allowing users to: search specific Corpora resources on the Internet query the web for concordances use available translation-engines in parallel. DOC HTML TXT PS PDF RTF BNC CETEM Público COMPARA Others Personal Corpora Custom Interface DEV Inter-user Communication ADM USER Administrator’s Tasks: Users, Groups and Disk Quotas Corpora Taxonomy (see box) Documentation Organization Access Service Statistics Virtual Desktop Custom Interface Tool Pool Concordance Engine Taggers Aligner (Semi-Auto) Corpora Bot Statistics Custom Tools Internet Terminology DB Medium: written, spoken, multimedia Domain: Engineering, medicine, etc. Genre: scientific, technical, informative, etc. Corpora Taxonomy Terminology Extraction Tool (Auto/Semi-Auto) Developer Task: Developer’s Tasks: Integrate Existing Tools/Resources Develop Additional Generic Tools Interact with Users/Administrator Develop Custom Tools for particular research needs Inter-User Communication Tagging and Aligning Cooperatively Messaging Service Exchange of Corpora Resources Provide on-line tutorials Provide links to: on-line teaching material bibliography and other resources Teacher’s Tasks:

12. Corpora Linguistics 23.08.0412 Working with the Corpógrafo Corpógrafo is a suite of integrated tools for INDIVIDUAL or GROUP research All research done ONLINE Each username/password = separate space on our server At present > anyone can work with it using 10 MB space for FREE BUT - you get an empty space + tools + tutorial!

13. Corpora Linguistics 23.08.0413 Corpora and Terminology Special Domain Corpora Terminology extraction Terminology databases Structuring of domain knowledge Further corpora and information retrieval

14. Corpora Linguistics 23.08.0414 Corpora Analysis Terminology Database Internet Text details

15. Corpora Linguistics 23.08.0415 Terminology Prescription or Description? Prescriptive > descriptive Paper > digital form Static > dynamic resources ‘Democratization’ of terminology ISO standards > socioterminology Knowledge structures increasingly recognized as structured but dynamic

16. Corpora Linguistics 23.08.0416 Perspectives of terminology users Domain experts and vested interests Translators Information retrieval Knowledge engineering  Standardized terminology  The ‘right word’  Finding information  Perfecting Google  Structuring knowledge  Finding it fast

17. Corpora Linguistics 23.08.0417 Bridging the Gap General linguists Translation teachers Translation students Corpus linguists Computational linguists Computer engineers Computer-phobia Computer-worship

18. Corpora Linguistics 23.08.0418 Focus of Corpógrafo Design priorities are to: –See the Big Picture –Create the Overall Framework –Get feedback from users –Develop according to real research needs –Fill in details and improve techniques as needed

19. Corpora Linguistics 23.08.0419

20. Corpora Linguistics 23.08.0420 File Manager Area where each individual or group can: –Upload texts to space on server –Convert various text formats to.txt –‘Clean’ them of unnecessary material –Check tokenization and sentence divisions –Register full information on source, domain and text type –Group – and re-group - texts into corpora

21. Corpora Linguistics 23.08.0421 General corpus analysis Concordancing tools allowing for –Concordancing at sentence level –KWIC concordancing –Collocations N-gram tool –Case-sensitive –Alphabetical or frequency ordering

22. Corpora Linguistics 23.08.0422 Corpora + TDB Choose corpus Choose related TDB = All terms, examples, definitions extracted (semi) automatically from corpus and transferred to TDB = All metadata on texts providing data can be automatically transferred to TDB

23. Corpora Linguistics 23.08.0423 Term extraction N-grams –Unfiltered –Filtered with restrictions on term in PT EN FR IT ES DE –Filtered with restrictions on term and context in PT EN FR IT ES DE –Singular + plural terms can be combined –Existing terms in TDB need not appear

24. Corpora Linguistics 23.08.0424 Term selection from n-grams Consultation of list of n-grams Check term status of each n-gram via underlying concordances Check sources Send to TDB

25. Corpora Linguistics 23.08.0425 Search for Candidates for Definitions and/or Semantic Relations Already possible via TDB Under development Research areas for Mestrado dissertations and research assistants –Expressions that find definitions –Expressions that find semantic relations

26. Corpora Linguistics 23.08.0426 TDB - Terminology database Databases are designed to be multilingual –Terms listed alphabetically + language tag –General data –Morphological data –Source metadata: Authors, texts etc –Definitions + search for candidates –Translation equivalents –Semantic relations

27. Corpora Linguistics 23.08.0427 Future developments General testing and improvement Development of new ideas or functions Isomorphic relationship between: –Research possibilities –Researchers’ needs –Our skills Coordination of individual corpus projects into bigger projects, when possible or necessary

28. Corpora Linguistics 23.08.0428 Theoretical questions / problems How large is a good domain corpus? Comparable corpora v. Parallel corpora? How much information does a database need – for information retrieval and knowledge engineering? How much does the user of a database need – for translation, teaching etc.?

29. Corpora Linguistics 23.08.0429 Corpógrafo and special domains Master’s in Terminology and Translation Terminology projects with the support of domain specialists in: –Engineering – Electronics, Mechanical Engineering –Geography - Population Geography, Natural Hazards – Fire, Floods, Earthquakes, Coastal Erosion, –Medicine - Kidney support machines, Neurology –Science – Genetics –Technology – GPS – Geographical Positioning Systems

30. Corpora Linguistics 23.08.0430 Corpógrafo and terminology/translation research Ongoing dissertations on aspects of: –Terminology – neologisms, definition searches, semantic relations, conceptual analysis –Corpora – text analysis, corpora construction –Technical writing > Electrical Appliances –Localization –Terminology in documentaries –Translation of Multimedia

31. Corpora Linguistics 23.08.0431 Linguateca Linguateca’s policy - all resources and tools freely available online Primary users - Portuguese and Brazilian Other users also welcome

32. Corpora Linguistics 23.08.0432 Polo CLUP/FLUP Bi- or multi-lingual in interest Corpógrafo available for experiments on a small scale to the general public Possibilities of future work on projects with users from other universities and other countries

33. Corpora Linguistics 23.08.0433 Corpógrafo team Belinda Maia - FLUP -Associate Professor Luís Sarmento - Linguateca, FCCN – Computer Engineer - Researcher-in-charge Luís Miguel Cabral - Linguateca, FCCN – Computer Engineer, Research assistant Débora Oliveira - Linguateca, FCCN – Research assistant Ana Sofia Pinto – FLUP – technical assistant

34. Corpora Linguistics 23.08.0434 Contacts If you are interested is finding out more, please contact me: Belinda Maia at bmaia@mail.telepac.ptbmaia@mail.telepac.pt Or Luís Sarmento at las@letras.up.ptlas@letras.up.pt The Corpógrafo can be used (with a username and password) at: http://www.linguateca.pt/corpografohttp://www.linguateca.pt/corpografo and http://poloclup.linguateca.pt/ferramentas/gc

35. Corpora Linguistics 23.08.0435