Presentation is loading. Please wait.

Presentation is loading. Please wait.

K.U. Leuven Leuven 2008-05-08 Morphological Normalization and Collocation Extraction Jan Šnajder, Bojana Dalbelo Bašić, Marko Tadić University of Zagreb.

Published byChristian Clifford Owen Modified over 9 years ago

Similar presentations

Presentation on theme: "K.U. Leuven Leuven 2008-05-08 Morphological Normalization and Collocation Extraction Jan Šnajder, Bojana Dalbelo Bašić, Marko Tadić University of Zagreb."— Presentation transcript:

1 K.U. Leuven Leuven 2008-05-08 Morphological Normalization and Collocation Extraction Jan Šnajder, Bojana Dalbelo Bašić, Marko Tadić University of Zagreb Faculty of Electrical Engineering and Computing / Faculty of Humanities and Social Sciences jan.snajder @ fer.hr, bojana.dalbelo @ fer.hr, marko.tadic @ ffzg.hr Seminar at the K. U. Leuven, Department of Computing Science Leuven 2008-05-08

2 K.U. Leuven Leuven 2008-05-08 Collocation Extraction Using Genetic Programming Bojana Dalbelo Bašić University of Zagreb Faculty of Electrical Engineering and Computing bojana.dalbelo @ fer.hr, Seminar at the K. U. Leuven, Department of Computing Science Leuven 2008-05-08

3 K.U. Leuven Leuven 2008-05-08 Outine  Collocations  Genetic programming  Results  Conclusion

4 K.U. Leuven Leuven 2008-05-08  (Manning and Schütze 1999) “...an expression consisting of two or more words that correspond to some conventional way of saying things.”  Many different deffinitions...  An uninterrupted sequence of words that generally functions as a single constituent in a sentence (e.g., stock market, Republic of Croatia). Collocation

5 K.U. Leuven Leuven 2008-05-08 Collocation Applications:  improving indexing in information retrieval (Vechtomova, Robertson, and Jones 2003)  automatic language generation (Smadja and McKeown 1990)  word sense disambiguation (Wu and Chang 2004),  terminology extraction (Goldman and Wehrli 2001)  improving text categorization systems (Scott and Matwin 1999)

6 K.U. Leuven Leuven 2008-05-08 More general term - n-gram of words – any sequence of n words (digram, trigram, tetragram) Collocation extraction is usually done by assigning each candidate n-gram a value indicating how strongly the words within the n-gram are associated with each other. Collocation

7 K.U. Leuven Leuven 2008-05-08 Association measures More general term - n-gram of words – any sequence of n words (digram, trigram, tetragram) Collocation extraction is usually done by assigning each candidate n-gram a value indicating how strongly the words within the n-gram are associated with each other. Collocation extraction

8 K.U. Leuven Leuven 2008-05-08 Association measures Examples:  MI (Mutual Information):  DICE coefficient:

9 K.U. Leuven Leuven 2008-05-08 Based on hypothesis testing:   2 :  log-likelihood: Association measures

10 K.U. Leuven Leuven 2008-05-08 Collocation extraction Example: digramAssoc.measure value stock market20.1 machine learning30.7 town Slavonski10.0 New York25.2 big dog7.2 new house7.4 White house16.2

11 K.U. Leuven Leuven 2008-05-08 Collocation extraction Example: digramAssoc.measure value machine learning30.7 New York25.2 stock market20.1 White house16.2 town Slavonski10.0 new house7.4 big dog7.2

12 K.U. Leuven Leuven 2008-05-08 Association measures extensions Extensions:

13 K.U. Leuven Leuven 2008-05-08 Evaluation of AMs  Needed: sample of collocations and non-collocations  F 1 measure:

14 K.U. Leuven Leuven 2008-05-08 Our approach based on genetic programming  Similar to genetic algorithm  Population  Selection  Fittness function  Crossover  Mutation  GP: Evolution of programs in the forms of trees

15 K.U. Leuven Leuven 2008-05-08 Fitness function  Used to evaluate the “goodness” of a particular solution  Solution with a higher fitness function has a higher probability that it will be copied into the next generation during the selection process

16 K.U. Leuven Leuven 2008-05-08 Crossover  Combines two (parent) solutions into a new (child) solution  Used to exchange genetic material (in our case – portions of code) between solutions

17 K.U. Leuven Leuven 2008-05-08 Mutation  Introduces new genetic material into a population by randomly changing a solution  It can help adding genetic material that was not initially in the population (consider a population where no solution contains the addition operator – this operator can never be “invented” solely through crossover)  Also used to randomly “jump” through the space of possible solutions, thereby avoiding local maxima

18 K.U. Leuven Leuven 2008-05-08 Genetic programming  Idea – evolution of association measures  Fitness function – F 1

19 K.U. Leuven Leuven 2008-05-08 Genetic programming  Idea – evolution of association measures  Fitness function – F 1  Specifics:  Parsimony pressure  Stopping conditions – maximal generalisations  Inclusion of known AMs in the initial population

20 K.U. Leuven Leuven 2008-05-08 Nodes and leaves OperatorsOperands +, -const *, /f(.) ln(|x|)N IF(cond, a, b) POS(W)

21 K.U. Leuven Leuven 2008-05-08 Examples MI:DICE coefficient:

22 K.U. Leuven Leuven 2008-05-08 One solution Heuristics H:

23 K.U. Leuven Leuven 2008-05-08 Recombination (crossover) parents children  Exchange of subtrees

24 K.U. Leuven Leuven 2008-05-08 Mutation Node insertion: Node removal:

25 K.U. Leuven Leuven 2008-05-08 Experiment  Collection of 7008 legislative documents  Trigram extraction – 1.6 million  Two samples of classified trigrams:  Each sample 100 positive + 100 negative examples

26 K.U. Leuven Leuven 2008-05-08 Generalisation Stopping conditions – maximal generalisations maximal generalisations

27 K.U. Leuven Leuven 2008-05-08 Experimental settings  We used three-tounament selection  We varied the following parameters:  probability of mutation [0.0001, 0.3]  parsimony factor [0, 0.5]  maximum number of nodes [20, 1000]  number of iterations before stopping [10 4, 10 7 ]  In total, 800 runs of the algorithm (with different combinations of mentioned parameters)

28 K.U. Leuven Leuven 2008-05-08 Results  About 20% of evolved AMs reach F 1 over 80% Figure shows F1 score and number of nodes

29 K.U. Leuven Leuven 2008-05-08 Results

30 K.U. Leuven Leuven 2008-05-08 Results Interpretation of evolved measures in not easy (M205): f(abc) f(a) f(c) * / f(abc) f(ab) f(c) - f(c) f(bc) f(b) -f(abc) + / + / N * f(b) + * ln f(c) f(b) * * N f(a) * f(abc) f(a) f(abc) f(a) f(c) * / f(bc) * f(bc) f(b) + / f(a) N AKO(vr(b)={X}) * (-14.426000) f(b) + / N * f(bc) f(b) -(2.000000) * ln ln / f(a) f(c) * (2.000000) * ln ln / N * ln * / f(bc) * f(bc) f(b) + / N * (-14.426000) f(b) + / N * f(abc) N f(a) * f(a) f(abc) f(a) f(c) * / f(bc) * f(abc) f(b) + / N * (- 14.426000) f(b) + / N * f(b) f(c) * ln ln / f(abc) f(a) f(c) * / f(c) * ln ln (2.000000) * ln ln / N * / N * / N * ln f(c) * / f(a) f(b) + * ln ln f(abc) f(abc) f(a) f(a) N AKO(vr(b)={X}) (-14.426000) f(b) + * / N * / N * ln f(c) * / f(a) f(b) + * ln ln * ln ln / f(abc) f(a) f(c) * / f(a) f(b) + * ln ln (2.000000) * ln ln / N * ln ln AKO(vr(c)={X}) N * AKO(vr(b)={X})  Verification on other collections

31 K.U. Leuven Leuven 2008-05-08 Results Some results are more easily interpretable (M13): (-0.423000) f(c) * f(abc) / f(a) * f(abc) f(b) - AKO(POS(b)={X}) f(abc) /

32 K.U. Leuven Leuven 2008-05-08 Results  96% of measures with F 1 over 82% contain operator IF with condition “second word is a stopword”.

33 K.U. Leuven Leuven 2008-05-08 Conclusion  Standard measures are imitated by evolution  Genetic programming can be used to boost collocation extraction results for a particular corpus and to “invent” new AMs  Futher reasearch is needed:  Other test collections (domains, languages)  Extraction of digrams, tetragrams...

34 K.U. Leuven Leuven 2008-05-08  Jan Šnajder, Bojana Dalbelo Bašić, Saša Petrović, Ivan Sikirić, Evolving new lexical association measures using genetic programming, The 46th Annual Meeting of the Association of Computational Linguistic: Human Language Technologies, Columbus, Ohio, June 15-20, 2008.

35 K.U. Leuven Leuven 2008-05-08 Thank you

Download ppt "K.U. Leuven Leuven 2008-05-08 Morphological Normalization and Collocation Extraction Jan Šnajder, Bojana Dalbelo Bašić, Marko Tadić University of Zagreb."

Similar presentations

10/01/2014 DMI - Università di Catania 1 Combinatorial Landscapes & Evolutionary Algorithms Prof. Giuseppe Nicosia University of Catania Department of.

10/01/2014 DMI - Università di Catania 1 Combinatorial Landscapes & Evolutionary Algorithms Prof. Giuseppe Nicosia University of Catania Department of.
CS6800 Advanced Theory of Computation

CS6800 Advanced Theory of Computation
Exact and heuristics algorithms

Exact and heuristics algorithms
Student : Mateja Saković 3015/2011.  Genetic algorithms are based on evolution and natural selection  Evolution is any change across successive generations.

Student : Mateja Saković 3015/2011.  Genetic algorithms are based on evolution and natural selection  Evolution is any change across successive generations.
Genetic Algorithms Contents 1. Basic Concepts 2. Algorithm

Genetic Algorithms Contents 1. Basic Concepts 2. Algorithm
On the Genetic Evolution of a Perfect Tic-Tac-Toe Strategy

On the Genetic Evolution of a Perfect Tic-Tac-Toe Strategy
Biologically Inspired AI (mostly GAs). Some Examples of Biologically Inspired Computation Neural networks Evolutionary computation (e.g., genetic algorithms)

Biologically Inspired AI (mostly GAs). Some Examples of Biologically Inspired Computation Neural networks Evolutionary computation (e.g., genetic algorithms)
Content Based Image Clustering and Image Retrieval Using Multiple Instance Learning Using Multiple Instance Learning Xin Chen Advisor: Chengcui Zhang Department.

Content Based Image Clustering and Image Retrieval Using Multiple Instance Learning Using Multiple Instance Learning Xin Chen Advisor: Chengcui Zhang Department.
Non-Linear Problems General approach. Non-linear Optimization Many objective functions, tend to be non-linear. Design problems for which the objective.

Non-Linear Problems General approach. Non-linear Optimization Many objective functions, tend to be non-linear. Design problems for which the objective.
1 Lecture 8: Genetic Algorithms Contents : Miming nature The steps of the algorithm –Coosing parents –Reproduction –Mutation Deeper in GA –Stochastic Universal.

1 Lecture 8: Genetic Algorithms Contents : Miming nature The steps of the algorithm –Coosing parents –Reproduction –Mutation Deeper in GA –Stochastic Universal.
Learning to Advertise. Introduction Advertising on the Internet = $$$ –Especially search advertising and web page advertising Problem: –Selecting ads.

Learning to Advertise. Introduction Advertising on the Internet = $$$ –Especially search advertising and web page advertising Problem: –Selecting ads.
Evolutionary Algorithms Simon M. Lucas. The basic idea Initialise a random population of individuals repeat { evaluate select vary (e.g. mutate or crossover)

Evolutionary Algorithms Simon M. Lucas. The basic idea Initialise a random population of individuals repeat { evaluate select vary (e.g. mutate or crossover)
Bruxelles, 2006-03-10 Computer Aided Document Indexing System (CADIS) with Eurovoc Bojana Dalbelo Bašić Faculty of Electrical Engineering and Computing.

Bruxelles, Computer Aided Document Indexing System (CADIS) with Eurovoc Bojana Dalbelo Bašić Faculty of Electrical Engineering and Computing.
Khaled Rasheed Computer Science Dept. University of Georgia

Khaled Rasheed Computer Science Dept. University of Georgia
Image Registration of Very Large Images via Genetic Programming Sarit Chicotay Omid E. David Nathan S. Netanyahu CVPR ‘14 Workshop on Registration of Very.

Image Registration of Very Large Images via Genetic Programming Sarit Chicotay Omid E. David Nathan S. Netanyahu CVPR ‘14 Workshop on Registration of Very.
Brandon Andrews.  What are genetic algorithms?  3 steps  Applications to Bioinformatics.

Brandon Andrews.  What are genetic algorithms?  3 steps  Applications to Bioinformatics.
Christoph F. Eick: Applying EC to TSP(n) Example: Applying EC to the TSP Problem  Given: n cities including the cost of getting from on city to the other.

Christoph F. Eick: Applying EC to TSP(n) Example: Applying EC to the TSP Problem  Given: n cities including the cost of getting from on city to the other.
Genetic Programming.

Genetic Programming.
1 Reasons for parallelization Can we make GA faster? One of the most promising choices is to use parallel implementations. The reasons for parallelization.

1 Reasons for parallelization Can we make GA faster? One of the most promising choices is to use parallel implementations. The reasons for parallelization.
Leuven, 2007-05-22 Computer Aided Document Indexing System for Accessing Legislation A Joint Venture of Flanders and Croatia Bojana Dalbelo Bašić Faculty.

Leuven, Computer Aided Document Indexing System for Accessing Legislation A Joint Venture of Flanders and Croatia Bojana Dalbelo Bašić Faculty.

Similar presentations

Ads by Google