1. Seed Generation and Seeded Version Space Learning Version 0.02 Katharina Probst Feb 28,2002

2. Seed Generation Type of Information Source of Information SL, TL sentence Informant AlignmentInformant Phrase Information Elicitation corpus, same as SL on TL SL POS sequence English parse (c,f) TL POS sequence English parse, TL dictionary X-side constraints English parse (f) Y-side constraints English parse, list of projecting features, TL dictionary (later: feature detection) XY constraints ---

3. Clustering ► Seed rules are “clustered” into groups that warrant attempt to merge ► Clustering criteria: POS sequences, Phrase information, Alignments ► Main reason for clustering: divide the large version space into a number of smaller version spaces and run the algorithm on each version space separately ► Possible danger: Rules that should be considered together (such as “the man”, “men”) will not be

4. The Version Space ► A set of seed rules in a cluster defines a version space as follows: The seed rules form the specific boundary (S). A virtual rule with the same POS sequences, alignments, and phrase information, but no constraints forms the general boundary (G): G boundary: virtual rule with no constraints S boundary: seed rules Generalizations of seed rules, less specific than rule in G

5. The partial ordering of rules in the version space ► A rule TR2 is said to be strictly more general than another rule TR1 if the set of f-structures that satisfy TR2 are a superset of the set of f- structures that satisfy TR1. It is said to be equivalent to TR1 if the set of f-structures that satisfy TR1 is the same as the set of f-structures that satisfy TR2. ► We have defined three operations that move a transfer rule to a strictly more general rule

6. Generalization operations ► Operation 1: delete value constraint, e.g. ((X1 agr) = *3pl) → NULL ► Operation 2: delete agreement constraint, e.g. ((X1 agr) = (X2 agr)) → NULL ► Operation 3: merge two value constraints to an agreement constraint ((X1 agr) = *3pl), ((X2 agr) = *3pl) → ((X1 agr) = (X2 agr)) [Note: if the first index is an X index and the second a Y index, this operation should only be performed if the feature is in the list of projecting features]

7. Merging two transfer rules At the heart of the seeded version space learning algorithm is the merging of two transfer rules (TR1 and TR2) to a more general rule (TR3): 1. Insert into TR3 all constraints that are both in TR1 and TR2 and remove them from TR1 and TR2. 2. Perform all instances of Operation 3 on TR1 and TR2 separately. 3. Repeat step 1. [Note: Operation 1 and Operation 2 are executed implicitly].

8. Seeded Version Space Algorithm 1. Remove duplicate rules from the S boundary 2. Try to merge each pair of transfer rules 3. A merge is successful only if the CSet of the merged rule is a superset of the union of the CSets of the two unmerged rules, where the CSet of a rule denotes the set of training sentences that are “covered”, i.e. translated correctly by the rule 4. Pick the successful merge that optimizes an evaluation criterion 5. Repeat until no more merges are found

9. Evaluating a set of transfer rules ► Initial thought: evaluate a set based on  The coverage of its rules, i.e. the union of its CSets  The size of the rule set ► Goal: maximize coverage and minimize set size ► Currently: merges are only successful if there is no loss in coverage, so size of rule set only criterion used ► Future(1): Coverage should be measured on a test set ► Future(2): Relax the constraint that a successful merge cannot result in loss of coverage

10. Next steps ► Compositionality, integration with transfer engine ► Exploring the space below the seed rules ► Specializing: we do not want a merge to be a final decision, want to allow for a rule to be “lowered” to a more specific rule ► What is the right inductive bias?