1. Introduction to FSM Toolkit Examples: Part I NLP Course 07

2. Example 1  Acceptor for “sheeptalk”: /baa+!/ Text Representation Symbols File (sheep.txt) (S.syms) 0 1 b eps 0 1 2 a a 1 2 3 a b 2 3 3 a ! 3 3 4 ! w 4 4 o 5 u 6 u 6 f 7 f 7 -Symbols w, o, u and f are needed for the 2 nd example. -eps symbol stands for possible future epsilon transitions.

3. Example 1  fsmcompile –i S.syms sheep.txt > sheep.fsa  fsmdraw –i S.syms sheep.fsa | dot –Tps > sheep.ps Image format: PostScript. For jpg write: Image format: PostScript. For jpg write: fsmdraw –i S.syms sheep.fsa | dot –Tjpg > sheep.jpg fsmdraw –i S.syms sheep.fsa | dot –Tjpg > sheep.jpg

4. Write an acceptor for “dogtalk”: /wouf!/

5. Example 2  Acceptor for “dogtalk”: /wouf!/ Text Representation Symbols File (dog.txt) (S.syms)  same as Ex.1 0 1 w eps 0 (sheep & dog share 1 2 o a 1 the same symbols file) 2 3 u b 2 3 4 f ! 3 4 5 ! w 4 5 o 5 u 6 u 6 f 7 f 7

6. Example 2  fsmcompile –i S.syms dog.txt > dog.fsa  fsmdraw –i S.syms dog.fsa | dot –Tps > dog.ps

7. Having the 2 fsa for “sheeptalk” and “dogtalk”, use the appropriate function to generate an acceptor that accepts a “sheeptalk” OR a “dogtalk”. Having the 2 fsa for “sheeptalk” and “dogtalk”, use the appropriate function to generate an acceptor that accepts a “sheeptalk” OR a “dogtalk”.

8. Example 3  fsmunion sheep.fsa dog.fsa > shORdg.fsa  fsmdraw –iS.syms shORdg.ps

9. Having the 2 fsa for “sheeptalk” and “dogtalk”, use the appropriate function to generate an acceptor that accepts a “sheeptalk” AND a “dogtalk”, using the Having the 2 fsa for “sheeptalk” and “dogtalk”, use the appropriate function to generate an acceptor that accepts a “sheeptalk” AND a “dogtalk”, using the constraint that sheep talks first! constraint that sheep talks first!

10. Example 4  fsmconcat sheep.fsa dog.fsa > shANDdg.fsa  fsmdraw –iS.syms shANDdg.ps

11. But the Society of Animals is always fair! This time let the dog to speak first…!!! But the Society of Animals is always fair! This time let the dog to speak first…!!!?

12. Example 5  Generate the following weighted FSM:

13. Example 5 Text Representation Symbols File (A.txt) (S2.syms) 0 1 red 0.3 eps 0 1 3 blue 0.7 red 1 0 2 green 0.4 blue 2 2 3 yellow 0.8 green 3 3 0.3 yellow 4 4 0.4 As before: fsmcompile, fsmdraw

14. Which is the path with the lowest cost?

15. Example 5  fsmbestpath A.fsa > B.fsa  fsmdraw –iS2.syms B.ps

16. Integrating the power of Perl with the FSM Toolkit

17. Perl & FSM Toolkit  Problem Definition: We have as input a file containing a single sentence of lower case words. “ hi nlp world” Goal: transform the above words into upper case using FSM. “ HI NLP WORLD”

18. Perl & FSM Toolkit  A Perl script (composition.pl) that: 1. Extracts the lower case words from the input file 2. Generates the corresponding transducer 3. Generates a second transducer that transforms each word to its’ upper case form 4. Compose the two transducers 5. Projects the output of the resulted transducer 6. Extracts the output of the above transducer by reading the appropriate file and prints the upper case sentence to the screen

19. #!/usr/bin/perl open (IN, $ARGV[0]) || die “error"; $rdln = ; @in_wrds = split(/\s+/,$rdln); close(IN); # write the files for the transducers open (OUT_T11, ">T11") || die "error"; open (OUT_T12, ">T12") || die “error"; @low_up_words=@in_wrds;$c=0; foreach $tmp (@in_wrds) { print OUT_T11 ($c,"\t",$c+1,"\t",$tmp,"\t",$tmp,"\n"); print OUT_T12 ($c,"\t",$c+1,"\t",$tmp,"\t",uc($tmp),"\n"); push (@low_up_words,uc($tmp)); #gather lower and upper case words $c++;} print OUT_T11 ($c,"\n"); print OUT_T12 ($c,"\n"); close(OUT_T1);close(OUT_T2);

20. # write symbols file $i=1; open (OUT_S12, ">S12") || die “error"; foreach $tmp (@low_up_words) { print OUT_S12 ($tmp,"\t",$i,"\n"); $i++;}close(OUT_S12); # call the FSM Library system ("fsmcompile -iS12 -oS12 -t T11.fst"); system ("fsmdraw -iS12 -oS12 T11.ps"); system ("fsmcompile -iS12 -oS12 -t T12.fst"); system ("fsmdraw -iS12 -oS12 T12.ps"); system ("fsmcompose T11.fst T12.fst > T12comp.fst"); system ("fsmdraw -iS12 -oS12 T12comp.ps");

21. system ("fsmproject -2 T12comp.fst > final_out.fsa "); system ("fsmdraw -iS12 final_out.ps"); system ("fsmprint -iS12 final_out"); # Finally, read the resulted file and extract the field of interest open (IN2, "final_out") || die "can not open the input file...\n"; $rdln2 = ; while ($rdln2 ne "") { @out_wrds = split(/\s+/,$rdln2); push (@up_wrds,$out_wrds[2]); $rdln2 = ; }close(IN2); # print the upper case content of the initial input file print (join(" ",@up_wrds),"\n");

22. Perl & FSM Toolkit First fst (T11.fst) Second fst (T12.fst) 0 1 hihi 0 1 hi HI 12nlpnlp 1 2 nlp NLP 23worldworld 2 3 world WORLD 3 3 Symbols File (S12) hi1 nlp2 world3 HI4 NLP5 WORLD 6

23. Perl & FSM Toolkit  Compose T11.fst and T12.fst system ("fsmcompose T11.fst T12.fst > T12comp.fst"); system ("fsmdraw -iS12 -oS12 T12comp.ps");

24. Perl & FSM Toolkit Project the output of the resulted transducer: system ("fsmproject -2 T12comp.fst > final_out.fsa "); Draw the final_out.fsa: system ("fsmdraw -iS12 final_out.ps"); Print a textual description of the above fsa: system ("fsmprint -iS12 final_out"); Read the textual this textual description using Perl: open (IN2, "final_out") || die "can not open the input file...\n"; $rdln2 = ;...

25. Perl & FSM Toolkit Textual description of final_out.fsa: 0 1 HI 12 NLP 23 WORLD 3

26. Simple extra exercises

27. Extras 1  Generate the following acceptor, determinize and minimize it

28. Extras 2  Generate the following transducers and find their composition