1. 1 Propositional Logic Syntax Acquisition Using Induction and Self-Organization Josefina Sierra Santibáñez Departamento de Lenguajes y Sistemas Informáticos Universidad Politécnica de Cataluña Spain

2. 2 Language Game: Guessing (Steels 1999) 1.Speaker: chooses a formula from a propositional language, generates a sentence for expressing the formula and communicates the sentence to hearer. 2.Hearer: tries to interpret the sentence generated by the speaker. If it can parse it using its lexicon and grammar, it extracts a meaning. 3.The speaker communicates the formula it had in mind to the hearer. They adjust their grammars to become sucessful in future language games. Success: speaker’s formula  hearer’s meaning

3. 3 Goals of the Experiments Observe the evolution of: –The communicative success: average of sucess- ful language games in the last ten language ga- mes played by the agents. –The internal grammars constructed by the individual agents. –The external language used by the population.

4. 4 Definite clause grammar: semantic, score, use s(right, 0.25, 20)  right s(light, 0.70, 50)  light s([P,Q], S, 12)  c1(P,S1,C1), s(Q,S2,C2), {S is S1  S2  0.01} c1(not, 0.80, 55)  not s([P,Q,R],S,3)  c2(P,S1,C1),s(Q,S2,C2),s(R,S3,C3), {S is S1  S2  S3  0.01} c2(and, 0.50, 35)  and Formula Meaning Sentence right  light [and, right, light] andrightlight  light [not, light] notlight

5. 5 Invention Generates a sentence E for a meaning M: –If M is atomic, it invents a new word E. –If M is a list, it tries to construct an expression for each of the elements in M using the agent’s grammar. 1.If it cannot construct an expression for an element using its grammar, it invents a new expression. 2.It concatenates the expressions associated with the elements of M randomly in order to construct a sen- tence E for the whole meaning M. Adds a new rule to the grammar: s(M, 0.01, 0)  E

6. 6 Adoption Communication fails because: The hearer cannot parse the speaker’s sentence: 1.The speaker communicates the formula it had in mind to the hearer. 2.The hearer adopts an association between that formula and the sentence used by the speaker. s(M, 0.01, 0)  E The hearer can parse the sentence, but its interpre- tation is not consistent with the speaker’s meaning. 1.Hearer decrase the scores of used associations. 2.It may adopt an association between the formula and the sentence used by the speaker.

7. 7 Induction The agents use some induction mechanisms to extract generalisations from the grammar rules learnt so far. The induction rules used in the experiments are based on the following rules proposed in (Kirby 2002): Simplification Chunk They are applied whenever the agents invent or adopt a new association.

8. 8 Simplification : scores (Vogt 05) and use counters r1  n(m1, S2, C2)  e1 r2  left(…m1…, S1, C1)  … e1 … Rule r2 is replaced with rule r3, of the form left(…X…, S3, 0)  … n(X,S,C)…, {S3 is S  0.01} or letf(…X.., S3, 0)  … n(X,S,C)…, {S3 is S  T  0.01} depending on whether S1 was a constant or a variable. In the second case, {S1 is T  sr1} is the arithmetic ex- pression on the right hand side of r1.

9. 9 Simplification : example 1 r1  s(right, 0.25, 16)  right r2  s([and,light,right], 0.10, 6)  andlightright Rule r2 is replaced with rule r3 r3  S([and,light,R], S, 0)  andlight, s(R, S3, C3), {S is S3  0.01} r4  s(light, 0.70, 25)  light Rule r3 is replaced with rule r5 r5  s([and,Q,R], S, 0)  and, s(Q, S2, C2), s(R, S3, C3), {S is S2  S3  0.01}

10. 10 Simplification : example 2 r1  s(right, 0.25, 16)  right r6  s([or,light,right], 0.10, 6)  orlightright Rule r6 is replaced with rule r7 r7  S([or,light,R], S, 0)  orlight, s(R, S3, C3), {S is S3  0.01} r4  s(light, 0.70, 25)  light Rule r7 is replaced with rule r8 r8  s([or,Q,R], S, 0)  or, s(Q, S2, C2), s(R, S3, C3), {S is S2  S3  0.01}

11. 11 Simplification : example 3 r1  s(right, 0.25, 16)  right r9  s([or,light,right], 0.10, 6)  lightorright Rule r9 is replaced with rule r10 r10  S([or,light,R], S, 0)  lightor, s(R, S3, C3), {S is S3  0.01} r4  s(light, 0.70, 25)  light Rule r10 is replaced with rule r11 r11  s([or,Q,R], S, 0)  s(Q, S2, C2), or, s(R, S3, C3), {S is S2  S3  0.01}

12. 12 Chunk I : scores (Vogt 05) and use counters r1  left(f(m1), S1, C1)  right(e1) , … r2  left(f(m2), S2, C2)  right(e2) , … A new category symbol n is created and rules added n(m1, 0.01, 0)  e1 n(m2, 0.01, 0)  e2 Rules r1 and r2 are replaced with rule r3, of the form left(f(X), S3, 0)  right  (n(X, S, C)), {S3 is S  0.01} or left(f(X), S3, 0)  right  (n(X,S,C)), {S3 is S  T  0.01}

13. 13 Chunk I : example 1 r1  s([and,Q,R], S, 18)  and, s(Q, S2, C2), s(R, S3, C3), {S is S2  S3  0.10} r2  s([or,Q,R], S, 23)  or, s(Q, S2, C2), s(R, S3, C3), {S is S2  S3  0.30} The following new rules are added to grammar c2(and, 0.01, 0)  and c2(or, 0.01, 0)  or Rules r1 and r2 are replaced with rule r3 r3  s([P,Q,R], S, 0)  c2(P, S1, C1), s(Q, S2, C2), s(R, S3, C3), {S is S1  S2  S3  0.01}

14. 14 Chunk I : example 2 r1  s([and,Q,R], S, 18)  and, s(Q, S2, C2), s(R, S3, C3), {S is S2  S3  0.10} r2  s([or,Q,R], S, 23)  s(Q, S2, C2), or, s(R, S3, C3), {S is S2  S3  0.30} Chunk cannot be applied to r1 and r2, because they place the expressions associated with the connectives and and or in different positions in the sentence.

15. 15 Chunk I : example 3 r1  s([and,Q,R], S, 18)  s(R, S3, C3), and, s(Q, S2, C2), {S is S2  S3  0.10} r2  s([or,Q,R], S, 23)  s(Q, S2, C2), or, s(R, S3, C3), {S is S2  S3  0.30} Chunk cannot be applied to r1 and r2, because they place the expressions associated with the arguments of the binay connective, Q and R, in different posi- tions in the sentence. Rules must agree on the positions of the expres- sions associated with the connectives and their arguments in the sentence.

16. 16 Chunk II : scores (Vogt 05) and use counters r1  left(f(X), S1, C1)  right  (n(X, S, C)), … r2  left(f(m1), S2, C2)  right  (e1), … Rule r2 is replaced with the following rule n(m1, 0.01, 0)  e1 right  (X) is the result of removing the arithmetic ex- pression {S3 is S  …} of the right hand side of r1.

17. 17 Chunk II : example 1 r1  s([P,Q,R], S, 25)  c2(P, S1, C1), s(Q, S2, C2), s(R, S3, C3), {S is S1  S2  S3  0.20} r2  s([iff,Q,R], S, 33)  iff, s(Q, S2, C2), s(R, S3, C3), {S is S2  S3  0.50} Rule r2 is replaced with the following rule c2(iff, 0.01, 0)  iff

18. 18 Chunk II : example 2 r1  s([P,Q,R], S, 25)  c2(P, S1, C1), s(Q, S2, C2), s(R, S3, C3), {S is S1  S2  S3  0.20} r2  s([iff,Q,R], S, 33)  s(Q, S2, C2), s(R, S3, C3), iff, {S is S2  S3  0.50} Chunk cannot be applied, because rule r1 places the expresion associated with the connective in first po- sition in the sentence and rule r2 places the expres- sion associated with the connective third position.

19. 19 Need for Coordination : The agents must reach agreements on how to name propositional constants and connectives a1: if  if a2: if  si order the expressions associated with the different components of non-atomic meanings consistently a1: not  un, 2pos: [not, right]  rightnot a2: not  un, 1pos: [not, right]  notright a1: if  bin, 2pos, inv: [if,right,light]  lightifright a2: if  bin, 2pos, noinv: [if,right,light]  rightiflight

20. 20 Self-organization : Coordinate agents’ grammars The agents construct a shared external language and prefer using the rules in that language over the rest in the rules in their individual grammars. The scores of the rules indicate the agents’ preferences meaning  sentence1 highest score competing sentences: sentence2, …, sentenceN sentence  meaning1 highest score competing meanings: meaning2, …, meaningN The score of a sentence (or meaning) is computed at generation (parsing) multiplying the scores of the rules involved (Vogt 2005).

21. 21 Score of a sentence (meaning): example s(right, 0.25, 20)  right c1(if, 0.50, 10)  if s(light, 0.70, 50)  light c2(if, 0.10, 15)  si s([P,Q,R],S,5)  c1(P,S1,C1),s(Q,S2,C2),s(R,S3,C3), {S is S1  S2  S3  0.10} s([P,Q,R],S,3)  c2(P,S1,C1), s(R,S3,C3),s(Q,S2,C2), {S is S1  S2  S3  0.01} Meaning: [if, right, light]  Generation Sentence: ifrightlight score: 0.50  0.25  0.70  0.10 Comp senten: silightright score: 0.10  0.25  0.70  0.01

22. 22 Coordination takes place at the third stage of a lan- guage game when the speaker communicates the meaning it had in mind to the hearer. hearer’s meaning  speaker’s meaning Speaker increases scores of rules  sentence decreases scores of rules  competing sentences Hearer increases scores of rules  meaning decreases scores of rules  competing meanings hearer’s meaning  speaker’s meaning Speaker and hearer decrease scores of rules they used for generating and interpreting the sentence.

23. 23 Reinforcement and Inhibition The rules used successfully are reinforced. The rules used for generating competing sentences or competing meanings are inhibited. The rules used for updating scores of grammar rules (Steels 1999) replace the original score S with S1  maximum(1, S + μ) if the score is increased S2  minimum(1, S  μ) if the score is decreased Purging: the rules that have been used more than 30 times and have scores  0.01 are removed from the agents’ grammars.

24. 24 Experiments Evolution Communicative Success: guessing game 5 agents, 15000 games about propositional formulas L={a, b, c, r, l, u} constructed using: , , , , 

25. 25 Guessing: negation  a1 s([not,Y], R)  2, f, s(Y,Q), {R is Q  1} a2 s([X,Y], R)  2, c1(X,P), s(Y,Q), {R is P  Q  1} c1(not, 1)  f a3 s([not,Y], R)  2, f, s(Y,Q), {R is Q  1} a4 s([not,Y], R)  2, f, s(Y,Q), {R is Q  1} a5 s([X,Y], R)  2, c1(X,P), s(Y,Q), {R is P  Q  1} c1(not, 1)  f

26. 26 Guessing: conjuction  (commut) a1 s([and,Y,Z], T)  3, dyp, s(Z,Q), s(Y,R), {T is Q  R  1} a2 s([X,Y,Z],T)  3, c2(X,P),s(Z,Q),s(Y,R), {T is P  Q  R} c2(and, 1)  dyp a3 s([X,Y,Z],T)  3, c1(X,P),s(Y,Q),s(Z,R), {T is P  Q  R} c1(and, 1)  dyp a4 s([X,Y,Z],T)  3, c4(X,P),s(Y,Q),s(Z,R), {T is P  Q  R} c4(and, 1)  dyp a5 s([X,Y,Z],T)  3, c4(X,P),s(Z,Q),s(Y,R), {T is P  Q  R} c4(and, 1)  dyp

27. 27 Guessing: implication  (non-com) a1 s([X,Y,Z],T)  1, c1(X,P),s(Y,Q),s(Z,R), {T is P  Q  R} c1(if, 1)  bqi a2 s([X,Y,Z],T)  1, c3(X,P),s(Y,Q),s(Z,R), {T is P  Q  R} c3(if, 1)  bqi a3 s([X,Y,Z],T)  1, c2(X,P),s(Y,Q),s(Z,R), {T is P  Q  R} c2(if, 1)  bqi a4 s([if,Y,Z], T)  1, bqi, s(Y,Q), s(Z,R), {T is Q  R  1} a5 s([if,Y,Z], T)  1, bqi, s(Y,Q), s(Z,R), {T is Q  R  1}