1. Domain Act Classification using a Maximum Entropy model Lee, Kim, Seo (AAAI unpublished) Yorick Wilks Oxford Internet Institute and University of Sheffield www.dcs.shef.ac.uk

2. Why are we reading this unpublished paper?  It proposes a pretty clear ML model using a standard method (ME) but which is novel in its application to dialogue and it is easy to see how to do better than them and gain some publishable traction.  Basically it tries to learn over DAs (Dialogue Acts) as well as conceptual content--of very much the type we propose.  It gets better DA figures than Webb by ML over both at once.  It suggests figures would be even better if they had measured the DEPENDENCE between the two.

3. Sample of the annotation they need for their classifier.  When is the changed date?  [System: ask_ref+change-date]  It’s December 5th.  [User: response+change-date {date=December 5th}]

4. Types of information annotated  System or User  Speech/Dialogue Act (from a set of 11, e.g. ask_if=YNQ)  Concept (from domain set: e.g. change-date, information-object, which function as n-place predicates)  Objects that are values of the predicate variable, e.g date=5 December)  ALSO actions in domain tied to instantiated predicates (e.g. Timeble:Insert:Date)

5. The overall classification task   To derive a general classifier assigning speech acts AND concepts at once, treating them as independent.   EVEN THOUGH they can be seen not to be   I.e. SA/DAs based on local evidence and sequence AND   Concepts based on local evidence and sequence   Big fat ME expression to do these all in one.   DA element not very different from Webb method: both used lexical evidence, POS tags and n-grams.

6. Key example of combination of local/global and SA/C information.  When is the changed date?  [System: ask_ref+change-date]  It’s December 5th.  [User: response+change-date {date=December 5th}]  Rather than  [User: inform+information-object {date=December 5th}]  BUT THIS CANT BE DONE WITHOUT LINKING SPEECH ACTS AND CONCEPTS

7. Results  SA/DA rising to 93% precision after 1000 turns;  Concepts rising to 90% slightly later  DA set seems very small (how compare Webb and DAMSL? His figures less good).

8. What can we take from this?  Cf. old arguments about limits on DA accuracy without semantic content.  Cf. Interactions local/global in Jelinek  BUT THEY DON’T ACUALLY DO IT, SO WHY THE BETTER FIGURES?