1. Automatic annotation of context and speech acts for dialogue corpora. K.Georgila, O.Lemon, Henderson, J., and J.D.Moore Basic idea of paper –Denser idea of annotation (for subsequent ML) –In COMMUNICATOR architecture What is proposed? –Annotation of each utterance with “full contextual state” as defined by the DM system

2. What does this “Information State” annotation mean? Annotation not just of linguistic features such as dialogue acts and semantic content but the setting of each “register”, or slot, relevant to the task/world that the dialogue corresponds to, e.g. DESTINATION-CITY for airline trip reservations. So each utterance is annotated with the whole state of the world/goals etc. at that point/state/

3. Problem in understanding what they propose Automated annotations are normally contrasted with hand-annotations as they are here; but HOW, even in principle, could a corpus be annotated by hand if that requires access to the register/slot contents as each utterance is “stepped through”? That is simply not what hand annotation means--there is no human intuitive skill being captured by this notion of annotation. how much is the annotated corpus, however produced, tied to the dialogue management system of the sort they advocate (since this is not “theory free” annotation)

4. In addition: They propose this for the 25 year old ATIS corpus Which is as dead as the WSJ in prose-- can any more be got from this corpse? Since the annotation is wholly dependent on their DM system----how can anyone not in their DM paradigm use the annotations? This dependence is not the norm (e.g. with syntactic annotation?!)

5. At best, this is only half the dialogue…… The ATIS corpus was originally annotated only for the user’s side of the dialogue. But how can the gold-standard annotation be provided for the other side of the dialogue--e.g. the register settings of the world/task on the alternate turns--how can these now be reconstructed? If they create a new DM to cover the data, how can the know the slot fillers are the same as what produced the original (Machine side response) data? Suppose they are false hypotheses….

6. What is the notion of “context” in use here? They seem to use “context” to mean “Information States” –Which is their own theoretical notion –Not explained till later in the paper than the (extensive) use of “context” –But I.S. is meant to explain “context” not vice versa! I.S. could be a trivial notion: I.e. the settings of all registers in combinations –So any system at all dealing with air travel bookings has a set of information states –Yet they use it confusingly to mean their (special) IS notion. –Either of these can be used to “model context” but they are not the same.