Research Design and Writing Carolyn Penstein Rosé Language Technologies Institute/ Human-Computer Interaction Institute and Eric Nyberg Language Technologies Institute “If you can not measure it, you can not improve it.” -Lord Kelvin

Plan for the Day Continue unit on Statistical Analysis

Endorsement of data analysis and scientific inquiry from Quincy "Gentlemen, you are about to enter the most fascinating sphere of police work: the world of forensic medicine."

Data Analysis According to Miss Marple Be nosey but discrete Be bothered by inconsistencies Consider the possibility of mysterious “factor X” Think about analogies with scenarios you have observed in the past Set a “trap” to confirm interpretation of data

Useful Data Analysis Quotes from Poirot If you are to be Hercule Poirot, you must think of everything. I am an imbecile. I see only half of the picture. Understand this, I mean to arrive at the truth. The truth, however ugly in itself, is always curious and beautiful to seekers after it. I will not give up. Hercule Poirot will *never* give up.

Data Analysis Advice from Sherlock Holmes “When you have eliminated all which is impossible, then whatever remains, however improbable, must be the truth.” Conon Doyle, A.

Eric’s Kojac Immatation…

Map Task The Map Task is a cooperative task involving two participants. The two speakers sit opposite one another and each has a map which the other cannot see. One speaker -- designated the Instruction Giver -- has a route marked on her map; The other speaker -- the Instruction Follower -- has no route. The speakers are told that their goal is to reproduce the Instruction Giver's route on the Instruction Follower's map. The maps are not identical and the speakers are told this explicitly at the beginning of their first session. It is, however, up to them to discover how the two maps differ. Giver Map Follower Map

Outcome Measure The main measure of task performance that has been used for the Map Task is in terms of how far the route that the follower has drawn deviates from the route shown on the giver's map. To reconstruct it, using the original A3 size maps, trace the giver's route on acetate marked with a one centimetre square grid, and impose it over the follower's map. The deviation score is the number of squares between the two routes.

What makes good dialogues good?

Hypothesis: Greater proportion of Receiver speech should correlate with good task performance  Fewer mistakes because Receiver confirms understanding

No evidence!

Still no evidence!

What’s your hypothesis? What theoretical framework can you draw from? What obstacles might there be to task success, and how might you detect those with the measures you have?

Questions?