Research Design and Writing

Carolyn Penstein Rosé

Language Technologies Institute/

Human-Computer Interaction Instituteand

Eric Nyberg

Language Technologies Institute

“If you can not measure it, you can not improve it.”-Lord Kelvin

Plan for the Day Continue unit on

Statistical Analysis

Endorsement of data analysis and scientific inquiry from Quincy

"Gentlemen, you are about to enter the most fascinating sphere of police work: the world of

forensic medicine."

Data Analysis According to Miss Marple Be nosey but discrete Be bothered by

inconsistencies Consider the possibility of

mysterious “factor X” Think about analogies with

scenarios you have observed in the past

Set a “trap” to confirm interpretation of data

http://www.youtube.com/watch?v=iMRN-i7KVxI

Useful Data Analysis Quotes from Poirot

If you are to be Hercule Poirot, you must think of everything.

I am an imbecile. I see only half of the picture.

Understand this, I mean to arrive at the truth. The truth, however ugly in itself, is always curious and beautiful to seekers after it.

I will not give up. Hercule Poirot will *never* give up.

Data Analysis Advice from Sherlock Holmes

“When you have eliminated all which is impossible, then whatever remains,

however improbable, must be

the truth.”Conon Doyle, A.

Eric’s Kojac Immatation…

Map Task The Map Task is a cooperative task involving two

participants. The two speakers sit opposite one another and each

has a map which the other cannot see. One speaker -- designated the Instruction Giver -- has

a route marked on her map; The other speaker -- the Instruction Follower -- has no

route. The speakers are told that their goal is to reproduce the

Instruction Giver's route on the Instruction Follower's map.

The maps are not identical and the speakers are told this explicitly at the beginning of their first session. It is, however, up to them to discover how the two maps differ.

Giver Map

Follower Map

Outcome Measure The main measure of task

performance that has been used for the Map Task is in terms of how far the route that the follower has drawn deviates from the route shown on the giver's map.

To reconstruct it, using the original A3 size maps, trace the giver's route on acetate marked with a one centimetre square grid, and impose it over the follower's map. The deviation score is the number of squares between the two routes.

What makes good dialogues good?

What makes good dialogues good?

What makes good dialogues good?

What makes good dialogues good? Hypothesis: Greater proportion of Receiver

speech should correlate with good task performanceFewer mistakes because Receiver confirms

understanding

No evidence!

Still no evidence!

What’s your hypothesis? What theoretical framework can you draw

from? What obstacles might there be to task

success, and how might you detect those with the measures you have?

Questions?