Entrainment on Backchannel‐ Preceding Cuesrlevitan/papers/acl_talk.pdf · • Does the way one speaker invites a backchannel aﬀect how her partner invites his next backchannel?

EntrainmentonBackchannel‐PrecedingCues

RivkaLevitan Agus;nGravano JuliaHirschberg

1

Entrainment

'Aretheirheadsoff?'shoutedtheQueen.

'Theirheadsaregone, if itpleaseyourMajesty!'thesoldiersshoutedinreply.

'That's right!' shouted theQueen. 'Can youplaycroquet?’

'Yes!'shoutedAlice.'Come on, then!' roared the Queen, and Alice

joined the procession, wondering verymuchwhatwouldhappennext.

‐‐Alice’sAdventuresinWonderland

2

Entrainment

'Aretheirheadsoff?'shoutedtheQueen.

'Theirheadsaregone, if itpleaseyourMajesty!'thesoldiersshoutedinreply.

'That's right!' shouted theQueen. 'Can youplaycroquet?’

'Yes!'shoutedAlice.'Come on, then!' roared the Queen, and Alice

joined the procession, wondering verymuchwhatwouldhappennext.

‐‐Alice’sAdventuresinWonderland

3

Backchannels

Speaker A: All right so I have a- a a nail on top Speaker B: okay Speaker A: with an owl in the lower left.

Backchannel:ashortsegmentofspeechuWeredbyalistenertoconveythatsheispayingaWenXon.

4

A.Gravano&J.Hirschberg,2009Backchannel‐invi,ngcuesintask‐orienteddialogue

T1: speech <pause> speech <pause> speech T2: <backchannel>

5

A.Gravano&J.Hirschberg,2009Backchannel‐invi,ngcuesintask‐orienteddialogue

T1: IPU <pause> IPU <pause> IPU T2: <backchannel>

6

Entrainmentonbackchannel‐precedingcues

•  DifferentspeakersdisplaydifferentcombinaXonsofcues.

•  Differentspeakersmayrealizeacueindifferentways.

7


•  DifferentspeakersdisplaydifferentcombinaXonsofcues.

•  Differentspeakersmayrealizeacueindifferentways.

•  Dospeakerpairsentrainonbackchannel‐precedingcues?

8


•  Threemeasuresofbackchannelcueentrainment:– Measure1:Cuesets– Measure2:CuerealizaXon– Measure3:LocalcuerealizaXon

9

10

TheColumbiaGamesCorpus• Twelvespontaneoustask‐orientedconversaXons• IneachconversaXon,twopeople,separatedbyacurtain,playedcomputergames.• ThirteenpeopleparXcipated;elevenparXcipatedintwosessionseach.• AllfeaturesextractedautomaXcallyinPraat.

Measure1:Similarcuesets

•  Dospeakersusesimilarsetsofcues?

Speaker1RisingintonaXonHigherpitch

LongerduraXon

Speaker2RisingintonaXonHigherintensityLongerduraXon

Changeinvoicequality____________________________________

2cuesincommon

11




LongerduraXon



2cuesincommon

12



Yes(pairedt‐test,p<0.05)


LongerduraXon



2cuesincommon

13

Measure2:CuerealizaXon

•  Dospeakersrealizetheircuesinsimilarways?

Foranyfeaturefmodelingcuec:

thissessionexhibitsentrainmentonc.

14

€

d1,2f < min(d1,*

f , d*,2f )


15


16


17


18


19


20


•  Pitch,intensityandvoicequalityshowsignificantlysmallerdifferencesbetweenpartnersthanbetweennon‐partneredpairs.

21


•  Pitch,intensityandvoicequalityshowsignificantlysmallerdifferencesbetweenpartnersthanbetweennon‐partneredpairs.

•  Convergence:ThedifferencesbetweenpartnersinintensityandpitcharesmallerinthelasthalfofaconversaXonthanthefirst.

22

Measure3:Localentrainment

•  Doesthewayonespeakerinvitesabackchannelaffecthowherpartnerinviteshisnextbackchannel?

23

IPU backchannel IPU backchannel





24

IPU

IPU

IPU

Pearson’scorrelaXon

IPU

IPU

IPU



25

IPU

IPU

IPU

Pearson’scorrelaXon

IPU

IPU

IPU

 SignificantcorrelaXonsformeanpitchandintensity(r=0.3).


•  Threemeasuresofbackchannelcueentrainment:– Measure1:Cuesets– Measure2:CuerealizaXon– Measure3:LocalcuerealizaXon

26


•  Threemeasuresofbackchannelcueentrainment:– Measure1:Cuesets✓– Measure2:CuerealizaXon– Measure3:LocalcuerealizaXon

27


•  Threemeasuresofbackchannelcueentrainment:– Measure1:Cuesets✓– Measure2:CuerealizaXon✓– Measure3:LocalcuerealizaXon

28


•  Threemeasuresofbackchannelcueentrainment:– Measure1:Cuesets✓– Measure2:CuerealizaXon✓– Measure3:LocalcuerealizaXon✓

29

AssociaXonwithdialoguesuccess

•  Threemeasuresofdialoguesuccess:– Score– ProporXonofinterrupXonsS1: <unfinished speech> S2: <speech>

– MeanbackchannellatencyS1: <speech> S2: <pause> <backchannel>

30


31


32


33

Conclusions

  Speakersusesimilarsetsofcues.  Theyrealizethecuesinsimilarways.

  Thisentrainmentislocalaswellasglobal.

  ThisentrainmentisposiXvelyassociatedwithtasksuccessandnegaXvelyassociatedwithpoorcoordinaXon.

34

FutureWork

•  Whoentrainstowhom?•  WhatistherelaXonshipbetweendifferenttypesofentrainment?

•  Isthisentrainmentassociatedwithmoreperceptualmeasuresoftasksuccess?

35

Thankyou!

Documents

Entrainment on Backchannel‐ Preceding Cuesrlevitan/papers/acl_talk.pdf · • Does the way one speaker invites a backchannel aﬀect how her partner invites his next backchannel?