The Effect of Miscommunication Rate on User Response Preferences Hua Ai, University of Pittsburgh

Thomas Harris, Carolyn Penstein Rosé, Carnegie Mellon University

1 Motivation – Smart Home

SecuritySystem

Voice-control Air conditioner

Remote Water

Monitoring

Smart Kitchen

Garage Door Sensors

Personalized Room Modes

2 Speech Modality

Eyes & hands free

Human-like natural interaction

However, the undesirable error rate and the high cost of error resolution degrade system performance and frustrate users

3 Wizard-of-Oz Experiments

However,How likely will the user follow the system prompts to resort direct manipulation?

4 Conclusions

Miscommunication Rate: Low error rate (30%)Mid error rate (50%)High error rate (70%)User actions: Direct ManipulationRepeatRephraseUser

Game

Stereo

•While user does not always follow the system’s advice, it does have an effect on their choice regardless of misrecognition rate

•Significant effect of system response to miscommunication on the likelihood of user direct manipulation (P<.05) and Repeat (P<.05)• Overall, users were significantly more likely to continue with speech interaction when the system ask the user to repeat (p < .01)

•While we would predict that users are more likely to resort to direct manipulation as miscommunication rate increased, our surprising finding was that users were most likely to resort to direct manipulation where communication success was least predictable. we found a significant effect of High versus Low Certainty on the likelihood of Direct Manipulation (p < .005) and likelihood of continuing with speech (p < .01)

Prompt users to switch to direct manipulation when continuous errors occur:

• Direct manipulation serves as an interpretation of user’s speech input to train the speech recognizer

• Direct manipulation makes the user feel in control, thus increases user satisfaction rate

Experimental Design:•3X2 within subjects Factorial Design•explore the separate and joint effects of 3 miscommunication rate and 2 system response to miscommunications (askRepeat and showMe) on user behaviorExperimental setups:•9 subjects, 4 females and 5 males, recruited on CMU campus.•Each subject is required to complete 10 tasks with a stereo while staying “alive” in an eyes & hands busy video game.

•Sample task: turn the stereo on, play a CD, skip over a song, etc. •Subjects are told that the stereo is an intelligent stereo that can accept commands either via direct manipulation or through speech commands. No vocabulary list is given.•Subjects are randomly assigned to the 3 miscommunication rate conditions, 3 people per group.•A wizard plays the pre-recorded system prompts to interact with the subjects. •The wizard decides the faked miscommunication points according to a pre-defined random sequence. Then either askRepeat or showMe prompts are given.•The subject can response by Direct Manipulation or speech (Repeat or Rephrase).

System prompts: Miscommunication condition:[askRepeat] “Could you repeat that?”[showMe] “Could you show me what do you mean?”

Normal condition:[ok] “OK, I’ll do that for you.”[Done] “I’m done!”

“Show & Tell”