Speech Guidelines

2 of 59

1. Errors

VUIs are error-prone due to speech recognition. Humans aren’t perfect speech recognizers, therefore,

machines aren’t either.

Goal: Reduce the number and severity of errors.

3 of 59

1. Errors

Use Specific Error Messages Limit Background Noise Allow the User to Turn Off the Input Device Provide an Undo Capability Use Auditory Icons Use Multi-Modal Cues for Errors If Applicable Don’t Assume People Hear Everything

4 of 59

Use Specific Error Messages

Bad Example System: “Say the departure date.” User: “Tomorrow.” System: “Say the departure date.” User: “I want to travel tomorrow.” System: “Say the departure date.”

5 of 59

Use Specific Error Messages

Good Example System: “Say the departure date.” User: “Tomorrow.” System: “I don’t understand that date. Say the

month, date and year. For example, say October 13th, 2003.”

User: “July 1st, 2003.”

6 of 59

Use Specific Error Messages

An adequate error message tells the user what is wrong, why and how to correct it.

7 of 59

Limit Background Noise

Background noise is input.

Computer hears the background, not the user.

8 of 59

Allow the User to Turn Off the Input Device

This reduces background noise errors.

For VUIs, allow the user to place the system in an ignore mode System ignores input until a keyword is

spoken, i.e. “I am back”.

9 of 59

Provide an Undo Capability

Build in ways for users to cancel out, go back and undo actions.

10 of 59

Use an Auditory Icon

Auditory Icons are sound clips with a message.

When errors occur, play an auditory icon to notify the user.

11 of 59

Use Multi-Modal Cues for Errors If Applicable

Use more than one mode to signal an error, if possible.

Play an auditory icon, display a message and speak a message.

12 of 59

Don’t Assume People Hear Everything

Just because the system spoke it, doesn’t mean the user heard it.

Say important information first or last to improve the likelihood of it being heard.

13 of 59

2. Feedback

During HCI, the user needs feedback from the computer.

When a user issues a command, the system should acknowledge that the user has been heard.

Users also want feedback when the system is busy.

14 of 59

2. Feedback

Supply Alternative Guesses Acknowledge the User’s Speech Show When It Is the User’s Turn to Talk Allow for Verification Use Non-Speech Audio for Transitions Use In-Progress Messages

15 of 59

Supply Alternative Guesses

Users may say one word, but the computer hears a different word. (IDEAL SOLUTION) i.e. User says “Boston” and the computer hears “Austin”. The computer should respond “Did you say Austin or

Boston?”

This is easier said than done because you have to know all the words that sound alike in order to accomplish this for a large vocabulary.

16 of 59

Supply Alternative Guesses

Repeat what the user said and allow the user to correct what was recognized. (REAL SOLUTION) i.e. User says “Boston” and the computer hears “Austin”. The computer should respond “You said Boston, is that

correct?”

17 of 59

Acknowlege the User’s Speech

When the user speaks, provide feedback that she was heard. Auditory Icon Go to the next option If the next option is time consuming, let the user know in

advance. i.e. “I heard you, let me process your request”

18 of 59

Show When It Is the User’s Turn to Talk

In a multi-modal user interface, provide the user with a visual cue that the computer is listening.

19 of 59

Show When It Is the User’s Turn to Talk

In a VUI, provide the user with: Prompt Auditory Icon

20 of 59

Allow for Verification

Users tend to verify more when using a speech interface versus a visual interface.

VUIs should allow the user to verify what is happening and what has happened.

21 of 59

Use Non-Speech Audio for Transitions

When the user issues a command that requires a transition, play an auditory icon to acknowledge the transition is under way.

Avoid non-speech feedback that sounds like equipment noise.

22 of 59

Use In-Progress Messages

If there is more than a 3 seconds delay between when the user issues a command and the system responds, issue an in-progress message.

For best results, your in-progress messages should be informative. i.e. tell the user their position in the wait queue when it

changes.

23 of 59

Use In-Progress Messages

Playing a musical auditory icon in the background doesn’t work alone, but it is better than nothing.

Combine the verbal message with music to have the best effect.

24 of 59

3. Confirmations

Confirmations are questions you ask of the user to be sure that the user has been heard correctly.

25 of 59

3. Confirmations

Use Confirmations Appropriately Ask for Clarifying Information Use Confirmations for Destructive or Predictable

Actions Be Specific

26 of 59

Use Confirmations Appropriately

Don’t over confirm You could overdo the confirmations by asking for a

confirmation for every input.

You have to balance the cost of making an error with the extra time and annoyance in requiring the user to confirm a lot of statements.

27 of 59

Ask for Clarifying Information

If the expected response has more than one known response, then you may want to clarify what the user said.

i.e. “Do you want to set up an appointment or contact the person by phone”

28 of 59

Use Confirmations for Destructive or Predictable Actions

If the user’s action is destructive, delete files, require a confirmation.

If the user’s input prone to errors, require a confirmation. i.e. the grammar has a lot of sound alike

words.

29 of 59

Be Specific

If the system doesn’t recognize what was spoken, be specific about what you need.

i.e. “Please repeat the date again” vs. “Please

repeat” “Do you mean December 3rd?” is not a good

example, unless you are fairly confident.

30 of 59

4. Social and Environmental Issues

Interfaces are used in the context of people’s work and their physical environment.

Therefore, you have to take into consideration where and under what conditions will your interface be used.

31 of 59

4. Social and Environmental Issues

Decide on Flexibility Consider Stress Consider Social Interaction

32 of 59

Decide on Flexibility

Flexibility with respect to different tasks and users

You can also consider this as adaptability.

33 of 59

Consider Stress

Speech recognition doesn’t work well when the user is stressed, i.e. yelling.

Therefore, you have to consider the stress level of the environment.

34 of 59

Consider Social Interaction

You have to consider how your interface will influence social interaction between people.

i.e. if your interface is being used by a salesperson while the person is speaking to a customer. Will your interface cause attention problems for either person?

35 of 59

5. Command-and-Control

VUIs that recognize a limited vocabulary of individual words and phrases spoken by the user.

36 of 59

5. Command-and-Control

User Constraints Be Brief and Terse

37 of 59

User Constraints

Limit the user’s input through specific prompts.

38 of 59

User Constraints

Bad dialouge: System: “Welcome to the XYZ Company. We look forward

to servicing your travel needs. What are the dates of travel that you would like me to check for?”

User: “We are interested in traveling the first week of July, say July 1st to July 5th”.

The system’s statement is too open. This is a natural dialouge that humans understand.

39 of 59

User Constraints

Good dialouge: System: “Welcome to the XYZ Company. Say

the departure date of travel. For example, say October 1st, 2003.”

User: “July 4th, 2003” System: “Thank you. Say the return date.”

40 of 59

Be Brief and Terse

People model the length of system speech. If the system is lengthy, then the user will tend to

be lengthy.

The length of user speech is directly proportional to the number of recognition errors. The longer you speak, the chances of errors

increases.

41 of 59

6. Conversation and Prompting

VUI are bound by conversation and prompting.

The success of your VUI is dependent upon the conversation and prompting.

42 of 59

6. Conversation and Prompting

Choose Appropriate Words Avoid Personal Pronouns in Prompts Change Voices Appropriately Use Small Steps Avoid Long Prompts or Menus Use Progressive Prompting Make Prompt Direct and Exact

43 of 59

6. Conversation and Prompting

Use Non-GUI Terms Allow Relative Dates Avoid Long Pauses Choose an Appropriate Speed Use Tapering Use Barge-In

44 of 59

Choose Appropriate Words

Use “say” if you want the user to speak.

Use “enter” if you want the user to press a key.

45 of 59

Avoid Personal Pronouns in Prompts

Bad Example “Tell me your credit card number.”

Good Example “Say your credit card number.”

46 of 59

Change Voices Appropriately

Different voices can be used to signal a mode change. i.e. language or different part of the application

If you don’t want to change mode, use the same voice throughout.

Warnings and errors should use a different voice.

47 of 59

Use Small Steps

Query users in small steps, but not too small. Mixed Initiatives

48 of 59

Avoid Long Prompts or Menus

This goes back to the 7 plus or minus 2 memory limitation.

Don’t overload the user’s memory.

49 of 59

Use Progressive Prompting

If the user’s input is not recognized, then progressively provide more specific prompts with each misrecognition.

For example: System: “What would you like to do?” (BAD) User’s input is not recognized.

50 of 59

Use Progressive Prompting

System: “You can check your account status, obtain market quotes or view a list of other actions.”

User’s input is not recognized. System: “Say one of the following:

Account Status, Quotes or List.”

51 of 59

Make Prompts Direct and Exact

Tell the user exactly what you expect them to say.

Get directly to the point.

52 of 59

Use Non-GUI Terms

For example: delete is a GUI term … remove or erase are VUI terms.

53 of 59

Allow Relative Dates

For example: next Friday, yesterday, tomorrow, next week, next

month, etc.

54 of 59

Avoid Long Pauses

People don’t like dead air in conversation.

Use auditory icons or speech to avoid long pauses.

55 of 59

Choose an Appropriate Speed

If the systems speaks fast, then the users will speak fast.

Users will mimic the speed of the computer.

56 of 59

Use Tapering

If you must provide the same message over and over use tapering. Reduce the prompt because the user is

learning.

For example: 1st time the user logs in – System: “Say the

street address, city and state of the business.”

57 of 59

Use Tapering

2nd time the user logs in – System: “Say the street, city and state.”

3rd time the user logs in – System: “Say the address.”

Tapering is an adaptive technique that works on the basis that people will learn the input appropriate syntax to speak.

58 of 59

Use Barge-In

Allow users to interrupt the computer’s speech. This is barge-in.

59 of 59

Use Barge-In

When a group of users have adapted to an VUI and they barge-in, they barge-in with 2 seconds of the introduction (maybe less).

So, if you have to change the options of the VUI, how do you notify the users if they barge-in within 1 second?