Inference Communications

Grammatical Inference real world implementations

..a new approach in practice

Callan Schebella

VP Business Development

Inference Communications

What is Grammatical Inference?

• Grammatical Inference is both: – a technology– a development methodology ←

• Grammatical Inference is the process of using machine learning techniques to automate the development of spoken dialog systems.

– any kind of speaker-independent system where grammar is of key importance

– In particular building speech recognition systems for call center environments ←

Classic Development Methodology

• Speech recognition engagements typically follow the following pattern:

1.Discovery (Requirements Analysis)

2.Simulation (Wizard of Oz)

3.Prototype

4.Pilot (limited functionality)

5.Deployment

6.Post Deployment

• Speech recognition engagements typically follow the following pattern:

1.Discovery (Requirements Analysis)

2.Simulation (Wizard of Oz)

3.Prototype

4.Pilot (limited functionality)

5.Deployment

6.Post Deployment

Grammatical Inference “Proof of Concept”

1–3 Days

Classic Development Methodology

Grammatical Inference - the methodology

• Grammars and call flow are developed right from the start and are developed concurrently.

– Sketch the call flow (usually with the customer) and then add the detail incrementally

– The sketch defines the underlying grammars• the ‘slots’ to be filled & the ‘types’ that will fill them• Typically 1 – 2 hours

– Example phrases are then provided to train and build the grammar automatically.

• “Grammar centric” Vs. “Call Flow centric”.– The aim is to get something you can talk to as quickly

as possible.

Grammatical Inference cont.

Grammatical Inference cont.

Define overall call flow

Grammatical Inference cont.

Define grammars in terms of slots

Grammatical Inference – in practice

• Where is it used?– Potentially, at every stage of the SDLC

• Requirements analysis– Collect customer requirements on the spot.

• Prototype development– Getting something to talk to

• Full application development– Traditional development processes

• Maintenance– Post deployment tuning

Continuous Speech Improvement

TestPrototype

Version X

Automated Learning

TestAutomated Learning

Initial protoype(s) developed in cooperation with customer and VUI experts.

Test with internal stakeholders via actual telephone calls.

Use Grammatical Inference to automate development of next version

Test with sample users via actual telephone calls.

Use Grammatical Inference to automate development of next version

TestVersion YAutomated Learning

Capture actual call data

Phase 1 - Define

Phase 2- Refine

Phase 3 - Tune

Focus on what the application should do

The aim of Phase 1 is to develop an interactive proof-of-concept application.

Focus on what the application should understand

Once a call flow is agreed the next step is to flesh out the grammars and back end linkages to provide a full natural language interface.

Focus on performance

Once call flow and grammars are complete a staged release can commence. Tuning the grammars is done via Inference’s Automated Learning technology.

Continue to use Grammatical Inference to improve performance and respond to changes in underlying application.

Human Factors

input

HumanFactors

Input

Still early days

• Focus to date has been on prototyping.– Many customers no longer satisfied with a

prepackaged application or audio file.

• A prototype:

1. Makes speech tangible– Nothing changes perception like an actual telephone

number.

2. Develops buy in from the stakeholder groups– They can interact with a prototype in their own time.

3. Highly effective at overcoming the key challenges in selling speech…

Grammatical Inference & Prototyping

• Common challenges:– Possibility: Does speech even work?

• “Company XYZ used speech recognition and they had problems.”

• Natural language – “They won’t believe it until they hear it!”

– Suitability: But my company is different. • “My customers don’t talk like that.”• “I need to talk to it”.• “I want some people to try it first” – Internal Selling

– Viability: Ok it works – but how much did that cost?• “What about the hidden costs?”• “I want to measure the performance”.

Different stages = different prototypes

• Recognize that different customers are at various stages of the process:

– Building prototypes for different purposes.– Tailoring each prototype to achieve the outcome

required.

• Proving the Concept• Assessing the Suitability• Proving the Viability

– Grammatical Inference is the ideal development methodology for getting answers to these questions with the minimum amount of cost and the minimum amount of time.

Case study #1 - Health Insurance

• New to speech• Required natural language• Had specific enquiries that were not common to many industries

Stakeholder Development work required

· 3rd Party Consultant Developing call center strategy

· IT Outsourcing partner & infrastructure provider

· Help Desk Mngr.

· COO

· Company Board

2 H

8 H

1 H

Initial prototype development using a grammatical inference methodology to create NL grammars.

Enhanced prototype using grammatical inference customize NL grammars (~100 training utterances).

Enhanced prototype with personalize caller data for the key stakeholders.

“Suitability”

“Viability”

“Possibility”

Case study #2 – Public Transport

• Had experience with speech, wary of promises• Huge number of diverse stakeholders• Customer charter – significant accessibility requirements

Stakeholder Development work required

· Outsourced Call Center provider

· Outsourced Call Center Technology Group

· Nuance

· Senior Management Team

· Company Board

8 H

8 H

Initial prototype development using grammatical inference to boot-strap grammars. Full context sensitive help, tapered prompts and DTMF fallback.

Switched to full recorded audio.

“Suitability”

“Viability”

“Possibility”

Case study #3 - Australia Post

• Massive organisation• Speech used in some areas – but not in others• ROI driven on a case-by-case basis (internal selling)

Stakeholder Development work required

· Help Desk Management

· Technical support group

· Senior Management Group

8 H

8 H

Initial prototype development using grammatical inference to boot-strap NLgrammars.

Switched to full recorded audio

Enhanced grammars with actual customer data. Added live agent support to facilitate the measurement of call parameters.

8 H

“Suitability”

“Viability”

“Possibility”

Conclusion

• Each customer was at a different point in the process. …by illustrating the required functionality the

customer was able to progress to the next stage.

• Very Cost Effective:– The most intensive demonstration only took 3 days

from start to finish.– The prototypes became the starting point for the real

system – evolutionary prototyping model.– The challenge is now to move from prototyping to full

application development.

Inference Communications