Speech-Enabled .NET Framework Application for CIMS

Murat Semerci

Çağdaş Kayra Akman

15.06.2004

Project Goal

Development of speech-enabled voice-only version of Electrical and Electronic Engineering Department’s Course

Information Management System (CIMS) which is accessed via telephone network.

Development Environment

• Microsoft Speech Application SDK Beta 4

• Microsoft Visual Studio .NET 2003 Enterprise Architect

• Microsoft SQL Server 2000

• Windows 2000 Professional Edition SP 4 (Intel Pentium III 800 MHz, 256 MB RAM)

Overview

SERVER CLIENT

SR Engine

TTS Engine

SQL Server

Prompts Database

USER

Basic Dialog Flow

User Name

Password

Intelligent Mode ?

Course Code Course Code/Course Name and Menu

YESNO

Menu

AnotherQuery ?

YES

END

START

Error Handling

• No Recognition: no speech detected or recognized phrase not covered by grammar

• Invalid Input: recognized input does not match with user’s data (UserName, Password Course ID, Menu)

• No Database Connection

SALT

• Speech Application Language Tags: extension of HTML and XML to support speech-enabled web applications

• Generated by Speech Application SDK

• Manipulated directly or via C# codebehind pages

• Supports client-side scripts to handle recognition or TTS events or prompts

Grammars

• XML codes complying to W3C’s SRGS (Speech Recognition Grammar Specification)

• Used by SR engine on the client side.

• Results returned to server using W3C’s SML (Semantic Mark-up Language)

Grammars

Grammars<rule id="UserName" scope="public"> <one-of>

<item><?MS_Grammar_Editor GroupWrap?>

<one-of><item>

<?MS_Grammar_Editor GroupWrap?><ruleref uri="Library.grxml#Digit7" type="application/srgs+xml"/>

</item><item>

<?MS_Grammar_Editor GroupWrap?><ruleref uri="Library.grxml#Digit10" type="application/srgs+xml"/>

</item></one-of><tag>$.Username = $.Username ? $.Username : {}; $.Username._value = $$._value</tag>

</item><item>

<?MS_Grammar_Editor GroupWrap?><ruleref uri="Library.grxml#Cancel" type="application/srgs+xml"/><tag>$.Cancel = $.Cancel ? $.Cancel : {}; $.Cancel._value = $$._value</tag>

</item></one-of>

</rule>

QA Controls

• Interact with Prompts Database, TTS Engine, SR Engine

• Prerecorded vs. TTS prompts

• Prompts generated by combining recorded prompts or parts of recorded prompts

• 4 different timeouts for recognition timing: Initial, Babble, End Silence, Max

DATABASE

• A collection of interrelated data

• DB Models– Entity - Relationship Model– Relational Model – Network Models– etc.

CIMS

• A Relational DB using MS SQL Server 2000

• Entities →Relations→Tables– Users– Courses– etc.

SQL

• String Query Language– Update, Insert, Delete etc.

• Operations on CIMS– Make Announcements– Give Assignments– etc.

Voice-Only CIMS

• Connect to CIMS Database

• Query the necessary tables– Ex : SELECT ID, Code FROM COURSES

WHERE Enabled='1‘

• Update the tables– Ex: UPDATE USERS SET LAST_ACCESS =

GETDATE() WHERE USERNAME = ‘9902959'

Regular Expressions

• Text Processing and String Manipulations – Pattern Matching– String Splitting – Substring Replacement.

• Regex Class by C#.NET

Regex-CIMS

• Free Labeling– Ex: hw#3, Homework 3, Spice hw 3 …

• Using prerecorded prompts– homework 3– Regex.Replace(name,@"(?<preamble>\w*\s*)

(H|h)\w*(W|w)([^09]*(?<no>\d*)).*","${preamble}"+"homework "+"${no}")

Conlusion

• An IVR system, operating side-by-side with online version of CIMS developed

• Opportunity to gain knowledge about emerging technologies

• Satisfaction of developing a working system with zero initial knowledge

What to do next?

• Windows Server 2003

• Microsoft Speech Server (MSS) (released in May 2004)

• Telephone card

• Deployment onto MSS

Thanks for listening

Murat Semerci

Çağdaş Kayra Akman