International Journal of Mini and Microcomputers, Vol. 10, No.2, 1988

A DISTRIBUTED COMPUTER SYSTEM FOR THE DEVELOPMENT OF VOICE

INTERACnVE APPLICA nONS

Claude Y. Laporte* and Jean-Louis Houle*'

SSI263 phonetic speech synthesizer circuit, manufacturedby Silicon Systems, contained in a single 24-pin CMOSintegrated circuit. It can synthesize speech in French andGerman as well as in English. The bit rate is approxima-tively 100 bits per second. The speech modules have twocommon features. First, each speech synthesizer circuit isinterfaced with the host CPU MC6809 through an identi-cal circuit: a MC6821 peripheral interface adapter (PIA)chip. Second, all speech modules have two outputs: a lowlevel output that can drive an amplifier and a high leveloutput that can deliver 1 W to an 8-ohm speaker, throughan LM-386 chip.

ABSTRACf

This paper describes the architecture of a distributed computer systemdesigned for the development of voice interactive applications. Thesystem is composed of two main elements: a microcomputer and afront-end processor. The front-end processor is a speech transactionprocessor used as a man-machine interface. It is designed to receiveverbal commands and deliver aural information and feedback to the

operator of the system.

Key Words

Microprocessor, voice command, distributed processing, development

system

INTRODUcnON1.2. The Speech Recognition Module

The speech recognition module is a speaker-dependentdevice built around the voice recognition chip set,VRC100-2, manufactured by Interstate Electronics Corpo-ration. The chip set consists of a 16-channel audio spec-trum analyzer, the ASA-16, and a 4K-bytes EPROM mem-ory circuit containing the processing algorithms. The sys-tem has been designed to recognize up to 200 words. Eachword must be neither shorter than 80 milliseconds norlonger than 1.25 second. A user must pause approximately160 milliseconds between words to allow the system torecognize word boundaries. The response time is a func-tion of the vocabulary size, since each word pronouncedmust be compared to every template stored in a RAM. Asan example, the response time, with a vocabulary of 50words, is 100 milliseconds. Figure 1 shows the block dia-gram of the speech recognition module: the circuit hasbeen assembled on two STD bus boards. An 8-bit MC6803microprocessor operating at a frequency of 2 MHz per-forms the operations related to the training of the systemor the recognition of words.

Speech technology is presently mature enough to startdesigning innovative applications. But since it involvesman-machine dialogues, it needs fine tuning before a sys-tem is left to an operator. In this paper, we describe adevelopment system that has been designed to properlyevaluate, develop, and validate such man-machine inter-faces. We describe the architecture of a front-end proces-sor and its associated firmware. We then describe itsapplications to radio frequency selection and robot con-trolled machining tasks.

I. THE FRONT -END SPEECH PROCESSOR

The speech processor designed around the widely usedSTD bus is composed of the following modules: a micro-computer board, a speech synthesis board, and a speaker-dependent speech recognition board. Since the hardware ofthese modules has been presented in previous publications[2, 3], we will only briefly describe the hardware of eachmodule and will present, in more detail, the software thathas since been developed. 1.2.1. Description of the development setup

We describe in this section the method followed torender the modules operational. Once the boards werewire-wrapped, we verified all the connections and installedthe chips. As expected, the system did not work completelythe first time. Since the programs stored on the EPROMare proprietary, we did not have access to the listing of thedifferent routines. It was therefore not possible to find theproblems with the boards configured as a speech recogni-tion system. We then improvised a debugging tool. Sincethe instruction set of the MC6803 processor is an en-hanced instruction set of the MC6800 microprocessor, weused a MC6800 microcomputer as a development system.We developed programs on the MC6800 microcomputerand downloaded them, through the PIA of the speechrecognition system, to the MC6803 microprocessor. Figure2 illustrates the setup. The MC6800 microcomputer wasfitted with a parallel interface board which uses a PIA as aparallel interface chip. On one end, a 4O-pin clip is mountedon top of the PIA, on the other end of a 4O-pin cable isattached a 4O-pin dual-in-line connector. This connector is

1.1. The Speech Synthesizer Modules

Four speech synthesizer modules were designed andassembled. The first module uses the pulse code modula-tion (PCM) technique. It is designed in such a way thatspeech can be digitized and synthesized at three differentsampling frequencies: 4, 6, and 8 kHz. The second moduleuses a continuously variable slope delta (CVSD) modula-tion circuit-the HC-55564 manufactured by Harris semi-conductor. The module can encode speech at different bitrates since it is equipped with an on-board clock samplingcircuit. A third speech module uses the linear predictivecoefficient (LPC) speech processor TMS5220 manufac-tured by Texas Instrument. The bit rate is approximatively1200 bits per second. Finally, the speech module uses the

*Physics Department, College Militaire Royal de Saint-Jean, Quebec,

Canada, JOJ lRO.**Electrical Engineering Department, Ecole Poly technique de Montreal.

Montreal, Quebec, Canada H3C 3A 7.

34

ANALOG BOARD

I~UT~[CROPHONE

ADDRESS BUS -

DATA BUS

CONTROL BUS~

24 PIN CABU

ADDR~S BUS DIGITAL BOARD

DATA~' I , 1

CONTROL BUc ..

~1-

PARAU.EL

INTERFACE

PIA

UPMC6803

EPROM

4K BYTES

RAM

BK BYTES

DATA BUS

ADDRESS BUS r

CONTROL BUSSTD BUS

Figure Speech recognition block diagram

inserted into the socket of the PIA of the speech recogni-tion system. The first program was written to exercise theentire address range of the MC6803. This allowed us toverify the proper operation of the address bus, theread/write, reset, and clock lines. We were also able tocheck all the address decoding logic circuits. Then we

wrote a routine to test the RAM chips and to verify theoperation of the PIA used to communicate with the micro-computer module of the STD bus. Satisfied that all thedigital circuits were operational, we wrote a loader, for theMC6803 processor, to download routines from the MC6800microcomputer to test the operation of the analog cir-

Figure 2. Interconnections between the 6800 microcomputer and the speech recognition system.

35

The MC6803 then jumps to the proper program and startsthe self-check routine by first verifying every RAM storagelocation; then it verifies the operation of each channel ofthe audio spectrum analyzer and the analog to digitalcoverter. Once these tests are completed, the MC6803processor transmits the results to the MC6809 processor.

1.2.2. Specch recognition system firmware

As mentioned above, the speech recognition algorithmsare stored on a 4K-byte EPROM. A set of 16 commandsallows the user to control the system. First, the self-testcommand verifies the operation for the analog and digitalcircuitries of the system. Second, the reset command ini-tializes input/output ports and flags. Third, a major com-mand is the train command, which creates the templates ofevery word of the vocabulary. It will also inform the user iftwo words sound too similar. The user can use this featureto reduce the possibility of misrecognition. When the userenters the training mode, the system asks how many wordshe wants to record and how many times he will repeat eachword. The system will then generate the template for everydifferent word spoken. The user is also requested to specifya reject threshold value. This value is used when the systemis in the recognition mode. When the user pronounces aword, the system generates a template and compares itwith the templates stored in a RAM during the trainingsession. The system selects the stored template which bestmatches the word just recorded: It computes a match scoreand compares it with the reject threshold. If the matchscore is equal to or higher than the reject threshold, thesystem will transmit to the host computer an index numberthat identifies the word recognized. Otherwise, the systemwill inform the host computer that it did not recognize aword with the precision requested. If the reject threshold isset too low, the system may make mis,takes (misrecogni-tion). Inversely, if the threshold is set too high, the systemmay not be able to identify a word (rejection). In the lattercase, the user is requested to repeat the word. The systemalso reports the index number of the word which achieved

cuitry. The download operation works as follows: first theMC6803 is reset; it then waits for data coming through thePIA. Next, the MC6800 sends the first byte to the PIA,activates the handshake line, waits for the acknowledgesignal, and repeats the procedure. The MC6803 processorlooks for a unique character to indicate that the downloadoperation is completed. Upon reception of this character,the MC6803 starts the execution of the program that it hasjust received. As shown on Figure 2, a signal generator isplugged to the microphone input of the speech recognitionmodule. This allowed ~s to verify the operation of theprogrammable gain amplifier. Then we measured the am-plitude of the signal coming out of each channel of theASA-16 spectrum analyzer. Finally, we verified the opera-tion of the analog to digital converter. Once all the irregu-larities were corrected, we were able to install the speechrecognition firmware.

All the communications between the speech recognitionmodule and the microcomputer module are done using aparallel interface circuit MC6821 peripheral interfaceadapter (PIA). Figure 3 shows the interconnections be-tween the MC6809 processor and the PIA of the speechrecognition module. Port A of the PIA is programmed toread data coming from the MC6803 while port B is pro-grammed to write data to the MC6803, lines CAl/CA2and CBl/CB2 are used as handshake lines. Figure 4illustrates the timing diagrams for the write and readoperations. The MC6809 initiates a write cycle by firststoring its data on port B, then it toggles the DOUT-RDYline (CB2). This line is tied to the interrupt request (IRQ)pin of the MC6803. The MC6803 then jumps to theinterrupt service routine, reads the data sitting at port B,and toggles the DOUT-REC line (CBl) of the PIA tosignify that the data transfer is completed. A similarsequence of operations is done when the MC6803 proces-sor wants to transmit data to the MC6809 processor. As anexample, when the MC6809 processor wants to check theworking condition of the speech recognition modules, ittransmits the command code 11 to the MC6803 processor.

Figure 3. Interconnections between the 6800 processor and the 6821 PIA of the speech recognition module

36

LJDOUT-RDY (6809)

DATA x

DOUT-REC (6803)

WRITE CYCLE TIMING

DjN=REC (6809) LJ

DATA x

READ CYCLE TIMING

Figure 4. Timing diagram for the transfer of data between the 6809 and 6803 processors.

the next best matching score, as well as giving its matchingscore. This way, a user is in a position to identify wordsthat may be misrecognized. The user may then modify hisvocabulary update command, mostly used when the sys-tem has problems identifying a particular word. The usermay then update the template. As an example, the usercould have been nervous when he trained the system forthe first time. As he gets more comfortable with this newinterface, his voice changes. Using the update command,he will adjust the template to the way he is now pronounc-ing. The set gain command allows the user to compensatefor the positioning of the microphone and for variation inthe loudness of the voice from one speaker to another.Since the templates are stored in RAM, the user may wantto save them before turning off the system. The uploadcommand transmits to a host computer the templates ofthe vocabulary. Inversely, the download command trans-mits to the speech recognition system the template re-quested. Using these two commands many subjects can usethe speech recognition system. The other commands allowthe user to read and write parameters like the rejectthreshold or the value of flags.

a CRT terminal and create, update, or modify a vocabu-lary. The monitor was very useful; it allowed us to discoverproblems that would have been hard to detect once anapplication program was written. The problem was thatthe rejection rate was too high, because new users weretense when they first trained the system-people are notused to talking to a machine. As the subjects becomefamiliar with the system, the error rate decreased substan-

tially.For our purposes, we have used only two RAM circuits,

out of a maximum of 8, to store the reference templates.This means that we can accommodate up to 50 words ofvocabulary .This was thought to be quite acceptable forour applications.

1.3. The Microcomputer Module

The microcomputer module uses an 8-bit MC6809 CPUdeveloped by Motorola. The MC6809 was selected becauseof its powerful instruction set and also because of previousin-house experience with Motorola's CPU and interfacecircuits. The board holds a 24-pin socket for a 2k-bytesscratchpad RAM and another 24-pin socket for a 2k-bytesEPROM. A 6850 asynchronous communications interfaceadapter (ACIA) has been used to communicate betweenthe MC6809 and the outside world. A 555 chip, wired asan astable multivibrator, has been used as a baud rategenerator. Another 555, wired as a monostable multivibra-tor, has been used as a reset circuit for the processor.Figure 5 shows the circuit diagram of the microcomputerboard.

Before we started developing any application programfor the speech recognition system, we decided to write amonitor that allowed us to familiarize ourselves with thisnew man-machine interface. The monitor program waswritten on an MC6809 development system and down-loaded on an EPROM. This chip was inserted on themicrocomputer module (see Section 2.3). The monitor al-lows the subjects to use any of the 16 commands, through

37

Circuit diagram of the microcomputer board.Figure 5.

The microcomputer module can be used together withthe speech recognition system as a training aid in order tofamiliarize users with this new man-machine interface. Wehave developed a monitor program that allows a user toenter in the training and recognition modes of operation asdescribed in Section 2.2. When the modules (speech recog-nition, speech synthesis, microcomputer) are configured asa front-end speech processor, the microcomputer board isfitted with another EPROM monitor. This monitor allowsthe master computer to send commands to the speechprocessor. The speech processor will then execute thecommand and return a date to the master processor, ifrequested. Since the user has a choice of four differentspeech synthesizer modules, he must, upon system initial-ization, inform the master processor. During the setupphase, the master computer will download, to the MC6809

module, the software routines for the configuration se-lected.

DESCRIP11ON OF THE MASTER COMPUTER2.

We have used, as a master computer, an 8-bit microcom-puter manufactured by Southwest Technical Products. Itsprocessor is an MC6809 chip developed by Motorola. TheSouthwest PC was configured with 56K bytes of RAMalthough up to 768K bytes are addressable. The main busis a 50-pin bus while the input/output bus has 30 pins. Weused floppy disks as secondary storage but hard disks arealso available. We also used off-the-shelf parallel and serialinterface modules. Unfortunately the 30-pin input/outputbus cannot accommodate direct memory access operations.The operating system, Flex, written by Technical Consul-

38

tant System, is easy to use and has all the utilities todevelop application programs. The user may also selectother operating systems like Uniflex or OS-9 from Mi-croware Inc.

A library of utilities was written in Assembler and inBasic. These programs allow the master computer to con-trol all the functions of the front-end speech processor. Auser can easily and rapidly develop an application andtest it.

target, using a joystick, while he is verbally directed,through a speech synthesizer module, to manually changefrequencies of radio sets. If the operator selects the secondprogram, he is first asked to train the speech recognitionsystem and then perform a series of voice activated fre-quency selections.

As a first experiment, volunteers were asked to performa series of ten manual frequency selections. The mastercomputer, through the speech synthesizer module, in-structed the subject to manually select a frequency. At thesame time, the subject was tracking a moving target with ajoystick. The master computer updated the position of thetarget, measured the position of the joystick, and the valueof the relative stress level of the subject. During a secondexperiment, the subjects were instructed to perform aseries of ten frequency selections using the speech proces-sor system. Results show that tracking a target whileperforming radio frequency selections, using a voice inter-active system, is more precise than the manual frequencyselection method, while stress level is not significantlychanged.

3. APPLICA DON OF THE SYSTEM TO RADIOFREQUENCY SELEcnON

The speech evaluation system is used in a project tocontrol radio frequency selection of an aircraft. Subjectswere asked to track a moving target on a CR T screen usinga joystick, while they were directed to make radio fre-quency selections. A test bench was assembled to partiallyreproduce the aircraft environment. It was divided intotwo parts: the front-end speech processor system and amaster computer. Figure 6 illustrates the block-diagram ofthe setup. On the right is the speech processor system withVHF and UHF radio controllers connected to an interfacemodule. Also shown are a headset and microphone con-nected to the speech synthesizer and the speech recogni-tion modules. This system is linked to a master computerthrough a serial interface port on the microcomputer mod-ule. On the left side of Figure 6 is the master computer: Itis the 8-bit microcomputer described in Section 3. A paral-lel interface module is used to measure the pilot stress leveland the position of the stick. Finally, a high resolutiongraphic screen (1024 by 1024 pixels) is used to display amoving target and the position of the tracking joystick.

Once the system is powered, a menu is displayed to theoperator. He may select one of the two familiarizationprograms. The first one asks the user to track amoving

4. APPLICATION IN A MACmNING ENVIRONMENT

In this application, the development system is beingused to control the operations of scaled-down machiningprocess. Figure 7 shows a block-diagram of the setup.Figure 8 is a photograph that shows the equipment used.Briefly, parts are brought on a tonveyor belt, manufac-tured by Rhino Robots Inc. [8], to a vision system, manu-factured by Micromint Inc. [9]. The task of this system isto identify the object. A computer-controlled robot arm,manufactured by Microbot Inc. [7], is then instructed topick up the part and place it on the pneumatic clamp of acomputer-controlled milling machine, manufactured byDyna Electronics Inc. [6]. The milling machine will thenperform a series of operations for this particular part.

Figure 6. Block-diagram of the test bench.

39

PNEUMATICCL..\MP

GRAPHICDISPLAY

SWT PC

MIU.ING

MACIUNEGRAPHIC

TRANSLATORSERIALPORT

SERIALPORT

ROBOT"'~

DIGITAL

CAMERA

II PARALLELPORT

CPU

6809FRONT-ENDPROCESSOR

SERIALPORT

STD BU S

SPEECHSYNTH.

SERIALPORT SPEECH

RECOGN

MEMORY

PROC.

MEMORY

SERIAL

PORT

SERIAL

PORT

Figure 7. Block diagram of the machining application.

showed that tracking a target. while performing radio fre-quency selection using a voice interactive system was moreprecise than the manual frequency selection method. Thevoice interactive system is presently being used in a projectto develop an industrial application.

REFERENCES

Figure 8. Vision System and other equipment

Once completed, the robot arm will pick up the part andstore it in a bin. The operator has full control of theoperations and uses the speech system to perform his task.

CONCLUSIONSs.

This system was used in a feasibility study to controlradio frequency selection in a military aircraft. Results

[11 Harvey, J.H., "Single-board Computers Boost System Throughput,"

Computer Design, Nov. 15, 1985, pp. 45-59.

[21 Laporte, C..Y., A Description of the Architecture of a Voice I/OEvaluation System and its Application in a Military Jet Aircraft,Proc. Speech Tech'85, Media Dimension, New York, pp. 313-316.

[31 Laporte, C.Y., and Houle, J.L., "Design of a Voice InteractiveSystem for Command Functions in Military Aircrafts," Proc. of

IASTED Int. Symposium: Applied Informatics, Innsbruck, Austria,

Feb. 18-20, 1986, pp. 23-28.

[41 Wiggins, R., and Brantingham, L., "Three-Chip System SynthesizesHuman Speech," Electronics, Vol. 51, No.18, pp. 109-116.

[51 Ciarca S., "Build 'a Third-Generation Phonetic Speech Synthesizer,"

Byte, Vol. 9 No. 3,1984, pp. 28-42.

[61 Dyna Electroni(;s, Inc., 2346 Walsh Ave., Santa Clara, California

95051.

[71 Microbot Inc., 453-H Ravendale Drive, Mountain View, California

94-43.

[81 Rhino Robot~ Inc., 3402 North Mat tis Ave., P.O. Box 4010, Cham-

paign, Illinois 61820.

[91.. Micromint Inc., 561 Willow Ave., Cedarhurst, New York.

40