Reading Aid for Visually Impaired

Veera Raghavendra, Anand Arokia Raj, Alan W Black, Kishore Prahallad, Rajeev Sangal

Language Technologies Research Center,International Institute of Information Technology, Hyderabad

www.iiit.ac.in

Agenda

• Motivation• Challenges• Architecture of RAVI• Sub Systems• What is working and progressing• Salient features of RAVI

Motivation

• Visually impaired should be able to access and interact with the computer as a normal person.

• Visually impaired should be able to access computers in their native language - Indian languages

• Multi-Lingual screen reader

Challenges• Tracing the system context information • Identifying the active application and extracting the

text content • Identifying the language or script notation • Language processing or script/font conversion • Text normalization • Invoking the right TTS system • Building a high quality TTS system for a language • Real-time response of the system

WordPad Microsoft Word

Microsoft Excel

Microsoft PowerPoint

Internet Explorer

Language Based TTS Selector

Font Converters, Text Normalization Module and LTS

Text, Font Name Text, Font Name

Text

Speech Output

Text, Font Name

Phonetic Notation of Input Text

Text to Speech (TTS) Engine

Accessibility proxy & Stub

System Wide Accessibility Event

TracerKeyboard Event Tracer

Microsoft Outlook

Architecture of RAVI

Sub Systems

• Text Information Retrieval• System wide• Application specific

• Speech Synthesis• Font Conversion• Text Normalization• Letter to Sound (LTS) Rules• TTS for Indian and English Language (s)

Text Information Retrieval

• System Information Retrieval• Microsoft’s Accessibility Technology is used to retrieve the system

information• It allows application to expose the type, name, location, and current

state of all objects• And notifies any Windows event that leads to a user interface

change.

• Application specific Information Retrieval• Identifying the Active Application on the desktop• Retrieving the Text information from the Active Application

• Microsoft Active Accessibility (MSAA)• Component Object Model / ActiveX Controls • Windows Message processing

Speech Synthesis• Font Conversion

• True Type Fonts• Uses 1-256 ASCII characters to represent characters

• Character representation is different from one font to other [even in the same language]

• Separate converter required for each font

• Proprietary fonts

• Unicode

• A universal character set

• provides a unique number for each character in a language

• Supports all platforms

• Supports all the languages..Continued

• Text Normalization• This is a Taj Mahal• Welcome to Dr.A.P.J Abdul Kalam• Cost of this pen is Rs.50.• Monday 14 January 2008, 19:48 GMT• I need P3 2.6Ghz, 2GB RAM, 300 GB SATA with 17” LCD

screen.

• Letter to Sound Rules• Written form of the word is different from spoken form in some

languages like Hindi and Tamil. • Eg: /r aa m a/ is spoken as /r aa m/• /aa ch I y aa/ cab be spoken as /aa s I y aa/

..continued

• TTS for Indian and English Language (s)• Indian Languages

• A character in Indian language scripts is close to a syllable • a syllable and can be represented as C*VC* where C is a consonant

and V is a Vowel• Syllable based synthesizer has been built for Indian Languages• Collect few thousand of sentences which are phonetically balanced• Record those sentences using native and well known speaker of the

language• Create syllable database from the recorded sentences• To reduce the database between 20-40 MB, select the syllable units

which are neutral

• English• Microsoft SAPI 5.0 (default for Windows XP)

What is working and progressing• Working

• System Information Retrieval• Application specific Information Retrieval

• Font Conversion• True Type Fonts• UNICODE

• Text Normalization• Letter to Sound Rules• Indian Language TTS for Telugu

• Progressing• Indian Language TTS for Hindi, Tamil, and Kannada

Salient Features of RAVI

• Voice guided installation support • Automatically starting at system boot up and option to remove start up • Character echoing while typing • Detailed help documentation for usage and keystrokes combinations • This software integrates itself with almost all the variants of Windows

operating system • This system identifies the language, based on the current context (font

name) and automatically switches to the respective speech engine • For producing English voice, we have used “Microsoft TTS” system and

for Indian Language voices, we have used ‘Indian Language TTS’ system developed at IIIT-Hyderabad

• This software supports applications like MS Word, Notepad, WordPad, PowerPoint, Outlook and Internet Explorer

Thanks

Questions?