CONNECT: A Personal Remote Messaging and Monitoring …crystal.uta.edu/~zaruba/Publications/J5.pdfstructure to relay messages, reminders, ques-tions, and in general any other multimedia

IEEE Communications Magazine • September 2005 1010163-6804/05/$20.00 © 2005 IEEE

TOPICS IN CONSUMERCOMMUNICATIONS AND NETWORKING

INTRODUCTION

In this article we are introducing CONNECT — asystem specifically designed to help people withdisabilities and the elderly in their everydayinteractions with their care providers, communi-ty organizers, medical personnel, and other indi-viduals wishing to communicate with them.CONNECT clients receive personal portabledevices (PPDs) with which they can be in touchwith their care providers. PPDs are devices withcapabilities to attach to wide area wireless net-works (generally cellular networks) and have suf-ficient resources to run the custom writtensoftware required by CONNECT. Thus, typicalPPD platforms are sophisticated cell phones, i.e.,PocketPC, SmartPhones, Symbian, RIM (e.g.,Blackberry), and Palm phones. Caregivers canthen use the CONNECT Web-page server infra-structure to relay messages, reminders, ques-tions, and in general any other multimediainformation to clients. Clients will have thePPDs customized to their specific needs andwishes. PPDs will consider the activities and spe-

cial needs of their users in determining whenand how to represent and display messages (forexample, displayed in a large character font or“read” to them over the phone speaker), andhow to obtain responses from them. VariousPPDs used in the CONNECT project are depictedin Fig. 1.

In the last couple of decades substantialinnovative work has made computer usability,particularly that of personal computers, anarea of major development. The user’s view(user interface) has advanced from text-based,mono-chrome displays to high-resolution, vivid-color devices; from punch-card keyboards tosophisticated pointing devices and voice recog-nition; and from keyboard clacking noise torich sounds and music. While many users bene-fit from low-cost, easy-to-use, powerful com-puters, many other potential users are lockedout because of vision or hearing impairments,muscle and motor difficulties. and many otherdisabilities that restrict their use of computers.With PCs becoming commonplace, Braille key-boards, text-to-speech devices, blow-tubes,pressure sensitive screens, and an impressiveamount of software (ranging from keyboarddrivers that reduce multiple simultaneous keypresses to sophisticated screen readers) extend-ed personal computing to more people withdisabilities [1, 2].

On a different front, wireless cellular phonesare playing a significant role in our everydaylives. At the same time, functions that were ear-lier those of personal digital assistants (PDAs),for example contact lists and calendar tracking,are merged into cellular phones. Operating sys-tems of PDAs are enhanced to merge with cel-lular chipsets (e.g., Windows PocketPC andSmartPhones), and operating systems designedfor phone-PDAs are also popular (e.g., Sym-bian). Additionally, data access on such devicesis becoming more affordable with both cellularwide infrastructures becoming available (suchas GPRS, CDMA data, and even WiMax), as

Gergely Záruba, Farhad Kamangar, Manfred Huber, and David Levine, The University of Texas at Arlington

ABSTRACT

In this article we introduce CONNECT, aninfrastructure running on small wireless devices(e.g., PDA phones) and servers that aids peoplewith disabilities in leading their everyday lives.We refer to the target (client) devices as Person-al Portable Devices (PPDs). The architecture ofthe CONNECT system is described, with emphasison the accessibility services provided to itsclients. As an example, the Reminder Service ofCONNECT is profiled where care providers andcommunity organizers (or general users whohave appropriate rights) can send timelyreminders to the PPD owners, giving them achance to respond to these messages. In con-junction with the CONNECT project, 600 PPDsare deployed to people with disabilities for eval-uation purposes.

CONNECT: A Personal RemoteMessaging and Monitoring System toAid People with Disabilities

ZARUBA LAYOUT 8/22/05 11:21 AM Page 101

IEEE Communications Magazine • September 2005102

well as hotspot-type of wireless access (e.g.,WiFi variants or Bluetooth). These technolo-gies enable mobile Internet access and makePDAs into powerful tools for networked com-puting and communication [3] with great poten-tial for people with disabilities [4]. But again,accessibility, especially for people with disabili-ties, becomes a severe problem since handlingthese small devices can be a burden for manypeople [4, 5].

Modern technology has certainly assistedpeople with disabilities, but often failed toadapt to their particular needs. While a num-ber of approaches have been developed todesign interface representations that can beadjusted to the needs of the user and of thedevice [6, 7], they frequently do not take intoaccount requirements arising from the particu-lar service provided and require significanteffort on the part of the content provider.CONNECT utilizes state-of-the-art technology toallow an individual to have more control overcustomizing their PPD to better suit theirneeds. As an example, a person with a specificdisability may require more time than usual torespond to a query or a message presented toher, making it important that message deliverysettings can be customized and personalized. Acaregiver (or any other person with sufficientrights to send a message) within the CONNECTsystem can use the tools provided to send ashort message and specify the actual deliverytime of the message to the client. The PPD ofthe message recipient is going to present themessage according to the individual prefer-ences of the recipient and present her with sev-eral response options. The caregiver who hadsent the message may check back at a latertime to see if the message was delivered andwhat the client’s response was.

UNIVERSAL ACCESSAs Tim Berners-Lee, the director of W3C(World Wide Web Consortium) [8], states: “Thepower of the Web is in its universality. Access by

everyone regardless of disability is anessential aspect.” Much of the recentwork toward providing accessibilityto information has been in the cre-ation of standards and initiatives inthe World Wide Web. These effortsrange from the straightforward —ensuring that one’s Web pages canbe “read” by people who cannot seeand navigated by people with motor-skill difficulty — to XML documentsthat have been designed with accessi-bility considerations, as well as Web-document languages that allow voiceinput to Web-documents.

Several World Wide Web stan-dards and initiatives have been pro-posed or accepted by the W3C andare the basis for development of alarger infrastructure for providinguniversal access. WAI, the WebAccessibility Initiative, strives to pro-vide accessibility through “technolo-gy, guidelines, tools, education andoutreach, and research and develop-

ment.” WAI [9] and W3C provide many usefulstandards and guidelines for creating HTMLWeb pages that are more accessible. Specifically,they recommend avoiding the “lowest commondenominator,” text-only pages, which is a com-mon solution to creating accessible Web pages.

While XML allows “rich” content documents,it does not inherently provide any accessibilityguarantees. XAG [10] is an initiative to provideXML document accessibility, since XML docu-ments may be dialect-content specialized (e.g.,for music and math there are MusicML andMathML, respectively) and may contain complexstructure. XAG suggests alternate representa-tions of pieces of XML documents where appro-priate and possible. A device-independencegroup (www-di [11]) attempts to ensure thatWeb access, whether viewing pages or morecomplex activities (such as e-commerce applica-tions, applets, etc.), are available on a widerange of devices such as phones and PDAs.Device independence is an important part of anyaccessibility infrastructure.

The multimodal interaction activity group atW3C [12] is developing standards for “multiplemodes of interaction,” including speech, pen,and gesturing, that adapt to devices as well asuser preferences. A framework is being devel-oped to describe component informationrequirements for data as well as an annotationmarkup language that will provide semanticmeaning to speech, natural text, and similarinputs. In this framework an interaction manag-er component coordinates data flowing frominput and output modality objects. This interac-tion manager will also interface to componentsthat know about user preferences and environ-mental issues. Another component may managesessions of multiple applications or changingmodes.

VoiceXML [13] is designed to facilitate creat-ing audio dialogs containing synthesized speech,music, natural speech, and recognition of spokeninput or conversations. Telephone key (DTMF)and other responses as well as voice mixed

nnnn Figure 1. PPDs used in CONNECT (PocketPC, SmartPhone, and Symbian).


IEEE Communications Magazine • September 2005 103

sources for input are also allowed. A speechrecognition grammar specification allows naturalor synthesized voice, telephone key tones, andother types of inputs to be specified and recog-nized as valid. User-agents (browsers, mediaplayers, screen readers, etc.) are addressed bythe UAAG, which are accessibility guidelines foruser-agents. These guidelines are designed forbrowsers, media players, and similar agents; theyare not designed for screen magnifiers or screenreaders or other assistive technologies. Current-ly, there is no way to certify “conformance” tothese guidelines; indeed, a single user-agent isnot expected to meet all requirements.

As W3C groups refine and detail their stan-dards, guidelines, and frameworks, they willimpact the creation, management, and accessibil-ity of all types of information, forms, and inter-actions, including Web or page-presentedinformation.

A CONNECT SCENARIOA general block diagram of CONNECT players isdepicted in Fig. 2. Below we describe a typicalscenario outlining CONNECT’s main features.

Joan, a hearing impaired person, is given aPPD, and thus she “registers” (through a Web-page) with the CONNECT infrastructure. (Thismay be done by others on her behalf.) Register-ing does not require any personal informationabout Joan. Joan either uses the PPD itself, ormore likely a Web page on a PC, to tell theCONNECT server as much about herself as shefeels comfortable with (e.g., where she lives, herdoctor’s and pharmacist’s names, description ofher disabilities, how she prefers to lead her day).She may add or modify her personal profile atany time.

Later that day her doctor is called out oftown on an emergency. The doctor’s assistantaccesses a Web-page, identifies himself, and

sends a message (e.g., by filling out a Web formas in Fig. 3) to all patients with appointmentsthe next day, telling them that the doctor is outof town and that their appointments have beenrescheduled to the following day and asking ifthat is acceptable. Joan’s pharmacist may alsosend her a reminder about mixing an old andnew medicine, telling her that she should takethe old medicine in the afternoon at 2 PM,rather than in the morning, as she used to.Joan’s PPD receives these messages through awireless access network (likely GPRS). Remem-bering that Joan is taking a nap for an hour,when she wakes up the PPD tells her (“remindsher”) that her doctor has rescheduled her andinforms her of the reminder from her pharma-

nnnn Figure 2. Block diagram of CONNECT.

ClientsHuman service

providers

Technical support

Internet

Wireless communicationprovider

Servers, databases, Web pages

nnnn Figure 3. CONNECT Web page to send reminder messages.



cist, and asks her to respond to the acceptabili-ty of the new time for the doctor’s appoint-ment.

Several features are novel in this scenario.The doctor’s assistant and the pharmacist donot require any special software to send thesemessages to a group of people; only a Webbrowser capable of accessing Web pages isneeded. Patient responses can be handledeither automatically or manually. For example,patients who are not available the next daymay be rescheduled using a simple Web pageinterface or, if desired, a software interface tothe doctor’s scheduling system can automatethis task. In one simple Web-based interface ahealth provider can inform a group of clientsabout something of interest to them. For exam-ple, they may be informed about new side

effects noticed in a new medicine, which is ofparticular interest only to patients taking thatmedicine.

CONNECT SYSTEM ARCHITECTUREIn this section we outline each of the buildingblocks (modules) of the CONNECT system (avisual overview is provided in Fig. 4). We willuse the term message to refer to informationexchanged between the CONNECT server andclient PPDs. We can divide the modules into thefollowing three distinct categories.

Server side modules deal with interfacingcaregivers, storing messages and message logs ofusers, user behaviors, and system events. Theyare also responsible for communicating with des-ignated client-side infrastructure modules.

nnnn Figure 4. Architectural overview of the CONNECT system.

CONNECT databaseServer side

Web server

Caregivers

GPRSWiFiBluetoothUSB Internet

Cellular/POTS

Clientside

Message gateway(MGI)

Remote update(RUI)

Call handler(CHI)

Shell process(SPI)

Delivery agent(DAI)

User

Sens

ors/

actu

ator

s

Process databaseProcess manager

(PMI)

Reminder serviceprofile

Reminder service(RMS)

User profileservice profile

User profileservice (UPS)

Sensorymonitoring profile

Sensory monitoringservice (SMS)

News serviceprofile

News service(NWS)

User and deviceprofiles

Phone manager(PMAN)

As W3C groups

refine and detail

their standards,

guidelines, and

frameworks, they

will impact the

creation,

management, and

accessibility of all

types of information,

forms, and

interactions,

including Web or

page-presented

information.



Infrastructure modules (client side) needto be present in all CONNECT devices. Theyare responsible for receiving and sending com-munications to the server; representing mes-sages in a time and format according to theuser profile; and providing a seamless upgradeprocess. (Infrastructure module name abbrevi-ations end with an I .) All cl ient modules(whether infrastructure or service) are imple-mented in C++ to reduce footprint andincrease speed.

Service modules (client side): There may beone or several such modules. Each module isresponsible for one type of service. Currentlythere are four implemented services:• Reminder Service sends and receives two-

way reminders.• User Profile Service modifies and synchro-

nizes user preferences.• News Service receives timely news feeds on

subscribed topics.• Monitoring Service monitors blood pres-

sure, heart rate, and other physiological sig-nals.

INFRASTRUCTURE MODULESProcess Manager (PMI) — The Process Man-ager (PMI) is the heart of the client-side mod-ules, and is responsible for scheduling all of theother modules. All communication betweenmodules goes through an SQLite database. Thisapproach guaranties that the message passingbetween modules is persistent. The samedatabase is used by all modules. Modules maycreate an entry in the database table of the pro-cess manager and then invoke the process man-ager as they exit. The PMI then reviews andupdates the database and invokes the appropri-ate other modules.

Message Gateway (MGI) — The MessageGateway (MGI) is responsible for the two-waycommunications with the servers. The MGI maybe invoked in two ways:• Through the PMI by the Call Handler to

look for new pending messages at the serv-er.

• By itself after an adaptive time-out to lookfor messages on its own.

Service modules may also invoke the MGI(through the PMI) if they have information torelay to the server side.

The communication between MGI and theserver uses secure HTTP (HTTPS) requests toensure privacy and authenticity between clientsand the server. The MGI is responsible forselecting the access service (e.g., GPRS, WiFi,Bluetooth to Internet, or USB to Internet) andestablishing a connection. If no connection tothe Internet existed previously, the MGI willinvoke the appropriate system calls to obtain anetwork address for the PPD, thus establishing aconnection.

Call Handler (CHI) — The Call Handler(CHI) is the only process continuously runningas a service (or daemon) on the phone. TheCHI is responsible for intercepting incomingphone calls and/or short messaging systemmessages, notifying the client about the need

to communicate with the server side. Sincedata connections must be established on thesedevices on-demand (partly to save batterypower) they are usually not addressable by astatic data network address (i.e., IP address).This means that data connections need to beinitiated by the client side. On the other hand,when messages become available, the serverside should invoke communications with thePPD. The solution to this problem is some-times referred to as a shoulder-tap. This meansthat the server side has to inform (out of bandto its TCP/IP stack) the PPD to obtain an IPaddress and send an HTTPS POST request tothe server. The shoulder-tap may be an incom-ing call from a well known phone number(which the CHI has to intercept and suppressso the user is not bothered by it) or a shortmessage that the CHI is watching. Dependingon the parameters of the tapping, the CHI isgoing to invoke the MGI through the PMI orthe Remote Update (RUI) module directly.

Delivery Agent (DAI) — The Delivery Agent(DAI) is the only module directly interactingwith the user. This design enables different ser-vice modules to appear to the user in a consis-tent fashion. The DAI is responsible for handlingall input/output (I/O) on the client device,including the (touch) screen, speakers and micro-phone. The DAI is invoked by the service mod-ules through the PMI. The DAI reads the userprofiles and determines how the informationneeds to be represented to the user; thus, it mayinvoke text-to-speech conversion or speech-to-text conversion, and handle any assistive devices(e.g., large external buttons or Braille readers)attached to the PPD device. After a userresponds to a message, the DAI forwards theresponse to the appropriate service module forfurther processing. A service module may in turnforward the user’s response to the MGI to berelayed to the server.

Shell Process (SPI) — The shell process maybe invoked to securely execute commandsreceived from the server on the PPD. The SPIwill parse any command messages and executethe commands (such as download, upload,copy file, or reset) contained in the batch. TheSPI may be used to upload and download userprofiles to the client as well as to downloadnew or replacement modules. Remote devicemonitoring (e.g., ensuring that the device isworking properly) may also be performedusing the SPI.

Remote Update (RUI) — The Remote Update(RUI) module, once invoked, will check theserver side (using HTTPS messages) to deter-mine if all the components’ versions are the lat-est. If there are components that need to beupdated, an HTTPS embedded FTP (file trans-fer protocol) process is started to download thelatest versions. The RUI is a stand-alone mod-ule, meaning it does not need to interact withany other modules; indeed, it is designed suchthat it can even be invoked manually to ensurethat the device remains useable and can recoverfrom an unexpected failure.

The MGI is

responsible for

selecting the access

service and

establishing a

connection. If no

connection to the

Internet existed

previously, the MGI

will invoke the

appropriate system

calls to obtain a

network address for

the PPD, thus

establishing a

connection.



SERVICE MODULES

Reminder Service (RMS) — The ReminderService (RMS) was the first service implementedto aid people with disabilities. With the reminderservice, caregivers can send timely remindermessages to the client. These messages are rep-resented to the client and then a response is col-lected and returned to the server. Messages canhave an arbitrary number of reply options.Replies (or no replies) are relayed back throughthe appropriate infrastructure modules to theserver side, where message logs will be updatedwith their results. Thus, the person sending themessage can observe whether the client hasresponded and the nature of the response. Mes-sages can be set up to be re-occurring and theirexpiration time is also adjustable. Only thosemessages are delivered whose priority is higherthan the user’s current activity. This ensures, forexample, that a user who is taking an afternoonnap is not awakened by a message informing herof an appointment tomorrow.

The reminder service can also be used by theuser to send reminder messages to other users.This allows users to compose and send their ownmessages on PPDs without accessing a Web page.

User Profile Service (UPS) — The User ProfileService (UPS) enables the user to modify heruser profile settings so her needs are accountedfor and incorporated in her PPD. User profilesare then checked for inconsistencies and areuploaded to the server, as well as stored on thePPD, for safekeeping and for caregivers (withsufficient rights) to verify them.

News Service (NWS) — Using the News Ser-vice (NWS), the user can subscribe to variousnews feeds available to the community. If arequested news message appears in the newsfeed, the server side will inform the PPD, whichwill relay and represent the news to the client.Using the news service, clients can be informedof upcoming social events or job opportunitiesthat may interest them.

Sensory Monitoring Service (SMS) — TheSensory Monitoring Service (SMS) enables thePPD to connect to a physical sensor interfacemodule. Currently both the serial, and a Blue-tooth-serial connection of the PPD may be usedto connect the sensor interface to the PPD. Thesensor interface is a small microcontroller-drivenbox that can convert and relay any analog (e.g.,ECG, body temperature) or digital (e.g., externalswitches) signals and drive actuators with analogor digital inputs. The SMS can interact with thedelivery agent to help the user input data intothe device. It may also be used to convert outputinto a form understandable by its user. Further-more, the sensory box can help in observing theuser’s behavior and notify caregivers about apossible fall, unused device, or any other prepro-grammed sensory event.

SERVER-SIDE MODULESComputers on the server side are located in securevaults for physical security and to comply withHIPAA (Health Insurance Portability and

Accountability Act) requirements. The only portsopen are ports associated with HTTPD and SSHaccess. Servers are placed at several locations(with more servers planned at many more places)sharing the load generated by providers and users.

Web Server — The Web server is based on amodified Apache server. The scripting languageused is PHP. The Web server requires secureauthentication from both clients (which is strictlyPPD and user profile related) and caregivers (byentering user name and password).

Phone Manager (PMAN) — The Phone Man-ager (PMAN) is the entity responsible for theshoulder-tap. It queries the CONNECT databasefor messages that have not been downloaded tothe PPDs, and puts them into an importanceorder based on delivery time and priority. SincePPDs can initiate data transfer on their own byestimating (based on past history) when they arelikely to have pending messages, PMAN may notbe invoked for all pending messages. When amessage becomes the first in PMAN’s impor-tance queue, PMAN will place a call to the PPD,where the CHI will intercept the call and invokethe PMI for further processing.

CONNECT Database — The CONNECT Databaserelies on a MySQL server maintaining records ofall messages and all user profiles and users. Itstables remember the status of all messages (i.e.,delivered to the phone, delivered to the user,response received and the nature of the response).The only two entities directly interfacing theMySQL server are the Web server and PMAN.

PROFILES: THE TOOLS OFACCESSIBILITY

Profiles are settings that influence how the PPDwill interact with its user. Thus, profiles carry themeans to adapt the PPDs to their users’ needs,wants and capabilities. For example, if the userprofile of the PPD indicates that the owner is avisually impaired person, then communicationswill need to produce either larger characters onthe screen and/or audible messages. Entry intothe device may also need to be adapted. Profilesare stored, interpreted, and transmitted in anXML (Extended Meta Language) format due toits hierarchical setup and human readability.Each stored profile has a top domain versionfield for upward compatibility.

There are three different kinds of profiles:• The User Profile contains information about

the user and her behavior.• The Device Profile contains information

about the device itself (i.e., the capabilitiesof the device).

• The Service Profiles contains informationabout how a user wishes to personalize agiven service, thus each service may have itsown profile.

USER PROFILEThe user profile maintains information aboutthe user, including information required forauthentication, such as the user’s al ias or

The Connect

Database relies on a

MySQL server

maintaining records

of all messages, all

user profiles and

users. Its tables

remember the status

of all messages.

The only two entities

directly interfacing

the MySQL server are

the Web server

and PMAN.



name and account detai ls . Details of useractivities and delivery preferences derivedfrom the special needs of the user are alsostored here.

Activit ies describe daily, weekly, andmonthly activities, or in general the scheduleof the user. There may be an arbitrary numberof overlapping activities in the user profile.Activities can be set up with a Unix-cron stan-dard-type feel, including a start time and anarbitrary duration. This means that periodicityof activities is easily established (as a cron-type description can include daily, weekly,work-daily, or any other periodicities). Eachactivity type includes a priority setting so thatunimportant communications do not disturbthe user in her current activity. For example, a“sleeping” activity may be set up with a highpriority for every night between 10 PM and 6AM, while nap-time may be established with amedium priority between 2 pm and 4 pm onweekends. The Delivery Agent is not autho-rized to initiate any communications with theuser if the priority of the communication isless than that of the current activity.

Delivery types can be defined and automati-cally derived from the special needs of the user.Thus a user with hearing difficulties can set thedevice to alert the user with a specified andadjustable amount of vibration, while a personwith visual impairments can set up a profile toinstruct the device to automatically convert thetext messages to speech before delivering them.Figure 5 shows a sample Web interface used tomodify delivery settings.

DEVICE PROFILEThe device profile is device-associated and thuscontains only information about the capabilitiesand specifications of the PPD. Informationstored contains the make and model of the

device, the available memory and CPU power,and the access network types the device is capa-ble of using. Additionally, this profile may con-tain the list, description, and properties of anyassistive technologies that are attached to thePPD (such as external sensors or sensory inter-faces, or I/O devices, i.e., Braille readers or eyemovement trackers).

SERVICE PROFILESEach service may have its own profile extend-ing the user and device profiles with settingsspecific to the service. As an example, theReminder Service will contain the people anddevices (peers) who have rights to sendreminders to this PPD and the peers to whomthe current PPD’s user has rights to send mes-sages. Furthermore, it contains the descriptionof how undelivered, expired, or unresponsivemessages will be escalated further, i.e., whatother person and/or device to contact and theway to contact it (e.g., by email, phone, oranother reminder) if and when a user fails torespond to a message.

PUTTING IT TOGETHERTo have a deeper understanding of how compo-nents work together we describe the interactionsof different modules, persons, and entities basedon a simple reminder example. Henry, a goodfriend of Joan, was just informed that Joan has aCONNECT device. Henry would like to remindJoan about their upcoming lunch meeting andknow whether or not Joan will be there.

Henry visits the CONNECT Web site where hesigns up for an account. He needs to requestpermission from Joan to send her messages, sohe fills out a Web form with his request andsubmits it. This generates a special remindermessage at the server, which is put into the

nnnn Figure 5. CONNECT Web interface to modify delivery settings.

The device profile is

device-associated

and thus contains

only information

about the capabilities

and specifications of

the PPD. Information

stored contains the

make and model of

the device, the

available memory

and CPU power, and

the access network

types the device is

capable of using.



CONNECT database. Since rights-requests areimportant messages, the PMAN module query-ing the database will pick it up and call Joan’sPPD. Joan’s PPD’s Call Handler intercepts thecall and instructs the Process Manager to startoff the Message Gateway. The MGI determinesthat the only available network is the GPRSnetwork to which it initiates connection. Afterthe connection is established, it sends an HTTPSPOST request to one of the CONNECT serversindicating its authenticity. The server replieswith any pending XML messages that Joan’sPPD has not received yet, including the requestfrom Henry. As the XML message is receivedand put in the persistent storage of the processdatabase, the PPD sends another POST mes-sage acknowledging the receipt of the message(and thus the status of the message in the logsand on the Web is updated). The PMI is theninstructed to handle the new XML message;thus it determines that the message is to be pro-cessed by the Reminder Service, marking themessage for the RMS, and as it exits it starts offthe RMS.

The RMS extracts and interprets thereminder and checks whether the current activi-ty’s priority is smaller than that of the message.Assuming that it is not (e.g., it is late at nightand Joan is asleep), it will query the User Profileto see what the closest time is that the messagecould be delivered. Before exiting, the RMSinstructs the PMI (through the process database)to reschedule itself to be awakened at the appro-priate time. The PMI will then schedule its ownreinstantiation at the time indicated by theRMS. When the PMI resurrects itself, it willlook into the process database and determinethat the RMS needs to be called. The RMS willreevaluate the priority of the message againstthe priority of the current activity (due to possi-ble changes in the profile). If the message can bedelivered, an XML message to the DeliveryAgent is assembled and the PMI is asked toinvoke the DAI. The DAI interprets the outputrelayed to it by the RMS. It will use the Userand Device Profiles to determine the mode ofdelivery. Suppose that Joan is visually impaired.In this case the DAI is going to invoke a localtext-to-speech engine to convert the messageand the instructions on how to answer the mes-sage. The DAI is going to determine how tonotify the user of the new message; as indicatedby Joan’s profiles, her PPD will vibrate fivetimes for 10 seconds and play an audible signalof her choice.

The message is then read out to Joan tellingher that Henry is requesting her to give him per-mission to send messages and that her possiblereplies are “yes” or “no.” She selects “yes” asthe answer. The DAI will return the response tothe RMS through the PMI. The RMS informsthe PMI to invoke the MGI; the MGI will estab-lish an appropriate network connection and

update the status of the message at the serverwith another HTTPS POST message. In thiscase, this will result in Henry obtaining rights tosend messages to Joan’s PPD. He can now enterhis reminder, which will propagate in a way simi-lar to the rights request message.

CONCLUSIONSIn this article we have presented an architec-tural overview of the CONNECT system devel-oped by a joint project of The University ofTexas at Arlington’s Department of ComputerScience, Sensor Logic, Inc., and C-CAD (Cen-ter for Computer Assistance to the Disabled)of Metropolitan Dallas. The system is current-ly being f ielded, with a total of 600 PPDsplanned to be deployed to people with disabil-ities by the end of the summer of 2005 to helptheir communications with their care pro-viders, family, and friends. Table 1 shows abreakdown of the disabilities of the currentusers of the system.

A preliminary analysis of the usage of thereminder service shows that persons who usedthe system sent an average of three messages perday, with the most avid users using the devicesbetween 10 and 30 times per day.

We have outlined the various architecturalelements of CONNECT and have shown how theyinteract. The robust design of CONNECT avoidsany lost messages and provides to its users aconstant but non time-consuming source ofmonitoring and messaging capabilities. OptionalPPD attachments can enable physical monitoringof the clients or their environments and sendtimely messages when predetermined eventsoccur. Although designed for people with dis-abilities, the system can be used as a basic infra-structure for any personal communications.There are several new services under develop-ment for the CONNECT system.

The current version of the CONNECT systemcan be deployed on Windows PocketPC 2003,Windows Mobile, and Symbian operating sys-tems. We are developing Windows XP, RIM,and Palm versions as well.

ACKNOWLEDGMENTSThe work presented in this article has been car-ried out in the collaborative project “CONNECT,”which is funded by a grant from the Texas Healthand Human Services Commission. The authorswould also like to acknowledge the Assist Labo-ratory Team, the Departments of Computer Sci-ence and Engineering, and the School of SocialWork at The University of Texas at Arlington(with extended thanks to Dr. Dick Schoech), andespecially Padma Sambath, whose contributionswere crucial to the success of the project. Oursincere gratitude also goes to Amanda Szucs. Tolearn more about the CONNECT project, pleasevisit http://assist.uta.edu

nnnn Table 1. Disabilities of CONNECT users.

Disability Cognitive Psychiatric Physical Hearing Vision Multiple

Users 34% 13% 17% 10% 18% 8%

Optional PPD

attachments can

enable physical

monitoring of the

clients or their

environments and

send timely

messages when

predetermined

events occur.

Although designed

with people with

disabilities in mind,

the system can be

used as a basic

infrastructure for

any personal

communications.



REFERENCES[1] http://www.abledata.com[2] D. Carlson et al., “Assistive Technology Survey Results:

Continued Benefits and Needs Reported by Americanswith Disabilities,” National Center for the Disseminationof Disability Research, 2001.

[3] G. D. Abowd, L. Iftode, and H. Mitchell, “The SmartPhone: A first Platform for Pervasive Computing,” IEEEPervasive Computing, vol. 4, no. 2, 2005, pp. 18–19.

[4] E. Bertini and S. Kimani, “Mobile Devices: Opportunitiesfor Users with Special Needs,” Springer Verlag: Human-Computer Interaction with Mobile Devices and Services,2003, pp. 486–91.

[5] V. Haberman, M. Jones, and J. Mueller, “Mobile Tech-nology, Compensatory Aids, and Usability Evaluations,”IEEE Pervasive Computing, vol. 4, no. 2, 2005, pp.82–83.

[6] M. DeMeglio, M. T. Hakkinen, and H. Kawamura,“Accessible Interface Design: Adaptive MultimediaInformation Systems (AIMS),” Springer Verlag: Comput-er Helping People with Special Needs, 2002, pp.406–12.

[7] G. Menkhaus, “An Architecture for Supporting Multi-Device, Client-Adaptive Services,” Annals of SoftwareEngineering, vol. 13, June 2002.

[8] http://www.w3.org/[9] http://www.w3.org/Talks/WAI-Intro/[10] http://www.w3.org/TR/xag[11] http://www.w3.org/2001/di/[12] http://www.w3.org/2002/mmi/[13] http://www.w3.org/TR/voicexml20/

ADDITIONAL READING[1] http://www.w3.org/2001/sw/WebOnt/[2] http://www.w3.org/TR/2003/PR-owl-features-20031215/

BIOGRAPHIESMANFRED HUBER [M] ([email protected]) is an assistantprofessor of computer science and engineering at the Uni-versity of Texas at Arlington (CSE@UTA). He received hisM.S. and Ph.D. degrees in computer science from the Uni-versity of Massachusetts, Amherst in 1993 and 2000,

respectively. He obtained his “Vordiplom” from the Univer-sity of Karlsruhe, Germany in 1990. He is the co-director ofthe Robotics program and of the Learning and PlanningLaboratory at CSE@UTA. His research interests are in rein-forcement learning, autonomous robots, cognitive systems,and adaptive human-computer interfaces. He is a memberof the ACM and the AAAI.

FARHAD A. KAMANGAR [M] ([email protected]) is a pro-fessor of computer science and engineering at the Universi-ty of Texas at Arlington (CSE@UTA). He received the Ph.D.and M.S. degrees in electrical engineering from the Univer-sity of Texas at Arlington in 1980 and 1977, respectively.He received his B.S. degree from the University of Teheran,Iran in 1975. His research interests include image process-ing, robotics, signal processing, machine intelligence, andcomputer graphics. He is the principal investigator of theCONNECT project at UTA, striving to help people with dis-abilities in their daily communications. He is a member ofthe ACM.

DAVID LEVINE [M] ([email protected]) has been at the Uni-versity of Texas at Arlington Department of Computer Sci-ence and Engineering since 1996. Previously he hadworked in industry. Most recently, his interests includeassistive technologies and systems for people with disabili-ties, grid and cluster computing, particularly applied to sci-ence, and streaming data representations and querying.His research has been partially supported by grants fromthe National Science Foundation, US National Laboratories,the State of Texas, and other federal and state agencies.He is a member of the ACM.

GERGELY V. ZARUBA [M] ([email protected]) is an assistantprofessor of computer science and engineering at the Uni-versity of Texas at Arlington (CSE@UTA). He received thePh.D. degree in computer science from the University ofTexas at Dallas in 2001, and the M.S. degree in computerengineering from the Technical University of Budapest,Department of Telecommunications and Telematics in1997. His research interests include wireless networks,algorithms and protocols, performance evaluation, andcurrent wireless and assistive technologies. He has servedon many organizing and technical program committees forleading conferences and has guest edited journals.


Documents

CONNECT: A Personal Remote Messaging and Monitoring …crystal.uta.edu/~zaruba/Publications/J5.pdfstructure to relay messages, reminders, ques-tions, and in general any other multimedia