View
214
Download
1
Tags:
Embed Size (px)
Citation preview
April 23, 2005
Mitsubishi Electric Research Laboratories
James L. FrankelSlide 1
Combadge: A Voice Messaging Device for the Masses
Berkeley UNIDO ConferenceInformation & Communications Technology (ICT) Workshop
April 23, 2005
James L. FrankelMitsubishi Electric Research Laboratories
Cambridge, Massachusetts
April 23, 2005
Mitsubishi Electric Research Laboratories
James L. FrankelSlide 2
Combadge
A speech-enabled communications device Functionality: Two-way voice messaging
with simple spoken commands and a one-button interface.
Platform: Basis for new handheld research
Goal: Bring state-of-the-art wireless communication and services to the less-wealthy in the world with a simple, low-cost device.
Advantages: Offers new services, yet is unimposing and non-intrusive, with low device and low ongoing infrastructure costs.
Contact: [email protected]
April 23, 2005
Mitsubishi Electric Research Laboratories
James L. FrankelSlide 3
Asynchronous Operation (1 of 2)
Users decide when to listen and respondMessages are sent to and from device when connected
Device can be very smallHas no display
Requires only one button
Need not reach from mouth to ear
In the future, it will be feasible to be packaged in a watch
Voice interface makes Combadge usable by illiterate users
Can use better compressionNo need for real-time compression
Can fully utilize available spectrum (packet switched)
April 23, 2005
Mitsubishi Electric Research Laboratories
James L. FrankelSlide 4
Asynchronous Operation (2 of 2)
Graceful degradation of service during network overload
Users less aware of dead spots in networkFunctional without any connectivity
Messages are cached in the Combadge
All functions that don’t require communication are useable
Reduces peak power demand, allowing much longer battery lifeSpeech recognition, compression and radio not used simultaneously
Can operate radio less frequently (it's like voice IM, not a phone)
Can use Internet for cheap global connectivity (like e-mail or IP telephony)
Makes group messaging easy
April 23, 2005
Mitsubishi Electric Research Laboratories
James L. FrankelSlide 5
SimpleSingle button, push-to-talk: no keypad, no display
Reduced manufacture cost and reduced power used
Simple interface using speech, e.g.:“New message for Peter""Play New", "Reply"
Talk immediately: no waiting for a dial tone, for someone to answer, or for a menuAfter adding another Combadge to the phonebook, there are no phone numbers to memorize
Everyone is identified by spoken name (or nickname)For children, restrictions applied on adding new Combadges
Optionally, no messages from people you don’t know
April 23, 2005
Mitsubishi Electric Research Laboratories
James L. FrankelSlide 6
Customer Base
Appeal to new users:
The less-privileged and less-educated in the world (including developing countries)
Designed for illiterate users
Lower cost device
Lower cost service
The cost conscious, such as youth (ages 8-14) and the elderly
Those irritated or intimidated by cell phones
Use cellular networks, but create a low bandwidth, low cost service
Use 802.11a/b/g for campus or village/town/city connectivity
Can use DakNet-like network for transport
April 23, 2005
Mitsubishi Electric Research Laboratories
James L. FrankelSlide 7
Interaction with Services and Other Devices
Open-ended opportunity to create new services, providing simple spoken interfaces to the entire digital universe
“Weather for Boston”
“Market price for rice”
“Calendar: Am I free Friday afternoon?”
“Traffic on the Mass. Pike”
Voice control of devices“House: Turn garage lights on”
“HVAC: Set living room temperature to 20 degrees Celsius”
Integration with e-mail, telephones, voice mail, etc.
April 23, 2005
Mitsubishi Electric Research Laboratories
James L. FrankelSlide 8
Hardware (Introduction)
Hardware component is code-named “Dilithium”
Back side of main board
April 23, 2005
Mitsubishi Electric Research Laboratories
James L. FrankelSlide 9
Hardware (Introduction)
Front side of main board
April 23, 2005
Mitsubishi Electric Research Laboratories
James L. FrankelSlide 10
Hardware (Daughterboard)
Daughterboard
April 23, 2005
Mitsubishi Electric Research Laboratories
James L. FrankelSlide 11
Hardware (Case Components)
Some Case Components
April 23, 2005
Mitsubishi Electric Research Laboratories
James L. FrankelSlide 12
Hardware (In Case)
Dilithium in Case
April 23, 2005
Mitsubishi Electric Research Laboratories
James L. FrankelSlide 13
Assembled Combadge
April 23, 2005
Mitsubishi Electric Research Laboratories
James L. FrankelSlide 15
Hardware (1 of 4)Processor is Intel XScale StrongARM running at 206 MHz
Moving to Intel XScale at 400 to 624 MHz and faster
MemorySDRAM: 64 Mbytes; Flash: 64 Mbytes
Integrated GSM/GPRS Modem for Wide-area NetworkingOn-board SIM Socket
Optional Daughterboard Provides One or Two Compact Flash (CF) Slots
802.11b Local Area NetworkingMany Other CF Peripherals (Ethernet, CF Memory Cards, Additional I/O Ports, CF Disk Drives)
Two On-board SiSonic Silicon-MEMS MicrophonesOn-microphone preampCan perform active noise cancellation
April 23, 2005
Mitsubishi Electric Research Laboratories
James L. FrankelSlide 16
Hardware (2 of 4)Flexible CODEC sampling rates
11.025, 22.05, 44.1 (CD), 8 (telephony), 16, 32, and 48 KHz
LED’sTwo banks of blue LED’s under the translucent side buttonsTwo bi-color LED’s on frontOne LED for bi-directional communication using LEDComm
Two-axis AccelerometerGesture detection
Vibrator (for silent new message indication)JTAG ConnectionUSB PortSerial Port with on-board RS232 driversTwo Stereo 2.5mm Phone Jacks for Audio In and Audio Out
April 23, 2005
Mitsubishi Electric Research Laboratories
James L. FrankelSlide 17
Hardware (3 of 4)Pushbuttons
Left and Right Push-to-Talk
Power On
Reset (Accessible through hole)
Real-time Clock
Dense component packing; Small overall size
Heavy use of BGA componentsProcessor, Four memory chips, and CPLD
Design of caseSolidWorks
SLA Master (Stereolithography)
Limited-run Rubber Molds
April 23, 2005
Mitsubishi Electric Research Laboratories
James L. FrankelSlide 18
Hardware (4 of 4)Hardware Revisions
Rev. 1Fabricated one deviceThis device has had a fruitful lifeStill functional today
Rev. 2Fabricated five devicesThese are the devices in the demo
Rev. 3Power management hardware addedReal-time clock addedGround planes to attenuate audio noise addedFabricated twenty-five devices to date
XScale Revision (StrongARM has been discontinued)
April 23, 2005
Mitsubishi Electric Research Laboratories
James L. FrankelSlide 19
Software (1 of 5)
InitializationJTAG Programming Utility
Initializes Flash memory using JTAG interface to StrongARM
Boot LoaderFirst Program running on StrongARM
Initializes memory and I/O devices
Provides debugging tools
Loads Operating System
Linux Operating SystemWe ported Linux 2.4.19 to Dilithium
Started with the Compaq “Familiar” Linux port
April 23, 2005
Mitsubishi Electric Research Laboratories
James L. FrankelSlide 20
Software (2 of 5)Linux Porting Issues
Our New Dilithium Architecture
New Flash memory chips
Custom Device Drivers
Accelerometer, buttons, LED’s
Combadge Voice-Messaging ApplicationInitial development on iPAQ PDA running Linux
Developed in Python, C, C++, and Shell Scripts
Voice RecognitionTwo Recognizers (Using SDX from SpeechWorks/ScanSoft):
One for speaker-independent tokens
One for speaker-dependent name tags such as the name given to phonebook entry
April 23, 2005
Mitsubishi Electric Research Laboratories
James L. FrankelSlide 21
Software (3 of 5)Grammar used for Combadge commands
Play new messages; Play again; Play next; Play previousNew message for <name>ReplyCreate contactPhonebookStatus all; Status ID; Status connection; Status messages; …Profile normal; Profile meeting; Profile silentVolume 1; Volume 9; Volume off; …Delete contact <name>; Delete all contactsShutdown; Restart; Configure MERL; Configure adhoc; Configure GPRS; …Version; Utility ping
April 23, 2005
Mitsubishi Electric Research Laboratories
James L. FrankelSlide 22
Software (4 of 5)Combadge application complexities
Heavily multi-threadedBarge in capabilityExtensive loggingGraceful handling of exceptional events
Power-down components when not usedAmplifierGSM/GPRS modem802.11b interface
More work is needed to cause Combadge to sleep to extend battery life when device is inactiveAudio messages are now PCM files; will transition to WAV filesGateway from voicemail system at MERL to Combadge
April 23, 2005
Mitsubishi Electric Research Laboratories
James L. FrankelSlide 23
Software (5 of 5)Voice messages are delivered using SMTP and IMAP
A custom “cbd” protocol is used to communicate from the Combadge to a “cbd” serverThe “cbd” server actually sends messages via SMTP and gets messages via IMAPSMTP is also used directly by the Combadge to verify valid phonebook entry addresses (using VRFY)
The Combadge application does the management of three categories of messages
Recorded to be sent, but not yet sent to serverReceived from server, but not yet heardReceived from server and already heard
The Combadge maintains a cache of messages in its own memoryCombadge is fully-functional without any connection to a network
April 23, 2005
Mitsubishi Electric Research Laboratories
James L. FrankelSlide 24
Deployment ConnectionsU. C. Berkeley
Eric Brewer
Divya Ramachandran, Graduate Student
Voice recognition for Tamil
Integration with Berkeley’s network transport for intermittent connectivity and long-distance 802.11b
Deployment in Tamil Nadu in India
Media Lab at MITSMART Group – EKG information transmission in ER or disaster situation
Mike Best – Potential developing world deployments
World Bank
April 23, 2005
Mitsubishi Electric Research Laboratories
James L. FrankelSlide 25
Server Environment
Server runs Linux with dhcpd, sendmail, imap (invoked by xinetd), and cbd (the Combadge server daemon)
April 23, 2005
Mitsubishi Electric Research Laboratories
James L. FrankelSlide 26
Research Directions (1 of 3)User studies in developing world deploymentsUser studies in deployments in urban/suburban settings in the United StatesInvestigate mesh networking
Combadge as an infrastructure-less voice messaging consumer appliance (like a walkie-talkie/FRS/GMRS)Forward messages through other Combadges toward the destinationAttention needed to patterns of physical location of Combadge over time (i.e., usual weekday daytime location, usual weekend daytime location, usual nighttime location)Utilize connection to Internet when present
April 23, 2005
Mitsubishi Electric Research Laboratories
James L. FrankelSlide 27
Research Directions (2 of 3)Develop services for Combadge users
Traffic reportingWeather informationSchedule/appointmentsStock quotes
Continue to Integrate with other Communication ParadigmsTelephone
Speech synthesisE-mailPagers
April 23, 2005
Mitsubishi Electric Research Laboratories
James L. FrankelSlide 28
Research Directions (3 of 3)Develop as an audio home appliance remote control
Audio and video systems
Security system
HVAC
Audio interface to use as an MP3 player
Utilize Dilithium platform for other MERL projectsMicrophone and audio processing server
April 23, 2005
Mitsubishi Electric Research Laboratories
James L. FrankelSlide 29
CreditsEarly work
Barry Perlman
David Anderson
Current workDaniel Bromberg