Embed Size (px)
Citation preview
Using Common Sense Reasoning to Create Intelligent Mobile Applications
Software Agents GroupMIT Media Lab
The PC vs. the Smart Phone
Current efforts to develop software for smart phones are based on taking PC applications
and making them smaller
The PC vs. the Smart Phone
The PC has an email application The smart phone has a smaller one
Unlike the PC, text entry on smart phones is very difficult
The PC vs. the Smart Phone
Because of the limited IO, the smart phone needs to be better at understanding the user
then the PC ever was
The PC vs. the Smart Phone
Unlike the PC which stays on its desk, people carry their smart phones with them everywhere
The PC vs. the Smart Phone
Because of this, the smart phone needs to be better at understanding the world then the PC
ever was
The PC vs. the Smart Phone
To understand the user and understand the world, smart phone applications need something
that computers have always lacked, common sense reasoning
This is a problem we are trying to solve
Introduction to Common Sense Reasoning
Over the past 3 years 12,000 people have entered common sense facts to a Web site called Open Mind
Introduction to Common Sense Reasoning
Open Mind now contains over 650,000 common sense facts about the world
Introduction to Common Sense Reasoning
Using this knowledge base we have created a semantic network of concepts and relationships called OMCSNet
Using Common Sense Reasoning in Mobile Applications
We are currently using OMCSNet in two cell phone based applications
Using Common Sense Reasoning in Mobile Applications
A language translation aid called GloBuddy 2
Using Common Sense Reasoning in Mobile Applications
And a text entry system capable of contextually relevant word completion
GloBuddy 2
GloBuddy 2
When Traveling in a foreign country, people often rely on phrase books. However:
-They only cover a very limited number of common situations
-Even a common situation will quickly deviate from the book’s script
GloBuddy 2
Translation Software and dictionaries currently exist for PDA devices
But users must write out the entire phrase they wish to translate, and look up each word individually
GloBuddy 2 7 / 14
GloBuddy 2 will translate phrases for the user into English
But GloBuddy can go beyond simple translation and understand the context of the user’s situation
GloBuddy 2
Instead of writing the entire phrase they wish to translate into a foreign language, GloBuddy asks the user to write the ideas they would like to talk
about.
GloBuddy then uses common sense reasoning to build a localized and contextually relevant
vocabulary of terms for the user
GloBuddy 2
1. GloBuddy asks the user to describe the ideas they would like to talk about
2. GloBuddy then uses OpenMind and OMCSNet to build a localized vocabulary of terms
3. Contextually relevant words and phrases are displayed to the user for use in conversation
GloBuddy 2
To return to our example in the introduction:
GloBuddy 2
Because GloBuddy can intelligently expand on the user’s translation request:
The amount of text entry is significantly reduced
And Conversations are more fluid
GloBuddy 2
There is a traditional trade off between a system’s breadth of information, and its depth of reasoning:
Breadth of Information
Dep
th o
f R
eason
ing
Phrase Book
Dictionary
Human Translator
GloBuddy 2
GloBuddy 2 is able to avoid this tradeoff due to its extensive knowledge of common sense facts
Breadth of Information
Dep
th o
f R
eason
ing
Phrase Book
Dictionary
Human Translator
GloBuddy 2
Word Completion
Using Common Sense Reasoning in Mobile Applications
Current approaches to reducing text entry on Smartphones are limited
Using Common Sense Reasoning in Mobile Applications
Using common sense reasoning, we can perform intelligent word completion based on the context of
the message
Using Common Sense Reasoning in Mobile Applications
Some technical notes:
• Unlike GloBuddy, OMCSNet has to be stored locally
• This approach is best used in addition to classic word completion, rather then replacing it
http://agents.media.mit.edu/projects/mobile
