SAB ReviewFebruary 2004Pervasive 2004April 2004 Using an Extended Episodic Memory Within a Mobile Companion Alexander Kröner, Stephan Baldes, Anthony Jameson,

SAB ReviewFebruary 2004 Pervasive 2004April 2004

Using an Extended Episodic Memory Within a Mobile Companion

Using an Extended Episodic Memory Within a Mobile Companion

Alexander Kröner, Stephan Baldes, Anthony Jameson, and Mathias Bauer

German Research Center for Artificial Intelligence (DFKI) GmbHStuhlsatzenhausweg 3, 66123 Saarbrücken, Germany

BMB+F 524-40001-01 IW C03 (SPECTER)


About this TalkAbout this Talk

• About Specter– Goals, example, challenges

• About the personal journal– User interface

• Adding value to the journal– Binding services to journal entries


What is Specter?What is Specter?

• Specter is about…– … context- and affect-aware personal assistance– … in instrumented environments– … using a long-term memory

• Major issues– Extension of perception– Learning about behavior and affect– Augmentation of decision making and effecting– Reflection and introspection


Example Scenario: Shopping AssistantExample Scenario: Shopping Assistant

• Technology– RFID enabled shelf and

basket– RFID-labeled Products– Integrates software of

projects Specter and REAL (Saarland University)

• Logging low-level actions– Product in shelf, removed

from shelf, etc.

• Applying logged information– Proposing alternative

products


Obtaining Data from the EnvironmentObtaining Data from the Environment

Stream of raw, low-level sensor data

UM

PersonalJournal

decisio

n supp

orte

pisode

retrieval

situationclassification

motion dataclassification

phys. dataclassification

planrecognition

objectclassification

locationclassification

actionrecognition

affectrecognition

motion profiling

abst

ract

ion

of d

ata

...

... ...

... ...

reflection/introspection


Obstacles in Obtaining DataObstacles in Obtaining Data

• Automatically retrieved input– Sensor input may be incorrect – Sensor input may be completely missing– Inference mechanisms are not perfect

• user input required

• Manual (user) input– Lack of motivation

• manual input has to be minimized• input value has to be maximized

– Recorded information may be sensible• user's trust has to be strengthened


Design PrinciplesDesign Principles

• Input has to be acquired collaboratively[addresses: manual input, lack of motivation, trust]

– filling the journal automatically– asking for additional information if required– providing control of information and bound services

• Provision of diverse, multiple benefits[addresses: maximizing value]

– providing the option of binding various services to journal content

• Support for diverse forms of collaboration[addresses: lack of motivation]

– providing various ways of entering data– enabling input/feedback in varying scenarios


Accessing the Personal JournalAccessing the Personal Journal

The personal journal user interface


Browser/Viewer ApproachBrowser/Viewer Approach

Browser• Goal: browsing the journal in a similar fashion like the Web

• Adopted concept of Web browser

• Browsed documents are generated on-the-fly by data viewers

• Browser and/or viewers can be exchanged

Viewers


Navigating the Personal JournalNavigating the Personal Journal

• Content requests– History (forward,

backward,…) – Bookmarks – Hyperlinks

• Reminder Points– Created by the user

during interaction with the instrumented environment

– Indicate the need to inspect points within the timeline during introspection


Adding ContentAdding Content

• Annotating entries– Free text comment,

references, categories, ratings

• Grouping entries– A group indicates that

some entries are related together

– For manual organization, and input for Specter


Collaborative Learning of the User ModelCollaborative Learning of the User Model

Binding services to the personal journal


Example: EC Card PurchaseExample: EC Card Purchase


Example: Assisting an EC Card PurchaseExample: Assisting an EC Card Purchase

• Goal: checking bank account automatically before payment is required

• Binding a service– Identifying situation where check is required

• Identifying relevant conditions, e.g., shop size, total prize of wares contained in basket

– Binding a service “account check” to that situation• To be stored as trigger in the user model

• Evoking a service– Comparing given situations with triggers from the

user model


Binding Services: WorkflowBinding Services: Workflow

User or Specter chooses a servicethat should be triggered automatically

Specter collects attributes from (potentially) relevant journal entries

User discards irrelevant attributesand/or adds relevant ones

Specter computes a rule and shows it to the user

User accepts or changes the rule

Specter adds the new rule as service trigger to the user model


Defining Decision TreesDefining Decision Trees

EC CardPurchase

Store PriceEdeka Hela

117,5€ > 117,5€

Price

– +

DayOfWeek

– +

TimeOfDay

– +

TimeOfDay

– +

So Sa...

112€ > 112€

5:25pm > 5:25pm 1:07pm > 1:07pm


Customizing Decision TreesCustomizing Decision Trees

EC CardPurchase

Store

Price DayOfWeek

Edeka Hela

117,5€ > 117,5€ So Sa...

userdef #1

TimeOfDay

otherwise[12am,2pm]

+ –

TimeOfDay

otherwise[12am,2pm]

+ –

TimeOfDay

otherwise[12am,2pm]

+ –

TimeOfDay

otherwise[12am,2pm]

+ –


Creating a Service TriggerCreating a Service Trigger


Conclusion and OutlookConclusion and Outlook

• Specter– Overview– Personal journal user interface– Binding services to journal content

• Next steps– Journal user interface

• Queries• Setting up plans• Additional viewers

– Identifying additional services

Documents

SAB ReviewFebruary 2004Pervasive 2004April 2004 Using an Extended Episodic Memory Within a Mobile Companion Alexander Kröner, Stephan Baldes, Anthony Jameson,