“Curious Places”

October, 2007

Key Centre of Design Computing and Cognition, University of Sydney

A Room that Adapts using Curiosity and

Supervised LearningKathryn Merrick, Mary Lou

Maher, Rob Saunders

Overview

Adaptable, Intelligent Environments

Curious Supervised Learning

A Curious, Virtual, Sentient Room

Limitations and Future Work

The computer for the 21st century Hundreds of computers per room Computers come and go (Weiser, 1991)

Adaptability is important at two levels: The middleware level The behaviour level

Adaptable, Intelligent Environments

Adaptable Middleware

Resource management and communication

Adaptability has been widely considered at this level Real time interaction Presence services Ad hoc networking

Intelligent Room Project

Gaia

BLIP Systems

Adaptable Behaviour

Adapting behaviour to human activities Supervised Learning The “Neural Network House” Data mining Considered in fixed domains

How can we achieve adaptive behaviour in response to changing hardware or software?

Adaptability by Curiosity and Learning

Curiosity adapts focus of attention to relevant learning goals

Learning adapts behaviour to fulfil goals Curious reinforcement learning Curious supervised learning

MyS

QL

Da

tab

ase

Projector

Rear project

ion screen

PC

Bluetooth blip nodes

Agent

Agent

Curious Information

Display

Curious Research

Space

Supervised Learning

“Learning from examples”

A supervised learning problem P can be represented formally by: A set S of sensed states A set A of actions A set X of examples Xi = (Si, Ai)

A policy π : S A

Complex, Dynamic Environments

Contain multiple learning problems P = {P1, P2, P3…}

Learning problems in P may change over time Addition of new problems Removal of obsolete problems

Aim to focus attention on states, actions and examples from a subset of problems Works by filtering

Identify potential tasks to learn or act upon

Compute curiosity values Arbitrate on what to filter

High curiosity may trigger learning or action

Low curiosity does not

Modelling Curiosity for Supervised Learning

S(t), X(t)

S(t)X(t)

Curiosity

Learning Action

Observations and events

Task Selection

Curiosity Value

Arbitration

The Curious Supervised Learning Agent

Past states, examples and actions are stored in an experience trajectory Y

Experiences may influence curiosity A(t)

S(t)

S(t), X(t)

Y(t-1)

sensors

effectors

L

Aπ(t) SL

π(t-1)

X(t)

M =

{ Y(t) U

π(t) }

π(t)

Y(t)

C

A university meeting room in Second Life Seminars and Meetings Tutorials Skype-conferencing

A Curious, Virtual, Sentient Room

Floor Sensors

SMART Board and Chairs

BLIP System

Lights

Virtual Sensors and Effectors

Meta-Sensors and Meta-Effectors

BLIP System provides an up-to-date list of current sensors and effectors and acts as an intermediary for communication

Agent does not communicate directly with sensors and effectors

Agent has a ‘sensor of sensors’ and an ‘effector of effectors’

The Curious Room Agent

Computational model of novelty used for curiosity

Table-based supervised learning using associations Learns accurately but Unable to generalise

Behaviour of the Curious Place

Avatar enters Lights go on

Avatar sits SMART Board on Lights off

Preliminary Evaluation

~6 repetitions by human controlled avatars required for learning

Can adapt to new devices

Can adapt simple behaviours to form more complex sequences

Limitations

Current prototype is proof-of-concept only, no significant empirical results yet

Issue of if/when/how to ‘forget’ behaviours Is an interface required for

manual editing or override of learned behaviours?

Future Work

Further work on curiosity models

Design a suite of experiments to test attention focus in Environments of increasing

complexity Dynamic environments More complex tasks