Resource-Bounded Machines are Motivated to be Effective, Efficient,

and CuriousBas R. Steunebrink, Jan Koutník, Kristinn R. Thórisson, Eric Nivel, Jürgen Schmidhuber

The Swiss AI Lab IDSIA, USI & SUPSI, Reykjavik University

The Main Argument

1. Explicitly acknowledge resource constraints2. Identify the constrained resources3. Design AI system to be driven by better and

better resource utilization4. Order activities around resource utilization5. Emergent result: effectiveness, efficiency &

curiosity

Fundamental Resources

Resource Compression Drive To improveEnergy Efficiency (unnamed) WorkInput Learning Curiosity PlayTime Effectiveness (unnamed) Dream

Resource Compression: Why?

• Less time & energy spent more reward (resources shared with other agents)

• Less time & energy spent more left for future tasks

• Compression of input = learning better prepared for unknown future

Driven by Resource Compression Progress

Minimize resource consumption through:1.Knowledge– Learn new more effective & efficient routines– Curious exploration to “fill knowledge gaps”

2.Architecture– Re-encode known routines for more effective &

efficient execution– Example: self-compilation

Work—Play—Dream Framework

• Utilize the kinds of activities afforded by the patterns of interaction with human teachers / supervisors / users

• Work: fulfill main purpose, no exploration, store interesting / unexpected events

• Play: curiosity-driven exploration, perform experiments, may still requires supervision

• Dream: analyze unprocessed events, self-compilation, task invention for Play

WPD Framework, cont.

• Work—Play—Dream are processes, not states• Can run in parallel, but sometimes not possible

due to resource/situational constraints• Combination of Work & Play leads to creativity• Dreaming may be a necessary side-effect for any

system constrained in computational power and memory– “Tired” = buffers reach capacity– “Dream” = process input history backlog

AERA: An Explicitly Resource-bounded Architecture

• Knowledge is operationally constructive• Model-based & model-driven, hierarchically• Simulation through forward chaining• Planning through backward chaining• Compilation of useful & reliable chains– Originally for scalability– Now realized to satisfy the architectural way of

achieving resource compression: re-encoding

How AERA does resource compression

• Consider 3 resources: Time, RAM, HDD• Time more precious than memory• Self-compilation leads to better resource

utilization• Thus AERA must be motivated to self-compile• Simple analysis of control values yields goals that

give rise to empirical testing of unstable models• Crux: scheduling needed Work, Play, Dream

Conclusion• AERA is being developed as a cognitive

architecture towards AGI– Based on many firm principles, but not curiosity– But all ingredients for curiosity are present– Learning & re-encoding both possible!– Thanks to self-compilation ability

• Resource usage compression = principled middle ground– No twisting of AERA or Theory of Curiosity– AERA still based on solid principles– Curiosity generalized to resources-bounded view