Embed Size (px)
Citation preview
Joshua Blue
Nancy Alvarado, PhDVisiting ScientistIBM Research and [email protected]
Sam S. AdamsIBM Distinguished EngineerIBM [email protected]
Bootstrapping semantics in any situated embodiment
The central problem: Computational Understanding
Barriers to natural language processing/understanding commonsense knowledgerecognizing plans and intentionsquality of speech recognition and text-to-speech
There is no artificial system that understands language, even though the research has been going on for 40 years.
upper ontology
Missing
Applications specific knowledge
1M rules
100M+ missing rules
10-100K rules/app
Although a computer has beaten the world chess champion, no computer has the commonsense of a six-year-old child. (AI Mag. Winter 1998 p. 25: P.Cohen et al. " The Darpa High Performance Knowledge Bases Project")
A different approach
Previous attempts to make computers understand humans have focused on peripheral issues (speech, vision, motor control, expert performance, etc).
In every case, whatever success was achieved was limited by the absence of true understanding by the computer.
Our approach is to focus on the central problem of understanding and then work out to the periphery
Understanding
(purgeAllCommands(finalize
isNumber(sunitSelectors removeAllEventsTriggered hasUnacceptedEdits) isNumber(canZapMethodDictionary removeAllEventsTriggered isMorphicEvent))
(sunitSelectors isMorphicEvent clone))
(purgeAllCommands(basicSize selfWrittenAsIll presenter)(selfWrittenAsIll presenter inspect))
What does this code do?
Where does meaning reside?
In your mindOriginal, reversed, and replaced Cyc "knowledge"
(implies(and (isa ?TRANSFER TransferringPossession)
(fromLocation ?TRANSFER ?FROM))
(isa ?FROM SocialBeing))
(implies(owns Fred ?X)(objectFoundInLocation ?X FredsHouse))
(seilpmi(dna (asi REFSNART? noissessoPgnirrefsnarT)
(noitacoLmorf REFSNART? MORF?))
(asi MORF? gnieBlaicoS))
(seilpmi(snwo derF X?)(noitacoLnIdnuoFtcejbo X? esuoHsderF))
(purgeAllCommands(finalize
isNumber(sunitSelectors removeAllEventsTriggered hasUnacceptedEdits)isNumber(canZapMethodDictionary removeAllEventsTriggered isMorphicEvent))
(sunitSelectors isMorphicEvent clone))
(purgeAllCommands(basicSize selfWrittenAsIll presenter)(selfWrittenAsIll presenter inspect))
Goals
Achieve common sense computing, human-style semantic processing in a computer system
Develop a computer system capable of passing a "Toddler Turing Test"
Achieve natural language acquisition to the level of a 3-year old child
Subgoals
Semantic ProcessingThe generation, validation, maintenance and application of a
network of concepts that together represent the observations, experiences, perspectives, ideas, goals, plans, actions and emotions of an entity or system
Autonomous Common Sense Knowledge AcquisitionMeaningful self discovery via experience of the characteristics and
behavior of both the system itself, its environment, and the relationship between them
Quantitative measurement of cognitive machine capabilitiesScalable autonomous learning, exploiting unique qualities of software
Engineering Goals
General purpose "mind"Universal cognitive engineIndependent of specific embodimentsSelf bootstrapping with minimal a priori knowledge
Specialized embodimentsDifferent bodies for different environmentsLimited interface between mind and body
–Sensors, effectors, hedonic events, affect"Mind meld" between systems possible
Overcome limitations to developmental learning by supporting large scale training and aggregation
Bootstrapping Approach
Seeking clues from "the first meaning"
Personal research into advanced knowledge representations over the past 20 years
"Interest Flow Ring Network" breakthrough in 1999 led to rejoining IBM Research to explore the possibilities of semantic processing
Two years of "skunkwork", synthesis of multidisciplinary research, collaborations with cognitive psychologists and emotion theorists, two generations of prototype systems
Project Background
= Class
= Instance
= Instance
TGT!
= Class MGP! =
Class
= Class= Instance
= Instance
Short term
InterestValue
Long term
InterestValue
STI
LTI
Node
Initial Insights
NeurophysiologyHebbian synaptic learningCalvin's synaptic population dynamics
Developmental NeurophysiologySynaptic density cycles during development
Knowledge RepresentationContinual spreading activation and distributed attentionOpen ended creation of conceptual metalevels
AI LessonsEscape the homunculus trap
Complexity TheoryExploit emergent behavior by biasing feedback loops
Synaptic Density, Visual Cortex (Huttenlocher 1990)
Resting Glucose Uptake(brain metabolism), Occipital Cortex (Chugani 1987)
Synaptic Density,Neocortex(Johnson 1997.Thatcher 1992)
"plasticity waves" motor visual* auditory olfactory vestibularcutaneous
* Staged onset of sensory functiondriving sensory integration(Turkewitz 1982)
150
100
50
0
% Adult Level
Months Years0 Birth 4 8 12 2 3-8 9-15 Adult
Eye focusand tracking(Johnson 1997)
Explicit Memory(Johnson 1997)Neuron
Density(Rakic 1995)
Language Acquisition(Johnson 1997)
Native Bias
Architectural Clues
Superstition and Forgetfulness
Synaptic Density,Neocortex(Johnson 1997.Thatcher 1992)
"plasticity waves"
150
100
50
0
% Adult Level
Months
Years
0 Birth 4 8 12 2 3-8 9-15 Adult
Hebbian synaptic learningTrial and error connectionsCo-occurence binding
Calvin's synaptic populationsRandomly searchingTrial connectionsPermanent connections
Result: Grounded Superstition
Huttenlocher's newborn die-offBrain at birth has adult number of synapses
~50% disconnect at 2-3 monthsThatcher's plasticity waves
Cyclical Cortical ReorganizationCycles roughly every 4 yearsExplains "world view" changes
Result: Mass Forgetfullness, aka Garbage Collection
Dynamic, Self-Similar, Multi-metalevel Knowledge
Representation
= Class
= Instance
= Instance
TGT!
= Class
MGP! = Class
= Class= Instance
= Instance
Short term
InterestValue
Long term
InterestValue
STI
LTI
Node
[Person]
[Person: Sam]
(Hit x,y)
[Person: Bill]
[Person: Sue]
All (Hit Sue,Bill)one (Hit Sue,Bill)
MetaConceptMetaRelatio
n
flood
ebb
The Homunculus TrapThe notion of a Homunculus, a little man inside the brain that does all the real thinking, has been a dangerous design trap throughout the history of AI. The danger is that you design a system in such a way as to place all the "real thinking" inside a black box, which of course is to be implemented in a future version. When you look at the requirements for such a black box, you find that they are the same or nearly the same as those for the entire system. All you have done is swept the really hard, unsolved problem into a corner of the design.
...
How to Escape the Trap
1 Generate lots(all?) grounded variations of concepts and associations
2 Test them constantly via motivated experience in environment
3 Regularly and aggressively remove "uninteresting" and "unused" concepts, venting the combinatorial explosion
Yes, this smells a lot like GA and evolutionary approaches, but its different because what's changing is the overall mental structure of a single individual
All aspects of the designmust be transparent,no black boxes
Exploiting Complexity
"Fixed""Crystalized"
"Dead""Total Order"
"Chaotic""Disintegrated"
"Dead""Total Disorder"
"Complex""Dynamic"
"Alive""Just right"
Joshua BlueDesign Point
Classical AI Systems and Most ApplicationsStrive for one shot, fixed point determinism
Query
Response
Start
Stop
Continue...
EmergeOur approach trades off determinism and control for expressivepower and adaptivebehavior
Implications
Focusing on the bootstrapping of semantics in the human infant will provide architectural clues
Superstition/Forgetfullness approach requires ongoing temporal experience
Experience requires embodied interaction with an environment
Multi-level emotional control system provides subsumption-style reactive flexibility, but at higher cognitive levels
Human levels of adaptive flexibility require a tradeoff on fixed-point determinism
Principles, Challenges and Risks
Minimum Innate Knowledge: no apriori knowledge baseSelf Organization: the system will independently acquire, organize, and generalize its own knowledge and experiences
Situated Embodiment: rich sensory and motor interaction with a rich, dynamic environment (that includes humans)
Affective Self Motivation: the system's attention and actions will be driven by its own needs, desires, and emotions
Longevity: long term, continuous, real-time operationTransparency: the structures in the resulting "mind" must be comprehensible to humans and be composable (unlike neural nets)
Temporality: all cognitive processing will take place embedded in the temporal experience of the system
Design Principles
Challenges
Finding the right architecture (innate vs emergent)Harnessing and steering emergent behaviorDeveloping sufficiently rich bodies and environments to enable "human-like" common sense acquisition
Implementing relatively unexplored constructs such as "self" and "consciousness"
Tuning the system's many parameters (self-tuning)Measuring incremental progress objectivelyOvercoming scale limitations of developmental approach
Achieving "mind meld" capability between systems
Risks
The Ultimate Grand ChallengeShortcuts in using high level sensors and effectors
Availability of sophisticated sensory/motor subsystems to enable rich interaction
Joshua Blue: Future
Phase 1 - Develop and Test 3rd Generation SystemImplement 3rd generation architectureDevelop several experimental bodies and environments of increasing sensory/motor complexity.Validate cognitive behavior experimentally using analogs of standard psychological tests.Demonstrate capability to merge MAINs and exploit combined experience.
Phase 2 - Develop and deploy Baby Joshuas for Large Scale Common Sense TrainingDevelop an online environment based on networked gaming platforms (e.g., Sony PS3) for the training
by humans of large numbers of Joshua systems in the common sense knowledge of a rich, virtual environment.
Recruit and maintain an online community of trainers using multi-user gaming competition techniques.Merge MAINs of trained systems and validate advances in cognitive development using standard tests.Demonstrate pre-verbal common sense reasoning using a "Baby Turing Test" which we will develop
Phase 3 - Develop and deploy Toddler Joshuas for Large Scale Verbal Language AcquisitionExtend online environment for the training by humans of large numbers of Joshua systems in acquiring
natural languages within a rich, virtual environment.Recruit and maintain an online community of language trainers using multi-user gaming competition
techniquesMerge MAINs of trained systems and validate advances in cognitive development using standard testsDemonstrate common sense reasoning and language acquisition using a "Toddler Turing Test" which
we will develop.
Joshua0
Second Generation SystemFirst Experimental Version
Implementation History
Joshua0 is the first version of the system fully capable of hosting multiple experimental models
Based on Sem interest flow network (Gen 1)Improvements
Subsystem architectureModular, tunable, traceableInitial embodiment system addedExtensive comments and helpSave and restore functions
High Level Architecture
Environment simulates physics and contains objectsObjects include bodiesBodies have minds and body partsBody parts have innate behavior, sensors and effectorsSensors and Effectors connect into the mindMind has a number of subsystems:
Interest Flow NetworkAssociatorOccurencerSequencerAttributorImaginationConsciousness
FeelerConcentratorArousalSensory SystemEffector SystemReaperGrouper
Knowledge RepresentationNodes
represent "concepts", no matter what levelhave long term and short term interestreceive and redistribute interest over wires and ringshave a ring "dual" which models its metalevel behavior
Wiresconductors of interest flow between nodeshave runtime conductance based on
–strength (frequency of reinforcement)–local valence: emotional relevancy to global valence
Ringsthe metamodeling dual of nodesmodel MGP relationships with other rings (Genotype/Phenotype)models a similarity group (basic definition of a concept)conduct and distribute interest in special ways
Distributed Attention
Short term
InterestValue
Long term
InterestValue
diffusion of interest from adjacent nodes
threshold varies based on lessened/heightened arousal/sensitivity
Repeated STIincreases LTI(entrainment)
STI
LTI
Very slow drain(LT forgetfullness)
The Metasemantic Affective Interest-flow Network is a specialized semantic network with spreading interest as a distributed attention mechanism
overstimulation -> STI goes refractory
"Short Term Memory"Interest Threshold
diffusion of interest to adjacent nodes
Node
The Attention Landscape
Extrapolate to a large network of such nodes
Short term
InterestValue
Long term
InterestValue
STI
LTI
Node Networ
k
"Awareness"InterestThreshold
Superrings
Subrings
Conceptual Metalevels
CurrentMetalevel
SuperMetalevel
SubMetalevelLower metalevel relationships
with Members, Instances, Players
Member relationships with higher metalevels: Sets, Categories, Classes, Roles
{Qualifier}
Specifies subring membership
Open-ended Concept MetalevelsRingStructure
Class-basedPerspective
Role-basedPerspective
Set-basedPerspective
ConceptPerspective
Instance
Class
Metaclass
Instance
Class
Metaclass
Instance
Class
Metaclass
Instance
Class
RolePlayer
RolePlayer
RolePlayer
RolePlayer
Concept
Concept
Concept
Concept
{x| x is qualified by expression Q1}
{x| x is qualified by expression Q2}
{x| x is qualified by expression Q3}
{x| x is qualified by expression Q4}
ee
ee
ee
ee
Q1
Q2
Q3
Q4
Qn = MAIN Qualifier Expression
subr
ings
supe
rrin
gs
MAIN Concept QualifiersAll concept nodes in the MAIN have qualifiers that specify conformance requirements for their subrings
Each new concept node is initially defined by a unique qualifier expression aasigned to it by the subsystem that generates the new node (Anti-H style)
Qualifiers are constraint objects that can determine membership in a setOriginally developed for OO CASE tools by Adams/Fogwell
Used in IBM Smalltalk engagements (version by Burbeck/Graham)Adapted for use in Joshua Blue
A qualifier can be specified asAn individual constraint on some attribute of a concept node
–node has emotional relevance >= 0.5 valence–node has > 1 outgoing wire–etc.
A set of constraints joined via logical operators–{ Q1 AND Q2 AND Q3 NOT Q4 }–{ Q1 OR Q2 }
A composite of other qualifiers–{[node1 has an outgoing wire to [node2 has 5 subrings]] AND Q3 }
Qualifer algebra enables concept similarity measures based on (mostly independent) quantifiable attributes of concept nodes
Qualifiers can be fuzzy
IncomingWires
OutgoingWires
Superrings
SubringsRelationships with Members, Instances, Players
Member Relationships with Sets, Categories, Classes, Roles
Concentrator
Sensors
Interest Flow in a Concept Node
generalizationinterest flowsup from subrings
specializationinterest flowsdown to subringsbased on a distribution function
specializationinterest flowsdown to nodefrom superring
generalizationinterest flowsup to superringsfrom node
IncomingWires
OutgoingWires
Superrings
Subrings
Wire thickness = assoc. frequencyWire color = emotional relevance
enlarged Long terminterest level (LTI)
Short terminterest level(STI)
ConsciousActivation
SubconsciousActivation
UnconsciousActivation
EmotionalRelevance
Node Ring twisted to form
STI + LTI = Total Interest
Concept Node/Ring Design
{Qualifier}
Specifies attributes for membership(set/type/category/class/role definition)
Lower metalevel relationships with Members, Instances, Players
Member relationships with higher metalevels: Sets, Categories, Classes, Roles
A concept's meaning consists of its sibling associations with other concepts as well as its metalevel relationships with other higher and lower level concepts
Subsystems IMind
Coordinates all subsystems in simulated timeInterest Flow Network
A spreading activation network of "concept" nodes and interest conductors (wires and rings)
Interest flows through the network, activating nodes, draining along wires and rings to activate other nodes.
ConsciousnessSets conscious and subconscious node interest thresholds and
maintains collections of nodes in both setsReaper
Aggressively removes uninteresting nodes, rings and wires from the network, simulating "plasticity events"
Multiple reapings can be scheduled with different thresholds, repetitions
MindBody Sensor
sEffectors
MAIN
Semantic Bootstrapping
Semantic Bootstrapping
MindBody Sensor
sEffectors
MAIN
Semantic Bootstrapping
MindBody Sensor
sEffectors
MAIN
Semantic Bootstrapping
MindBody Sensor
sEffectors
MAIN
Interest-based Attention/Awareness
MindBody
Sensors Effectors
MAINTot
al N
ode
Inte
rest
Lev
el
Low
High
Subconscious Threshold
Conscious Threshold
Subconscious Nodes
Unconscious Nodes
Conscious Nodes =ov
er ti
me
CurrentExperience
A
BC
Subsystems II
AssociatorGenerates associations between all conscious nodesUses these associations to create new wires and reinforce existing
wires between these nodesSystem Mood
Maintains global emotional stateReacts to emotional content of conscious nodesImpresses emotional relevancy on nodes, rings and wires
System ArousalMaintains global state of arousalGlobally effects node activation thresholdsSharpens/dulls concentrationDirectly driven by body physiology
Focus: Associator
System Mood/Valence
Experiencer
Associator
CurrentExperience
A
BC
A
B
C
nodes andexisting wires
newly created wire with system valence
existing wires strengthened and valence adjusted
The Associator uses the Anti-homunculus strategy to generate grounded but unproven semantic associations (interest flows) between the concept nodes currently being experienced. New wires are very weak and start with System Mood. Existing wires between current nodes are reinforced and their valence is adjusted based on the System Mood. This process also emulates Hebbian synaptic dynamics by generating possibly interesting connections between nodes that are tested over time.
Taking a Functionalist Approach
Human emotion is not unitary or modular -- it is implemented as multiple interacting systems with different functions.
Physiology need not be duplicated as long as functionality is achieved.
Functionality depends upon taking a whole system view, recognizing dependencies.
A "control system" metaphor is useful because emotion has a regulatory function and includes feedback mechanisms.
Social Control
Emotional Activation
Unattended Cognition
Conscious Cognition
Deep Affective Control
Coordination of Mind/Body
Feedback Mechanism
Signs, language
Subjective affect
Appraisals
Valence
Arousal
Basis for Homeostasis
Social distance
Sense of self
Attention
Experience in world
Metabolism &body function
Cortical
Subcortical
FamiliarityCompetenceAutonomyControl, etc
Emotion and Motivation in Joshua's Architecture
Two primary dimensions of affect:Valence -- evaluates survival and goal-relevant
significance of experience.Arousal -- controls activity level of system.
These are sufficient to describe nearly all affective phenomena observed in humans, including "basic" emotions and Sloman's secondary & tertiary emotions.
These provide a single regulatory mechanism across levels of the system.
Positive Valence
Negative
Valence
High Arousal
Low Arousal
Open Mouth Brows UpEyes Wide
Face Relaxed
Brows downEyes narrow
Lips tense
Mouth droops Smile
Duchenne Smile
Sadness
JoyExcitementAnger
CalmApathy
Interest
ShockFear
Panic
Disgust
DistressBoredom
Annoyance
DisapprovalDiscontent
ApprovalLiking
Ecstasy
EnjoymentAmusement
ThrillSurprise
Anxiety
Contentment
Lip Curl
Despair Peace
Attachment
Happiness
Oblique Brows
Mouth Stretch
Oblique Brows
Smile
Brows Down
Open Mouth
Closed Mouth
Open Mouth
Duchenne Smile
Closed Mouth
Affect Coordinate Space
~80% of human emotional response modeled in two dimensions
Designing Emotional Feedback
Affect-guided Interest Flow
Interest flow pattern during negative affect
Interest flow pattern during positive affect
How you feel influences what you think about
Interest-guided Affect
What you think about influences how you feel
EtEt -2 Et -1
A BC D
GK
MCAt t
E?t +1
Ct? t?
GR
E?t +2
AC
Good Bad
Interest flow Interest flow Prediction/PlanPrediction/Plan
V = System Mood/Valence
Vt =Vt -2 = Vt -1 = V?t +1 = V?t +2 =
shifts shifts
Past Present Future
Subsystems III
ExperiencerCaptures co-occurence of conscious nodesGenerates "experiences/moment rings" based on this similarityMaintains collection of recent experiences/moment rings, which
form the "breadth of attention" Sequencer
Looks across the Experiencer's recent momentsCaptures potentially interesting sequences of conscious node
activations over timeGenerates "sequence rings" based on this similarity, which can
model procedures, state machines, and plansConcentrator
"sprays" extra interest on most active nodes
Interest-based Concentration
MindBody
Sensors Effectors
MAIN
Subconscious Threshold
Conscious Threshold
Subconscious Nodes
Unconscious Nodes
Arousal influences breadth of interest distribution
Interest-sprayermimics Focus
Valence influences selection of focal concepts(?)
Tot
al N
ode
Inte
rest
Lev
el
Low
High
over
tim
e
Subsystems VI
SensoryMaintains all connections between body sensors and
"sensor nodes" in the networkMostly passive since sensors are driven from the
environment/bodyEffector
Maintains all connections between "effector nodes" in the network and effectors in the body
Drives effector triggering based on effector node activation levels
Sensors and EffectorsSensor Node
Sensory Event Nodes
Sensor Fires
createsandinjectsinterest
InternalSensation
Mind
Body
Effector Node
Effector Event Nodes
IntentionalActivationEffector Fires Executor
creates andinjects interest
Environment
"Hopsee" Experimental ModelFinding the right design was trickyBalance of simplicity and experimental strengthBased on a "minimal" body
an eye that detects visual change onlya foot that moves the body
–hop effector–hopped proprioceptor
1D infinite sidewalk/tape of white or black tiles (1's or 0's)"Turing Machine" level model, minimal but sufficent
Laboratory
Joshua1
Third Generation System
Comparison with 2nd Generation
2nd Generation 3rd Generation
Additions/Improvements:Hedonic motivation, Experience models, Anticipation/Predicition,Goals, Planning, Execution, Generalization/Specialization, Analogy
Architectural OverviewEmbodied MindSituated BodyDual Semiosis
ProprioceptionEnvironmental
Blackboard-style integration (MAIN)
Universal RepresentationDeep Affective ControlContinuously AdaptiveSelf tuningEmergent self consciousness
Focus: MAIN
Reaper
Planner
Executor
PlanStep
Goalie(sub)Goal
Sequencer
Hedonic
Effectors
Sensors
ExperiencerExperience
AnticipatorPredictorPredictionExpectation
Generalizer
Specializer
Associator
Concentrator
Imagineer
Analogizer
M
A
I
N
current nodes
new or adjusted wires
current nodes
new subnodes
current nodes
new supernodes
current nodes
better new supernodes
current nodes
better new subnodes
nodes and wires
current nodes
candidate sequence rings for plans
longer/stronger sequence rings
current nodes
new experience/moment rings
experience/moment rings
new sequence rings
experience/moment rings for goals
stronger experience/moment rings
pain/pleasure/relief event nodes
sensory event nodes
effector event nodes
candidate sequence rings for predictions
longer/stronger sequence rings
HQ
Q
Q
Q
H
H
Q
Q
HH
H
H
= Quality improvement feedback loop
= Anti-homunculus strategy at work
Awarenesschanging nodes
current nodes
effector activation
Focus: System Mood(Valence)
System Mood(Valence)
Planner
Executor
PlanGoalie
Anticipator
Experiencer
Step
Associator
(sub)Goal
Hedonic
System Arousal
Interest-flow
CurrentExperience
A
BC
Pained State
Pain Event
Pleasure Event
Relief Event
ExpectationPredictorPrediction
shifts
determines interesting nodes
adjusts
continual decrease
point decre
ase
poin
t inc
reas
e
point incre
ase
increase as valence goes neg
increase with big valence changes
incr
ease
with
big
aro
usal
dro
p
decrease arousal goes neg
high valence = optimism
, patience
low valence = pessim
ism, im
patience
decr if abandonedincr if succeed
decr if failed
incr if succeed
decr if fail
incr if desire future
decr if fear future
Novelty = decr slight but cumul.
Familiarity = incr slight but cumul.
adju
st e
mot
iona
l rel
evan
ce o
f wire
s
Focus: System Arousal
System Arousal
System Mood/Valence
Planner
Concentrator
Plan
Goalie
Anticipator
Predictor
Step
BodyPrediction
(sub)Goal
EffectorsSensors
CurrentExperience
A
BC
Expectation
incr
ease
as
vale
nce
goes
neg
increase with big
valence changes
incr
ease
with
big
arou
sal d
rop
decr
ease
aro
usal
goe
s ne
g
Hedonic
Pained State
Pain Event
Pleasure Event
Relief Event Metabolic Parameters(hunger, thirst, fatigue etc.)Physiological attributes thatrequire body-scale actionfor regulation (eat, drink,rest, etc.)
raises and lowers awareness
high input increases
large drop in input decreases
high = fast reaction
low = slow reaction
higher = shorter patience
higher = increase decisiveness
heavy action in future incr
Prep for action
High = sharpen
Low = dull
adju
sts
as n
eede
d
drops suddenlyincreases
increasesincreases
Affective Interest-flow Feedback
Et
DG
MA
System Mood(Valence)
Interestflow
biases
adjusts
shifts attentionto other nodes
System Arousal
adjusts awarenessthresholds that"experience" nodes
bi-drectional influences
Focus: Sequence Ring Lifecycle
EtEt -2 Et -1
A BC D
GK
MCAt t
Planner
Executor
Plan
Goalie
Anticipator
Predictor
Experiencer
Step
Prediction
(sub)Goal
Sequencer
Experience (Et)MAIN
candidate sequenceswith non-terminalsimilarity to currentexperience generalized
sequenceslongersequences
Expectation
new, groundedsequences basedon experience
Q
Q
Q
candidate sequenceswith non-initialsimilarity to currentsubgoal
strengthenedor weakenedplan sequencesbased onexperience
Q
Innate Goals
Reaper uninterestingsequences
= Quality improvement feedback loop
Anticipation Process
Anticipator
Predictors
Predictions
Experience (Et )
Expectations
MAIN
~=
EffectorsSensorsEnvironment
failed
Experience (Et-1)
predicted
stre
ngth
en p
redi
ctor
wai
t/ret
ry o
r ab
ando
n
wea
ken
pred
icto
r
next
pre
dict
ion
Q
Q
Q
longerpredictivesequences
candidatepredictivesequences
Q
partial prediction
gene
raliz
e pr
edic
tion/
pred
icto
r
Q
Q
moregeneralpredictivesequences
Q = Quality improvement feedback loop
Forward-chaining "inferencing"Shapes up quality in predictive cause/effect understanding
Grounded in environmental experience
Multiple quality feedback loopsSimultaneous Predictor - Prediction breadth is a parameter
Et -1
BKC
E
t
DG
MA
Time
Goalie/Planner/Executor Process
Planner
Executor
Plans
GoalieSteps
Subgoals
Experience (Et )
Goal
MAIN
~=
EffectorsSensorsEnvironment
failed
Experience (Et-1)
satisfied
wait/retry or abandon
weaken plan
strengthen plannext step
Q
Q
Q
candidateplanningsequences
longerplanningsequences
Q
Q
generalize plan/step
partial satisfaction
moregeneralplanningsequences
Q
Q = Quality improvement feedback loop
Goal driven motivationBackward-chaining "inferencing"
Shapes up quality in exploiting cause/effect understanding
Grounded in environmental experience
Multiple quality feedback loopsSimultaneous Plan/Subgoal breadth is a parameter
Subgoal effectors are "fired" in order to change environment
Et -1
BKC
Et
DG
MA
Time
Anticipator/Planner Symmetry
Planner
Executor
Plans
Goalie
Anticipator
Predictors
StepsPredictions
Subgoals
Experience (Et )
Expectations
Goal
MAIN
~= ~=
EffectorsSensorsEnvironment
failed failed
Experience (Et-1)
predicted satisfied
stre
ngth
en p
redi
ctor
wai
t/ret
ry o
r ab
ando
n
wea
ken
pred
icto
r
next
pre
dict
ion
wait/retry or abandon
weaken plan
strengthen plannext step
Q
Q Q
longerpredictivesequences
candidatepredictivesequences
candidateplanningsequences
longerplanningsequences
Q Q
partial prediction
gene
raliz
e pr
edic
tion/
pred
icto
r
Q
Q
moregeneralpredictivesequences
Q
generalize plan/step
partial satisfaction
moregeneralplanningsequences
Q
Q = Quality improvement feedback loop
Et -1
BKC
Et
DG
MA
Time
“The best way to predict the future…”*
*“…is to create it.” – Alan Kay
Planner
Executor
Plans
Goalie
Anticipator
Predictors
StepsPredictions
Subgoals
Experience (Et )
Expectations
Goal
MAIN
~= ~=
EffectorsSensorsEnvironment
failed failed
Experience (Et-1)
predicted satisfied
stre
ngth
en p
redi
ctor
wai
t/ret
ry o
r ab
ando
n
wea
ken
pred
icto
r
next
pre
dict
ion
wait/retry or abandon
weaken plan
strengthen plannext step
Q
Q Q
longerpredictivesequences
candidatepredictivesequences
candidateplanningsequences
longerplanningsequences
Q Q
partial prediction
gene
raliz
e pr
edic
tion/
pred
icto
r
Q
Q
moregeneralpredictivesequences
Q
generalize plan/step
partial satisfaction
moregeneralplanningsequences
Q
Q = Quality improvement feedback loop
Et -1
BKC
Et
DG
MA
TimePredict the FuturePredict the Future
Reality CheckReality Check
Create the FutureCreate the Future
"Mind-Meld" Capability
MindBody
Sensors Effectors
MAIN
MindBody
Sensors Effectors
MAIN
MindBody
Sensors Effectors
+ =
MergedMAIN
Possible because:Representation is comprehensible to humans (unlike neural networks)
Continuously grounded cognitionIdentical bodies in different environments
Continual sequence competitionAlso implies programmability
Testing and Scalable Training
Our Approach to TestingIncremental testing against performance metrics derived from human/animal behavior.
Minimum competences established in three domains: learning, social cognition, language acquisition and processing.
Develop appropriate bodies and environments for specific tests, but use the same "mind"
Hopsee test bed can show that the system develops and exploits a mental model of its environment.
Ultimately, Joshua will pass a Toddler Turing Test which we will develop
The Toddler Turing Test
Adult cognition rests on an experiential foundation that is in place by roughly age 3.
This foundation includes almost everything encompassed by the term "common sense" but excludes formal learning and logic.
Because current AI efforts have mastered logic-based learning, efforts need to focus on acquiring the missing experiential foundation upon which logic operates.
Minimal Competences
Associative Learning -- Purposefully explore the environment, recognize reward and punishment, avoid danger, form goals and seek to satisfy them.
Social Cognition -- Understand and effectively interact with others, form a self schema, engage in observational learning, demonstrate empathy and attachment.
Language Acquisition -- Acquire and use language flexibly to accomplish goals.
Scalable Learning
Assumes "mind meld" capabilityExploits web-enabled gaming platforms for simulating and managing rich training environments (PS3 target)
Exploits both multiuser gaming phenomenon and A-Life enthusiasm
Designed to appeal to "Creatures" and Tamagotchi users (> 1 million)
Harnesses 1000's of tutors and teachers for Joshua's education
Example: Creatures
A-life environment for breeding, raising and playing with artificial creatures, "Norns"
Simplified models of emotion, metabolism, geneticsCreatures freely explore and learn own their own
Over 1 million copies soldLarge, thriving internet communities of users
>300 sitesBreeding clubsAbuse sitesRecovery sitesOpen source development of characters and objects
"A very great vision is needed, and the onewho has it must follow it as the eagle seeksthe deepest blue of the sky"
- Crazy Horse
Backup
Design Patterns forComplexity Reduction
Three Powerful Patterns
Whole ValueThing-GroupThingMetatype-Genotype-Phenotype
Discovered by numerous OO Design practitioners during the 2nd decade of Object Technology (late 1980's, early 1990's)
Each provides a unique and powerful mechanism for reducing design complexity
When all three can be applied simultaneously to the same design, the result is usually a serveral orders of magnitude reduction in complexity
Whole ValueReduces design complexity by moving burden of semantic mapping from the designer to the system itself
The "big idea" behind Object Orientation
Electronics
Domain Semantics
MathLogic Data
IdentityLanguag
e Domain Semantics
MathLogic Data
Identity
Language
Electronics
Electronics
?
Data+Functions=Programs
Programs=Object Ensembles
x=f(y)
if a>b then
Domain Semantics
MathLogicData Identit
yLanguage
Thing-GroupThing
Self-similar behavioral decompositionSimilar to "composite" patternReduces complexity of scale
IndustryCompany + Commerce + TransportationFactory + ShippingShop Floor + WarehousingWork Cell + Material HandlingMachine + Adjustable FixtureValves, Motors, Switches
vs
Machine-likeat different scales
Texas InstrumentsSemiconductor Wafer Fab(GemWorks)
Metatype-Genotype-Phenotype
Generalization of Class-Instance relationshipsDirect modeling of conceptual metalevelsReduces complexity of instance configurationsFundamental framework for systems with modifiable and open-ended ontologies
I= Instance
Class
MetaClass
Class Class
I I I I I I I I I
MetatypeGenotypePhenotype
ClassInstance
ClassInstance
What is the class of a metaclass? Supplier MetaContract
Supplier ContractCustomer MetaContractCustomer Contract
GE Capital Vendor Financial Systems
ClassInstance
MGP
MGP
MGP
Implications of MGP
Note: conceptual metalevels are orthogonal to concept (inheritance)hierarchies
Humans routinely deal with 3 metalevels the current level of attention the level above (the generalization of current level) the level below (specific cases of current level)
Simultaneously working with 4 levels is uncommon, and 5+ is very rare
ExamplesOrganism/Organ/Tissue/Cell/Biomolecules/Atoms/...Social Organization Hierarchies
Implications of MGP
Complexity can be dramatically reduced by shifting it to a higher metalevel
Conservation Rule? We don't think so
GenotypePhenotype
Com
ple
xit
y
MGP
MGP
MGP
Exploiting Complex Adaptive Systems in
Design
Self Organization and Complexity
Feedback networks among interacting behavioral entities trigger the emergence of complex macro-level behavior, and a new entity is created at the metalevel
The Complexity Continuum
"Fixed""Crystalized""Dead""Total Order"
"Chaotic""Disintegrated""Dead""Total Disorder"
"Complex""Dynamic""Alive""Just right"
"Entropy""External Forces"
"Emergence""Adaptive"
"Self-organization""Autocatalysis"
Forces Interacting
Exploiting Complexity
"Fixed""Crystalized""Dead""Total Order"
"Chaotic""Disintegrated""Dead""Total Disorder"
"Complex""Dynamic""Alive""Just right"Joshua Blue
Design Point
Classical AI Systems and Most ApplicationsStrive for one shot, fixed point determinism
Query
Response
Start
Stop
Continue...
EmergeOur approach trades off determinism and control for expressivepower and adaptivebehavior
Finding the "Edge of Chaos"
Frozen System
s
Chaotic System
s
Complex
Adaptive
Systems
Chaotic System
s
Frozen System
s
Where the wild things are
Complex
Adaptive
Systems
Chaotic System
s
Frozen System
s
Multiple Complexity Continua
Every complex adaptive system (CAS)has its own continuum
Interacting CAS create new CAS at higher metalevel
Chaotic
Complex
Frozen
Chaotic
Complex
Frozen
Chaotic
Complex
Frozen
Chaotic
Frozen
Chaotic
FrozenChaotic
Frozen
Frozen
Complex
Chaotic
Exploiting Multiple CAS in Joshua's design
Design major subsystems as CASIntegrate subsystems by designing interactions between CAS
Repeat as necessaryChaotic
Complex
Frozen
Chaotic
Complex
Frozen
Chaotic
Complex
Frozen
Chaotic
Frozen
Chaotic
Frozen
Chaotic
Frozen
Frozen
Complex
Chaotic
