Mohamed Oubbati Embodied Intelligence

EmbodiedIntelligencewhenBRAIN,BODY,andENVIRONMENTinteract

MohamedOubba?

March,2012

1Dr.Oubba?,March2012

EmbodiedIntelligencewhenBRAIN,BODY,andENVIRONMENTinteract


Intelligence


Canwedesignintelligence?


Imeanhuman‐likeintelligence,notjustthe“appearance”ofintelligence




YES





YESBUT1.weneedtoknowhow.

2.weneedtoolstoimplementthisknow‐how.7Dr.Oubba?,March2012


YESBUT1.weneedtoknowhow.

2.weneedtoolstoimplementthisknow‐how.

Thistalkisaboutthesetwopoints



Whatisintelligence?

In1921,theJournalofEduca?onalPsychologyasked14expertsfordefini?ons


Whatisintelligence?

Result:14differentdefini?ons!

In1921,theJournalofEduca?onalPsychologyasked14expertsfordefini?ons


Whatisintelligence?

Intelligenceisdifficulttodefineandunderstand


Whatisintelligence?

Intelligenceisdifficulttodefineandunderstand

Maybeoneshouldtrytobuildar?ficialintelligentsystemsinordertounderstandintelligence


Ar?ficialIntelligence


• “…thestudyofhowtomakecomputersdothingsatwhich,atthemoment,peoplearebeber”

A.I.researchis

(Rich&Knight,1991)

• “…theartofcrea?ngmachinesthatperformfunc?onsthatrequireintelligencewhenperformedbypeople“

(Kurzweil,1990)

• Themoderndefini?onofA.I.is“thestudyanddesignofintelligentagents”



ThecurrentAIhasproducedsuccessinthenarrow:well‐definedproblems,withverifiableresults,inpredictableenvironments,usingwell‐understoodtechniques.



In1997,DeepBluebeatKasparov.

SomeAI programsareextremelysuccessfulatwhattheyarespecialized.



‐ naturallanguageunderstanding‐ generalvision‐ understandingamovie‐ wri?ng

AlthoughmanyAIsystemsarebeyondhuman‐levelcompetenceatsomeindividualtasks,AIhasfailed(miserably)inmanydomainslike


FailureofthecurrentAI


ToovercomethecurrentAIproblemswemustreturnbacktotheoriginalgoalofAI:

„createsystemsthatarehumanlyintelligent“


FailureofAI


ToovercomethecurrentAIproblemswemustreturnbacktotheoriginalgoalofAI:

„createsystemsthatarehumanlyintelligent“


FailureofAI

aimstocreateacomputer‐basedintelligenceatleastassmartasahuman.

(since2001)



•  Conventional AI(alsocalledNarrowAI,AppliedAI,WeakAIorClassicAI)

(alsocalledStrongAI,Human‐levelAI,RealAI,…)

Conventional AIvs.AGI

Intelligenceistheabilitytoachievepre‐definedgoals,underconstraints.

Intelligenceistheabilitytoexpandanddevelopnewabili?es.


aimstocreatemachinesthatactasiftheywereintelligent,i.e.theappearanceofintelligence.

aimstocreate,approach,andexceedhumanintelligence.



•  Conventional AI(alsocalledNarrowAI,AppliedAI,WeakAIorClassicAI)

(alsocalledStrongAI,Human‐levelAI,RealAI,…)

Conventional A.I.

PROBLEMSPACE

[AdaptedfromDr.Goertzel]Conventional AIvs.AGI


mostpeopleworkonconventional AIConventional AIvs.AGI


mostpeopleworkonconventional AI...maybebecauseworkingonAGIisrisky:

1.  Maybeitis“impossible”.

2.  Itisdifficulttogetfundingforrisky„ambi?ous“researchprojectsthataimtogotowardshuman‐levelintelligence.

3.  Achievinghuman‐levelintelligencemeansthepossibilitytoachieveSuperintelligencethatcouldbeoutofcontrol.




Iconsiderthefollowingques?ons



YES


YES

wehaveasolidproofthatitispossible—namelyourselves.




Howcanarobotlearnlikeachild?


Howcanarobotlearnlikeachild?

bytryingtoreconstructtheprocessesofdevelopmentofthechild'smind.


ChildDevelopment


Childdevelopmentisaprocesseverychildgoesthrough.

Itincludes

physicaldevelopment,personaldevelopment,socialdevelopment,andcogni?vedevelopment.

childdevelopment


Childdevelopmentisaprocesseverychildgoesthrough.

Itincludes

physicaldevelopment,personaldevelopment,socialdevelopment,andcogni?vedevelopment.

childdevelopment


Frombirth,babiesbegintoac?velygatherandprocessinforma?onfromthesurroundingworld,andusethedatatodeveloppercep?onandthinkingskills.

childdevelopment

Cogni?veDevelopment


years

World0

World1

World2

World3

Cogni?veDevelopment

Theprocessofunderstandingtheworldchangesasafunc?onofage&experience

childdevelopment


years

World0

World1

World2

World3

Cogni?veDevelopment

Theprocessofunderstandingtheworldchangesasafunc?onofage&experience

Howdoesthishappen?

childdevelopment


JeanPiaget(1896‐1980)

(Psychologist)

LevVygotsky(1896‐1934)

(Psychologist)

Howchildren’scogni?ondevelop?Twotypicalsourcesofinspira?on

childdevelopment



(Psychologist)

Piagetwasinterestedinhowachildadaptstoitsenvironment.

(hedescribedthisabilityasintelligence)

childdevelopment



(Psychologist)

Piagetwasinterestedinhowachildadaptstoitsenvironment.

(hedescribedthisabilityasintelligence)

Adapta?on

Assimila?on Accomoda?on

childdevelopment


Assimila?on

Addingnewexperiencestotheexis?ngones,andreinforcethem.

childdevelopment

Accommoda?on

Adjus?ngexis?ngexperiencestodealwithnewsitua?ons.

Piaget’sassump?ons


Developmenttakesplaceindis?nctstagesofcogni?vedevelopment.

childdevelopmentPiaget’sassump?ons



Sensorimotor(~Birth to 2 yrs) achievingsimplegoalsinasensorimotorspace





Preopera?onal(~2 to 6 yrs) Represen?ngtheworld,andgobeyondtheconnec?onofsensorimotorinforma?on.






ConcreteOpera?onal(~ 7-12 yrs) reasoninglogicallyaboutevents.






ConcreteOpera?onal(~ 7-12 yrs) reasoninglogicallyaboutevents.

FormalOpera?onal(adolescence to adult)

reasoninginamoreabstract,idealis?c,andlogicalway.



Piagetdidnotemphasizeontheroleofsocialinterac?on.




(Psychologist)

Mostofachild’scogni?veskillsevolvefromsocialinterac?ons

childdevelopment



(Psychologist)

Mostofachild’scogni?veskillsevolvefromsocialinterac?ons

childdevelopment

Developmentoccurswithinthe„zoneofproximaldevelopment“


rangeoftasksthataretoocomplextobemasteredalonebutcanbeaccomplishedwithguidance.

Vygotsky’sassump?on

ZoneofProximalDevelopment

childdevelopment


Childcan ChildcannotZoneofProximal

Development

Vygotsky’sassump?on


Learning/Development

54

childdevelopment


56

Socialengagement

Individu

alcapabili?e

s


Sensorimotor

Preopera?onal

ConcreteOpera?onal

FormalOpera?onal

Dr.Oubba?,March2012

…sohowcanwecreateasystemthatexhibitsautonomousdevelopment?



Weneed„informal“defini?onsforintelligence,anyway…..



Iwillcallthemsignsofintelligence

Weneed„informal“defini?onsforintelligence,anyway…..


•  Learn(understand)fromexperience.

•  Usereasoningtosolveproblems.

•  Adaptquicklyandsuccessfullytonewsitua?ons.

•  Usetheacquiredknowledgetomanipulatetheenvironment.

[R. J. Sternberg, editor. Handbook of Intelligence. Cambridge University Press, 2000.] [U. Neisser, et. al., Intelligence: Knowns and unknowns. American Psychologist, 51(2):77–101, 1996.]

SignsofIntelligence(possiblytheiraremanyother)

Howtocreateautonomousdevelopment?


•  Learn(understand)fromexperience.

•  Usereasoningtosolveproblems.

•  Adaptquicklyandsuccessfullytonewsitua?ons.

•  Usetheacquiredknowledgetomanipulatetheenvironment.

SignsofIntelligence(possiblytheiraremanyother)



Themostcommonideaofthesesignscanbesummarizedas

intelligenceisapropertyofasystemthatisinterac?ngsuccessfullywithaproblem,oranenvironment.

SignsofIntelligence



Tocreateautonomousdevelopmentwehavetodesign1)adevelopmentallearningprocess,2)thatinteractswithrealenvironments.



controlarchitectureThatcanlearnanddevelop

Environment

Thatsupportsthisdevelopment

BrainPhysicalbody

Interac?ngwiththeenvironmentrequiresaphysicalbody

environment



Howimportantisthebodytodevelopment?



Thebodyfacilitateslearningfromthephysicalenvironment

iCub‐robot(EU‐projects)

Theroleofthebody

Explora?on,Experimenta?on,...



RobotKismet(MIT)

Imita?on,Learningfromdemonstra?on,etc.

Thebodyfacilitateslearningfromsocialenvironment

TheroleofthebodyHowtocreateautonomousdevelopment?


Buttheactual„classic“robo?cists(conven?onalAI)neverdisputedtheimportanceofthephysicalbodytotheirresearch.



Somepeoplebelievethatjustconnec?ngacomputertosensorsandplacingitonwheelsissufficienttocreatethe„body“whichisrequiredforintelligence.



Intelligencerequiresabodywhichhastobeabletoembodycogni?veprocesses,thataffectsandbeaffectedbytheenvironment.

Theroleofthebody




Intelligencerequiresabodywhichhastobeabletoembodycogni?veprocesses,thataffectsandbeaffectedbytheenvironment.

Thisistheno?onofEmbodiment

Theroleofthebody




Embodimentvs.Body

Thefundamentaldifferenceliesinthedegreeofinterac?onbetweenthebodyandthebrain(controlprogram).



Let’sconsideranembodiedagent...


Dr.Oubba?,March2012

assumethatthisagentisinterac?ngwithacomplexenvironment


Dr.Oubba?,March2012

Conventional AIseeksanexactrepresenta?onoftheexternalworldinordertoachievethedesiredtask.


Dr.Oubba?,March2012

Theroleofthebody

TheBodyhereisusedjustasanobserver...



Dr.Oubba?,March2012

Theroleofthebody

However,theexternalworldistoocomplex,andrequiresnotjustcon?nualobserva?on,butalsoreasoningandadapta?on.




Dr.Oubba?,March2012

Theroleofthebody


Furthermore,Knowledge(abouttheenvironment)isnoteverything.



Dr.Oubba?,March2012

Theroleofthebody

Onlybylearningfrominterac?ontheagentwillbeabletounderstandtheenvironment,andusethisknowlegetoachieveitsgoals.


Dr.Oubba?,March2012

Howtodesignadevelopmentallearningprocess?



DevelopmentalArchitectureHowtocreateautonomousdevelopment?


Evensimplecreaturesdiscovernewstrategiesandtechniquestogetsurvivalresources,ortofightpredators.



Backtoouragent…


Dr.Oubba?,March2012 83

Whatdrivestheagenttolearn?




Task

…bygivingitatask?



Notonly


DevelopmentalArchitecture

…bygivingitatask?



Task

Notonly



…bygivingitatask?...buthowtoprogramanagentwithoutknowingthe

task!?



Task



Mo?va?on?



Task



Mo?va?on? Thismightwork...



Task

Mo?va?oniswhatgetsustoact.

Psychology



Therearetwomaincategoriesofmo?va?on

Intrinsicmo?va?on

doingsomethingjustbecauseitisenjoyable.(eg.curiosity)

doingsomethingbecauseitleadstoexternalrewards.(e.g.hunger)

Extrinsicmo?va?on



sa?sfac?on+

‐consump?on

(InspiredbythePSItheoryproposedbythepsychologistDietrichDörner)

Amo?veasacontainer

Thefillingstateofthecontainerrepresentsthedegreeofsa?sfac?on.



Gatheringresources+

‐ energyconsump?on

Extrinsicmo?va?on

Hunger



ExploringNovelty+

‐ Predictabilityoftheenviroment

Intrinsicmo?va?on

Curiosity



Hun

ger

Curiosity


BodyMind



Curiosity


Hun

ger

Mo?veswilldrivetheagenttolearnasmuchaspossibleabouttheenvironment.



Curiosity


Hun

ger

Theneedsofthebodyandthoseofthemindmaycompeteincertaincircumstances.



Curiosity


Hun

ger

Associa?ngimportancetoonemo?veovertheotherisconstantlyre‐evaluatedbyexperience.



Curiosity


Hun

ger

Learninghowtobalancebetweenmo?veswillprovidetheagenttheabilitytosolveafacedprobleminseveralways.

Thus,moreflexibility(authority)inhandlingsitua?ons.



howtoar?culatepreferencesamongmo?ves?



howtoar?culatepreferencesamongmo?ves?


Weconsiderthisasamul?objec?veproblem.




Weproposeahierarchicaladap?ve‐cri?cdesign(ACD)thatallowsanembodiedagenttoadaptitsbehaviorinpresenceofseveralmo?ves.

Howtosolvemul?objec?veproblems?



Theproposedarchitectureismainlybasedonreservoircompu?ng.


Reservoir

Onlytheoutputconnec?onsaretobetrained






Cri?c1

Cri?c2

Cri?cN

Ahigherlevelreservoirmayserveasacri?ctolearnseparatevaluefunc?onsforeachmo?vesimulatneously.

High‐levelACD





Cri?c1

Cri?c2

Cri?cN

1.  Thiswouldhaveanadvantagetolearnfromseparatestatespacesspeeduplearning.

High‐levelACD





Cri?c1

Cri?c2

Cri?cN

2.Theabilitytolearnfromheterogenousdatawiththesameweights(i.e.memoryisintherecurrentac?va?on,notonlyinthesynap?cweights)thereservoirwouldbeabletocopewiththeconflic?ngdemandsimposedbythemo?ves.

High‐levelACD





Low‐levelACDs

Atalowlevelseveralreservoirsmayhavetheroletolearnop?malbehaviorfromthecon?nuousanduncertainstatespacetosa?sfytheselectedmo?ves.

γ





Low‐levelACDsγ

Thus,ouragentrunsseveralACDsinparallelthatlearnfromdifferentstatespaces.




HierarchicalACDwithreservoircompu?ngtosolvemul?objec?veproblems



M.Oubba?,J.Uhlemann,andG.Palm(2012),Adap7velearningincon7nuousenvironmentusingactor‐cri7cdesignandecho‐statenetworks.FROMANIMALSTOANIMATS(SAB12).


ACDwithreservoircompu?ng:preliminaryresults



Currently,weareworkingonthisdevelopmentallearningsystem




Weimaginehowtheagentexplorestheenvironmentinordertosa?sfyistinborndrives.Weimaginehowitcon?nuallywantstomastertheenvironment,andhowitdoeasistbesttosuccessfullyfacechallenges.Weimaginehowthisagentasksforsocialinterac?ontolearnfromhumans,whatitcannotlearnindependently...Thisiswhatwewanttorealize.










OncethefirstAGIisbuilt,itisdifficulttopredictwhatfurtherdegreesofintelligencecanbebuilt.





Ifthesegoalsarenotorientedtoourneeds,itwouldleadtounpredictable,andprobablydangerous,consequencesforoursurvival.







Danke


Science

Mohamed Oubbati Embodied Intelligence