Exploring Robotic Minds: Emergentist Account for Non-Reductive Consciousness Jun … · 2020-01-06 · Exploring Robotic Minds: Emergentist Account for Non-Reductive Consciousness

Exploring Robotic Minds: Emergentist Account for Non-Reductive Consciousness

Jun TaniKAIST

Emergence in My Research History

• Original research interest– How can compositionality be developed through iterative

learning of sensory-motor experiences?• Synthetic neurorobotics experiments (1993~).

Phenomenarelatedtophenomenology

ThankstolateJ.Goguen andlateF.Varela

Homunculus and Symbol Grounding Problems

Categorizer

Sensory-motor patternsin the physical world

FSMHomunculus

Physical World

Sensory-motor

Fast

Slow

Synthesis of Embodied NeuroDynamic Sys.

Top-Down intentionality

Bottom-up perceptual reality

• Upward and downward causation.– Development of spatio-temporal

hierarchy and compositionality.• Correspondences to some of

phenomenological questions.– Consciousness, free will

Emergence!!

Phenomenological Questions

WilliamJames(priortophenomenology…)

Consciousnessappearsonlyintermittentlyinthestream…

Whatistheessentialmechanismforautonomouslyswitchingbetweenconsciousnessandunconsciousness?

Questforstructureofconsciousness!!

Merleau-Ponty

Inseparableentityofthesubjectandtheobjectastheyaredeeplyentangledeachother.

Then,whydowefeelthesubjectivityofourselves?Self-Consciousness?

Free Will

• The essential question concerning free will is that if we suppose that everything proceeds deterministically by following the laws of physics, what is left that enables our will to be free? (Thomas Hobbes)

• On the other hand, if free will is built on randomness and probability, the origin of free will turns out to be just noise…

Free-Decision Task by Benjamin Libet

Readiness PotentialDelayed!!

How is free will originated?How can it be consciously aware?

Libet 550ms before (Single button push)

Soon et al 5000ms before(Left/right button push)

Consciously aware with delay

Some Backbone Ideas• Predictive coding.

– Internal model accounting for own past as well as future.

– Causality from intention to perception.– Inference of intention from perception.

• Development of spatio-temporal hierarchy. – Downward causation.

• Spatial-temporal constraints → functional hierarchy– Objectifying experience.

• Segmentation and chunking

Predictive-Coding for Learning, Generation and Recognition

(Rao & Ballard, 1999; Tani, 1999, 2003; Friston, 2005; Clark, 2015)

All three processes of learning, generation and recognition of patterns can be achieved by a single principle of prediction error minimization.

Recurrent Neural Network (RNN) (Jordan, 1986; Elman, 1991)

• RNN as a neurodynamic model.– Nonlinear dynamic system

• It can learn to extract temporal structures from exemplar sequential data.– Self-organization of latent dynamic structures

RNN Models

Internal state(t)

Outputs (t+1)

Inputs (t)

Recurrent NN

Internal state(t+1)

Fully-Connected Recurrent NN

Inputs

Outputs

NonlinearDynamicalSystems

𝑥"#$ = f(𝑥", 𝑝)

Differenceequation

Differentialequation

�̇� = 𝑓(𝑥, 𝑝)

DifferentClassesofAttractors

• Parameterbifurcation• Initialsensitivity

Predictive Coding by RNNPB (Tani, 2003)

PB

ProprioceptionVision Play1: Rolling ball left and right.(Supervised teaching.)

ｔ

PredictionError

Intention: Play1

RNN

ｔ

TargetPlay1 Vision

Proprioception

PB

ProprioceptionVision Play2: Lift up ball and drop it.

ｔ

Prediction

TargetPlay2

Modulateweights and PB

Error

Intention: Play2

ｔ

LearningOptimizing synaptic weights shared by all patterns as well as corresponding PB for each pattern.

PB plays a role of bifurcation parameter to switch action pattern.

Recognition of Change in Perceptual Patterns by Error Regression

PB

ProprioceptionVision

ｔ

Prediction

Perception

Error

Play1

ｔ

Modulate PB

Play2

The situation change is recognized!!

Now Predict future

No weight change!!

Actual Robot Test after Learning

(Ito, Noda, & Tani, 2005)

Continuous perceptual flow is segmented into a sequence of primitives by parameter bifurcation

which is caused by autonomous shift of PB.

Rolling Holding and dropping

PB

Arm joint angles

SensationBall position

PredictionBall position

Phenomenological Analysis (Tani, J. of Consci. Studies, 1998, 2004)

• Continuous flow of experience is segmented into episodic sequence by using prediction error.

• This is the process for objectifying one’s own experience.• This prediction error causes consciousness.

– Analogous to Heidegger’s analysis on “carpenter and hammer”.• If there were a higher level, it could manipulate such

episodes compositionally.– The robot could become narrative on own experience.

Therefore, let’s explore development of hierarchy.

Possible Hierarchies in Brains

PFC

SMA PMC

M1

Pons

Spinal cord

Motor Hierarchy(Felleman & Van Essen)

Vision Hierarchy

Multiple Timescale RNN (MTRNN) (Yamashita & Tani, 2008)

Intended image

Perceptual reality

.Intention space

Multiple Timescale RNN (MTRNN) (Yamashita & Tani, 2008)

Clusters of intentions developed

. ........

. ........

. . .. . .. ..

Intended image

Perceptual reality

.

Intention space

Generation

slow

fast

Intention

Generation

slow

fast

Intention

Recognition

slow

fast

Intention

Perceptual reality

ERROR!!

Recognizing the situation change by modifying intention.

PredictionReconstruction

Body&Environment

Closed-loopformentalsimulation

MTRNNarchitectureusedinroboticsexperiments

Middlenet

Highernet

In/outnet

Byvaryingintention

SMA

V1

PFC-SMA-IPL Interaction

IntermediateFast

Fast

PFC

1sec

PFC cell (Hoshi et al, 2000)

1sec

M1 cell

Rostral-Caudal Gradient of Time-Scales(Kiebel & Friston, 2008; Badle, 2008; Hasson et al., 2008; Murray et al., 2014)

IPL

Slow

Vision Prediction

Posture Prediction

Error!!

Intention M1

Three Different Goal-Directed Tasks Are Simultaneously Trained

Task 1

Task 2

Task 3

Interactive Tutoring Video

(Nishimoto & Tani 2009)

Developmental Interactive Tutoring

After the 1st tutoring

After the 2nd tutoring

After the 3rd tutoring (One more)

Seq 0: Teach

Step

0 50 100 150 200 250 300 350

Mot

or V

alue

0.0

0.2

0.4

0.6

0.8

1.0

Seq 0: closed loop

Step

0 50 100 150 200 250 300 350

Mot

or V

alue

0.0

0.2

0.4

0.6

0.8

1.0

(a) S

essi

on 1

(b) S

essi

on 2

(c) S

essi

on 3

Seq 0_a: Slow Context

Step

0 50 100 150 200 250 300 350

PC V

alue

-1.0

-0.5

0.0

0.5

1.0

1.5


Step

0 50 100 150 200 250 300 350

PC V

alue

-1.0

-0.5

0.0

0.5

1.0

1.5


Step

0 50 100 150 200 250 300

PC V

alue

-1.0

-0.5

0.0

0.5

1.0

1.5


Step

0 20 40 60 80 100 120

PC V

alue

-1.0

-0.5

0.0

0.5

1.0

1.5


Step

0 50 100 150 200 250 300

PC V

alue

-1.0

-0.5

0.0

0.5

1.0

1.5


Step

0 50 100 150 200 250 300

PC V

alue

-1.0

-0.5

0.0

0.5

1.0

1.5

Seq 0: robot generation

Step

0 20 40 60 80 100 120

Mot

or V

alue

0.0

0.2

0.4

0.6

0.8

1.0

Seq 0: Teach

Step

0 50 100 150 200 250 300 350

Mot

or V

alue

0.0

0.2

0.4

0.6

0.8

1.0

Prop1Prop2Prop3Prop4Vision1Vision2

Seq 0: Teach

Step

0 50 100 150 200 250 300 350M

otor

Val

ue0.0

0.2

0.4

0.6

0.8

1.0

Seq 0: Teach

Step

0 50 100 150 200 250 300

Mot

or V

alue

0.0

0.2

0.4

0.6

0.8

1.0

Seq 0: Teach

Step

0 50 100 150 200 250 300

Mot

or V

alue

0.0

0.2

0.4

0.6

0.8

1.0

Seq 0: Teach

Step

0 50 100 150 200 250 300

Mot

or V

alue

0.0

0.2

0.4

0.6

0.8

1.0

Seq 0: Teach

Step

0 50 100 150 200 250 300

Mot

or V

alue

0.0

0.2

0.4

0.6

0.8

1.0

Teaching Mental Imagery Actual Action

Session1

Session2

Session3

Developmental Learning Process

Observed Characteristics in Development

• A set of primitives are developed first and then combinations of them are developed later.

• Mental imagery is developed first followed by physical actions.– Analogy to song-bird learning.

• Codevelopmental processes between tutors and robots.

0 50 100 150 200 250 300

PC V

alue

-1.0

-0.5

0.0

0.5

1.0

1.5

Step

0 50 100 150 200 250 300

PC V

alue

-2

-1

0

1

2

0 50 100 150 200 250 300 350

Mot

or V

alue

0.0

0.2

0.4

0.6

0.8

1.0

0 50 100 150 200 250 300 350

Mot

or V

alue

0.0

0.2

0.4

0.6

0.8

1.0

0 50 100 150 200 250 300 350

PC V

alue

-2

-1

0

1

2

Step

0 50 100 150 200 250 300 350

PC V

alue

-2

-1

0

1

2

PC1PC2PC3PC4

0 50 100 150 200 250 300

Mot

or V

alue

0.0

0.2

0.4

0.6

0.8

1.0 Prop1Prop2Prop3Prop4Vision1Vision2

0 50 100 150 200 250 300

Mot

or V

alue

0.0

0.2

0.4

0.6

0.8

1.0 Prop1Prop2Prop3Prop4Vision1Vision2

0 50 100 150 200 250 300

PC V

alue

-2

-1

0

1

2

Step

0 50 100 150 200 250 300

PC V

alue

-2

-1

0

1

2

Teac

hG

ener

atio

nSl

ow c

onte

xtFa

st c

onte

xt

Teac

hG

ener

atio

nSl

ow c

onte

xtFa

st c

onte

xt

0 50 100 150 200 250 300

Mot

or V

alue

0.0

0.2

0.4

0.6

0.8

1.0

0 50 100 150 200 250 300

Mot

or V

alue

0.0

0.2

0.4

0.6

0.8

1.0

Teac

hG

ener

atio

nSl

ow c

onte

xtFa

st c

onte

xt

(a) Task 1 (b) Task 2 (c) Task 3

PC1PC2PC3PC4

All 3 task sequences at the end of the final tutoring session

“Kinetic Melody” by Luria

More Complex Task ExampleTetsuya Ogata et al., (2017)

video

Next Question

How can probabilistic structure be learned?

Origin of Free Will

Learning to imitate probabilistic transitions of actions by MTRNN

50%

50%

50%

50%

50%

50%

Center RightLeft

(Namikawa et al., PLoS Compt 2011)

Learning to imitate probabilistic transition of actions.

Slow

FastMedium

MTRNNVideo

Test Generation After Learning

Video

50%50%

50%

50%

50%

50%

Center RightLeft

Probability Changed

What sorts of internal mechanisms were developed?

Neuro-Dynamics Developed

Time steps

High

Middle

Motor

Chaos!!Positive Lyapunov exponent

Spontaneous Selection of Actions by Cortical Chaos

Environment

Cortical Chaos in the PFC

Motor act

Visual perception

These results suggest:

• The probabilistic structures latent in event transitions were successfully reconstructed by developing the corresponding chaotic dynamic structures in the higher level.

• The cortical chaos which can select next action spontaneously can be considered as the source of free will.

Deterministic Dynamics or Stochastic One?

Fluctuated spatio-temporal patterns

What is between them?(Murata & Tani, 2015)

Deterministic chaotic Stochastic neurodynamicsFree energy minimization by Friston or using Variational

Bayes.Dichotomy

A Question Remained• Although actions are generated spontaneously in

sequences by chaos, how can own free decisions of selecting those actions be consciously aware?

Delayed awareness of own intention?

The answer will be different by assuming deterministic dynamics and stochastic one!!

(Choi&Tani,2016)

MultipleSpatioTemporal ScalesRNN(MSTRNN)GenerationandRecognitionofVideoby

PredictiveCoding

48X54X1 Layer 140X40X6τ: 2

Layer 2Max pool20X20X6

Layer 314X14X50τ: 5



Layer 6 SoftMax1X1XN

Projectionofvisualimage Intention

HigherLower

Fastdynamics Slowdynamics

Localconnectivity Globalconnectivity

CNN-Like

Error

Modify

LearningandGenerationofWholeBodyMovementPatterns

• 6primitivepatternsdemonstratedby5subjectswererecorded.

• Test-1:all6primitivepatternsby5subjectswerelearnedasindependentpatterns.

• Test-2:Twodifferentconcatenatedsequencesusingtheseprimitivesweretrained.– Forwardseq:A->B->C->D->E– Reverseseq:E->D->C->B->A

• Test-3:Imitativesynchronizationwithtargetpatterns.– Recognitionbytheerrorregression

Test-1:Learning6primitivesby6subjects

6primitivesvideo

Higherleveltrajectories Lowerleveltrajectories

Test-2:LearningConcatenatedSequencesForwardconcatenatedsequencevideo

High

Low

(Choi&Tani,preparation)

RecognitionbyErrorRegressionImitativeSynchronizationwithTarget

48X54X1

Layer 140X40X6τ: 2





Layer 6 SoftMax1X1XN

Projectionofvisualimage Intention

HigherLower

Error!! Modify!!

ImitatetargetTargetinput

(Choi&Tani,2016)

ImitationofTargetVisualInputsbyBottom-UpErrorRegression

VIDEO

now futurepast

Rewritethepast.Postdiction!!

Tani-ArmRobo-2003-video

(Murataetal.,2015)

VIDEO

Postdiction Prediction

(Tani,NeuralNetworks,2003)

Look Ahead Prediction of Future and Postdiction of Past

The present is “born” via dynamic interplay between Zukunft—looking ahead future for possibility andGewesenheit—regressing past for reflection. (Martin Heidegger)

Postdiction(Shimojo)

Prediction

Image

Intention

Modification byregression Look ahead

Now

ExperienceError!!

VIDEO

(Tani, 2003)

ExplorationofsubjectiveexperiencePuttinghumaninroboticexperimentloop

VIDEO

Novel movement patterns can emerge via spontaneous mutual interactions among different brain areas, body, and environment.

1. A set of movement patterns is pre-trained.2. Attempt to tutor new patterns by physically guiding the

robot while performing the error regression.

Co-Development of Novel Actions

Mapping from Intention Space to Patterns

Intent 1 1.0

Inte

nt 2

1.0

0.0

Learned Cycle Learned Cycle

mot

or

mot

or

LearnedReach

mot

or

mot

or

mot

or Motor1

Motor2

Motor3Motor4

Novel

Novel

Structural Relationship between Free Will and Consciousness

EnvironmentHuman

Cortical Chaos in the PFC

Motor act

Error!!Visual perception

Unconscious free will

Intention modified

Intention for generating an action can be aware only via postdiction!!

Circular causality!!

Delayed conscious awareness

Delayed Awareness of Own Free Wills

• When the higher level suddenly attempts to drive the lower peripheral parts by free wills, the lower level may not be ready for it.

• The gap between these two levels may generate prediction error.• The very efforts to minimize the error causes consciousness with

delay.

Self-Consciousness (Tani, OUP book, p172, p249-250, 2016)

Subjectivity

• Self-Consciousness is an experience of “first person awareness” of own subjectivity.

• The subjectivity is exemplified by the top-down pathway of anticipating the actional outcome.

• The objectivity is exemplified by the bottom-up recognition of the perceptual reality.

• Then, self-consciousness arises through sensation of the subjectivity being differentiated in the process of minimizing a gap generated between these two.

Objectivity

Self-Consciousness as sense of differentiated

Modified!!

Structure self-organized via accumulated experiences

My new book has been just published from Oxford

Univ. Press.

Possible material for parallel group discussion


perceptual image

internal flow

error

Intention

Goal

Modify!

FutureNow

prediction

Min!Modify!

Modify!


Perceptual image error

Intention

Goal

Modify!!

Future

experience flow

errorerror

Past

predictionPostdiction (Shimojo)

internal flow Modify!

Now

Min!Min!Min!

The present is “born” via dynamic interplay between Zukunft—looking ahead future for possibility andGewesenheit—regressing past for reflection. (Martin Heidegger)

Now

Goal

Intention

error


Goal

Intention

Goal

Intention

Goal

Intention


Goal

IntentionIntention

GoalGoal

Intention

Documents

Exploring Robotic Minds: Emergentist Account for Non-Reductive Consciousness Jun … · 2020-01-06 · Exploring Robotic Minds: Emergentist Account for Non-Reductive Consciousness