Projectivities and continuous action spacesfeat. Lena

Reinaldo Uribe M

Sept 30, 2013

w − l space as projective transformation from policyvalue/cost space

1

D

−D

Policy V

alu

e

Episode Length

w − l space as projective transformation from policyvalue/cost space

1

D

−D

Policy V

alu

e

Episode LengthD

−D

w

l

w − l space as projective transformation from policyvalue/cost space

1

D

−D

Policy V

alu

e

Episode Length

w − l space as projective transformation from policyvalue/cost space

1

D

−D

Policy V

alu

e

Episode LengthD

−D

w

l

Extension to continuous spacesSample task: two states, continuous actions

s1a1 ∈ [0, 1]r1 = 1 + (a1 − 0.5)2

c1 = 1 + a1

s2

a2 ∈ [0, 1]r2 = 1 + a2c2 = 1 + (a2 − 0.5)2

Extension to continuous spacesSample task: two states, continuous actions

s1a1 ∈ [0, 1]r1 = 1 + (a1 − 0.5)2

c1 = 1 + a1

s2a2 ∈ [0, 1]r2 = 1 + a2c2 = 1 + (a2 − 0.5)2

Extension to continuous spacesSample task: two states, continuous actions

Policy Space (Actions)

0

a2

1

0 a1 1

Extension to continuous spacesSample task: two states, continuous actions

Policy Values and Costs

Policy v

alu

e

Policy cost

4

4

Extension to continuous spacesSample task: two states, continuous actions

Policy Manifold in w − l

l

w

D/2

D/2