Fast cooling for a system of stochastic oscillators

Yongxin Chen, Tryphon Georgiou and Michele Pavon

We study feedback control of a system of coupled nonlinear stochastic oscillatorsin a force field. We first consider the problem of asymptotically driving the systemto a desired steady state corresponding to lower thermal noise. Among the feedbackcontrols achieving the desired asymptotic transfer, we find that the most efficientone from an energy point of view is characterized by time-reversibility. We alsoextend the theory of the Schrödinger bridges to this model thereby steering thesystem in finite time and with minimum effort to a target steady-state distribution.The system can then be maintained in this state through the optimal steady-statefeedback control. The solution, in the finite-horizon case, involves a space-timeharmonic function ϕ and − logϕ plays the role of an artificial, time-varying potentialin which the desired evolution occurs. This framework appears extremely general andflexible and can be viewed as a considerable generalization of existing active controlstrategies such as macromolecular cooling. In the case of a quadratic potential, theresults assume a form particularly attractive from the algorithmic viewpoint as theoptimal control can be computed via deterministic matricial differential equations.An example involving inertial particles illustrates both transient and steady stateoptimal feedback control.

Keywords: Stochastic oscillator, steady-state, cooling, Schrödinger bridge, stochastic control,

reversibility.

I. INTRODUCTION

Cold damping feedback is employed to reduce the effect of thermal noise on the motion ofan oscillator by applying a frictional force, which is historically one of the very first feedbackcontrol actions ever analyzed1. It was first implemented in the fifties on electrometers [34].Since then, it has been successfully employed in a variety of areas such as atomic forcemicroscopy (AFM) [32], polymer dynamics [2, 11] and nano to meter-sized resonators, see[13, 33, 40, 45]. These new applications also pose new physics questions as the system isdriven to a non equilibrium steady state [1, 28, 38, 41].

Another important issue is the following: As one can extract a net useful work from thesedevices, therefore sometimes called Brownian motors [42], the question of their efficiencyarises. In some specific examples, it has been argued that the latter can be studied viastochastic control [14]. Nevertheless, it may be fair to state that, in spite of the flourishingof these applications and cutting edge developments, the interest in these problems in thecontrol engineering community has been shallow to say the least.

The fact is, as we argue below, that these problems may be cast in the framework of theclassical theory of Schrödinger bridges for diffusion processes [49] where the time-intervalis finite or infinite. The connection between finite-horizon Schrödinger bridges and the socalled “logarithmic transformation” of stochastic control of Fleming, Holland, Mitter et al.,

1 “In one class of regulators of machinery, which we may call moderators, the resistance is increased by

a quantity depending on the velocity”, James Clerk Maxwell, On Governors, Proceedings of the Royal

Society, no. 100 (1868), 270-282.

arX

iv:1

411.

1323

v1 [

mat

h-ph

] 5

Nov

201

4

2

see e.g. [17], has been known for some time, see e.g. [8, 9, 37]. Excepting some special cases[14, 15], however, the optimal control is not provided by the theory in an implementableform and a wide gap persists between the simple constant linear feedback controls used inthe laboratory and the Schrödinger bridge theory which requires the solution of two partialdifferential equations nonlinearly coupled through their boundary values [49]. Only veryrecently some progress has been made in deriving implementable forms of the optimal controlfor general linear stochastic systems [4–7]; for Markov chains and Kraus maps of statisticalquantum mechanics implementable solutions of the Schrödinger systems have been recentlypresented in [21].

In this paper, continuing the work of [4–7], we study a general system of nonlinearstochastic oscillators. For this general model, we prove optimality of certain feedback con-trols which are given in an explicit or computable form. We highlight the relevance of optimalcontrols on examples of stochastic oscillators. To this end, we begin by discussing two basicparadigms, an electromechanical system and polymer dynamics, and highlighing similaritiesin the corresponding models (Section II). In Section III we introduce the system of non-linear stochastic oscillators and establish a fluctuation-dissipation relation in steady-statescorresponding to cooling (Proposition 2). In Section IV, we characterize the most efficientfeedback law which achieves the desired cooling and relate optimality to reversibility of thecontrolled evolution. In Section V, we show how the desired cooling can be accomplished infinite time using a suitable generalization of the theory of Schrödinger bridges. The latterresults are then specialized in the following section to the case of a quadratic potential wherethe equations become linear and the results of [5] lead to implementable optimal controls.Optimal transient and steady state feedback controls are illustrated in one example involvinginertial particles in Section VII.

II. BASIC EXAMPLES

We begin with two examples of physical systems where it is desirable to regulate thestate distribution using suitable control input and, thereby, bringing those to a lower effectivetemperature; their mathematical models are quite similar. Besides the two examples outlinedbelow, velocity dependent feedback control (VFC) has been implemented to reduce thermalnoise of a cantilever in atomic force microscopy (AFM) [32] and in dynamic force microscopy[46].

A. Feedback Cooling of the Normal Modes of a Massive Electromechanical System

To observe quantum behaviour and investigate decoherence in macroscopic mechanicalresonators requires cooling to ultralow temperatures, so that the thermal energy becomescomparable to the quantum energy. In [48], cooling of the ton-scale resonant bar gravi-tational wave detector AURIGA is described. The bar resonator motion is detected by acapacitive transducer followed by a dc-SQUID amplifier. The detector is modelled by threecoupled low-loss resonators: two mechanical ones (the bar and the plate of the capacitivetransducer) and an LC electrical circuit. AURIGA employes a cooling feedback strategywhich is equivalent to a frictional force on the oscillators.

In a suitable approximation, each oscillator is described by the Nyquist-Johnson model

3

[1, Eq. 3]

LdIs(t)

dt+ Is(t) [R +Rd] +

qs(t)

C=√

2kBT0Rν(t), Is(t) =dqs(t)

dt, (1)

where ν(t) is a Gaussian white noise process, i.e., the formal derivative of a Wiener processw. Here, Rd expresses the viscous damping on the oscillator due to the feedback loop. Themodel can be written in the form of a stochastic differential equation (SDE)

dqs(t) = Is(t)dt (2a)

dIs(t) = −R +RdL

Is(t)dt−1

LCqs(t)dt+ u(qs, Is)dt+

√2kBT0Rdw(t) (2b)

where we introduce an external control input source u (in units of current/time) and expressthe stochastic input directly in terms of the Wiener process (i.e., by replacing the formalν(t)dt by dw(t)).

B. Regulating polymer dynamics

In polymer dynamics [11], the macromolecule is described by a Hamiltonian

H(x, p) =1

2〈p,M−1p〉+ V (x),

where M stands for the direct sum

M = M1 ⊕ · · · ⊕MN , Mk = mkI3, k = 1, . . . , N.

Here x and p are 3N -dimensional vectors with entries the 3-dimensional positions and mo-menta of the N hard building blocks of the macromolecule, and V (x) is the internal po-tential of macromolecule. We use v = M−1p to denote the corresponding velocities. Ran-dom collisions between solvent water molecules and building blocks of macromolecule aremodeled by the formal derivative of a Wiener process, namely Gaussian white noise. The6N -dimensional stochastic process (x′, p′)′ of positions and momenta obeys the equation

dx = Hp(x, p)dt, (3a)

dp = −Hx(x, p)dt+ fdt+ u(x, p)dt+ Γdw(t), (3b)

where Hx denotes the (column) vector of partial derivatives with respect to the entries of x,and similarly for Hp, f denotes a frictional force, u a position-momentum dependent control,and w(t) a vector-valued Wiener process. Note that system (3) is quite similar to (2),although, in this case, depending on the potential and the frictional force the model may benonlinear. For simplicity, we let f = −γv, u = −κp, with scalar γ > 0, κ > 0. The controlhere acts like a frictional force on the macromolecule. This control drives the system to thenon-equilibrium steady state

ρ̄(x, p) = C exp

[−H(x, p)

kTeff

]where the effective temperature Teff = [γ/(γ + κ)]T0 is lower than the actual thermostattemperature T0.

4

III. A SYSTEM OF STOCHASTIC OSCILLATORS

Consider a mechanical system in a force field coupled to a heat bath. More explicitly,consider, as in [24], the following generalization of the Ornstein-Uhlenbeck model of physicalBrownian motion [35]:

dx(t) = v(t) dt, x(t0) = x0 a.s. (4a)

Mdv(t) = −Bv(t) dt−∇V (x(t))dt+ ΣdW (t), v(t0) = v0 a.s.. (4b)

Here x(t) and v(t) take values in Rn = R3N . The potential V ∈ C1 (i.e. continuouslydifferentiable), is bounded below and tends to infinity for |x| → ∞. The noise processW (·) is a standard n-dimensional Wiener process independent of the pair (x0, v0). M , Band Σ are n × n matrices with M symmetric, positive definite. Matrices B and Σ neednot be symmetric. The phase space probability density ρt(x, v) represents the state of thethermodynamical system at time t. Notice that we allow for both potential and dissipativeinteraction among the particles, with velocity coupling and with dissipation described by alinear law. Models in Section V correspond to the situation where M , B and Σ are diagonalmatrices. Model (4), however, may also describe a system of N stochastic oscillators withvelocity coupling trough first neighbour interaction. Different spatial arrangements such asa closed ring and a linear array may be accommodated in this frame, see [24, Section 6].Consider, for instance, a ring of N -oscillators with x0 = xN described by the scalar equations

dxk = vkdt, (5a)

mkdvk =

(−γvk−1 − βvk − γvk+1 −

∂V (x)

∂xk

)dt+ σkdW, (5b)

where σk ∈ R1×n. Then, (5) can be put in the form (4) defining

M = diag(m1, . . . ,mN), B =

β γ 0 γγ β γ 0 00 γ β γ 0· · · · · ·· · · · · ·γ · · · γ β

, Σ =σ1···σN

.As is well known, further applications of this basic model of dissipative processes is found,besides thermodynamics, in nonlinear circuits with noisy resistors [47], in chemical physics,in biology, etc.

According to the Gibbsian postulate of classical statistical mechanics, the equilibriumstate of a system in contact with a heat bath at constant absolute temperature T and withHamiltonian function H is necessarily given by the Maxwell-Boltzmann distribution law

ρMB = Z−1 exp

[− HkT

](6)

where Z is the partition function (we assume here and throughout the paper that V is suchthat exp

[− HkT

]is integrable on Rn×Rn). Correspondingly, the main mathematical concept

relevant to a stochastic characterization of equilibrium is that of an invariant probabilitymeasure. For our model (4), the Hamiltonian function is of course given by

H(x, v) =1

2〈v,Mv〉+ V (x),

5

where 〈·, ·〉 denotes the Euclidean scalar product in Rn. The partition function is here justa normalization constant. It is therefore important to find conditions for a given system toobey the Maxwell-Boltzmann distribution in equilibrium. In [24], the following generaliza-tion of the Einstein fluctuation-dissipation relation was established.

Proposition 1 The Maxwell-Boltzmann distribution with density (6) is an invariant mea-sure for (4) if and only if

ΣΣ′ = kT (B +B′). (7)

Condition (7) will be assumed throughout the paper to ensure that the uncontrolled evo-lution of the system of stochastic oscillators tends to the equilibrium state. Proposition 1ensures the Maxwell-Boltzmann character of the invariant measure if it exists, but it doesnot guarantee its existence. In the case of B satisfying (7), the connection between existenceof an invariant measure and complete controllability of the associated deterministic system(

ẋMv̇

)=

(v

−∇V (x)− (B −B′)v

)+

(0

(B +B′)

)u (8)

has been investigated in [3, 24]. In the case of a quadratic potential, in Müller’s terminologyas quoted in [50], this means that damping in the corresponding deterministic system ispervasive. Besides (7), we shall then assume in the rest of the paper that the system (8) iscompletely controllable.

Consider again the system of stochastic oscillators (4) and let ρ̄, given by

ρ̄(x, v) = C exp

[−H(x, v)

kTeff

], (9)

be a desired thermodynamical state with Teff < T , T being the temperature of the thermo-stat. Consider the controlled evolution

dx(t) = v(t) dt, x(t0) = x0 a.s. (10a)

Mdv(t) = −Bv(t) dt− Uv(t)dt−∇V (x(t))dt+ ΣdW (t), v(t0) = v0 a.s., (10b)

where B and Σ satisfy (7) and U is a constant n × n matrix. We have the followingfluctuation-dissipation relation.

Proposition 2 Under condition (7), the probability density ρ̄(x, v) in (9) is invariant forthe controlled dynamics (10) if and only if the following relation holds

T − TeffT

ΣΣ′ = kTeff [U + U′] . (11)

Proof. The proof is given in Appendix A. 2

If V is smooth and U satisfies condition (11), by hypoellipticity, ρ̄ is the unique invariantdensity for (10) [25, 29, 30]. Moreover, from any initial condition ρ0(x, v), the density ρt(x, v)of (10) converges exponentially to ρ̄ [30]. Thus, such a control −Uv achieves asymptoticallythe desired cooling. Finally, observe that U satisfying (11) always exist. For instance, if werequire U to be symmetric, it becomes unique and it is explicitly given by

Usym =1

2

[T − TeffkTTeff

ΣΣ′]. (12)

6

IV. OPTIMAL STEERING TO THE STEADY STATE AND REVERSIBILITY

It is interesting to investigate which of the feedback laws −Uv which satisfy (11) andtherefore drive the system (10) to the desired steady state ρ̄, does it more efficiently. Fol-lowing [7, Section II-B], we consider therefore the problem of minimizing the expected inputpower (energy rate)

Jp(u) = E {u′u} (13)over the set of admissible controls

Up ={u(t) = −M−1Uv(t) | U satisfies (11)

}. (14)

Observe that, under the distribution ρ̄dxdv, x and v are independent. Moreover, E{vv′} =kTeffM

−1. Hence

E {u′u} = E{v′U ′M−2Uv

}= kTeff trace

[M−1U ′M−2U

].

Let Π be a symmetric matrix and consider the Lagrangian function

L(U,Π) = kTeff trace[M−1U ′M−2U

]+ trace

(Π(kTeff [U + U

′]− T − TeffT

ΣΣ′)

)(15)

which is a simple quadratic form in the unknown U . Taking variations of U , we get

δL(U,Π; δU) = kTeff trace((M−1δU ′M−2U +M−1U ′M−2δU + ΠδU + ΠδU ′

)).

Setting δL(U,Π; δU) = 0 for all variations, which is a sufficient condition for optimality, weget M−2UM−1 = Π which implies that M−1U equals the symmetric matrix MΠM . Thuswe get the symmetry condition

U∗M−1 = M−1(U∗)′. (16)

We wish to investigate next the relation between the optimality condition (16) and reversibil-ity. It has been shown by Nelson [35, 36], see also [23], that the Markov diffusion process(10) admits, under rather mild conditions, a reverse-time stochastic differential given by

dx(t) = v(t) dt,

Mdv(t) = −Bv(t) dt− Uv(t)dt−∇V (x(t))dt− ΣΣ′M−1∇v log ρt(x(t), v(t)) + ΣdW−(t).

Here dt > 0, ρt is the probability density of the process in phase space and W− is a standard

Wiener process whose past {W−(s); 0 ≤ s ≤ t} is independent of(x(t)v(t)

)for all t ≥ 0.

Consider first the uncontrolled situation (U = 0) in equilibrium, namely with the Maxwell-Boltzmann distribution (6). Then, the process is reversible if the forward and reverse-timetransition mechanisms coincide. Since the diffusion coefficient is the same, we only need tocheck the relation between the forward and the backward drift fields. From (4), the forwarddrift field of v is

bv+(x, v) = −M−1Bv −M−1∇V (x).By (7), we also have

bv−(x, v) = −M−1Bv −M−1∇V (x)−M−1ΣΣ′M−1∇v log ρMB(x, v)

= −M−1Bv −M−1∇V (x) +M−1 ΣΣ′

kTM−1Mv = −M−1Bv +M−1(B +B′)v −M−1∇V (x)

= M−1B′v −M−1∇V (x).

7

If B is symmetric, we getbv−(x, v) = b

v+(x,−v)

which is equivalent to reversibility. We proceed to study reversibility in the steady state ρ̄under the condition that B, besides satisfying (7), is symmetric and U satisfies (11). Thenew forward and backward drift fields of v are given by

bv+(x, v) = −M−1(B + U)v −M−1∇V (x) (17a)

and by

bv−(x, v) = −M−1(B + U)v −M−1∇V (x)−M−1ΣΣ′M−1∇v log ρ̄(x, v)

= −M−1(B + U)v −M−1∇V (x) +M−1 ΣΣ′

kTeffM−1Mv,

respectively. Writing

ΣΣ′

kTeff=

(TeffT

+T − Teff

T

)ΣΣ′

kTeff= 2B + U + U ′,

we get

bv−(x, v) = −M−1(B + U)v −M−1∇V (x) +M−1(2B + U + U ′)v= M−1(B + U ′)v −M−1∇V (x). (17b)

We conclude that if U is symmetric, the evolution in the steady state ρ̄ is reversible. IfM = mI is a scalar matrix, this condition coincides with the optimality condition (16)! Inthis case, efficiency of the Brownian motor or Maxwell demon [31] which uses the velocitymeasurements to convert heat from the thermostat into work, can be viewed as solvingminimum entropy problem (see next section) and coincides with reversibility. Consider nowthe relative entropy of the state ρt with respect to the steady state ρ̄

H(ρt, ρ̄) =∫Rn

∫Rn

logρt(x, v)

ρ̄(x, v)ρt(x, v)dxdv.

Adapting a standard calculation [22, (2.21)] to this degenerate diffusion case

d

dtH(ρt, ρ̄) = −

1

2

∫Rn

∫Rn

(∇v log

ρt(x, v)

ρ̄(x, v)

)′M−1ΣΣ′M−1

(∇v log

ρt(x, v)

ρ̄(x, v)

)dxdv. (18)

Let us introduce the electrochemical potential

µs := H(x, v) + kTeff log ρt(x, v)

and the corresponding forces

Φ(x, v, t) = −∇vµs = Mv + kTeff∇v log ρt(x, v) = kTeff∇v logρt(x, v)

ρ̄(x, v).

We see from (18) that, when displaced away from ρ̄, they drive the system back to the steadystate.

8

V. FAST COOLING FOR THE SYSTEM OF STOCHASTIC OSCILLATORS

Consider now the same system of stochastic oscillators subject to an external forcerepresented by the control action u(t):

dx(t) = v(t) dt, (19a)

Mdv(t) = −Bv(t) dt+ u(t)dt−∇V (x(t))dt+ ΣdW (t), (19b)

with x(t0) = x0 and v(t0) = v0 a.s. Here u is to be designed by the controller in order toachieve the desired cooling at a finite time t1. More explicitly, we seek to steer the systemof stochastic oscillators to the desired steady state ρ̄ given in (9) in finite time. Let U be thefamily of adapted, finite-energy control functions such that the initial value problem (19) iswell posed on bounded time intervals and such that the probability density of xu(t1) is givenby (9). More precisely, u ∈ U is such that u(t) only depends on t and on {xu(s); t0 ≤ s ≤ t}for each t > t0, satisfies

E{∫ t1

t0

u(t)′u(t) dt

}

9

since the endpoints marginals at t = t0 and t = t1 are fixed. Observe now that, under Pu,by Ito’s rule [27],

d logϕ(x(t), v(t), t) =∂ logϕ

∂t+ v · ∇x logϕ+ (−βv −

1

m∇xV + u) · ∇v logϕ

+σ2

2∆v logϕ(x(t), v(t), t)dt+∇v logϕ(x(t), v(t), t)σdWt.

Using this and (22) in (23), we now get

I(Pu) = EPu[∫ t1

t0

1

2σ2u · udt− logϕ(x(t1), v(t1), t1) + logϕ(x(t0), v(t0), t0)

]= EPu

[∫ t1t0

(1

2σ2u · u

−[∂ logϕ

∂t+ v · ∇x logϕ+ (−βv −

1

m∇xV + u) · ∇v logϕ+

σ2

2∆v logϕ

](x(t), v(t), t)

)dt

−∫ t1t0

∇v logϕ(x(t), v(t), t)σdWt]

= EPu[∫ t1

t0

(1

2σ2u · u− u · ∇v logϕ(x(t), v(t), t) +

σ2

2‖∇v logϕ(Xt, t)‖2

)dt

]= EPu

[∫ t1t0

1

2σ2‖u− σ2∇v logϕ(x(t), v(t), t)‖2dt

], (24)

where we have used the fact that the stochastic integral has zero expectation. Then theform of the optimal control follows

u∗(t) = σ2∇v logϕ(x(t), v(t), t). (25)

It turns out that u∗ is in feedback form, so that the optimal solution is a Markov process aswe know from the general theory [26]. If the pair (ϕ, ϕ̂) satisfies the Schrödinger system

∂ϕ

∂t+ v · ∇xϕ+ (−βv −

1

m∇xV ) · ∇vϕ+

σ2

2∆vϕ = 0, (26)

∂ϕ̂

∂t+ v · ∇xϕ̂+∇v

[(−βv − 1

m∇xV

)ϕ̂

]− σ

2

2∆vϕ = 0. (27)

with boundary conditions

ϕ(x, v, t0)ϕ̂(x, v, t0) = ρ0(x, v), ϕ(x, v, t1)ϕ̂(x, v, t1) = ρ1(x, v)

then Pu∗ is the solution of the Schrödinger bridge problem. The optimal evolution steeringthe stochastic oscillator from ρ0 to ρ1 with minimum effort is given by

dx(t) = v(t) dt,

dv(t) = −βv(t) dt− 1m∇xV (x(t))dt+ σ2∇v logϕ(x(t), v(t), t)dt+ σdW (t).

We observe that −σ2 logϕ(x, v, t) plays the role of an artificial potential generating theexternal force which achieves the optimal steering. Notice, moreover, that − logϕ(x, v, t) is

10

nothing but the value function [18] of the stochastic control problem starting the oscillatorat the point (x, v) ∈ R2n at time t < t1. Indeed, suppose now we replace the initial timewith t < t1 and have all measures concentrated at the point (x, v) at time t. Let ϕ1(x, v) =ϕ(x, v, t1) the value of the space-time harmonic function we get from the Schrödinger systemat τ = T when in the initial boundary condition is a δ is concentrated at (x, v) . Considerthe stochastic control problem to minimize the functional

J(u) = E[∫ t1

t

1

2σ2u · udτ − logϕT (x(T ), v(T ))

], (28)

subject to

dx(t) = v(t) dt, x(t) = x a.s.

dv(t) = −βv(t) dt− 1m∇V (x(t))dt+ u(x(t), v(t), t) + σdW (t), v(t) = v a.s..

over feedback controls u such that the system of differential equations has a weak so-lution. Then the solution is precisely the Schrödinger bridge Pu∗ relative to the inter-val [t, t1] with u

∗ given by (25) and initial density concentrated on (x,v). Moreover,S(x, v, t) = − logϕ(x, v, t) = infuJ(u) is the value function of the control problem. Itsatisfies the Hamilton-Jacobi-Bellman equation

∂S

∂t+ v · ∇xS + infu

[(−βv − 1

m∇xV + u

)· ∇vS +

1

2σ2‖u‖2

]+σ2

2∆vS = 0,

S(x, v, T ) = − logϕT (x, v).

Notice that when E[ϕ(x∗(t), v∗(t), t)] and E[− logϕ(x∗(t), v∗(t), t)] are finite on the timeinterval of interest, then logϕ(x∗(t), v∗(t), t) is a submartingale with respect to the familyFWt (i.e., the σ-algebra generated by the past of W (t)), namely, it is conditionally increasingon the optimal evolution. When the potential V is quadratic, one can expect S to be aquadratic form in (x, v).

VI. THE CASE OF A QUADRATIC POTENTIAL

When the potential V is simply a quadratic form

V (x) =1

2x′Dx,

with D a symmetric, positive definite n×n matrix, the dynamics of the stochastic oscillator(4) becomes linear and we can directly apply the results of [5]. This is precisely the situationconsidered in [1, 48]. We proceed to show that it is possible to design a feedback controlaction which takes the system to the desired (Gaussian) steady state

ρ̄ = C exp

[−

12mv′v + 1

2x′Dx

kTeff

]at the finite time t1. We specialize to the case where M = mI, B = βI, and Σ = σI, forsimplicity. We write the uncontrolled dynamics

dx(t) = v(t) dt, (29)

dv(t) = − βmv(t) dt− 1

mDx(t)dt+

σ

mdW (t).

11

asdξ = Aξdt+ BdW, (30)

where

ξ =

(xv

), A =

(0 I− 1mD − β

mI

), B =

(0σmI

).

Notice that the pair (A,B) is controllable. Once again, introducing a control input u(t),we want to minimize

E{∫ t1

t0

1

2u(t)′u(t) dt

}under the controlled dynamics

dx(t) = v(t) dt, (31)

dv(t) = − βmv(t) dt− 1

mDx(t)dt+

1

mu(t)dt+

σ

mdW (t), (32)

with x(t0) = x0, and v(t0) = v0 a.s. Then, applying [5, Theorem 8], we get that the optimalsolution is

u∗(t) = −σB′Π(t)ξwhere Π(t) is the solution to the following system of Riccati equations

Π̇(t) = −A′Π(t)− Π(t)A+ Π(t)BB′Π(t), (33)

Ḣ(t) = −A′H(t)− H(t)A− H(t)BB′H(t), (34)which are coupled through their boundary solutions by

1

kTdiag{D, mI} = H(t0) + Π(t0) (35)

1

kTeffdiag{D, mI} = H(t1) + Π(t1). (36)

Since control effort is required to steer the system to a lower-temperature state, Π(t) willbe non-vanishing throughout. The precise form of the optimal control is in [5, Theorem 8].

VII. EXAMPLE

The example is based on the linear model

dx(t) = v(t)dt

dv(t) = −v(t)dt− x(t)dt+ u(t)dt+ dw(t)

which corresponds to taking m = 1, β = 1, σ = 1, and D = 1. This is an academic example,and we assume units so that k = 1. Accordingly, we want to steer and maintain the systemstarting from an intial temperature (in consistent units) of T = 1

2to a final temperature

Teff =116

. Although the figures have been chosen for convenience, the example demostratesa typical response of the system in Figure 1 and the nature of the corresponding controlinputs in Figure 2.

12

FIG. 1: Inertial particles: trajectories in phase space

Thus, we seek an optimal u(t) as a time-varying linear function of x(t) to steer thesystem from a normal distribution in phase space with zero mean and covariance

kT diag{D, mI}−1 = 12

[1 00 1

]to a final distribution with zero mean and covariance

kTeff diag{D, mI}−1 =1

24

[1 00 1

],

over the time window [0, 1]. Thereafter, the distribution of x(t) remains normal maintainingthe covariance via a choice of u(t) which is a linear, time-invariant function of v(t), namelyu(t) = −Uv(t), with now the scalar constant U satisfying (11). The figures show thetrajectories of the inertial particles in phase space as a function of time and the respectivecontrol effort. The transition is effected optimally, using time-varying control, whereas att1 = 1, the value of the control switches to the time-invariant linear function of v(t) whichmaintains thereafter the distribution of (x(t), v(t)) at the desired level.

Appendix A: Proof of Proposition 2

Recall [25, 30] that a smooth probability density ρ yields an invariant measure for aMarkov diffusion process if and only if it annihilates the formal adjoint of the correspondinginfinitesimal generator L. Here

L =n∑j=1

vj∂xj −n∑j=1

(M−1(B + U)v −M−1∇V (x)

)j∂vj +

1

2

n∑i,j=1

(M−1ΣΣ′M−1

)ij∂vi∂vj

13

FIG. 2: Inertial particles: control effort u(t)

Thus, the condition becomes to satisfy the stationary Fokker-Planck equation

0 = L∗(ρ) =

−v · ∇xρ+M−1 [(B + U) v +∇V (x)] · ∇vρ+ trace(M−1(B + U))ρ

+1

2

n∑i,j=1

(M−1ΣΣ′M−1

)ij

∂2ρ

∂vi∂vj

We get

L∗(ρ̄) =ρ̄

kTeff

[trace

(M−1

(kTeff(B + U)−

1

2ΣΣ′

))− v′

((B′ + U ′)− 1

2kTeffΣΣ′

)v

].

The invariance of ρ̄ is then equivalent to have for all v ∈ Rn

kTeff trace

(M−1

(kTeff(B + U)−

1

2ΣΣ′

))= v′

1

2(kTeff(B + U +B

′ + U ′)− ΣΣ′) v, (A1)

Taking v = 0, we get

trace

(M−1

(kTeff(B + U)−

1

2ΣΣ′

))= 0

which in turn implies that the form in the right-hand side of (A1) is identically zero.This, together with (7), gives (11). Conversely, if (11) holds, we get from (7) thatthe matrix

(kTeff(B + U)− 12ΣΣ

′) is skew-symmetric. From this it follows that alsoM−1/2

(kTeff(B + U)− 12ΣΣ

′)M−1/2 is skew symmetric. Hence it has zero trace and weget

trace

(M−1/2

(kTeff(B + U)−

1

2ΣΣ′

)M−1/2

)= trace

(M−1

(kTeff(B + U)−

1

2ΣΣ′

))= 0.

Thus the left-hand side of (A1) is zero and so is the right-hand side because of (11)-(7).Hence, equality holds and ρ̄ is invariant.

14

Appendix B: Relative entropy for stochastic oscillators measures

Let Ω := C([t0, t1],R2n) denote the family of 2n-dimensional continuous functions, letW(x,v) denote Wiener measure on Ω starting at the point (x, v), and let

W :=

∫W(x,v) dxdv

be stationary Wiener measure. Let D be the family of distributions on Ω that are equivalentto W . For Q,P ∈ D, we define the relative entropy H(Q,P ) of Q with respect to P as

H(Q,P ) = EQ[logdQ

dP].

By Girsanov’s theorem [19, 20, 27], under Q ∈ D, the coordinate process Xt(ω) = ω(t)admits the representation

dXt = βQt dt+ dWt, β

Qt is Ft − adapted

where Ft are σ- algebras of events observable up to time t. Moreover,

Q

[∫ t1t0

‖βQt ‖2dt

15

be the diffusion coefficient matrix. We compute the Radon-Nikodym derivative dPnu

dPn0us-

ing Girsanov’s theorem [25, 27]. Let W0 be Wiener measure starting with distributionρ0(x, v)dxdv of (x0, v0) at t = t0. Since W0, P

nu and P

n0 have the same initial marginal, we

get

dP nudW0

= exp

[∫ t1t0

Θ−1βPnut ·Θ−1dXt −

∫ t1t0

1

2βPnut ·Θ−2β

Pnut dt

], P nu a.s.,

dW0dP n0

= exp

[−∫ t1t0

βPn0t ·Θ−1dXt +

∫ t1t0

1

2βPn0t Θ

−2βPn0t dt

], P n0 a.s.⇒ P nu a.s..

Thus

dP nudP n0

= exp

{∫ t1t0

(Θ−1β

Pnut −Θ−1β

Pn0t

)·Θ−1

(dxtdvt

)+

1

2

∫ t1t0

[βPn0t ·Θ−2β

Pn0t − β

Pnut ·Θ−2β

Pnut

]dt

}= exp

{∫ t1t0

(Θ−1β

Pnut −Θ−1β

Pn0t

)·(dZtdWt

)+

1

2

∫ t1t0

(βPnut − β

Pn0t

)·Θ−2

(βPnut − β

Pn0t

)dt

}= exp

{∫ t1t0

[Θ−1

(0u

)]·(dZtdWt

)+

1

2

∫ t1t0

(0u

)·Θ−2

(0u

)dt

}= exp

{∫ t1t0

1

σu · dWt +

∫ 10

1

2σ2u · udt.

}.

Notice, in particular, that this Radon-Nikodym derivative does not depend on n. Let Puand P0 be the measures in D corresponding to the situation when there is no noise in theposition equation (n =∞). Then

dPudP0

=

∫ t1t0

1

σu · dWt +

∫ 10

1

2σ2u · udt.

Assuming that the control satisfies the finite energy condition

E[∫ t1

t0

u · udt]

16

REFERENCES

[1] M. Bonaldi, L. Conti, P. De Gregorio et al, Nonequilibrium steady-state fluctuations in activelycooled resonators, Phys. Rev. Lett., 103 (2009) 010601.

[2] Y. Braiman, J. Barhen, and V. Protopopescu, Control of Friction at the Nanoscale, Phys.Rev. Lett. 90, (2003), 094301.

[3] C. I. Byrnes and C. F. Martin, An integral-invariance principle for nonlinear systems, IEEETrans. Aut. Contr., 40, pp.983–994, 1995.

[4] Y. Chen and T.T. Georgiou, Stochastic bridges of linear systems, preprint, http://arxiv.org/abs/1407.3421.

[5] Y. Chen, T.T. Georgiou and M. Pavon, Optimal steering of a linear stochastic system to afinal probability distribution, Aug. 2014, http://arxiv.org/abs/1408.2222, submitted forpublication.

[6] Y. Chen, T. Georgiou and M. Pavon, Optimal steering of inertial particles diffusing anisotrop-ically with losses, Oct. 2014, http://arxiv.org/abs/1410.1605, submitted to ACC 2015.

[7] Y. Chen, T.T. Georgiou and M. Pavon, Optimal steering of a linear stochastic system to a finalprobability distribution, part II, Oct. 2014, http://arxiv.org/abs/1410.3447, submittedfor publication.

[8] P. Dai Pra, A stochastic control approach to reciprocal diffusion processes, Applied Mathe-matics and Optimization, 23 (1), 1991, 313-329.

[9] P.Dai Pra and M.Pavon, On the Markov processes of Schroedinger, the Feynman-Kac formulaand stochastic control, in Realization and Modeling in System Theory - Proc. 1989 MTNSConf., M.A.Kaashoek, J.H. van Schuppen, A.C.M. Ran Eds., Birkaeuser, Boston, 1990, 497-504.

[10] A. Dembo and O. Zeitouni, Large deviations techniques and applications, Jones and BartlettPublishers, Boston, 1993.

[11] M. Doi and S. F. Edwards, The Theory of Polymer Dynamics, Oxford University Press, NewYork, 1988.

[12] R. S. Ellis, Entropy, Large deviations and statistical mechanics, Springer-Verlag, New York,1985.

[13] I. Favero and K. Karrai, Nat. Photon. 3, 201 (2009).[14] R. Fillieger and M.-O. Hongler, Relative entropy and efficiency measure for diffusion-mediated

transport processes, J. Physics A: Mathematical and General 38 (2005), 1247-1255.[15] R. Fillieger, M.-O. Hongler and L. Streit, Connection between an exactly solvable stochastic

optimal control problem and a nonlinear reaction-diffusion equation, J. Optimiz. Theory Appl.137 (2008), 497-505.

[16] M. Fischer, On the form of the large deviation rate function for the empirical measures ofweakly interacting systems, Bernoulli, 20 (4), (2014), 1765-1801.

[17] W.H. Fleming, Logarithmic transformation and stochastic control, in: W. Fleming and L.Gorostiza, eds., Advances in Filtering and Optimal Stochastic Control, Lecture Notes in Con-trol and Inform. Sciences, Vol. 42, Springer, Berlin, 1982, 131-141.

[18] W.H. Fleming and R.W. Rishel, Deterministic and Stochastic Optimal Control, Springer-Verlag, Berlin, 1975.

[19] H. Föllmer, in: Stochastic Processes - Mathematics and Physics , Lect. Notes in Math. 1158(Springer-Verlag, New York,1986), p. 119.

[20] H. Föllmer, Random fields and diffusion processes, in: Ècole d’Ètè de Probabilitès de Saint-Flour XV-XVII, edited by P. L. Hennequin, Lecture Notes in Mathematics, Springer-Verlag,New York, 1988, vol.1362,102-203.

[21] T. T. Georgiou and M. Pavon, Positive contraction mappings for classical and quantumSchroedinger systems, 2014, arXiv:1405.6650v2, submitted for publication.

http://arxiv.org/abs/1407.3421http://arxiv.org/abs/1407.3421http://arxiv.org/abs/1408.2222http://arxiv.org/abs/1410.1605http://arxiv.org/abs/1410.3447http://arxiv.org/abs/1405.6650

17

[22] R. Graham, Path integral methods in nonequilibrium thermodynamics and statistics, inStochastic Processes in Nonequilibrium Systems, L. Garrido, P. Seglar and P.J.Shepherd Eds.,Lecture Notes in Physics 84, Springer-Verlag, New York, 1978, 82-138.

[23] U.G.Haussmann and E.Pardoux, Time reversal of diffusions, The Annals of Probability 14,1986, 1188.

[24] D.B.Hernandez and M. Pavon, Equilibrium description of a particle system in a heat bath,Acta Applicandae Mathematicae 14 (1989), 239-256.

[25] N. Ikeda and S. Watanabe, Stochastic Differential Equations and Diffusion Processes, North-Holland, 1981.

[26] B. Jamison, The Markov processes of Schrödinger, Z. Wahrscheinlichkeitstheorie verw. Gebiete32 (1975), 323-331.

[27] I. Karatzas and S. E. Shreve, Brownian Motion and Stochastic Calculus (Springer-Verlag, NewYork, 1988).

[28] K. H. Kim and H. Qian, “Entropy production of Brownian macromolecules with inertia”,Phys. Rev. Lett., 93 (2004), 120602.

[29] W. Kliemann, Transience, recurrence and invariant measures for diffusions, in R. S. Bucy andJ. M. F. Moura (Eds.) Nonlinear Stochastic Problems, Reidel, 1983, 437-454.

[30] W. Kliemann, Recurrence and invariant measures for degenerate diffusions, Ann. Prob., 15(1987), 690-707.

[31] H. S. Leff and A. F. Rex (eds.) Maxwell’s Demon 2, Institute of Physics, Bristol, 2003.[32] S. Liang, D. Medich, D. M. Czajkowsky, S. Sheng, J. Yuan, and Z. Shao, Ultramicroscopy, 84

(2000), p.119.[33] F. Marquardt and S. M. Girvin, Optomechanics, Physics 2, 40 (2009).[34] J. M. W. Milatz, J. J. Van Zolingen, and B. B. Van Iperen, Physica (Amsterdam)19, 195

(1953).[35] E. Nelson, Dynamical Theories of Brownian Motion, Princeton University Press, Princeton,

1967.[36] E. Nelson, Stochastic mechanics and random fields, in Ècole d’Ètè de Probabilitès de Saint-

Flour XV-XVII, edited by P. L. Hennequin, Lecture Notes in Mathematics, Springer-Verlag,New York, 1988, vol.1362, pp. 428-450.

[37] M.Pavon and A.Wakolbinger, On free energy, stochastic control, and Schroedinger processes,Modeling, Estimation and Control of Systems with Uncertainty, G.B. Di Masi, A.Gombani,A.Kurzhanski Eds., Birkauser, Boston, 1991, 334-348.

[38] M. Pavon and F. Ticozzi, On entropy production for controlled Markovian evolution, J. Math.Phys., 47, 06330, doi:10.1063/1.2207716 (2006).

[39] M. Pavon and F. Ticozzi, Discrete-time classical and quantum Markovian evolutions:Maximum entropy problems on path space, J. Math. Phys., 51, 042104-042125 (2010)doi:10.1063/1.3372725.

[40] M. Poot and H. S. J. van der Zant, Mechanical systems in the quantum regime, PhysicsReports, 511 (5) (2012), 273-335.

[41] H. Qian, “Relative entropy: free energy associated with equilibrium fluctuations and nonequi-librium deviations”, Physical Review E, 63 (2001), p. 042103.

[42] P. Reimann, Brownian motors: noisy transport far from equilibrium, Phys. Rep. 361, (2002)57.

[43] E. Schrödinger, Über die Umkehrung der Naturgesetze, Sitzungsberichte der Preuss Akad.Wissen. Berlin, Phys. Math. Klasse (1931), 144-153.

[44] E. Schrödinger, Sur la théorie relativiste de l’electron et l’interpretation de la mécaniquequantique, Ann. Inst. H. Poincaré 2, 269 (1932).

[45] K. C. Schwab and M. L. Roukes, Phys. Today 58, No. 7, 36 (2005).[46] J. Tamayo, A. D. L. Humphris, R. J. Owen, and M. J. Miles, Biophys., 81 (2001), p. 526.

18

[47] H. N. Tan and J. L. Wyatt, Thermodynamics of electrical noise in a class of nonlinear RLCnetworks, IEEE Tran. Circuits and Systems 32 (1985), 540-558.

[48] A. Vinante, M. Bignotto, M. Bonaldi et al., Feedback Cooling of the Normal Modes of aMassive Electromechanical System to Submillikelvin Temperature, Physical Review Letters101 (2008), 033601.

[49] A. Wakolbinger, Schroedinger bridges from 1931 to 1991, in Proc. of the 4th Latin AmericanCongress in Probability and Mathematical Statistics, Mexico City 1990, Contribuciones enprobabilidad y estadistica matematica, 3 (1992), 61-79.

[50] H. Wimmer, Notes for a lecture course on Lyapunov’s equation, Facoltà di Ingegneria, Uni-versità di Padova, 1987.

19

I IntroductionII Basic examplesA Feedback Cooling of the Normal Modes of a Massive Electromechanical SystemB Regulating polymer dynamics

III A system of stochastic oscillatorsIV Optimal steering to the steady state and reversibilityV Fast cooling for the system of stochastic oscillatorsVI The case of a quadratic potentialVII ExampleA Proof of Proposition ??B Relative entropy for stochastic oscillators measures REFERENCES