Robotic Radiotherapy: An optimal, adaptive control approach. › ~opo140030 › media › presentations › … · Radiotherapy: An optimal, adaptive control approach. Olalekan Ogunmolu

RoboticRadiotherapy:An optimal,

adaptivecontrol

approach.

OlalekanOgunmolu

PublicationsUntil Now

Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search

Fluence MapOptimization

Results

Solution

Solution:Soft-Robot PositionCorrecting Systems

Experiment Setup

Kalman Filtering

Sensor Fusion

SystemIdentification

3DOFControl

Vision-based HeadPose Estimation

AdaptiveNeuroControl

ControlContributions

Neural NetworkModel

Control gains

Results

References

Robotic Radiotherapy: An optimal,adaptive control approach.

Olalekan Ogunmolu

Department of Electrical Engineering

The University of Texas at Dallas, Richardson, TX

Department of Radiation OncologyUniversity of Texas Southwestern Medical Center, Dallas, TX

January 10, 2019

Olalekan OgunmoluRobotic Radiotherapy: An optimal, adaptive control approach.


adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Introduction

This page is left blank intentionally.



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Publications Until Now

Publications Under reviewOlalekan Ogunmolu, Xuejun Gu, Steve Jiang, Nicholas Gans. Nonlinear SystemsIdentification Using Deep Dynamic Neural Networks”. Under review at Physical ReviewApplied, American Physical Society, Submitted September 2018.

Olalekan Ogunmolu, Michael Folkerts, Dan Nguyen, Nicholas Gans, Steve Jiang. DeepBOO 2.0: End-to-End Training of Beam Orientation Selection Policies for IntensityModulated Radiation Therapy. Under publication invitation review at International Journalof Robotics Research (IJRR), 2018.

Accepted PublicationsOlalekan Ogunmolu, Michael Folkerts, Dan Nguyen, Nicholas Gans, and Steve Jiang. DeepBOO: Automating Beam Orientation Selection in Intensity Modulated Radiation Therapy.To appear at The 13th International Workshop on the Algorithmic Foundations of Robotics(WAFR), Merida, Mexico. December 2018.

Olalekan Ogunmolu, Nicholas Gans, Tyler Summers. Minimax Iterative Dynamic Game:Application to Nonlinear Robot Control Tasks. IEEE/RSJ International Conference onIntelligent Robots and Systems (IROS), Madrid, Spain. October 2018.

Olalekan Ogunmolu, Dan Nguyen, Xun Jia, Weiguo Lu, Nicholas Gans, and Steve Jiang.Automating Beam Orientation Optimization for IMRT Treatment Planning: A DeepReinforcement Learning Approach. Selected for Oral Presentation at the John R. CameronYoung Investigators Symposium – 60th Annual Meeting of the American Association ofPhysicists in Medicine, Nashville, TN (AAPM). July 2018.


https://arxiv.org/abs/1610.01439


https://www.springer.com/series/15556

https://www.springer.com/series/15556

http://algorithmic-robotics.org/wafr2018/


adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Publications Until Now

Accepted Publications (Cont’d)Yara Almubarak, Joshi Aniket, Olalekan Ogunmolu, Xuejun Gu, Steve Jiang, Nicholas Gans,and Yonas Tadesse, Design and Development of Soft Robots for Head and Neck CancerRadiotherapy. SPIE: Smart Structures + Nondestructive Evaluation, (SPIE), Denver, CO,U.S.A. March 2018.

Olalekan Ogunmolu, Adwait Kulkarn, Yonas Tadesse, Xuejun Gu, Steve Jiang, and NicholasGans. Soft-NeuroAdapt: A 3-DOF Neuro-Adaptive Pose Correction System For Framelessand Maskless Cancer Radiotherapy. IEEE/RSJ International Conference on IntelligentRobots and Systems (IROS), Vancouver, BC, Canada. September 2017. DOI:10.1109/IROS.2017.8206211.

Olalekan Ogunmolu, Xuejun Gu, Steve Jiang, and Nicholas Gans. Vision-based control of asoft-robot for Maskless Cancer Radiotherapy. IEEE Conference on Automation Science andEngineering (CASE), Fort-Worth, Texas, August 2016. DOI: 10.1109/CoASE.2016.7743378.

Olalekan Ogunmolu, Xuejun Gu, Steve Jiang, and Nicholas Gans. A Real-TimeSoft-Robotic Patient Positioning System for Maskless Head-and-Neck Cancer Radiotherapy.IEEE Conference on Automation Science and Engineering (CASE), Gothenburg, Sweden,August 2015. DOI: 10.1109/CoASE.2015.7294318.


http://spie.org/conferences-and-exhibitions/smart-structures/nde?SSO=1



http://ecs.utdallas.edu/~opo140030/media/Papers/CASE2016.pdf

http://ecs.utdallas.edu/~opo140030/media/Papers/CASE2016.pdf

http://ecs.utdallas.edu/~opo140030/media/Papers/CASE2015/CASE2015.pdf

http://ecs.utdallas.edu/~opo140030/media/Papers/CASE2015/CASE2015.pdf


adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Background




adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Background

Cancers of the head and neck (H&N)



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Treatment Options

Radiotherapy treatment is the most effective



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Radiotherapy types

Conventional Radiation Therapy

Conformal Radiation Therapy (CFRT)

Intensity-Modulated Radiation Therapy (IMRT)

Left: Conventional radiotherapy.Middle: Conformal radiotherapy (CFRT) without intensity modulation.Right: CFRT with intensity modulation. Reprinted from Webb (2001).



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Frame-based Radiotherapy Treatment

Requires head landmarks + mechanical immobilizationmechanisms

Aim ionizing radiation at abnormal tissues



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Frameless (Completely non-invasive) RT

High-dose volumes with complex shapes [Adler and Cox (1995)]

c©Accuray Inc.



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Image Guided Radiotherapy

c© Wiersma et al. (2009)



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Image-Guided SRS and SB Radiotherapy

The Novalis ExacTrac Module

c©Novalis



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Radiation Therapy Drawbacks

Frame-based immobilization

LINAC angle misalignments cause negative dosimetryeffects [Xing (2000)].Does not encourage fractionated treatments

Frameless RT

Incompatible with conventional LINACs used at themajority of cancer centers.

Novalis ExacTrac Robotic Tilt Module

Only uses pre-treatment imagesProvides just rigid motion compensation



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Beam Orientation Optimization

During treatment planning, a beam orientationoptimization problem (BOO) is separately solved

Radiation is delivered from ≈ (5− 15) different beamorientations during IMRT

BOO determines the best beam angle combinations fordelivering radiation.

Process of determining beamlets’ intensities is termedfluence map optimization (FMO)



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Cyberknife Radiation Therapy

c©Accuray Inc.



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Radiation Therapy Treatment Plan

Reprinted from “IMRT Optimization Algorithms. David Shepard. Swedish Cancer Institute. AAPM 2007.”



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Our approach

Automate the beam search problem

Drawing ideas from

pattern recognition;

monte carlo evaluations;

game simulations; and

approximate dynamic programming

Ultimate goal is real-time beam angle prediction given atarget volume



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Game Tree Simulation

For bd possible move sequences for a robot-patient setup

where b = cardinality of discretized angles; and

d = number of beam angles to choose from b.

Suppose b = 180 and d = 5, we have 1805 possible searchdirections

Exhaustive search becomes real-time infeasible.



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

State Representation: Network Input



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Separated Target Volume Organs and PTV Masks



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Methods

Suppose the first player is p1, and the second player is p2

p1 chooses its action under a (stochastic) strategy,πp1 = {πp1

0 , πp11 , . . . , π

p1

T } ⊆ Πp1

minimizing the game’s outcome ζ

p2’s actions are governed by a policyπp2 = {πp2

0 , πp21 , . . . , π

p2

T } ⊆ Πp2

maximizing ζ in order to guarantee an equilibrium solutionfor a game without saddle point.

Πpi is the set of all possible nonstationary Markovianpolicies



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Methods

One may write the optimal cost-to-go value function forstate s in stage t, with horizon length T as

V ∗t (s) = infπp1∈Πp1

supπp2∈Πp2

E

[T−1∑i=t

Vt(s0, f (st , πp1 , πp2))

],

s ∈ S , t = 0, . . . ,T − 1

where the terminal value V ∗T (s) = 0, ∀ s ∈ S ;

πp1 and πp2 contain the action/control sequences{ap1

t }0≤t≤T and {ap2t }0≤t≤T

Each player generates a mixed strategy determined byaveraging the outcome of individual plays.



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Methods

The saddle point strategies for an optimal control

sequence pair {ap∗1

t , ap∗2t } can be recursively obtained by

V ∗t (s) = V ∗t (st , πp1t , π

p2t ) = min

πp1∈Πp1max

πp2∈Πp2V ?t (st , π

p1 , πp2)

∀st ∈ S, πp1 ∈ Πp1 , πp2 ∈ Πp2 .

such that

V ?p1≤ V ? ≤ V ?

p2∀ {πp1

t , πp2t }0≤t≤T .

where V ?pi

are the respective optimal values for each player.



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Game Tree Simulation

Network roll-out policy then efficiently guides the tree’sgame, Γ toward a best-first set of beam angle candidates

Best-first leaf node encountered is the child node with thehighest reward in the tree

Essentially, a sampling-based lookout algorithm

Focuses learning on regions of the state space that arelikely to have a good fluence



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Methods: Fluence Map Optimization (FMO)

Suppose X is the total discretized of voxels of interest(VOI ’s) in a target volume

Suppose B1 ∪ B2 ∪ . . . ∪ Bn ⊆ B represents the partitionsubset of a beam B

where n is the total number of beams from which radiationcan be delivered

Suppose further that Dij(θk) is the matrix that describeseach dose influence, di .



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Methods: FMO problem definition

The pre-calculated dose term is given byAx = {

∑swsvsDs

ijxs | Dij ∈ Rn×l , n > l}

Let ws = {w s , ws} be the respective underdosing andoverdosing weights for the OARs and PTVs

We propose the following cost

1

vs

∑s∈OARs

‖(bs − w sDsijxs)+‖2

2 +1

vs

∑s∈PTVs

‖(wsDsijxs − bs)+‖2

2

(1)



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Methods: FMO

The objective, subject to nonnegative bixel intensityconstraints is the minimization problem

min1

2‖Ax− b‖2

2 subject to x ≥ 0.

with Lagrangian

L(x,λ) =1

2‖Ax− b‖2

2 − λTx.

Introduce the auxiliary variable z, we have

minx

1

2‖Ax− b‖2

2, subject to z = x, z ≥ 0,



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Methods: FMO

Solving the x and z sub-problems, we have

x, z-updates

xk+1 =(

ATA + ρI)−1 (

ATb + ρzk − λk)

zk+1 = Sλ/ρ

(xk+1 + λk

)where Sλ/ρ(τ) = (x − λ/ρ)+ − (−τ − λ/ρ)+. λ isupdated as

λk+1 = λk − γ(zk+1 − xk+1), (2)

where γ is a parameter that controls the step length.



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Results



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Results: Dose Wash Plot



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Results: Dose Wash



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Results: Clinically Optimized DVH Curve



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Results: Deep BOO Optimized DVH Curve



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References




adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References




adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

The Immobilization Problem




adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

The Immobilization Problem

Cervino et al. (2010) Liu et al. (2015) Ostyn et al. (2017)

Feasibility evaluation 4-D robotic stage couch 6-DoF robotic couch



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Drawbacks of current solutions

Rigid patient’s body assumption

Non-compliant immobilization devices

Lack of embodied intelligence and the morphologicalcomputation property in immobilization devices.

Attenuation of x-rays or irradiation photon beams



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Solution: Soft-Robot Position Correcting Systems

Eliminate rigid frames and metallic rings

Maskless and frameless positioning system X

Eliminate attenuation of X-Ray beams

Separate electromechanical components from patientactuation XExplore soft polymer actuators X

Control design

Feedback control + optimal regulation + robustness todisturbance X

Adapt for changing parameters in system’s dynamicswithin control law



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Vision-based 1-DOF Control

Left blank intentionally



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Experiment Setup



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Head Pose Estimation: Sensing Hardware



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Sensors’ Noise Floor

0 500 1000 1500 2000

Time(sec)

680

700

720D

ep

th (

mm

)Kinect Xbox Tracking Noise on a Static Target at 684mm

Ground Truth = 680mm; Covariance = 21.5022

100 200 300 400 500 600 700 800 900 1000

Time(sec)

995

1000

1005

1010

De

pth

(m

m)

Kinect v1 Position Estimation of a Static Object

Ground Truth = 960mm; Covariance = 11.4707

Case for Sensed Observation Filtering



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Optimal head state estimation and sensor fusion

Given the measurement sequence from either sensorz(1), · · · , z(j),

find the observation estimates x(i) that minimize themean-square error from true measurement, i .e. ,

x(i |j) = arg minx(i |j)∈Rn

E{(x(i)− x)T (x(i)− x)|z(1), · · · , z(j)}

Suppose we define the estimate error’s covariance as

P(i |j) , E{(x(i)− x(i |j))T (x(i)− x(i |j)) |Z j}. (3)



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

State estimation with Kalman filters

Assume a state model, F(k), common to either sensor,

And denote the state vector as x(k) = [d(k), d(k)]T

where d(k) is the displacement of the head above theactuator

For a discretized time interval ∆T between measurements,we define the state

x(k) = F(k)x(k − 1) + B(k)uk + Gkwk (4)

with

F =

[1 ∆T0 1

](5)



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Kalman filters

Since there is no control in this case, (4) becomes

xk = Fkxk−1 + Gkwk (6)

where wk is the effect of an unknown input and Gk appliesthat effect to the state vector, xk .

Gk is modeled as uncontrolled forces accelerating the head

Head’s acceleration, ak , is a random scalar; ak ∼ N (0, σa)

Setting Gk = I2x2 and w(k) ∼ N (0,Q(k))

set Q(k) to a random walk sequence, Wk = [ ∆T 2

2 ,∆T ]T



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Kalman Filters Design

Such that

Q = WWTσa2 =

∆T 4

4

∆T 3

2∆T 3

2∆T 2

σa2. (7)

Set the transfer matrix from the estimates, x(k), toobservations, z1(k) and z2(k) according to

zs = Hs(k)x(k) + vs(k) s = 1, 2 (8)

where Hs(k) =[1, 0

]Tand vs(k) ∼ N (0, σ2

rs)note that v1(k), v2(k), and w(k) are independent anduncorrelated in time.



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

KF Priori and Posteriori State Estimates

Prediction Phase:

xk|k−1 = Fxk−1|k−1 + Bkuk

Pk|k−1 = FkPk−1|k−1FkT + Qk (9)

Update Phase:

K(k) = P(k |k − 1)H(k)T [H(k)P(k |k − 1)H(k)T + R(k)]−1

x(k |k) = x(k |k − 1) + K(k)(z(k)−H(k)x(k|k − 1))

P(k|k) = (I−K(k)H(k))P(k |k − 1) (10)



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Filtering Results

200 400 600 800 1000 1200 1400 1600 1800 2000 2200

Samples

670

680

690

700

710

Ob

se

rva

tio

n in

m

m

Kinect Xbox Tracking Noise on a Static Target at 684mm

Observation

200 400 600 800 1000 1200 1400 1600 1800 2000 2200

Samples

682

684

686

688

Ka

lma

n u

pd

ate

s in

m

m

Kalman Filter Estimates with Kinect Xbox. [Truth: 684mm, R: 70mm2

]

Estimates

100 150 200 250 300 350 400

Time(Secs

645

650

655

De

pth

(m

m)

Kinect v1 Tracking Noise on a Static Target at 651mm

Observation

100 150 200 250 300 350 400

Time(secs)

649

650

651

652

653

KF

E

stim

ate

s

(m

m)

Recursive KF Results of Raw Observation in Real Time

Priori

Posteriori

[Left]: Xbox’ observation filtering results. [Right]: KF results of Kinect v2’s observation



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Global fusion of local tracks

Communicate each updated prediction via Unix namedpipes

Again, assume a state model common to both sensors

Fuse both updates with a variance-weighted average ofeach local track as follows,

x(F )(k |k) = P(F )(k |k)N∑

s=1

[P(s)−1(k|k)x(s)(k|k)

]

where P(F )(k |k) =

[N∑

s=1

P(s)−1(k |k)

]−1

.



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Sensor Fusion Results

Track-to-track fusion of both sensors’ local track estimates.



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Vision-based Control




adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

System Identification

From I/O data, estimate a model

Find set of optimal model parameters via the minimization,

G (t) = arg minθ

VN(θ,ZN)

where VN(θ,ZN) =∑K

k=1

∑ni=1

1

2(yi (k)− yi (k))2,

and ZN = {u(1) · · · u(N), y(1) · · · y(N)}After a least squares minimization, we derive a state-spacerealization,

x(k + Ts) = Ax(k) + Bu(k) + Ke(k)

y(k) = Cx(k) + Du(k) + e(k) (11)



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

LQG Design

Posing the cost

J =K∑

k=0

xT (k) Q x(k) + u(k)T R u(k) + 2x(k)T N u(k)

K is the terminal sampling instant,Q is a symmetric, positive semi-definite matrix thatweights the n-states of the A matrix,N specifies a matrix of appropriate dimensions thatpenalizes the cross-product between the input and statevectors,R is a symmetric, positive definite weighting matrix on thecontrol vector u

we can obtain u as ∆u = arg min∆u

J

∆u is a future control sequence



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Closed-loop control (Full state observer).

B(k)u(k)

Kopt

y(k)++

+∫

C (k)x(k) +

+

A(k)

w(k)

y(k)

v(k)

−+

B(k)++

+∫

C (k)

y(k)

A(k)

-Kobs

ε = y(k)− y(k)

x(k)

−1

y(k)

x(k + 1) = A(k)x(k)− Kobs [C (k)x(k)− y(k)] + B(k)u(k).



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

1-DOF Control Results

20 30 40 50 60 70 80 90 100

Time (seconds)

685

690

695

700

705

710

Head P

ositio

n(m

m)

Experiment I Reference

Kinect Fused yhat(kT)

LQG Estimated yhat(kT)

10 20 30 40 50 60 70 80 90 100

Time (seconds)

680

690

700

710

Head P

ositio

n (

mm

)

Experiment IIReference



10 20 30 40 50 60 70 80 90

Time (seconds)

680

685

690

695

He

ad

P

ositio

n (m

m)

Experiment III

Reference



LQG Controller on mannequine head.



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References


Left blank intentionally



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References


Hardware Description



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Vision-based head pose estimation

Acquire face point cloud from stereocamera

Find edges of 2D planar regions in scene

structure (Torr and Zisserman, 2000)

bound resulting plane indiceswith their 2D convex hull

Extract face and face-height neighborsinto a predefined 2D prismatic model(Rusu, R. et. al, 2008)

Cluster extracted points based withEuclidean Clustering (Rusu Thesis)

[Top-left]: Dense point cloud of the scene.[Top-right]: Downsampled cluttered cloud of the scene.[Bottom-left]: Using RANSAC, we searched for 2D plane candidates inthe scene and compute the convex hull of found planar regions. Wethen extrude point indices in the hull to a prismatic polygonal modelto give the face region.[Bottom-right]: An additional step clusters the resultant cloud basedon a user-defined Euclidean distance.



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Head Pose Estimation

Goal: Compute optimal translation and rotation of thehead

from a model point set X = {−→x i} to a measured point setP = {−→p i}, where Nx = Np = 3,point −→x i ∈ X has the same index as −→p i ∈ Pset three points on the table scene as model points

Obtain centroid of segmented cloud from the previousscene

set this as measured point

Compute the covariance matrix, Σpx , of P and Xextract cyclic components of the skew-symmetric matrix as∆



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Head Pose Estimation (Besl & McKay, ’92)

Form symmetric 4× 4 matrix Q(Σpx)

Q(Σpx) =

[tr(Σpx) ∆T

∆ Σpx + ΣTpx − tr(Σpx)I3

]Optimal rotation quaternion, qR , is maximum eigenvalueof Q(Σpx)

Optimal translation vector is −→q T = −→µ x − R(−→q R)−→µ p

where µx and µp are the mean of point sets X and Prespectively

Face pose is [qT , qR ] = {x , y , z , θ, φ, ψ} w.r.t the worldframe



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Control Proposals

Solve a state feedback and feedforward regulation problem

An adaptation model based on head pose estimate givenpast states and past control inputs:

ZN = {u(k), u(k − 1), . . . u(k − nu), y(k), . . . , y(k − ny )}Let a persistently exciting input signal uex ∈ L2 ∩ L∞excite the system’s nonlinear modes

Control Design Goals

Stabilize system states, y =[x , y , z , θ, φ, ψ

]T



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Control Design Goals

Provide closed loop tracking given a desired trajectory, r

Robustify system to (non-)parametric uncertainties

changing head shapes, size and other anatomic/tumorvariations

Indirect MRAC system. (Source mdpi.com)



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Model Reference Adaptive Control

Model head and bladder dynamics asy = Ay + BΛ (u− f (y)) + w(k)

A, Λ unknown, B, sgnΛ knownf (y) , nonlinear function to be adapted forx , tuple containing past controls and current outputs

Approximate f (y) by a neural network with continuousmemory states

f (u(k − d), y(k),w(k)) is realized with a long-short termmemory cell (Horchreiter and Schmidhuber, ’91, ’97)purpose: remember good adaptation gains



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Assumptions

A dynamic RNN with N neurons, ϕ(x), existsmaps from a compact input space u ⊂ U to y ⊂ Y on theLebesgue integrable functions within [0,T ] or [0,∞)

f (y, u) is exactly ΘTΦ(y)f has coefficients Θ ∈ RN×m and a Lipschitz-continuous vectorof basis functions Φ(y) ∈ RN

Inside a ball YR with known, finite radius R,an ideal neural network (NN) approximation f (y) : Rn → Rm, isrealized to a sufficient degree of accuracy, εf > 0;

Outside YR ,the NN approximation error can be upper-bounded by a knownunbounded, scalar function εmax(y);‖ε(y)‖ ≤ εmax(y), ∀ y ∈ YR ;

There exists an exponentially stable reference modelym = Amym + Bmr



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Adaptive Neuro-Control Scheme

Set control law in terms of parameter estimates from theneural network weights and Lipschitz basis functions

Φ(y) = {y(k−d), · · · , y(k−d−4),u(k−d) · · ·u(k−d−5)}i .e. network looks back in time by 5 time steps at everyinstant, and then makes a prediction

Derive adaptive adjustment mechanism from Lyapunovanalysis for Adaptive Control (Parks, P., 1966)

u = KTy y︸︷︷︸

state feedback

+ KTr r︸︷︷︸

optimal regulator

+ f (y, u)︸︷︷︸approximator



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Controller formulation

Ky and Kr are adaptive gains to be designed

Term Contributions

KT

y y term keeps the states of the approximation sety ∈ BR stable,

KT

r r term causes the states to follow a givenreference trajectory

Function approximator f (y,u) ensures states thatstart outside the approximation set y ∈ BR

converge to BR in finite time



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Adaptive Control Formulation

Assume model matching conditions

such that Ky = Ky , and Kr = Kr (ideally)

Realize the approximator as f (y) = ΘTΦ(y) + εf (y)

ΘT denotes the vectorized weights of the neural networkΦ(y) denotes the vector of lagged inputs and output,εf (y) is the approximation error.Φ(y) = {y(k−d) · · · y(k−d−4),u(k−d) · · ·u(k−d−5)}



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Neural Network Model

Neural Net Architecture

input: lagged vector of pastobservations and currentcontrol actions

repeat × 3

pass input through anlstm cellfollowed by 30%dropout

output will be controlpredictions directly fed as valvevoltages



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Lyapunov Redesign

Theorem:Given correct choice of adaptive gains Ky and Kr , theerror state vector, e(k) with closed loop time derivative e,is uniformly ultimately bounded, and the state y willconverge to a neighborhood of r.

Proof:

Choose a Lyapunov function candidate V in terms of the

generalized error state space e, gains, KT

y , KT

r , andparameter error εf (y(k)) space

V(e, Ky , KT

r ) = eTPe + tr(KT

y Γ−1y K

T

y |Λ|) + tr(KT

r Γ−1r Kr |Λ|)



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Stability proof

V(e, Ky , Kr ) = eTPe + eTPe + 2tr(KTy Γ−1y

˙Ky |Λ|)

+ 2tr(KTr Γ−1r

˙Kr |Λ|)

=[Ame + BΛ[∆K

Tr r + ∆K

Tx x]]T

Pe + . . .

eTP[Ame + BΛ[∆K

Tr r + ∆K

Tx x]]

+ . . .

2 tr(∆KTx Γ−1

x˙Kx |Λ|) + 2 tr(∆KT

r Γ−1r

˙Kr |Λ|)



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Stability Analysis

= eT (P Am + ATmP)e + 2eTPBΛ

(K

Ty y + K

Tr r)

+2tr(

KTy Γ−1y

˙Ky |Λ|)

+ 2tr(

KTr Γ−1r

˙Kr |Λ|)

= −eTQe− 2eTPBΛεf (y) + 2eTPBΛKTy y

+ 2 tr(

KTy Γ−1y

˙Ky

)+ 2eTPBΛK

Tr r + 2 tr

(∆KT

r Γ−1r

˙Kr

)Notice xT y = tr

(y xT

)from trace identity



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Stability Analysis Cont’d

Therefore,

V(·) = −eTQe− 2eTPBΛεf

+ 2 tr(

KTy (Γ−1

y˙Ky + yeTPBsgn(Λ)

)|Λ|

+ 2 tr(

KTr (Γ−1

r˙Kr + reTPBsgn(Λ)

)|Λ|

where for a real-valued x , we have x = sgn(x)|x |.first two terms will be negative definite for all e 6= 0

since Am is Hurwitz

other terms will be identically null if we choose theadaptation laws

˙Ky = −ΓyyeTP Bsgn(Λ), ˙Kr = −Γr reTP B sgn(Λ)



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Proof Main Matter

,

V(·) = −eTQe− 2eTPBΛεf

≤ −λlow‖e‖2 + 2‖e‖‖PB‖λhigh(Λ)εmax

λlow , λhigh ≡ minimum and maximum characteristic rootsof Q and Λ respectively.

V(·) is thus negative definite outside the compact set

χ =(

e : ‖e‖ ≤ 2‖PB‖λhigh(Λ)εmax (y)λlow (Q)

)thus, we conclude that the error e is uniformly ultimatelybounded.

i.e. y(t)→ 0 as t →∞



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Experimental Results

Head 3-DOF pose made up of tuples {z(k), θ(k), φ(k)}i .e. roll, pitch and yawsample from the parameters of the network

Set f (·) to the fully connected layer of samples from the network

Control law is implemented on the ROS middleware

Gains Ky and Kr found by solving the ODEs iteratively

using a single step of the integral of the solutions to˙Ky (t), ˙Kr (t)

implemented using Runge-Kutta Dormand-Prince 5ODE-solver available in the Boost C++ libraries



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Results

Sample from trained network parameters

Set f (·) to fully connected layer of network

ym is computed based on

ym(t) = eAmtym(0) +∫ t

0eAm(t−τ)Bm r(τ)dτ

Set ym(0) = y(0) at t = 0

For a nonnegative Q and a positive definite P,

the pair (Q,Am) will be observableso that the dynamical system is globally asymptoticallystable.

Pick a positive definite, Q = diag(100, 100, 100) for thedissipation energy

Set Λ = I3×3



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Results

Solving the general form of thelyapunov equation, we have

P =

−1705002668 0 00 −170500

2668 00 0 −170500

2668

Solenoid valves operate in pairs

two valves create a difference inair mass within each IAB at anygiven timeset

B =

1 1 0 0 0 00 0 1 1 0 00 0 0 0 1 1

B maps to the 3-axescontrollers[

uz uθ uψ]T

non-zero terms are themax. duty-cycle tovalves based on thesoftware configurationof the NI RIO PWMgenerator



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Results

10 20 30 40 50 60 70 80time (secs)

5

10

15

20

heig

ht (m

m) z-motion correction

z-setpoint

z

10 20 30 40 50 60 70 80time (secs)

0.5

1

1.5

2

pitch angle

(deg)

pitch motion correction

pitch set-point

pitch motion20 40 60 80 100 120 140 160 180

time (secs)

70

72

74

76

78

80

ro

ll (

de

g)

roll motion correction

roll setpoint

roll motion

[Left]: Goal command: (z , θ, φ) = (2.5mm, 0.25◦, 35◦) to(14mm, 1.6◦, 45◦)T . [Right]: Head roll tracking.



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Transition Slide

Page left blank intentionally.



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Steve Webb. Intensity-Modulated Radiation Therapy. Instituteof Physics Publishing Ltd, Bristol and Philadelphia, 2001.

Rodney D Wiersma, Zhifei Wen, Meredith Sadinski, KarlFarrey, and Kamil M Yenice. Development of a framelessstereotactic radiosurgery system based on real-time 6dposition monitoring and adaptive head motioncompensation. Physics in Medicine & Biology, 55(2):389,2009.

L. Xing. Dosimetric effects of patient displacement andcollimator and gantry angle misalignment on intensitymodulated radiation therapy. Radiother Oncol, 56(1):97–108, 2000.

Laura I Cervino, Todd Pawlicki, Joshua D Lawson, and Steve BJiang. Frame-less and mask-less cranial stereotactic



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

radiosurgery: a feasibility study. Physics in medicine andbiology, 55(7):1863, 2010.

Xinmin Liu, Andrew H Belcher, Zachary Grelewicz, andRodney D Wiersma. Robotic Stage for Head MotionCorrection in Stereotactic Radiosurgery. In AmericanControl Conference (ACC), 2015, pages 5776–5781. IEEE,2015.

Mark Ostyn, Thomas Dwyer, Matthew Miller, Paden King,Rachel Sacks, Ross Cruikshank, Melvin Rosario, DanielMartinez, Siyong Kim, and Woon-Hong Yeo. Anelectromechanical, patient positioning system for head andneck radiotherapy. Physics in Medicine & Biology, 62(18):7520, 2017.

Eulalie Coevoet, Adrien Escande, and Christian Duriez.Optimization-based inverse model of soft robots with



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

contact handling. IEEE Robotics and Automation Letters, 2(3):1413–1419, 2017.

James M Bern, Grace Kumagai, and Stelian Coros.Fabrication, modeling, and control of plush robots. InIntelligent Robots and Systems (IROS), 2017 IEEE/RSJInternational Conference on, pages 3739–3746. IEEE, 2017.

Isuru S Godage, Gustavo A Medrano-Cerda, David T Branson,Emanuele Guglielmino, and Darwin G Caldwell. Dynamicsfor variable length multisection continuum arms. TheInternational Journal of Robotics Research, 35(6):695–722,2016.

Federico Renda, Michele Giorelli, Marcello Calisti, MatteoCianchetti, and Cecilia Laschi. Dynamic model of amultibending soft robot arm driven by cables. IEEETransactions on Robotics, 30(5):1109–1122, 2014.



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Rongjie Kang, David T Branson, Emanuele Guglielmino, andDarwin G Caldwell. Dynamic modeling and control of anoctopus inspired multiple continuum arm robot. Computers& Mathematics with Applications, 64(5):1004–1016, 2012.

Federico Renda, Vito Cacucciolo, Jorge Dias, and LakmalSeneviratne. Discrete cosserat approach for soft robotdynamics: A new piece-wise constant strain model withtorsion and shears. In Intelligent Robots and Systems(IROS), 2016 IEEE/RSJ International Conference on, pages5495–5502. IEEE, 2016.

Federico Renda and Lakmal Seneviratne. A Geometric andUnified Approach for Modeling Soft-Rigid Multi-bodySystems with Lumped and Distributed Degrees of Freedom.2018 IEEE International Conference on Robotics andAutomation (ICRA), pages 1567 – 1574, 2018.



adaptivecontrol

approach.

OlalekanOgunmolu


Research Overview

Treatment Types

Frame-based RT

Cyberknife

Drawbacks

BOO

Overview

Search


Results

Solution


Experiment Setup

Kalman Filtering

Sensor Fusion


3DOFControl




Neural NetworkModel

Control gains

Results

References

Juan Carlos Simo and Loc Vu-Quoc. On the dynamics in spaceof rods undergoing large motions—a geometrically exactapproach. Computer methods in applied mechanics andengineering, 66(2):125–161, 1988.


Documents

Robotic Radiotherapy: An optimal, adaptive control approach. › ~opo140030 › media › presentations › … · Radiotherapy: An optimal, adaptive control approach. Olalekan Ogunmolu