Embed Size (px)
Citation preview
A Configurable Deep Network for Clinical Trial Analysis
Jim O’ Donoghue, Mark Roantree, Martin Van Boxtel
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979
International Joint Conference on Neural Networks
12th July 2015, Killarney Ireland
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979
Background + Motivation
Algorithms + The CDN
Experiments + Results
Future Work + Conclusions
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 3
2
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 4
2
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 5
2
High-Dimensional
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 6
2
High-Dimensional
Variable Interactions
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 7
2
High-Dimensional
Variable Interactions
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 8
2
High-Dimensional
Variable Interactions
Hyper-Parameter Selection
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 9
2
High-Dimensional
Variable Interactions
Hyper-Parameter Selection
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 10
4
Class
Connection Weights
Class
Input Features
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 11
5
Connection Weights
Class
Input Features
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 12
6
Connection Weights
Input Features
Learned Features
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 13
7
Class
Connection Weights
Input Features
Learned Features
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 14
8
Data Transform
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 15
8
MySql
File System
Configurable Deep Network Framework
Data Transform
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 16
8
MySql
File System
Grid
Algorithm
Configurable Deep Network Framework
Data Transform
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 17
8
MySql
File System
Grid
Algorithm
Configurable Deep Network Framework
e1, e2, … , en
Data Transform
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 18
8
MySql
File System
Grid
Algorithm
Configurable Deep Network Framework
Query
e1, e2, … , en
Data Transform
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 19
8
MySql
File System
Grid
Algorithm
Configurable Deep Network Framework
Final Model
Query
e1, e2, … , en
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 20
9
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 21
10
CDN
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 22
11
Total: (3,441 X 1,835) 14 Study Sub-Sections Subset Chosen: (523 X 556) Forgetful – yes/no
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 23
12
To Choose:
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 24
12
To Choose:
learning rate α
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 25
12
To Choose:
learning rate α weight decay term λ
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 26
12
To Choose:
learning rate α weight decay term λ training iterations t
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 27
13
The Grid:
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 28
13
The Grid:
α, λ: [0.001, 0.003, 0.009, … , 0.1, 0.3, 0.9]
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 29
13
The Grid:
α, λ: [0.001, 0.003, 0.009, … , 0.1, 0.3, 0.9]
t: [100, 1000, 10000]
0
5
10
15
20
25
30
35
40
45
50
Valid. Cost
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 30
14
100 1,000
10,000
0.3046
Training Iterations
Categorical Continuous Lambda 0.009 0.003 0.001
Alpha 0.9 0.3 0.09 0.003
0
5
10
15
20
25
30
35
40
45
50
Valid. Cost
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 31
14
100 1,000
10,000
0.3046 0.2815
Training Iterations
Categorical Continuous Lambda 0.009 0.003 0.001
Alpha 0.9 0.3 0.09 0.003
0
5
10
15
20
25
30
35
40
45
50
Valid. Cost
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 32
14
100 1,000
10,000
0.3046 0.2815
Training Iterations
Categorical Continuous Lambda 0.009 0.003 0.001
Alpha 0.9 0.3 0.09 0.003
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 33
17
To Choose:
Last layer nodes h(1)n
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 34
17
To Choose:
Last layer nodes h(1)n
The Grid:
[10, 30, 337, 900, 1300, 2000]
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 35
15
Parameter Initialisation:
− 4 6
𝑓𝑎𝑛_𝑖𝑛 + 𝑓𝑎𝑛_𝑜𝑢𝑡, + 4
6
𝑓𝑎𝑛_𝑖𝑛 + 𝑓𝑎𝑛_𝑜𝑢𝑡
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 36
18
0
10
20
30
40
50
60
70
80
90
Valid. Cost
Categorical Continuous
Nodes 10 30 100 337 900 1300 2000
0.232
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 37
18
0
10
20
30
40
50
60
70
80
90
Valid. Cost
Categorical Continuous
Nodes 10 30 100 337 900 1 1300 2000
0.232 0.291
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 38
18
0
10
20
30
40
50
60
70
80
90
Valid. Cost
Categorical Continuous
Nodes 10 30 100 337 900 1 1300 2000
0.232 0.291
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 39
19
Lambda @ 0.03
10 200 3567
10 200 3567
10 200 3567
10 2000 200 3567
10 200 3567
10 200
10 30
10 100 337
Alpha 0.001 0.01 0.9 Steps 3000 1000 100
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
10 337
10 100 337
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 40
21
Extensible network -> easier modelling Constituent models can be used to select a starting point for deep learning configurations
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 41
20
Gaussian units More accurate Inference Mapping learned features to original
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 42
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 43
![MPC8548E Configurable Development System … Configurable Development System Reference Manual, ... [4:0] ... MPC8548E Configurable Development System Reference Manual,](https://img.pdfslide.us/doc/110x75/5af028337f8b9ac62b8e4c0e/mpc8548e-configurable-development-system-configurable-development-system-reference.jpg)
