Stream flow rate prediction

Stream flow rate prediction

x = weather data

f(x) = flow rate

11

12

13

14

15

16

0 1 2 3 4 5 6Years of Data Transfered

MA

E (

m^

3/s)

No Transfer Wilmot Sharpe Sharpe & Wilmot Shubenacadie

2

Transfer Learning and MTL

Instance Space X

TrainingExamples

TestingExamples

(x, f(x))

Model ofClassifier

h

Inductive Learning Systemshort-term memory

h(x) ~ f(x)

DomainKnowledge

long-term memoryRetention &ConsolidationInductive

Bias SelectionKnowledgeTransfer

3

Transfer Learning and MTL

Instance Space X

TrainingExamples

TestingExamples

(x, f(x))

Model ofClassifier

h

h(x) ~ f(x)

DomainKnowledge

long-term memoryRetention &ConsolidationKnowledge

Transfer

f2(x)

x1 xn

f1(x) fk(x)

Multiple Task Learning (MTL)

InductiveBias Selection

4

Multiple Task Learning (MTL)

f2(x)

x1 xn

f1(x)

Task specificrepresentation

Common internalRepresentation

Common feature layer

fk(x)

Multiple hypotheses develop in parallel within one back-propagation network [Caruana, Baxter, Silver 93-95]

An inductive bias occurs through shared use of common internal representation

Knowledge or Inductive transfer to primary task f1 (x) depends on choice of secondary tasks