• Home

  • Documents

  • Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text,...
107
Deep Learning and Sta.s.cs: Connec.ons Padhraic Smyth Chancellor’s Professor Departments of Computer Science and Sta.s.cs University of California, Irvine [email protected]

Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

  • Upload
    others

  • View
    0

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

DeepLearningandSta.s.cs:Connec.ons

PadhraicSmythChancellor’sProfessorDepartmentsofComputerScienceandSta.s.csUniversityofCalifornia,[email protected]

Page 2: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:2

AIResearch10to20yearsago

LogicandAutomatedReasoning

KnowledgeRepresenta.on

MachineLearningNatural

LanguageProcessing

SpeechRecogni.on

ComputerVision

GamePlaying

SearchAlgorithms

Robo.cs

Page 3: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:3

AIResearchin2019

LogicandAutomatedReasoning

KnowledgeRepresenta.on

NaturalLanguageProcessing

SpeechRecogni.on

ComputerVision

GamePlaying

SearchAlgorithms

Robo.cs

DeepMachineLearning

Page 4: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:4

(Russakovskyetal,2015)ImageNetLargeScaleVisualRecogniBonChallenge

Page 5: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:5

FigurefromKevinMurphy,Google,2016

Page 6: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:6

FigurefromKevinMurphy,Google,2016

Deepneuralnetworks

Page 7: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:7

Page 8: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:8

DeepNetworksforDetecBngSkinCancer

FromEstevaetal,Nature,2017

Page 9: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:9

Microsoft/IBM Benchmarks for Speech Recognition

Source: https://www.economist.com/node/21710907/sites/all/modules/custom/ec_essay

Page 10: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:10

Microsoft/IBM Benchmarks for Speech Recognition

Source: https://www.economist.com/node/21710907/sites/all/modules/custom/ec_essay

Page 11: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:11

2016 IEEE Conference on Acoustics, Speech, and Signal Processing

Page 12: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:12

FromKodish-Wachsetal,AMIASymposium,2018

Page 13: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:13

PedestrianDetecBon:AlgorithmsandHumans

Algorithms

HumanAnnotators

FromZhangetal,CVPR2016

Page 14: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:14

APerspecBveonDeepLearning•  Deeplearning(DL)research:

–  Highvisibilitysuccessesinvision,speech,text,game-playing–  Fundingagencies,companies,students,publicareinaweofdeeplearning–  Companiesaredrivingalotoftheinterest–  Highlyempirical–li[leguidancefromtheory–  Fewlinks(todate)tostaBsBcsorstaBsBcalthinking

•  Academicresearchcanplayakeyrole–  Computerscience,staBsBcs,mathemaBcs,etc–  Provideguidance:wheredoesDLworkwell?Andnotsowell?

•  ObjecBveempiricalanalyses•  Developmentofprinciplesandtheory

–  Providebalancetothe“hype”

Page 15: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:15

OutlineofToday’sTalk

•  Keyideasindeeplearning

•  LinkstostaBsBcalthinking

•  LimitaBonsofcurrentdeeplearning

•  OpportuniBesfornewideasanddirecBons

Page 16: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:16

PredicBveModeling

f=blackbox predicBonoftargetyinputsx

parametersθ

Goalistolearnamodelfromtrainingdatatopredictyvalues

Machinelearning:emphasisonpredic.onsofySta.s.cs:emphasisonmodelsandparameters

Page 17: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:17

Training Data: D = {xi, yi}, i = 1, . . . , N

Model: yi ⇡ f(xi;✓✓✓)

d-dimensionalinputvector

targetvalue

funcBonalformofthemodel

p-dimensionalparametervector(unknown)

Loss: �

�yi, f(xi;✓✓✓)

�

Model’sPredicBon

IdealTarget

Page 18: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:18

TheThreeComponentsofPredicBveModeling

1.Predic.onModelf:WhatfuncBonalformshouldwechooseforf?

2.LossFunc.onHowdowecomparef’spredicBonstotruey?

3.Op.miza.onGivenfandalossfuncBon,howcanwelearnf’sparameters

Page 19: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:19

TheThreeComponentsofPredicBveModeling

1.Predic.onModelf:WhatfuncBonalformshouldwechooseforf?

2.LossFunc.onHowdowecomparef’spredicBonstotruey?

3.Op.miza.onGivenfandalossfuncBon,howcanwelearnf’sparameters

Page 20: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:20

ExamplesofPredicBonModels

-100 -50 0 50 1000

0.2

0.4

0.6

0.8

1

LinearRegression

Logis.cRegression

f(x;✓✓✓) = ✓0 + ✓1x1 + ✓2x2 + . . .+ ✓dxd

=dX

j=0

✓jxj = ✓

✓

✓

Tx

f(x;✓✓✓) = P (y = 1|x;✓✓✓)

=1

1 + e�z, z = ✓✓✓Tx

z

1

1 + e�z

Page 21: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:21

LogisBcRegressionasaSimpleNeuralNetwork

x1

x2

x3

+1

Each“edge”inthenetworkhasanassociatedweightorparameter,θj

f(x;✓✓✓) = P (y = 1|x;✓✓✓)

=1

1 + e�z, z = ✓✓✓Tx

Page 22: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:22

ANeuralNetworkwithOneHiddenLayer(from1990’s)

Herethemodellearns3differentlogisBcfuncBons,eachonea“hiddenunit”andthencombinestheoutputsofthe3tomakeapredicBonMorecomplexthanlogisBcfuncBon,manymoreparameters

x1

x2

x3

+1HiddenLayer

Output

Inputs

f(x;✓✓✓)

Page 23: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:23

DeepLearning:ModelswithMoreHiddenLayers

Usethisideatorecursivelybuild“deepmodels”withmulBplehiddenlayers

x1

x2

x3

+1

Veryflexible,highlynon-linearfuncBonsCanhavedifferenttypesofnon-lineariBes,skiplayers,etc

HiddenLayer1

HiddenLayer2

Output

Inputs

f(x;✓✓✓)

Page 24: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:24

Figurefromh[p://parse.ele.tue.nl/

ExampleofaNetworkforDigitClassificaBon

MathemaBcallythenetworkisjustadifferenBablefuncBon…butaverycomplicatedone

EachoutputisanesBmateofaclassprobabilityP(c=k|x),implementedviaamulBnomiallogisBcfuncBon

Inputpixels,nofeatureextracBon

Page 25: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:25

DeepNetworkarchitectureforGoogLeNetimagerecogni.onnetwork27layers,millionsofparameters

PixelInputs

Output

Page 26: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:26

ABriefHistoryofNeuralNetworks…•  ThePerceptronEra:1950sand60s

–  GreatopBmismwithperceptrons(linearmodels)....–  ...unBlMinsky,1969:perceptronshadlimitedrepresentaBonpower–  Hardproblemsrequirehiddenlayers....buttherewasnotrainingalgorithm

•  TheBackpropaga.onEra:Late1980stomid-90’s–  InvenBonofbackpropagaBon–trainingofmodelswithhiddenlayers–  Wildenthusiasm(intheUSatleast)....conferences,funding,etc–  Mid1990’s:enthusiasmdiesout:trainingdeepNNsishard

•  TheDeepLearningEra:2010-present–  3rdwaveofneuralnetworkenthusiasm–  Whathappenedsincemid90’s?

•  NowpracBcaltotraindeepnetworks• MuchlargerdatasetsandgreatercomputaBonalpower•  FastopBmizaBontechniques+othergoodengineeringtricks

Page 27: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:27

FigureadaptedfromEfronandHasBe,Computer-AidedStaBsBcalInference,2016

Page 28: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:28

FeatureextracBonFigureadaptedfromEfronandHasBe,Computer-AidedStaBsBcalInference,2016

Page 29: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:29

FigureadaptedfromEfronandHasBe,Computer-AidedStaBsBcalInference,2016

Rec.fiedLinearUnit(ReLu)Connec.onswithlinearsplines(e.g.,EckleandSchmidt-Heiber,2018)

Page 30: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:30

FeatureextracBon LogisBcModelFigureadaptedfromEfronandHasBe,Computer-AidedStaBsBcalInference,2016

Page 31: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:31

MachineLearningbeforeDeepModels

FigurefromMarc’Aurelio-Ranzato

Page 32: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:32

DeepConvoluBonalNetwork

FigurefromPeemanetal,2012

Keypoint:end-to-enddifferen.abilityallowsfeatures(convolu.onalfilters)tobelearned,removesneedforhand-cradedfeatureextrac.on(densewordembeddingsplaythesamerolefortext)

Page 33: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:33

ConvoluBonalFiltersforImageData

FigurefromMarc’Aurelio-Ranzato

Keyidea:LearnsuchfiltersinadiscriminaBvefashion

Page 34: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:34

ExamplesofLearnedSpaBalFiltersinPixelSpace

Page 35: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:35

GeneralizedLinearModels

(e.g.,logisBc)

Page 36: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:36

RecursiveGLMs

Define a latent feature via a GLM:

[Mohamed, 2015; Tran et al, 2018]

Page 37: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:37

Define GLM on latent feature:

RecursiveGLMs

Define a latent feature via a GLM:

[Mohamed, 2015; Tran et al, 2018]

Page 38: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:38

BuildingNeuralNetsfromRecursiveGLMs

Page 39: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:39

BuildingNeuralNetsfromRecursiveGLMs

Page 40: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:40

BuildingNeuralNetsfromRecursiveGLMs

HIDDENUNIT/NEURON

HIDDENLAYER

ACTIVATIONFUNCTION

WEIGHTMATRIX

Page 41: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:41

DeepNeuralNetworks

Page 42: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:42

DeepNeuralNetworks

……..

Page 43: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:43

DeepNeuralNetworks

……..

Page 44: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:44

DeepNeuralNetworks

…....ineffectdoingregressionwithlearnedfeaturesorbasisfuncBons

…..

Feature Extractor Statistical Model

NEW REPRESENTATION OF FEATURES COMPUTED BY NN LAYERS

Page 45: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:45

DeepNeuralNetworkRepresentaBons•  Itmaybeusefultoviewdeepnetworksastrainablefeatureextractorswith

staBsBcalmodelsas“back-ends”’•  ThisviewencouragesthemixingofdeterminisBcDNNrepresentaBons

(“embeddings”)withconvenBonalstaBsBcalmodels

•  Examples–  Deepsurvivalmodels–  Neuralpointprocessmodels–  RecurrentnetworkmodelsforBme-series

Page 46: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:46

RecurrentNetworksandState-SpaceModels

Page 47: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:47

RecurrentNetworksandState-SpaceModels

RNNstructureissimilartostate-spacemodelsinstaBsBcse.g.,Kalmanfilters,hiddenMarkovmodels,andsoon

RNN:nodistribuBonalassumpBonsonstatevariables->moreflexibilityState-spaceapproach:be[ercharacterizaBonofuncertainty

Page 48: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:48

TheThreeComponentsofPredicBveModeling

1.Predic.onModelf:WhatfuncBonalformshouldwechooseforf?

2.Lossfunc.onHowdowecomparef’spredicBonstoy?

3.Op.miza.onGivenfandalossfuncBon,howcanwelearnf’sparameters

Page 49: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:49

Loss: �

�yi, f(xi;✓✓✓)

�

Model’sPredicBon

IdealTarget

Page 50: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:50

Loss: �

�yi, f(xi;✓✓✓)

�

Example: Squared Error � =

�yi � f(xi;✓✓✓)

�2

Example: Log Loss � = log

1

P (yi|xi;✓✓✓)

Page 51: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:51

Loss: �

�yi, f(xi;✓✓✓)

�

Example: Squared Error � =

�yi � f(xi;✓✓✓)

�2

Example: Log Loss � = log

1

P (yi|xi;✓✓✓)

Page 52: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:52

Empirical Loss:

L(✓✓✓) =

NX

i=1

�

�yi, f(xi;✓✓✓)

�

funcBonalformofthemodel

p-dimensionalparametervector(unknown)

sumovertrainingdatapoints

FocusisongetngpointesBmatesofθ,byminimizaBonofriskSimplemodels:lossisconvex,opBmizaBoncanbestraighuorwardDeepnetworkmodels:thelossisnon-convex,difficulttoopBmize

EmpiricalRisk

Page 53: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:53

TheThreeComponentsofPredicBveModeling

1.Predic.onModelf:WhatfuncBonalformshouldwechooseforf?

2.Lossfunc.onHowdowecomparef’spredicBonstoy?

3.Op.miza.onGivenfandalossfuncBon,howcanwelearnf’sparameters

Page 54: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:54

GradientDescent

Scalarlearningrate:Howfarwemove

Vectorgradient:Direc.onwemoveUpdatedp-dimensional

parametervector

Currentparametervector

Simplegradientmethodsarethe“workhorse”ofmachinelearningNewton(2ndorder)methodsarerarelyused

……….requiresinversionofp x p Hessianmatrix,O(p3)

✓(k+1) = ✓(k) � � rL(✓)

Page 55: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:55

rL(✓) =NX

i=1

rLi(✓)

rL(✓) ⇡ N

m

mX

j=1

rLj(✓)

FullGradient:

Stochas.cGradient:

ApproximaBonofthefullgradient

Randomsampleofmdatapoints(“mini-batch”)

IntuiBon:form << N,wecanmakemanyfastnoisyupdatesCanleadtosublinearconvergenceforlargeN

Page 56: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:56

StochasBcGradientin2dParameterSpace

Gradientsteps

Stochas.cgradientsteps

Empiricallyworksverywellonlargedatasets:sometheoreBcalsupportAnapplicaBonofRobbins-Monro(1951)stochasBcapproximaBonmethodUsefulforstaBsBcalmodelfitngingeneral(notjustfordeeplearning)

e.g.,Wangetal,2015;Chenetal,2016

Page 57: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:57

CONNECTIONSTOSTATISTICS

Page 58: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:58

TheThreeComponentsofPredicBveModeling

Model+LossFunc.on+Op.miza.onMethod

Thefunc.onalformoff

Howwemeasurethequalityofthemodel’spredic.ons

Thealgorithmthatfindstheparametersthatminimizeempiricalrisk

Deeplearningwaspresentedasanop.miza.onproblemWhereissta.s.cslurking?

Page 59: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:59

Empirical Loss with Regularization:

L(✓✓✓) =

NX

i=1

�

�yi, f(xi;✓✓✓)

�+ �R(✓✓✓)

EmpiricalRiskMinimizaBon

FindtheparametersthatminimizeempiricalriskontrainingdataThisdirectlycorrespondstomaximizinglikelihood:

Squarederrorloss->GaussianlikelihoodforregressionLogloss->binomial/mulBnomiallikelihoodforclassificaBon

ImplicaBon:implicitcondiBonalindependenceassumpBonoverdata

Page 60: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:60

Empirical Loss with Regularization:

L(✓✓✓) =

NX

i=1

�

�yi, f(xi;✓✓✓)

�+ �R(✓✓✓)

StrengthofregularizaBon

RegularizaBononparameters

EmpiricalRiskMinimizaBonwithRegularizaBon

TheregularizaBontermcanbeinterpretedas(minus)alogprior RL2(✓✓✓) =

X✓2j

RL1(✓✓✓) =X

|✓j |

InaddiBon,DLtechniquessuchasdropoutcanbeinterpretedasaformofprior–generalizestoabroad“dropoutfamily”(seeBhadraetal,ArXiv2019andNalisnicketal,ICML2019)

Gaussianprior

Laplacianprior

Page 61: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:61

Lookslikeadeterminis.cproblem?

Page 62: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:62

Lookslikeadeterminis.cproblem?

Page 63: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:63

Lookslikeadeterminis.cproblem?

Page 64: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:64

Minimizedbysetngf(x;θ)toE[y|x],ateveryx

Lookslikeadeterminis.cproblem?

Page 65: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:65

Minimizedbysetngf(x;θ)toE[y|x],ateveryx

Lookslikeadeterminis.cproblem?

Conclusion:op.miza.onproblemisreallyasta.s.cales.ma.onproblem

Page 66: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:66

TheBias-VarianceTradeoff

Expectedfutureerror=ModelBias2+ModelVariance+IntrinsicUncertaintyApproximaBon

errorEsBmaBon

errorLowerbound

Note:thedecomposi.onaboveisop.mis.c:assumesfuturedataisfromsamedistribu.onastrainingdata

Page 67: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:67

Page 68: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:68

FromNeal,Mi[al,etal,ArXiv,2019

UnexpectedBias-VarianceTrendswithDNNs

Page 69: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:69

ClassProbabiliBesForbothMSEandlog-losstheopBmalpredicBonatanyxisE[y|x]

ForK-aryclassificaBon,yisaK-dimensionalindicatorvector

i.e.,theopBmalpredictorforclasskistheprobabilityofthatclassSodeepnetworkswillproduceesBmatesofclassprobabiliBes…intheory,givenenoughdataandassumingnolocalminima(Note;thisisapropertyofthelossfuncBon,notdeepnetworks)

E[yk|x] = 1 P (yk = 1|x) + 0 P (yk = 0|x)= P (yk = 1|x)

Page 70: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:70

ExampleofTestBedData:CIFARImageClassificaBon•  Anexampleofawidelyused

datasetindeeplearningresearch–  Upto100classes–  50,000imagesfortraining–  10,000imagesfortest

•  Studiesongeneraliza.on,op.miza.on,etc,odenusethisdataset

Page 71: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:71

DeepNetworksareozenMiscalibrated(CIFARdata)

PredictedasTigerwithP(y|x)=0.99

PredictedasTelevisionwithP(y|x)=0.99

Page 72: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:72

DeepNetworksareozenMiscalibrated(CIFARdata)

NetworkofDepth5 NetworkofDepth110

FigurefromGuoetal,ICML2017

PredictedasTigerwithP(y|x)=0.99

PredictedasTelevisionwithP(y|x)=0.99

Page 73: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:73

Page 74: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:74

…….

ExpectedlosswithrespecttoP(x)…forthetrainingdata

Page 75: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:75

2 4 6 8 10 12 14

X values

4

6

8

10

12

14

16

Y va

lues

True E[y|x] functionObserved data

2 4 6 8 10 12 14

X values

4

6

8

10

12

14

16

Pred

icte

d an

d tr

ue Y

val

ues Model 95% confidence intervals

Model prediction for E[y|x]True E[y|x] function

WhatwillhappenwhenweextrapolatebeyondP(x)?

Page 76: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:76

2 4 6 8 10 12 14

X values

4

6

8

10

12

14

16

Y va

lues

True E[y|x] functionObserved data

2 4 6 8 10 12 14

X values

4

6

8

10

12

14

16

Pred

icte

d an

d tr

ue Y

val

ues Model 95% confidence intervals

Model prediction for E[y|x]True E[y|x] function

WhatwillhappenwhenweextrapolatebeyondP(x)?

Page 77: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:77

FromTatemetal.,Nature2004.(seealsoresponsele[ersath[p://faculty.washington.edu/kenrice/naturele[er.pdf)

Generalizingfrom100mOlympicWinningTimes

Page 78: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:78

FigurefromKevinMurphy,Google,2016

Deepneuralnetworks

Howwelldothesemodelsextrapolatetonewtypesofimages?

Page 79: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:79

Page 80: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:80

FromRechtetal,ICML2019

AccuracyofImageNetClassifiersonNewImageNetData

Page 81: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:81

Page 82: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:82

ADeepNeuralNetworkforImageRecogniBonFromNguyen,Yosinski,Clune,CVPR2015

Page 83: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:83

ADeepNeuralNetworkforImageRecogniBon

ImagesusedforTraining NewImages

FromNguyen,Yosinski,Clune,CVPR2015

Page 84: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:84

ADeepNeuralNetworkforImageRecogniBonFromNguyen,Yosinski,Clune,CVPR2015

Page 85: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:85

ADeepNeuralNetworkforImageRecogniBonFromNguyen,Yosinski,Clune,CVPR2015

Page 86: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:86

0 10 20 30 40 50 60 70 80 900

2000

4000

6000

8000

10000

12000

14000

AGE

MO

NTHL

Y IN

COM

E

DecisionBoundary

Poorextrapola.onfortestpointslikethis….

Page 87: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:87

Non-RobustnessinDeepImageClassificaBon

FigurefromEngstrometal,ICML2019

Page 88: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:88

ExternalversusInternalValidaBonFromZechetal.,PLOSMedicine,2018

AUCsontestdatafromhospitalnotusedinmodeltraining(“external)

AUCsontestdatafromhospitalsusedinmodeltraining(“internal”)

Page 89: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:89

ExternalversusInternalValidaBonFromZechetal.,PLOSMedicine,2018

AUCsontestdatafromhospitalnotusedinmodeltraining(“external)

AUCsontestdatafromhospitalsusedinmodeltraining(“internal”)

Page 90: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:90

BayesianAssessmentofBlackBoxModels

•  Scenario–  Black-boxpredicBon(e.g.,neuralnetwork)hasbeentrained,parametersarefixed,wecanonlyquerythemodel

–  Wewishtoevaluateitsperformance(accuracy,calibraBon,precision,etc)onlineinanewenvironment

(NewworkinSmyth/SteyversgroupatUCIrvine)

ResultswithdeepnetworksonCIFARimageclassifica.on

Page 91: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:91

BayesianAssessmentofBlackBoxModels

•  Scenario–  Black-boxpredicBon(e.g.,neuralnetwork)hasbeentrained,parametersarefixed,wecanonlyquerythemodel

–  Wewishtoevaluateitsperformance(accuracy,calibraBon,precision,etc)onlineinanewenvironment

(NewworkinSmyth/SteyversgroupatUCIrvine)

N=100queries N=500queries N=10,000queries

ResultswithdeepnetworksonCIFARimageclassifica.on

Page 92: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:92

BayesianAssessmentofAccuracyandCalibraBon(OngoingworkinSmyth/SteyversgroupatUCIrvine)

ResultsonCIFARimageclassificaBondataset

Page 93: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:93

BayesianAssessmentviaRankingandAcBveLearning(OngoingworkinSmyth/SteyversgroupatUCIrvine)

palm treewardrobe

motorcyclesunflowerkeyboard

Most Accurate

0.0 0.2 0.4 0.6

lizardseal

ottershrew

boy

Least Accurate

Bayesianrankingbypredictedclass

ClasswithLeastAccuratePredicBons

ClasswithLeastCalibratedPredicBons

Bayesianac.velearning

Page 94: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:94

THEOVERFITTINGQUESTION

Page 95: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:95

FromPoggioetal,2018:TheoryofdeeplearningIII:thenon-overfitngpuzzle

LackofOverfitngofDeepNetworksonCIFAR-10

Moreparametersthandata

Page 96: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:96

LackofOverfitng:DifferentNetworks,DifferentData

FromNeyshaburetal,2018;TowardsunderstandingtheroleofoverparametrizaBoningeneralizaBonofneuralnetworks

CIFAR-10Data SVHNData

MNISTData

Page 97: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:97

OverfitngintheDLLiterature•  Standardbias-variancetheoryseemsnottoapply

–  DLmodelscaninterpolatethedata(zerotrainingerror)butsBllgeneralizewellontestdata

•  Trainingerror(orloss)tendstoodenbemuchlowerthantesterror–  ThisistradiBonallyanindicatorofoverfitng….butnothere

•  Variousemergingconjecturesandtheories–  e.g.minimum-norminterpolatorsgeneralizewellinoverparametrizedregime(seeBelkinetal(2018,2019),HasBeetal(2019)

…..butverymuchsBllanopenproblem

Page 98: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:98

The“DoubleDescent”TheoryBelkinetal.,Reconcilingmodernmachinelearningandthebias-variancetradeoff,2018

Page 99: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:99

“DoubleDescent”onMNISTData

RecentworkfromstaBsBcsthatconfirmsthesetheories:HasBeetal,ArXiv,2019

Belkinetal.,Reconcilingmodernmachinelearningandthebias-variancetradeoff,2018

Page 100: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:100

CONCLUDINGCOMMENTS

Page 101: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:101

CauBonaryNotesaboutDeepLearning

•  Verylargeamountsoflabeleddataneeded(forclassificaBonproblems)

•  ExtrapolaBonproperBesareunpredictable

•  ModelbuildingandopBmizaBoncanbecomplex(significanthumaneffort)

•  InterpretabilityandexplanaBonaredifficult•  Relianceonempirical“folkwisdom”ratherthanprinciplesandtheory

Page 102: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:102

QuesBonsworthaskingforAIApplicaBons

1.  Ismachinelearninganappropriateapproach?

2.  Ifso,isdeeplearningthebestapproach?

3.  HowdowebuildmodelsthatgeneralizewelltonewsituaBons?

Page 103: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:103

Scullleyetal,NIPS2015Conference

Page 104: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:104

ConcludingComments

•  Deeplearninghasachievedimpressiveresultsinpamernrecogni.on–  ParBcularlyusefulwithhigh-dimensionalsignals(images,speech,text)

•  Manyfounda.onalideasaregroundedinsta.s.cs(includingothertopicswedidnotdiscuss:fairness,adversarial/robustlearning,reinforcementlearning,…)

Page 105: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:105

ConcludingComments

•  Deeplearninghasachievedimpressiveresultsinpamernrecogni.on–  ParBcularlyusefulwithhigh-dimensionalsignals(images,speech,text)

•  Manyfounda.onalideasaregroundedinsta.s.cs(includingothertopicswedidnotdiscuss:fairness,adversarial/robustlearning,reinforcementlearning,…)

•  However,deeplearninghasblindspots

–  e.g.,reportedempiricalaccuraciesmaybeopBmisBc

•  Asdeepmachinelearningisappliedmorebroadlyweneed–  Robustprinciplesandtheorytoguidemodel-building–  ObjecBvediagnosisandevaluaBonmethodsforpracBBoners

Page 106: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:106

THANKYOUFORLISTENINGQUESTIONS?

Page 107: Deep Learning and Stascs: Connecons · – High visibility successes in vision, speech, text, game-playing – Funding agencies, companies, students, public are in awe of deep learning

PadhraicSmyth:MonashUniversity,July2019:107

AddiBonalReading•  Efron,Bradley,andTrevorHasBe.ComputerAgeSta;s;calInference.Cambridge

UniversityPress,2016.(Chapter18:NeuralNetworksandDeepLearning).

•  Goodfellow,Ian,YoshuaBengio,AaronCourville.DeepLearning.Cambridge:MITPress,2016

•  Jordan,MichaelI.,andTomM.Mitchell.Machinelearning:Trends,perspecBves,and

prospects.Science349.6245(2015):255-260.

•  Taddy,Ma[.TheTechnologicalElementsofAr;ficialIntelligence.No.w24301.NaBonalBureauofEconomicResearch,2018.

•  Brynjolfsson,Erik,andTomMitchell.Whatcanmachinelearningdo?WorkforceimplicaBons.Science358.6370(2017):1530-1534.

•  Breiman,L.(2001).StaBsBcalmodeling:Thetwocultures.Sta;s;calScience,16(3),199-231.


Adap3ve)MediaStreaming) over)Emerging)Protocols) · Conclusions • DASH/2.0) – HTTP/2.0)(SPDY))~HTTP/1.1)with) persistent connecons & requestpipelining) – Implicitly)solves)HeadKofKLineblockingproblem

Adap3ve)MediaStreaming) over)Emerging)Protocols) · Conclusions • DASH/2.0) – HTTP/2.0)(SPDY))~HTTP/1.1)with) persistent connecons & requestpipelining) – Implicitly)solves)HeadKofKLineblockingproblem

Documents
Deep Learning Lab Course 2017 (Deep Learning …ais.informatik.uni-freiburg.de/teaching/ws17/deep...Deep Learning Lab Course 2017 (Deep Learning Practical) Labs: (Computer Vision)

Deep Learning Lab Course 2017 (Deep Learning …ais.informatik.uni-freiburg.de/teaching/ws17/deep...Deep Learning Lab Course 2017 (Deep Learning Practical) Labs: (Computer Vision)

Documents
Complaints StascsComplaints Stascs Khyber Pakhtunkhwa Informaon Commission Date: 02-03-2020 Stascs Date: 01-07-2019 AND 02-03-2020 Days: All S.no Number Date Name Department District

Complaints StascsComplaints Stascs Khyber Pakhtunkhwa Informaon Commission Date: 02-03-2020 Stascs Date: 01-07-2019 AND 02-03-2020 Days: All S.no Number Date Name Department District

Documents
CS 149: Understanding Stascs Using Baseballemeyers.scripts.mit.edu/emeyers/wp-content/uploads/CS149... · 2017-08-25 · Analyzing Baseball Data with R • Available as an e-book

CS 149: Understanding Stascs Using Baseballemeyers.scripts.mit.edu/emeyers/wp-content/uploads/CS149... · 2017-08-25 · Analyzing Baseball Data with R • Available as an e-book

Documents
Insightson Connecons...Editorial)! Over%the%nextfew%days%people%from%all%across%the% world%will%be%converging%on%Orlando%for%IBM Connect2014.%The%fiveCday%conference%is%an%

Insightson Connecons...Editorial)! Over%the%nextfew%days%people%from%all%across%the% world%will%be%converging%on%Orlando%for%IBM Connect2014.%The%fiveCday%conference%is%an%

Documents
Deep Learning Early Work Why Deep Learning Stacked Auto Encoders Deep Belief Networks CS 678 – Deep Learning1

Deep Learning Early Work Why Deep Learning Stacked Auto Encoders Deep Belief Networks CS 678 – Deep Learning1

Documents
Deep Learning for Laser Based Odometry Estimation - …juxi.net/workshop/deep-learning-rss-2016/papers/Nicolai - Deep... · Deep Learning for Laser Based Odometry Estimation ... deep

Deep Learning for Laser Based Odometry Estimation - …juxi.net/workshop/deep-learning-rss-2016/papers/Nicolai - Deep... · Deep Learning for Laser Based Odometry Estimation ... deep

Documents
HR Analycs...(Bangalore), Loyola Instute of Business Administraon and Chennai Business School. His teaching at MBA instutes focuses on Business Stascs, Quantave Methods and HR …

HR Analycs...(Bangalore), Loyola Instute of Business Administraon and Chennai Business School. His teaching at MBA instutes focuses on Business Stascs, Quantave Methods and HR …

Documents
Deep Energy - · PDF file2. Deep Energy. Deep Energy. DEEP ENERGY. NASSAU. DEEP ENERGY DEEP ENERGY. The Deep Energy is Technip’s latest state-of

Deep Energy - · PDF file2. Deep Energy. Deep Energy. DEEP ENERGY. NASSAU. DEEP ENERGY DEEP ENERGY. The Deep Energy is Technip’s latest state-of

Documents
20151216 Computer Vision Meetup Deep Learning Deep Learning - Rene...Deep Learning . René Donner Deep Learning Overview 2 ... coursera ... 20151216 Computer Vision Meetup Deep Learning

20151216 Computer Vision Meetup Deep Learning Deep Learning - Rene...Deep Learning . René Donner Deep Learning Overview 2 ... coursera ... 20151216 Computer Vision Meetup Deep Learning

Documents
ursi2013 deboer pdfVersion - National Radio Astronomy ... · • Phaseddevelopmentoftechniques/ instrumentaonforredshiedCO – piggybacktohigherz – upperlimits>detecon>stascs>imaging

ursi2013 deboer pdfVersion - National Radio Astronomy ... · • Phaseddevelopmentoftechniques/ instrumentaonforredshiedCO – piggybacktohigherz – upperlimits>detecon>stascs>imaging

Documents
FELLOWSHIP IN COSMETIC GYNECOLOGY - Fellowship in Cos… · August 22 - 24, 2019 | Dubai, UAE COURSE CONTENTS l History and Stascs. l Consultaon and Paent Selecon. l Paent Pain and

FELLOWSHIP IN COSMETIC GYNECOLOGY - Fellowship in Cos… · August 22 - 24, 2019 | Dubai, UAE COURSE CONTENTS l History and Stascs. l Consultaon and Paent Selecon. l Paent Pain and

Documents
Stascs for Genomics (140.688) - Biostatistics - Departments - …jleek/teaching/genomics/flow... ·  · 2010-05-05Publicaons using the keyword “ flow cytometry” from 7:36 AM

Stascs for Genomics (140.688) - Biostatistics - Departments - …jleek/teaching/genomics/flow... ·  · 2010-05-05Publicaons using the keyword “ flow cytometry” from 7:36 AM

Documents
Using&Mul*path&TCP&to&enable& …ew2017.european-wireless.org/wp-uploads/2017/05/... · Mul/path!Networks! Mobile devices have mulplewireless& connecons&& Mul/path!Networks! ... MPTCP

Using&Mul*path&TCP&to&enable& …ew2017.european-wireless.org/wp-uploads/2017/05/... · Mul/path!Networks! Mobile devices have mulplewireless& connecons&& Mul/path!Networks! ... MPTCP

Documents
ACKNOWLEDGMENTS...Survey (MICS 2009) by the Bangladesh Bureau of Stascs (BBS), with parcipaon from the Department of Public Health and Engineering. Compared to previous surveys, MICS

ACKNOWLEDGMENTS...Survey (MICS 2009) by the Bangladesh Bureau of Stascs (BBS), with parcipaon from the Department of Public Health and Engineering. Compared to previous surveys, MICS

Documents
HIGHLANDS & ISLANDS CONNECTIONS · Highlands and Islands Connecons - a pilot project exploring community based mental health area. The Highlands and Islands' Project has taken a community-led,

HIGHLANDS & ISLANDS CONNECTIONS · Highlands and Islands Connecons - a pilot project exploring community based mental health area. The Highlands and Islands' Project has taken a community-led,

Documents
Stascal Thinking at the Introductory Level Stascs and ... › ~nolan › talks › beijing › StatThinking-4pp.pdfUnited States v. Kristen Gilbert (adapted from Cobb and Gehlbach,

Stascal Thinking at the Introductory Level Stascs and ... › ~nolan › talks › beijing › StatThinking-4pp.pdfUnited States v. Kristen Gilbert (adapted from Cobb and Gehlbach,

Documents
How Deep is Deep Ecology?

How Deep is Deep Ecology?

Documents
Product data sheet NORVI IIOT...*AE-02 / AE-03 Models Only + 0-10V 0-10V 0-10V ESP32 WROOM IIOT n GND 24+ ADS1115 connecons IC Type ADS 1115 Communication I2C IO16 - IO17 Module Address

Product data sheet NORVI IIOT...*AE-02 / AE-03 Models Only + 0-10V 0-10V 0-10V ESP32 WROOM IIOT n GND 24+ ADS1115 connecons IC Type ADS 1115 Communication I2C IO16 - IO17 Module Address

Documents
growth - African Union · • 6th Internaonal UNWTO Conference on Tourism Stascs “Measuring Sustainable Tourism”, Philippines, 21-23 June 2017 • 2nd World Conference on Tourism

growth - African Union · • 6th Internaonal UNWTO Conference on Tourism Stascs “Measuring Sustainable Tourism”, Philippines, 21-23 June 2017 • 2nd World Conference on Tourism

Documents
ANNUAL REPORT - Province of Manitoba · Alena Lukes, PhD Acng Director and Chief Operang Officer Vital Stascs Agency. 7 Monsieur Dave Wright Sous-ministre de la Jusce et sous-procureur

ANNUAL REPORT - Province of Manitoba · Alena Lukes, PhD Acng Director and Chief Operang Officer Vital Stascs Agency. 7 Monsieur Dave Wright Sous-ministre de la Jusce et sous-procureur

Documents
Othe Deep,Deep Love of Jesus

Othe Deep,Deep Love of Jesus

Documents
Stascs - Lex Jansen7/10/2013 VI Open Summary’of’Efficacy’Endpoint ’ Back’ ground’ Problem Reason’ Discussion’ Soluons ’ 15OCT2013 PhUSE’2013’:’IS05’ 13 Characteristic

Stascs - Lex Jansen7/10/2013 VI Open Summary’of’Efficacy’Endpoint ’ Back’ ground’ Problem Reason’ Discussion’ Soluons ’ 15OCT2013 PhUSE’2013’:’IS05’ 13 Characteristic

Documents
Magnec fields and turbulence in molecular clouds · centroid‐velocity (CV) in translucent fields • Departure from Gaussian stascs • Comparison of plane‐of‐sky projecon

Magnec fields and turbulence in molecular clouds · centroid‐velocity (CV) in translucent fields • Departure from Gaussian stascs • Comparison of plane‐of‐sky projecon

Documents
DNS Privacy - RIPE Network Coordination Centre · DNS over TLS • DNS queries to resolver via (authenNcated) TLS connecons • Requires “tuning” for DNS over TCP/TLS

DNS Privacy - RIPE Network Coordination Centre · DNS over TLS • DNS queries to resolver via (authenNcated) TLS connecons • Requires “tuning” for DNS over TCP/TLS

Documents
Deep calls unto deep

Deep calls unto deep

Documents
MM BPM - AnythingFlows · 2021. 4. 18. · MM Series: Flanged Connecons Carbon Steel, Stainless Steel & Alloy Construcon. The MM series are spring loaded, self actuated, direct-operated,

MM BPM - AnythingFlows · 2021. 4. 18. · MM Series: Flanged Connecons Carbon Steel, Stainless Steel & Alloy Construcon. The MM series are spring loaded, self actuated, direct-operated,

Documents
Australian Horticulture Statistics Handbook Vegetables · Stascs Handbook This Handbook is an analysis that combines all available data on producon, internaonal trade, processing

Australian Horticulture Statistics Handbook Vegetables · Stascs Handbook This Handbook is an analysis that combines all available data on producon, internaonal trade, processing

Documents
Research Computing - USC Viterbi School of Engineering · 2017-08-12 · Research Computing USC Viterbi School of Engineering New Faculty Orienta:on Presenta:on ... connecons to HPC

Research Computing - USC Viterbi School of Engineering · 2017-08-12 · Research Computing USC Viterbi School of Engineering New Faculty Orienta:on Presenta:on ... connecons to HPC

Documents
Water and Jobs - UNESCO · 45.2% of countries do not produce any gender stascs related to water (gender water data is among the least available of naonal-level indicators according

Water and Jobs - UNESCO · 45.2% of countries do not produce any gender stascs related to water (gender water data is among the least available of naonal-level indicators according

Documents