Physics with Neural Networksfeindt/Freiburg.pdf · Physics with Neural Networks Michael Feindt Seminar Uni Freiburg Nov. 2005 Address all these topics and build a professional robust

Physics with Neural Networks Michael Feindt Seminar Uni Freiburg Nov. 2005

Prof. Dr. Michael FeindtCETAInstitut für Experimentelle KernphysikUniversität Karlsruhe

Universität Freiburg, 23. November 2005

Physics with Neural Networks


ContentsHypothesis testing / classificationNeural networks

Classification, conditional probability densitiesNeuroBayes (robust regularised Bayesian neural network)Applications in DELPHI, CDF II, outside physics

Unusual but powerful network applications: Training with MC/data mixturesTraining on data onlyTraining with weightsTraining with background subtraction Multidimensional regression using NeuroBayes


Hypothesis testing

Cut in a test-statistic:Accept hypothesis H0, if t<t(cut)

Error of 1. kind:P1(true hypothesis will be rejected)

Error of 2. kind:P2(wrong hypothesis is accepted)


Hypothesis testing


Hypothesis testing

A statistical method is thebetter the nearer it reachesthe point (1,1) in thepurity-efficiency-plot

Optimal choice of workingpoint according to particulartask: How does the total errorof the analysis scale withε und P?

0.7 0.8 0.9 1.0

Optimalworking point

Different cuts in t

Different cuts in t


Determining the working point (scan through cuts on network output)


Construction of a test statistic:How to make 100 dimensions one real number…


Neural Networks

Neural networks:Self learning procedures, copied from nature

ParietalCortexFrontal Lobe

Motor Cortex

Temporal Lobe

Brain Stem

OccipitalLobe

Cerebellum


Neural Networks

The information(the knowledge, the expertise)is coded in the connectionsbetween the neurons


Neural networks

Basis function


Neural networks – transfer functions


Neural networks - training


f t

Probability that hypothesisis correct (classification)or probability densityfor variable t

t

How it works: training and applicationHistoric or simulateddata

Data seta = ...b = ...c = .......t = …!

NeuroBayesNeuroBayes®®TeacherTeacher

NeuroBayesNeuroBayes®®ExpertExpert

Actual (new real) data

Data seta = ...b = ...c = .......t = ?

ExpertiseExpertise

Expert system


Naïve networks and criticizmWe‘ve tried that but it didn‘t give good results

- Stuck in local minimum- Learning not robust

We‘ve tried that but it was worse than our 100 person-yearsanalytical high tech algorithm

- Selected too naive input variables- Use your fance algorithm as INPUT !

We‘ve tried that but the predictions were wrong- Overtraining: the net learnt statistical fluctuations

Yeah but how can you estimate systematic errors?- How can you with cuts when variables are correlated?


Address all these topics and build a professional robust and flexible NeuralNetwork package for physics, insurance, bank and industry applications:NeuroBayes®

<phi-t>: Foundation out of University of Karlsruhe, sponsored by exist-seed-programme of thefederal ministery for Education and ResearchBMBF


2000-2002 NeuroBayes®-specialisationfor economy at the University of Karlsruhe

Oct. 2002: GmbH founded, first industrial projects

June 2003: Removal into new office199 qm IT-Portal Karlsruhe

Exclusive rights for NeuroBayes®

Personell currently4 full time staff (all from HEP) anda number of associated people,Prof. consultance z.B. by Prof. Dr. Volker Blobel,Economic/legal/marketing- expertise present

Customers: Badische Versicherungen, dm drogerie markt, Otto Versand

History


NeuroBayes® principle

NeuroBayes® Teacher:Learning of complex relationships from existingdata bases

NeuroBayes® Expert:Prognosis for unknown data

Output

Input

Sign

ific

ance

cont

rol

Postprocessing

Preprocessing



Bayesian RegularisationUse Bayesian arguments to regularize network learning:

PosteriorPosterior EvidenceEvidence

LikelihoodLikelihood PriorPrior

Learn only statistically relevant information, suppress statistical noise










Ramler-plot (extended correlation matrix)


Ramler-II-plot (visualize correlation to target)


Visualisation of single input-variables


Visualisation of correlation matrix

Variable 1: Training target


Visualisation of network performance

Purity vs. efficiency

Signal-effiziency vs. total efficiency(Lift chart)


Visualisation of NeuroBayes network Topology




Example DELPHI kaon ID ClassificationClassificationIdentification of kaons:

For Phi reconstruction:For Phi reconstruction:Efficiency twice as good Efficiency twice as good at the same background at the same background level (i.e. S/B doubled)level (i.e. S/B doubled)

Green: better of two Green: better of two competing standard competing standard algorithms (>50 men algorithms (>50 men years)years)

NeuroBayesNeuroBayes®®::Finds who is right under Finds who is right under which circumstanceswhich circumstances


NeuroBayes:Increase of efficiencyat same purity as best cut basedselection:Improve significancefrom 18.4 to 22.1

NeuroBayes:Increase of efficiencyat same purity as best cut basedselection:Improve significancefrom 18.4 to 22.1

Optimisation of Bs reconstruction for Bs oscillation analysisOptimisation of Bs reconstruction for Bs oscillation analysis


Neural Nets for B flavour taggingNeural Nets for B flavour tagging

NeuroBayes track net:Does a track come from a B-decay or not?Tracknet 1 – Vertexnet – Tracknet 2

NeuroBayes track net:Does a track come from a B-decay or not?Tracknet 1 – Vertexnet – Tracknet 2


Aim: Bayesian estimator for a singlemultidimensional measurement .

"Components of may be correlated."Components of should be correlated to t or its uncertainty. "All this should be learned automatically in a robust way from data bases"containing Monte-Carlo simulations or historical data.

Aim: Bayesian estimator for a singlemultidimensional measurement .

"Components of may be correlated."Components of should be correlated to t or its uncertainty. "All this should be learned automatically in a robust way from data bases"containing Monte-Carlo simulations or historical data.

Note: Conditional probability density contains much more information than just the mean value, which is determined in a regression analysis. It also tells us something about the uncertainty and the form of thedistribution, in particular non-Gaussian tails.

Note: Conditional probability density contains much more information than just the mean value, which is determined in a regression analysis. It also tells us something about the uncertainty and the form of thedistribution, in particular non-Gaussian tails.

xr

xr

xr

)|( xtfr

ConditionalConditional probabilityprobability densitydensity reconstructionreconstruction::


Conditional probability densitiesin particle physics

What is the probability density of the true B energy in this event taken with the DELPHI detector at LEP IIat this beam energy, this effective c.m. energythese n tracks with those momenta and rapidities in the hemisphere, which are forming this secondary vertex with this decay length and probability, this number of not well reconstructed tracks, this neutral showers,etc pp )|( xtf

r

t

xr


Conditional probability densitiesin investment bankingWhat is the probability density for a price change of equity A in the neyt 10 days…

that made this and that price movement in thelast days and weeks…

is so much more expensive than the n-days moving average…

but is so much less expensive that the absolute maximum…

has this correlation to the crude oil price…and the Dow Jones index…and etc. pp.

)|( xtfr

t

xr


Prediction of the completeprobability distributionwith the < phi-t > NeuroBayes neural network

t

)|( xtfr

ModeExpectation value

Standard deviationvolatility

Deviations fromnormal distribution,e.g. crash probability


Conditional probability densities f(t|x)

Conditional probability densities f(t|x) are functions of x, but also depend on marginal distribution f(t).

Conditional probability densities f(t|x) are functions of x, but also depend on marginal distribution f(t).

Inclusive distribution(Bayesian Prior)

Inclusive distribution(Bayesian Prior)

Conditional probability density for a special case x

(Bayesian Posterior)

Conditional probability density for a special case x

(Bayesian Posterior)

Marginal distribution f(t)Marginal distribution f(t)


Classical ansatz:f(x|t)=f(t|x)

approximately correctat good resolution

far away fromphysical boundaries

Bayesian ansatz:takes into accounta priori- knowledge f(t):•Lifetime never negative•True lifetime exponentiallydistributed






No selection:Improved resolution

No selection:Improved resolution

NeuroBayes phi-direction

Best ”classical" chi**2- fit

(BSAURUS)

NeuroBayes phi-direction

Best ”classical" chi**2- fit

(BSAURUS)

Resolution of azimuthal angle of inclusively reconstructed B-hadrons in the DELPHI-

detector

first neural reconstruction of a direction

Resolution of azimuthal angle of inclusively reconstructed B-hadrons in the DELPHI-

detector

first neural reconstruction of a direction

Direction of B-mesons (DELPHI)

After selection cut on estimated error:Resolution massively improved, no tails

==> allows reliable selection of good events

After selection cut on estimated error:Resolution massively improved, no tails

==> allows reliable selection of good events


Some applications in high energy physicsDELPHI:Kaon, proton, electron idOptimisation of resolutions inclusive B- E, phi, Theta, Q-valueB** enrichmentB fragmentation functionLimit on B_s-mixingB0-mixingB- F/B-asymmetryB-> wrong sign charmCDF: (Work in Progress)Electron ID, muon ID, kaon/proton IDOptimisation of resonance reconstruction (X, Y , B_s) Spin parity analysisB-Tagging for top, Higgs, etc. B-Flavour Tagging for mixing analyses


Applications of NeuroBayes in Economy> Medicine and Pharma research

e.g. effects and undesirable effects of drugsearly tumor recognition

> Banks e.g. Credit-Scoring (Basel II), Finance time seriesprediction, valuation of derivates, risk minimisedtrading strategies, client valuation

> Insurancese.g. risk and cost prediction for individual clients, probability of contract cancellation, fraud recognition, justice in tariffs

> Trading chain stores: turnover prognosis for individual articles/stores

Necessary prerequisite:Historic or simulated data must be available!


Risk analysis for a car insuranceBGV

Results for a the Badischen Gemeinde-Versicherungen:

since May 2003: radically new tariff for young drivers!

New variables added to calculation of the premium.Correlations taken into account.

Risk und premium up to a factor of 3 apart from each other!Even probability distribution of height of can be predicted

Premature contract cancellation also well predictable


The ‘‘unjustice‘‘ of insurance premiums

The majority of customers (with lowrisk) are paying too much.

Less than half of the customers (withlarger risk) do not pay enough, someby far not enough.These are currently subsidised by themore careful customers.

Anza

hl K

unde

n

Ratio of the accident risk calculated using NeuroBayes®to premium paid (normalised to same total premium sum):

Risiko/Prämie


Prediction of contract cancellationfor an insurance

The predictionreally holds:

Test on a a newstatistic year


Bank customers contract cancellation

Prognosis of prematurecontract cancellation

Massive enrichment possible.

CRM measures introduced


Near Future TurnaroundPredictions for Chain Stores

1. Time series modelling2. Correction and error estimate

using NeuroBayes


The <phi-t> mouse game:or:

even your ``free will´´ is predictable

www.phi-t.de/mousegame


Direct competition…Data-Mining-Cup 2005:Task: Internet shop: Prognosis of customerswho will not pay

531 Participantsfrom 176 Universitiesfrom 41 countries.

6 Karlsruhe studentsall using Phi-TNeuroBayes®:TOP RANKINGS: Positions 2,3,4,5,6,7!

www.data-mining-cup.de


How does NeuroBayes compare to other neural networks?How does NeuroBayes compare to other neural networks?

depends on problem.Usually superiorbecause of robustpreprocessing and Bayesian regularisation.

(does not meanthat Jetnet couldnot also do it ifpreprocessing isdone by hand)

depends on problem.Usually superiorbecause of robustpreprocessing and Bayesian regularisation.

(does not meanthat Jetnet couldnot also do it ifpreprocessing isdone by hand)Example: electron ID for CDF IIExample: electron ID for CDF II


Hadron collider: No good MC for backgrounds availableMC for resonance production with different JPC assumptions

Idea: take background from sidebands in datacheck that network cannot learn mass

Hadron collider: No good MC for backgrounds availableMC for resonance production with different JPC assumptions

Idea: take background from sidebands in datacheck that network cannot learn mass


background-likehardly loose any signalbackground-likehardly loose any signal

signal-likesignal-like

X

soft NeuroBayesselection

X(3872) analysis:X(3872) analysis:


Hard NeuroBayes cut:Very clean X(3872) signalHard NeuroBayes cut:Very clean X(3872) signal


Training with weights:Neural Network Spin Parity AnalysisTraining with weights:Neural Network Spin Parity Analysis

Use data from sidebands as background sample

Use phase space decay MC with modelled pT distributionas signal sample

Calculate squared amplitudes for specific spin-parity assignmentsand use as weights in training

Hard cut on neural network trained by correct hypothesis shouldincrease signal over background


Very hard NeuroBayes cuts:Very hard NeuroBayes cuts:

A good hypothesis for X A good hypothesis for X

JPC=1—(ππ)s hypothesisJPC=1—(ππ)s hypothesis


Very hard NeuroBayes cuts:Very hard NeuroBayes cuts:

a not so good hypothesis for Xa not so good hypothesis for X

JPC=1—(ππ)s hypothesisJPC=1—(ππ)s hypothesis


Hadron collider: Fast resonance S/N optimisation without MC:

Idea: Training with background subtractionSignal: Peak region weight 1

Sideband region with weight -1Background: Sideband region with weight 1

Hadron collider: Fast resonance S/N optimisation without MC:

Idea: Training with background subtractionSignal: Peak region weight 1

Sideband region with weight -1Background: Sideband region with weight 1

works very well!

also for Y(2S)and Y(3S) !Although just trained on Y(1S)

works very well!

also for Y(2S)and Y(3S) !Although just trained on Y(1S)


works very well also for Y(2S) and Y(3S) !Really learn to separate resonance from combinatorial backgroundworks very well also for Y(2S) and Y(3S) !Really learn to separate resonance from combinatorial background


Making MC for hadronic background without specific model:Multidimensional correlated regression using NeuroBayesMaking MC for hadronic background without specific model:Multidimensional correlated regression using NeuroBayes

Use data in non-resonance region as signalUse phase space MC as background

Train NeuroBayes network,NN output O is Bayesian a posteriori probability that event stemsfrom signal (i.e. data distribution) rather than phase space MC:

O=P(S) with P(S)+P(B)=1Calculate weight W= P(S)/P(B) = O/(1-O) Phase space MC events with this weight W look like data!MC modelling of complicated background is possible!Opens new roads for likelihood fits


Some kinematical variableDistributions(J/psi pi+pi- selection)

Black: real dataRed: weighted phase space MC


Documents

Physics with Neural Networksfeindt/Freiburg.pdf · Physics with Neural Networks Michael Feindt Seminar Uni Freiburg Nov. 2005 Address all these topics and build a professional robust