1

Methods of Experimental Particle Physics

Alexei Safonov

Lecture #25

2

Today• Brief reminder on upper limits discussion

from last time• More on parameter estimation• Combining measurements• Alternatives to ML• Walk through and interpret the CMS

results

3

One Sided Limits• It is typical in HEP to look

for things that are at the edge of your sensitivity• You frequently can’t “see”

the signal, but you still want to say that the cross-section for that new process can’t be larger than X

• Also very useful information for theorists and model builders as your results can rule out a whole bunch of models or parameter space of the models

• Can do it for either Bayesian or Frequentist methods • Most of the time fairly straightforward – either

construct a one-sided intervals with known coverage or calculate the integral from 0 to x in Bayesian case

4

Some good to remember numbers• Upper 90% and 95%

limit on the rate of a Poisson process in the absence of background when you observe n events• If you observe no

events, the limit is 3 events

• In Bayesian case, this would also be true for any expected B rate

5

Practical Parameter Estimation• You usually calculate –log(L)

• Assuming you are doing a measurement, the minimum of –log(L) is the maximum of L, so that gives the most likely parameter value

• Changing – log(L) by +/-1/2 (think of taking a log of a gaussian distribution – it will give you (x-x0)2/2s2 so x shifted by sigma gives ½) gives you 1 sigma deviation in the parameter (68% C.L.)

• With the MLS method, you vary it by 1 instead:

• MINUIT is the most used minimization package in HEP (it is part of ROOT), easy to use in simple cases, some experience required for more complex ones

6

Upper Limits Calculations• In case of upper limits, you will need to

integrate the distribution to find the point where 95% of the integral is accumulated• Any numeric integration method will do as the

integral is usually not too complicated

• I usually just keep halving the bin size until I get accuracy well below a fraction of a percent• Surely can be done more

efficiently, but if it takes no time who cares

7

Combining Measurements• Having likelihood function

makes combining measurements very easy• For a combined measurement,

the joint likelihood is a product of the likelihoods:

• Obvious if you consider one of the measurements as a prior and take into account that both measure the same value

• Correlated systematics may need to be accounted for • The product/limit will

be a factor sqrt(2) more narrow/lower• Kind of like doubling the

sample

8

Correlated Systematics• Having correlated systematics is not

unusual• E.g. the luminosity (L) measurement is

usually 100% correlated

• If the correlation is not 100%, include a prior correlation matrix

• The prior P in this case should be constructed to include the correlation matrix

• Can be easily done using Monte Carlo generation: generate pairs of x1 and x2 following the required correlation

9

Un-Binned Likelihood• Binned likelihood is easy to

interpret and gives consistent and predictable outcomes• But there is a choice of bin size

and strictly speaking you are loosing information by clamping things together

• Can you avoid that?• Decrease the size of the bins, in the extreme

limit - unbinned likelihood:

• where nu is the function value at the point where the data point happened to occur

• Is there a problem with this?

10

Un-Binned Likelihood• Key difference: binned likelihood knows about

areas where there is no data• It will try to push the fit function down to account for

the lack of data points

• Unbinned likelihood doesn’t account for empty “bins”• If you have areas in the histogram that are included

in the fit and which are

• One can correct for it (modified unbinned likelihood):

• The last term accounts for the empty bins

11

WALK THROUGH AND COMMENT ON THE CMS OBSERVATION PAPER

http://arxiv.org/pdf/1303.4571.pdf

12

Expected Upper Limits

• Mean expected • 95% C.L. upper limit on s/s(SM) in the

absence of Higgs • The p-value if Higgs exists versus its mass

13

Signal in Two Photon and ZZ Channels

• Pretty compelling, right?

14

Observed Upper Limits

• Three main channels• What can you tell about upward downward

fluctuations that happened in data based on these plots?

15

Two Less Important Channels

• How do you interpret these?

16

Upper Limit and Hypothesis Testing

• This includes all channels• Left is upper limit• Right is the “p-value” for the Higgs

hypothesis (not null hypothesis)

17

Local p-value

• Two channels

18

Combined Channels

19

Signal Strength and Mass

• q on the right is the test statistic (it’s the likelihood ratio based analysis)

20

Signal Strength: Channels

21

Next Time• Review ongoing and future analyses at

the LHC