25
1 Iterative dynamically stabilized (IDS) method of data unfolding (*) *arXiv:0907.3791) Bogdan MALAESCU CERN PHYSTAT 2011 Workshop on unfolding

1 Iterative dynamically stabilized (IDS) method of data unfolding (*) (*arXiv:0907.3791) Bogdan MALAESCU CERN PHYSTAT 2011 Workshop on unfolding

Embed Size (px)

Citation preview

Page 1: 1 Iterative dynamically stabilized (IDS) method of data unfolding (*) (*arXiv:0907.3791) Bogdan MALAESCU CERN PHYSTAT 2011 Workshop on unfolding

1

Iterative dynamically stabilized (IDS) method of data unfolding (*)

(*arXiv:0907.3791)

Bogdan MALAESCUCERN

PHYSTAT 2011Workshop on unfolding

Page 2: 1 Iterative dynamically stabilized (IDS) method of data unfolding (*) (*arXiv:0907.3791) Bogdan MALAESCU CERN PHYSTAT 2011 Workshop on unfolding

2

Outlook

• Introduction: main effects to deal with• Additional problems in practice• An iterative unfolding method • A complex example• Discussion and conclusions

Page 3: 1 Iterative dynamically stabilized (IDS) method of data unfolding (*) (*arXiv:0907.3791) Bogdan MALAESCU CERN PHYSTAT 2011 Workshop on unfolding

3

Introduction: detector effects, folding and unfolding

Example of transfer matrix (MC)

Aij

ij

1

; ijij NBins

kjk

AP true spectrum data P

A

• Folding:

• Unfolding of detector effects (acceptance corrected afterwards)• Unfolding is not a simple numerical problem

Must use a regularization method.

Resolution+

Distortion

Page 4: 1 Iterative dynamically stabilized (IDS) method of data unfolding (*) (*arXiv:0907.3791) Bogdan MALAESCU CERN PHYSTAT 2011 Workshop on unfolding

4

Problems in practice: fluctuations due to background subtraction

• A “standard” unfolding could propagate large fluctuations into precise regions of the spectrum

• The uncertainties of the data points must be taken into account in the unfolding! (used to compute the significance of data-MC differences in each bin)

Folding

UnfoldingBackground subtraction

Page 5: 1 Iterative dynamically stabilized (IDS) method of data unfolding (*) (*arXiv:0907.3791) Bogdan MALAESCU CERN PHYSTAT 2011 Workshop on unfolding

5

Problems in practice: transfer matrix simulation perfect

• Key: use the significance of data-MC differences in each bin

New structure(not simulated)

MC - improved normalization

MC - standard normalization

Detector simulation (folding): systematic uncertaintyNew structures in data:• must also be corrected for detector effects• could bias MC normalization (needed in the unfolding, for data-MC

comparison)

Page 6: 1 Iterative dynamically stabilized (IDS) method of data unfolding (*) (*arXiv:0907.3791) Bogdan MALAESCU CERN PHYSTAT 2011 Workshop on unfolding

6

Ingredient for the unfolding procedure: a regularization function

• Used to “measure” significance in the (bin by bin) comparison of experimental data and MC simulation

• Allows one to perform a different treatment of fluctuations and significant new structures in data

• Important for the dynamical regularization of fluctuations• Depends (monotonously) on the absolute data – MC difference, their

uncertainties and a parameter l (scale factor)

• Behavior at small/large parameter values is important, but the exact choice of the function is not critical

• Used at all the steps of the unfolding procedure, with different values for l

2

( , , ) 1x

f x e

Page 7: 1 Iterative dynamically stabilized (IDS) method of data unfolding (*) (*arXiv:0907.3791) Bogdan MALAESCU CERN PHYSTAT 2011 Workshop on unfolding

7

Model for the test of the method

Transfer matrix model: • For the folding• Fluctuated matrix used for

the unfolding

Reconstructed MC

Generated MC

Resolution effect

Systematic transferof events

Page 8: 1 Iterative dynamically stabilized (IDS) method of data unfolding (*) (*arXiv:0907.3791) Bogdan MALAESCU CERN PHYSTAT 2011 Workshop on unfolding

8

Generated MC

Data

New Structures

Data -Reconstructed MC

Data -Generated MC

Model for the test of the method

Reconstructed MC

Generated MC+ New Structures Truth Data

Page 9: 1 Iterative dynamically stabilized (IDS) method of data unfolding (*) (*arXiv:0907.3791) Bogdan MALAESCU CERN PHYSTAT 2011 Workshop on unfolding

9

• First estimation of the number of events in data, corresponding to structures simulated by MC:

2

2 2

MCd D

k k k kMC

MCD

k k kMC

Nd d B r

N

Nd d r

N

1

1 , ,n

MC MCD D k k k

kNN N f d d d

1

( )n

MC dD k k

k

N d B

• A better estimation:

• The same method at the level of (corrected spectrum/ generated MC)

# data ev., in the bin k

# background subtraction fluctuation ev., in the bin k

ITERATIONS

Ingredients for the unfolding procedure: the MC normalization procedure

Page 10: 1 Iterative dynamically stabilized (IDS) method of data unfolding (*) (*arXiv:0907.3791) Bogdan MALAESCU CERN PHYSTAT 2011 Workshop on unfolding

10

•Relative improvement of the normalization: (ND – ND

MC)/ND

•The number of iterations is important only in the unstable region•The size of the unstable region depends on the amplitude of fluctuations in background subtraction

Study performed directly on data!

50 iterations (at most)

λN Choice λN

StableUnstable

Ingredients for the unfolding procedure: the MC normalization procedure

Page 11: 1 Iterative dynamically stabilized (IDS) method of data unfolding (*) (*arXiv:0907.3791) Bogdan MALAESCU CERN PHYSTAT 2011 Workshop on unfolding

11

Ingredients for the unfolding procedure: one step of the unfolding method

1

; ijij n

kjk

AP true spectrum data P

A

1

1 , ,, , k k k kj

n

j k k k kj

Mu

Cd

jM k

jC

u f d d f dN

d d dt PN

B

1

ijij n

ikk

AP

A

Folding:

Unfolding matrix (like d’Agostini method):

1

1

n

i ik kk

n

j kj kk

r P t

t P r

By construction:

Unfolding: compare data and reconstructed MC spectra

General equation

Only approximate for spectra other than MC

Fluctuation in background subtraction

True MC Significant difference (unfolded)

Not significant difference (fixed)

Aij

ij

Page 12: 1 Iterative dynamically stabilized (IDS) method of data unfolding (*) (*arXiv:0907.3791) Bogdan MALAESCU CERN PHYSTAT 2011 Workshop on unfolding

12

1st step of the unfolding methodL Choice:

(all differences between data and reconstructed MC spectra treated as not significant)

Reconstructed MC

Generated MC+ New Structures Truth Data

Data

New Structures

Data -Reconstructed MC

Data -Generated MC

Corrected spectrum

Corrected spectrum - generated MC

If one would choose lL=0 …

Page 13: 1 Iterative dynamically stabilized (IDS) method of data unfolding (*) (*arXiv:0907.3791) Bogdan MALAESCU CERN PHYSTAT 2011 Workshop on unfolding

13

Ingredients for the unfolding procedure :Comparison of the corrected spectrum and generated MC:• Estimation of large fluctuations in background subtraction:

not significant deviations, with large uncertainties

1 , ,uj j j j

MCD

j j j

S

MC

B f u u u

Nu u t

N

• Transfer matrix improvement: use significant structures

The folding matrix (P), describing detector effects, stays unchanged. Only the generated MC spectrum is improved.

, , , 1;MCij ij j j j ijMC

D

MCu D

j j j jMC

M

NA A f u u u P pour i n

N

Nu u B t

N

Normalization procedure

Page 14: 1 Iterative dynamically stabilized (IDS) method of data unfolding (*) (*arXiv:0907.3791) Bogdan MALAESCU CERN PHYSTAT 2011 Workshop on unfolding

14

The Iterative Unfolding Method• 1st unfolding, where the large fluctuations due to background

subtraction are kept unchanged

1)Estimation of large fluctuations due to background subtraction

2)Transfer matrix improvement (hence of the unfolding probability matrix)

3)Improved unfolding

Dynamical regularization: from the treatment of fluctuations in each bin, at each step of the procedure

When should the iterations stop? • Comparison of data and reconstructed MC • Study the number of needed iterations, with toys

Choice of parameters used at different steps, with a model for data. One can (in general) give up some of the parameters (by performing a maximal unfolding & transfer matrix modification).

Page 15: 1 Iterative dynamically stabilized (IDS) method of data unfolding (*) (*arXiv:0907.3791) Bogdan MALAESCU CERN PHYSTAT 2011 Workshop on unfolding

15

Results after iterations

Data – improved reconstructed MC

Estimation of background fluctuations

Data -Reconstructed MC

New structures

Page 16: 1 Iterative dynamically stabilized (IDS) method of data unfolding (*) (*arXiv:0907.3791) Bogdan MALAESCU CERN PHYSTAT 2011 Workshop on unfolding

16

Unfolding Result

New Structures

Initial reconstructed MC

Initial generated MC + New Structures Truth Data

Data

Data -Initialreconstructed MC

Data -Initialgenerated MC

Corrected spectrum

Corrected spectrum - Initial generated MC

• Statistical uncertainties propagated using pseudo-experiments (“toys”).

Page 17: 1 Iterative dynamically stabilized (IDS) method of data unfolding (*) (*arXiv:0907.3791) Bogdan MALAESCU CERN PHYSTAT 2011 Workshop on unfolding

17

Discussion

Studied but not discussed:• N bins data N bins result (rebinning in the

unfolding or afterwards)• Effect of rebinning on correlations• Effect of regularization on uncertainties and

correlations (see Kerstin’s talk)• Treatment of bins with negative number of

events (data)• Empty bins in MC• Preventing the existence of negative bins in the

improved generated MC

Page 18: 1 Iterative dynamically stabilized (IDS) method of data unfolding (*) (*arXiv:0907.3791) Bogdan MALAESCU CERN PHYSTAT 2011 Workshop on unfolding

18

Conclusion

• New general method for the unfolding of binned data

• Can treat problems that were not considered previously

• Dynamic regularization procedure, bin by bin at each step

• This method allows one to keep some control of bin to bin correlations in the unfolded spectrum

• Root code is available

Page 19: 1 Iterative dynamically stabilized (IDS) method of data unfolding (*) (*arXiv:0907.3791) Bogdan MALAESCU CERN PHYSTAT 2011 Workshop on unfolding

19

Backup

Page 20: 1 Iterative dynamically stabilized (IDS) method of data unfolding (*) (*arXiv:0907.3791) Bogdan MALAESCU CERN PHYSTAT 2011 Workshop on unfolding

20

Zoom on the narrow resonance region

Page 21: 1 Iterative dynamically stabilized (IDS) method of data unfolding (*) (*arXiv:0907.3791) Bogdan MALAESCU CERN PHYSTAT 2011 Workshop on unfolding

21

Simplified example:• Reduced effects of the transfer matrix• Smoother « bias », without structures• No « deeps » in the spectrum• No important fluctuations from background subtraction • Statistics reduced by a factor 20

A simple example for the use of the unfolding method

Data uncertainties

Data - Finalreconstructed MC(after one iteration)

Data -Initialreconstructed MC

Page 22: 1 Iterative dynamically stabilized (IDS) method of data unfolding (*) (*arXiv:0907.3791) Bogdan MALAESCU CERN PHYSTAT 2011 Workshop on unfolding

22

Simplified unfolding method:• Standard normalization for the MC• No estimation of left fluctuations (from background subtraction)• 1st unfolding with λ = λL ( = 1.5, justified by a study (see next))• One iteration with λU= λM=0

Effect of the 2nd unfolding

Effect of the 1st unfolding

Data uncertainties

A simple example for the use of the unfolding method

Page 23: 1 Iterative dynamically stabilized (IDS) method of data unfolding (*) (*arXiv:0907.3791) Bogdan MALAESCU CERN PHYSTAT 2011 Workshop on unfolding

23

• Use (data – reconstructed MC) as bias with respect to the generated MC, in order to build « generated data » (toys)

• Folding with the matrix Aij • (Do not) Fluctuate the folded data• Unfolding with the matrix A’ij (Aij fluctuated)• Compare the result with the « generated data »

A test with known « generated data » (before folding)

No extra data fluctuations: test systematic effectsWith statistical data fluctuations: stability test

Data uncertainties

Data -Initialreconstructed MC

Data - Finalreconstructed MC(after one iteration)

Page 24: 1 Iterative dynamically stabilized (IDS) method of data unfolding (*) (*arXiv:0907.3791) Bogdan MALAESCU CERN PHYSTAT 2011 Workshop on unfolding

24

Bias measurement after unfolding (without statistical

fluctuations of folded data)

Result – generated data (1st step)

Result – generated data (2nd step)

Bias measurement after unfolding (without statistical

fluctuations of folded data) in large bins

•The 1st unfolding provides a good result•λL = 1.5 : very small bias and reduced correlations with respect to the case λL = 0

Data uncertainties

A test with known « generated data » (before folding)

Page 25: 1 Iterative dynamically stabilized (IDS) method of data unfolding (*) (*arXiv:0907.3791) Bogdan MALAESCU CERN PHYSTAT 2011 Workshop on unfolding

25

• Diagonal uncertainties after the 1st unfolding: larger in the non trivial case (less correlations between the bins)

Uncertainties after 1st unfolding λL = 0

Uncertainties after 1st unfolding λL = 1.5

Data uncertainties

A simple example for the use of the unfolding method