View
4
Download
0
Category
Preview:
Citation preview
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010
The CMS SoCware and Common Tools
Sudhir Malik Fermilab/University of Nebraska‐Lincoln, U.S.A.
Talk at the Dept. of Physics UPRM, 1 December, 2010
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010 2
I • Framework basics and EventDataModel • Looking at files • The Data Formats • Wri>ng a framework module • Running the module II • PAT – Physics Analysis Toolkit • Places to look for informa>on
Outline
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010 3
The CMS Framework and the Event Data Model
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010
Basic Concept
4
The soHware of CMS is called CMSSW:
• Modular architecture of the framework
• Flexible structure of the Event Data Model (EDM) based on the event:
‐ Single en>ty in memory: edm::Event container
‐ Data are uniquely iden>fied within an Event
‐ Modular Event Content
• Data processing steered via Python job configura>ons
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010
Flow of the Data
5
Modules are configurable and communicate via the Event
hUps://twiki.cern.ch/twiki/bin/view/CMS/WorkBookCMSSWFramework
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010
Framework modules
6
Source
• Reads the event in a root file OutputModule
• Write events to a file. Can use filter decisions EDAnalyzer (read)
• Reading data only • Crea>ng histograms • the standard use case
EDProducer (read/write)
• You want to create new products • You want to share your reconstruc>on code with others
EDFilter (read/write)
• you want to know if an object could be produced • you want to control the analysis flow or make skimming
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010
Non‐EventData
7
• Examples of Framework Services and setups are ‐ Geometry, Calibra>on, MessageLogger
• Types are:
‐ Service
Constantly available
‐ ESSource Provides data which have an IOV (Interval of Validity)
‐ ESProducer Creates Products when IOV changes
Topic of its own. No further details in this tutorial!
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010
CMSSW job configuraUon
8
import FWCore.ParameterSet.Config as cms
process = cms.Process("EXAMPLE")
process.source = cms.Source("EmptySource")
process.maxEvents = cms.untracked.PSet( input =
cms.untracked.int32(100)
)
process.int = cms.EDProducer("IntProducer",
ivalue = cms.int32(2)
)
process.test = cms.EDAnalyzer("IntTestAnalyzer", valueMustMatch = cms.untracked.int32(2)
)
process.Tracer = cms.Service("Tracer")
process.path = cms.Path( process.int * process.test)
$ cmsRun example_cfg.py One executable
Data processing is steered via configura>on file wriUen using Python script language: hUps://twiki.cern.ch/twiki/bin/view/CMS/WorkBookConfigFileIntro
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010
Input ‐ Source
9
process.source = cms.Source("PoolSource”,
fileNames = cms.untracked.vstring("file:input.root”)
)
INPUT
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010
Output ‐ OutputModule
10
process.out = cms.OutputModule("PoolOutputModule", fileName = cms.untracked.string("output.root")
)
OUTPUT Process.out.outputCommands = cms.untracked.vstring(
"keep *", "drop *_*_*_HLT", "keep FEDRawDataCollec>on_*_*_*”
)
Configurable Event Content: What to drop? What to keep?
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010 11
The Data Formats in edm files
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010
Files can be inspected with ROOT
12
root ‐l
[] TFile f(“AOD.root”)
[] new TBrowser()
Data inside the event are called “Product”
moduleLabel : productInstanceLabel : processName
Example: recoTracks_generalTracks_RECO
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010
FWLite gives access to classes
13
Automa>c library loading
[] gSystem‐>Load("libFWCoreFWLite”)
[]AutoLibraryLoader::enable()
[] new TBrowser()
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010
Storing and managing Products in the Event
14
The Products can be read and analyzed by an EDAnalyzer (read only), stored in the Event using an EDProducer (read/write), you can also make a filtering using an EDFilter
module (read/write). These are the basic steps of data processing
More details on these Framework modules will come later
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010
What is stored in the event files?
15
In CMSSW a set of standard data formats is defined, they are collec>ons of several ed‐Products managed centrally in CMSSW • RAW:
• Data like they come from the detector
• RECO (Reconstruc>on):
• Output of the event reconstruc>on
• AOD (Analysis Object Data):
• Subset of data needed for standard analysis
• RAWSIM, RECOSIM, AODSIM:
• with addi>onal simula>on informa>on
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010
What are the stored products?
16
vector "tcMet" "" "RECO." vector "muons" "" "RECO." vector "muonsFromCosmics" "" "RECO." vector "muonsFromCosmics1Leg" "" "RECO." vector "par>cleFlow" "" "RECO." vector "par>cleFlow" "electrons” "RECO." vector "ak5PFJets" "" "RECO."
edmDumpEventContent
Handleon> muons; Event.getByLabel(”muons”,muons );
reco::MuonCollec>on is a typedef for vector
Access the single
Product in the framework module
C++ class type product alias label process name
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010
Accessing Event Data
17
# by module and default product label Handleon> muons; iEvent.getByLabel(”muons", muons );
# by module and product label Handle par>cleFlow; iEvent.getByLabel("par>cleFlow", ”electrons" , par>cleFlow_electrons );
We can access the products
in the module using the Handle
Framework modules are wriUen in C++ , you can find a basic C++ guide at:
hUps://twiki.cern.ch/twiki/bin/view/CMS/WorkBookBasicCPlusPlus
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010 18
Event VisualizaUon
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010
VisualizaUon Tool
19
In CMS there are visualization tools, one of them is:
Fireworks is the light weight event display for analysis. It can be installed on your laptop. You can find it at: https://twiki.cern.ch/twiki/bin/view/CMS/WorkBookFireworks Try out the video tutorial! http://cern.ch/cms-sdt/fireworks/demo.mov Also check ( for all tutorials) https://twiki.cern.ch/twiki/bin/viewauth/CMS/Tutorials
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010
Fireworks
20
Put here a picture of Fireworks and Iguana ???
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010 21
WriUng your own framework module
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010
The Release Area
22
Preparing the environment
crea>ng your local area $ cmsrel CMSSW_3_8_7 $ cd CMSSW_3_8_7/src
sezng run>me variables $ cmsenv
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010
WriUng an EDAnalyzer
23
$ cd CMSSW_3_8_7/src $ mkdir Tutorial $ cd Tutorial
Create the skeleton of an EDAnalyzer
$ mkedanlzr DemoAnalyzer $ cd DemoAnalyzer & scram b
The skeleton of the EDAnalyzer has been created and put in DemoAnalyzer/src/DemoAnalyzer.cc
Find more details on WorkBookWriteFrameworkModule: hUps://twiki.cern.ch/twiki/bin/view/CMS/WorkBookWriteFrameworkModule
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010
The Source Code
24
In the header:
The methods beginJob(), analyze() and endJob() are called for each event!
private: virtual void beginJob() ; virtual void analyze(const edm::Event&, const edm::EventSetup&); virtual void endJob() ;
void DemoAnalyzer::analyze(const edm::Event& iEvent, const edm::EventSetup& iSetup) {
using namespace edm; Handleon> tracks; iEvent.getByLabel("generalTracks", tracks);
for(TrackCollec>on::const_iterator itTrack = tracks‐>begin(); itTrack != tracks‐>end(); ++itTrack)
{ int charge = itTrack‐>charge(); }
}
Get the TrackCollecUon and loop on all the tracks in the
event
DEFINE_FWK_MODULE(DemoAnalyzer);
Define the Module!
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010
TFileService
25
// access the TFileService edm::Service fs; // create your histogram TH1F * h_pt = fs‐>make( "pt" , "p_{t}", 100, 0., 100. ); // fill it h_pt‐>Fill( pt ); // create subdirectories if you like TFileDirectory subDir = fs‐>mkdir( "mySubDirectory" );
We can create ROOT histograms and store them using the TFileService, a framework service.
# make the TFileService known to the config process.TFileService = cms.Service("TFileService",
fileName = cms.string("histo.root") )
hUps://twiki.cern.ch/twiki/bin/view/CMS/SWGuideTFileService
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010
HistoAnalyzer
26
plotJets = cms.EDAnalyzer("CandViewHistoAnalyzer", src = cms.InputTag(”ak5CaloJets"), histograms = cms.VPSet( cms.PSet( itemsToPlot = cms.untracked.int32(5), # plots the first 5 jets min = cms.untracked.double(0.0), max = cms.untracked.double(200), nbins = cms.untracked.int32(50), name = cms.untracked.string("jet %d E_{T} [GeV/c]"), descrip>on = cms.untracked.string("jet_%d_et"), plotquan>ty = cms.untracked.string("et") ) )
)
We can get histograms without wri>ng C++ code
hUps://twiki.cern.ch/twiki/bin/view/CMS/SWGuideHistogramU>li>es
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010
ConfiguraUon Files
27
• Usually placed in a package’s python/ or test/ • Can be checked for completeness doing
python myExample_cfg.py (Python interpreter)
• Can be run using cmsRun
cmsRun myExample_cfg.py
If you are Python addicted...and if not yet maybe you will be soon ;‐) ... you may want to inspect your config file in python interac>ve mode:
$ python –i config_file_cfg.py # to inspect the process path called “path” >>>process.path
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010
ConfiguraUon Files
28
Naming conven>ons:
– Defini>on of a single object: _cfi.py – A configura>on fragment: _cff.py – A full process defini>on: _cfg.py
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010
How to import objects
29
• To fetch all modules from some other module into local namespace
from Subsystem.Package.Foo_cff import *
(looks into Subsystem/Package/python/Foo_cff.py)
• To load everything from a python module into your process object you can say:
process.load(‘Subsystem.Package.Foo_cff’)
• Don’t forget that all imports create references, not copies: changing an object at one place
changes the object at other places
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010
Sequences and Paths
30
Sequence:
• Defines an execu>on order and acts as building block for more complex configura>ons and contains modules or other sequences.
trDigi = cms.Sequence(siPixelDigis + siStripDigis)
Path:
• Defines which modules and sequences to run. p1 = cms.Path(pdigi * reconstruc>on)
EndPath:
• A list of analyzers or output modules to be run aHer all paths have been run.
outpath = cms.EndPath(myOutput)
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010
Filters in Paths
31
• Each path corresponds to a trigger bit
• When an EDFilter is in a path, returning False will cause the path to terminate
• Two operators ~ and ‐ can modify this. 1. ~ means not. The filter will only con>nue if the filter
returns False.
2. ‐ means to ignore the result of the filter and proceed regardless
jet500_1000 = cms.Path( ~jet1000filter + jet500filter + jetAnalysis )
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010
Inspect your configuraUon: the ConfigBrowser
32
Tree View Graphical representa>on
Property View Box
hUps://twiki.cern.ch/twiki/bin/view/CMS/SWGuideConfigBrowser
You can inspect your config file using a graphical tool as well: the ConfigBrowser
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010 33
Break for quesUons
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010 34
PAT (Physics Analysis Toolkit)
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010
PAT
35
• It serves as well tested and supported common ground for group and user analyses.
• It facilitates reproducibility and comprehensibility of analyses, • It is an interface between the some>mes complicated EDM and the simple mind of the common user.
• You can view it as a common language between CMS analysts: • If another CMS analyst describes you a PAT analysis you can easily know what he/she is talking about
PAT provides a very quick start for beginners
PAT is a toolkit as part of the CMSSW framework aimed at performing analysis
It provides:
• data format • common modules
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010
PAT ‐ DATA Formats
36
Representa>on of reconstructed physics par>cles
pat::Candidate (pat::Jet, pat::Photon, pat::Muon,etc..) There is a base class common to all kind of “Par>cles”: the reco::Candidate
It provides access:
• kinema>cs (pt, mass, eta, phi, etc. ) • underlying componenents (link to track, superclaster, etc.) • naviga>on among the daughters (to access the daughter par>cles and their aUributes )
The pat::Object inherits from the reco:Candidate
you can add extra informa>ons to pat Candidates wrt reco Candidates such as:
• Isola>on • MC matching • Trigger matching
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010
PAT ‐ DATA Formats 2
37
This is the hierarchy of pat::Candidates
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010
PAT ‐ Common modules
38
PAT provides a series of modules common to different analysis task, such as:
‐ cleaning ‐ to remove disambigui>es on the iden>fica>ons of par>cles in the event
‐ MC matching ‐ to associate PAT objects with generator objetcs
‐ Trigger matching ‐ to associate PAT objects with trigger objects
Moreover PAT provides a set of tools to performe easily the configura>on of the Workflow PAT Tools
These tools have been conceived to be common to different kind of analysis and, at the same >me, to be customized according to the specific analysis requirement. They are well documented in SWGuidePATTools:
hUps://twiki.cern.ch/twiki/bin/view/CMS/SWGuidePATTools
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010
PAT workflow
39
PAT workflow is organised in a principal sequence and a parallel one for associa>ng trigger informa>ons.
The main steps of the patTuple produc>on are:
‐ PAT preproduc>on
‐ Candidate Produc>on
‐ Candidate Selec>on
‐ Candidate Disambigua>on
‐ PAT Trigger Event
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010 40
Browsing configuraUon
The PAT workflow is defined in python configura>on files.
We can learn how it is structured inspec>ng it by looking at which are the modules involved and in which way they are related to each other, at which values parameters are set and so on.
Some tools have been implemented to inves>gate workflow configura>on and the pat::Tuple produced:
• edmConfigEditor (hUps://twiki.cern.ch/twiki/bin/view/CMS/WorkBookPATConfigEditor) • edmDumpEventContent
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010 41
EdiUng configuraUon
The produc>on of an user‐defined PAT analysis consists of three steps:
1. Start from the standard configura>on file. 2. Apply the PAT tools to change the configura>on of the standard configuring file according to the specific needs of your analysis. 3. Replace parameter values according to the needs of your analysis.
To browse and edit configura>on file you can use a graphical tool: the edmConfigEditor
NOTE: You may want to learn more about ConfigEditor and PAT Tools, look the SWGuideConfigExercise page:
SWGuidePATConfigExercise hUps://twiki.cern.ch/twiki/bin/view/CMS/SWGuidePATConfigExercise
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010
The ConfigEditor ‐ Import Configura>on
42
STEP 1: Click on Import configuraUon to choose a standard PAT config file to start from
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010
ConfigEditor ‐ Apply tool
43
STEP 2: Click on Apply PAT tool and choose a tool to add to the config file, sezng its parameters
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010
ConfigEditor ‐ Replace parameters
44
STEP 3: Click on a module and replace a parameter value in the Property view on the right
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010
ConfigEditor ‐ The resul>ng code
45
In the leH corner inspect the produced code
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010 46
PAT Tutorial
A PAT Tutorial was held at Fermilab from 15 to 19 of November, 2010
You can find updated documenta>on, lectures and exercise sec>ons at the PAT Tutorial Twiki page created for this occasion :
hUps://twiki.cern.ch/twiki/bin/view/CMS/WorkBookPATTutorial
A next tutorial will be held at CERN/EVO from March of 2011!
You can always get help from PAT Team wri>ng on the AnalysisTools hypernews: hn‐cms‐physTools@cern.ch
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010 47
Helper uUliUes
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010
Finding Code
48
How can we know where an object is defined?
• If you already know where to look, you might use cvs browser:
hUp://cmssw.cvs.cern.ch/cgi‐bin/cmssw.cgi/CMSSW
• In all other 99% of the cases you might use lxr browser:
hUp://cmslxr.fnal.gov/lxr/
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010
Handling Source Code
49
• To add a package from the cms repository type:
addpkg PhysicsTools/UUliUes [tag]
• If no tag is given the default one from the release is taken
• List which packages are in your release area:
showtags –r
Test Release based on: CMSSW_3_6_2
Base Release in: /afs/cern.ch/cms/sw/slc5_ia32_gcc434/cms/cmssw/CMSSW_3_6_2
Your Test release in: /afs/cern.ch/user/d/decosa/scratch0/CMSSW_3_6_2
‐‐‐ Tag ‐‐‐ ‐‐‐ RelTag ‐‐‐ ‐‐‐‐‐‐‐‐ Package ‐‐‐‐‐‐‐‐
V05‐09‐14‐02 V05‐09‐14‐00 DataFormats/PatCandidates
V07‐13‐15‐05 V07‐13‐15‐03 PhysicsTools/PatAlgos
‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐
total packages: 2 (2 displayed)
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010
How to check what is going on?
50
In CMSSW a Service module, called ‘Tracer’, helps us giving trace of each step of the processing this is a clean way to understand what’s happening
process.trace = cms.Service(‘Trace’)
process.p = cms.Path(... + trace + ...)
The output:
++++source
Begin processing the 1st record. Run 1, Event 1, LumiSec>on 1 at 09‐Sep‐2008 10:30:22 CEST
++++finished: source
++++ processing event:run: 1 event: 1 >me:5000000
++++++ processing path:genera>on_step
++++++++ module:randomEngineStateProducer
++++++++ finished:randomEngineStateProducer
++++++++ module:VtxSmeared
++++++++ finished:VtxSmeared
process.trace = cms.Service(‘Trace’)
process.p = cms.Path(... + trace + ...)
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010
Support and documentaUon
51
• There is the WorkBook:
hUps://twiki.cern.ch/twiki/bin/view/CMS/WorkBook
• The SWGuide:
hUps://twiki.cern.ch/twiki/bin/view/CMS/SWGuide
• LXR:
hUp://cmslxr.fnal.gov/lxr/
• PAT Tutorial hUps://twiki.cern.ch/twiki/bin/view/CMS/WorkBookPATTutorial
• Many, many hypernews lists
• Thanks to my colleagues ‐ Annpaola DeCosa and Benedikt Hegner for providing helping with slides
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010 52
Backup slides
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010
Cloning
53
• Some>mes you need to add a module which has almost the same parameter as another one
• You can copy the module and change the parameters that need to be modified
import ElectroWeakAnalysis.ZReco.zToMuMu_cfi as zmumuzToMuMuGolden = zmumu.zToMuMu.clone( massMin = cms.double(40) )
• Changing while cloning should be preferred wrt clone + later replace as it is a much safer prac>ce.
import ElectroWeakAnalysis.ZReco.zToMuMu_cfi as zmumu zToMuMuGolden = zmumu.zToMuMu.clone( massMin = cms.double(40) )
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010
Sequence Operators
54
“+” as ‘follows’:
• Use if the input of the previous module/sequence is not required
trDigi = cms.Sequence(siPixelDigis + siStripDigis)
“*” as ‘depends on’:
• If module depends on previously created products
p1 = cms.Path(pdigi * reconstruc>on)
• Enforced and checked by schedulerCombining:
• By using () grouping is possible
(ecalRecHits + hcalRecHits) * caloTowers
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010
EventContentAnalyzer
55
At every posi>on in the schedule one can inspect what products can be accessed:
We obtain a similar thing using the command line:
edmDumpEventContent
process.dump = cms.EDAnalyzer(‘EventContentAnalyzer’)
process.p = cms.Path(... + dump + ...)
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010
The Output
56
• Standard event contents are defined centrally: ConfiguraUon.EventContent.EventContent_cff
• Output files are wriUen via the PoolOutputModule
• Details on selectEvents in the SWGuide:
hUps://twiki.cern.ch/twiki/bin/view/CMS/SWGuideEDMPathsAndTriggerBits
cms.OutputModule("PoolOutputModule”, outputCommands = RECOEventContent.outputCommands, fileName = cms.untracked.string('TTbar_cfi_GEN_SIM_DIGI.root'), SelectEvents = cms.untracked.PSet( SelectEvents = cms.vstring('*Electron:HLT’) )
)
malik@fnal.gov @Dept. of Physics, UPRM, 1 December 2010
Processing steps
57
There are quite a few standard steps in simula>on:
• GEN (generator + generator level objects like genJets) • SIM (energy deposits in the detector material)
• SIMDIGI (actual detector response + noise )
• L1 (Level1 trigger)
• DIGI2RAW (conversion into RAW data format)
• HLT (High Level Trigger) • RAW2DIGI (conversion from RAW to digi data format)
• RECO (reconstruc>on)
Recommended