Embed Size (px)
Citation preview
LIGO-G040107-00-Z LSC Meeting March 2004
Search Pipelines for Binary Inspiral
Duncan BrownInspiral Working Group
University of Wisconsin-Milwaukee
LIGO-G040107-00-Z
LIGO-G040107-00-Z LSC Meeting March 2004
S2 Inspiral Pipeline
lalapps_tmpltbank
LIGO-G040107-00-Z LSC Meeting March 2004
S2 Inspiral Pipeline
lalapps_inspiral
LIGO-G040107-00-Z LSC Meeting March 2004
S2 Inspiral Pipeline
lalapps_inca
LIGO-G040107-00-Z LSC Meeting March 2004
S2 Inspiral Pipeline
lalapps_inca
LIGO-G040107-00-Z LSC Meeting March 2004
S2 Inspiral Pipeline
lalapps_inspiral
LIGO-G040107-00-Z LSC Meeting March 2004
S2 Inspiral Pipeline
lalapps_inca
LIGO-G040107-00-Z LSC Meeting March 2004
S2 Inspiral Pipeline
lalapps_inca
LIGO-G040107-00-Z LSC Meeting March 2004
S2 Inspiral Pipeline
LIGO_LW XML file
LIGO-G040107-00-Z LSC Meeting March 2004
Pipeline Infrastructure Requirements
• Ensure that all data is analyzed
• Automate pipeline as much as possible
• Provide flexible pipeline for testing and tuning
• Allow easy construction of complex workflows
• Simple reusable infrastructure
• Easy to debug
LIGO-G040107-00-Z LSC Meeting March 2004
Pipeline Implementation
• Condor to manage job submission to cluster
• lalapps code to execute components of pipeline» Use LAL functions for GW analysis
• Condor DAGman to manage execution of pipeline
• Standard file types for I/O» Read AS_Q and calibration from frame data» Writes triggers as LIGO_LW XML» Can write r(t), x2(t), PSD, filter data as frames
LIGO-G040107-00-Z LSC Meeting March 2004
Creation of the DAG
• Simple Python modules in lalapps to build scripts that write pipeline
• lalapps/src/lalapps/pipeline.py» Read segwizard files» Manipulate science segments (union, intersection, inverse)» Create Condor Jobs and DAGs
• lalapps/src/inspiral/inspiral.py» Construction of DAG nodes specific to inspiral
• lalapps/src/inspiral/inspiral_pipe.in» Use building blocks to construct pipeline
LIGO-G040107-00-Z LSC Meeting March 2004
Putting It All Together
data = pipeline.ScienceData()data.read(‘segwizard.txt’,2048)data.make_chunks(length,overlap,isplay)dag = pipeline.CondorDAG(‘mydag.dag’)datafind_job = pipeline.LSCDataFindJob()inspiral_job = inspiral.InspiralJob()
for seg in data: df = pipeline.LSCDataFindNode() df.set_start(seg.start()) df.set_end(seg.end()) for chunk in seg: insp = inspiral.InspiralNode() insp.set_start(chunk.start()) insp.set_end(chunk.end()) insp.add_parent(df)
dag.write()
LIGO-G040107-00-Z LSC Meeting March 2004
Putting It All Together
data = pipeline.ScienceData()data.read(‘segwizard.txt’,2048)data.make_chunks(length,overlap,isplay)dag = pipeline.CondorDAG(‘mydag.dag’)datafind_job = pipeline.LSCDataFindJob()inspiral_job = inspiral.InspiralJob()
for seg in data: df = pipeline.LSCDataFindNode() df.set_start(seg.start()) df.set_end(seg.end()) for chunk in seg: insp = inspiral.InspiralNode() insp.set_start(chunk.start()) insp.set_end(chunk.end()) insp.add_parent(df)
dag.write()
LIGO-G040107-00-Z LSC Meeting March 2004
Putting It All Together
data = pipeline.ScienceData()data.read(‘segwizard.txt’,2048)data.make_chunks(length,overlap,isplay)dag = pipeline.CondorDAG(‘mydag.dag’)datafind_job = pipeline.LSCDataFindJob()inspiral_job = inspiral.InspiralJob()
for seg in data: df = pipeline.LSCDataFindNode() df.set_start(seg.start()) df.set_end(seg.end()) for chunk in seg: insp = inspiral.InspiralNode() insp.set_start(chunk.start()) insp.set_end(chunk.end()) insp.add_parent(df)
dag.write()
LIGO-G040107-00-Z LSC Meeting March 2004
Putting It All Together
data = pipeline.ScienceData()data.read(‘segwizard.txt’,2048)data.make_chunks(length,overlap,isplay)dag = pipeline.CondorDAG(‘mydag.dag’)datafind_job = pipeline.LSCDataFindJob()inspiral_job = inspiral.InspiralJob()
for seg in data: df = pipeline.LSCDataFindNode() df.set_start(seg.start()) df.set_end(seg.end()) for chunk in seg: insp = inspiral.InspiralNode() insp.set_start(chunk.start()) insp.set_end(chunk.end()) insp.add_parent(df)
dag.write()
LIGO-G040107-00-Z LSC Meeting March 2004
Putting It All Together
data = pipeline.ScienceData()data.read(‘segwizard.txt’,2048)data.make_chunks(length,overlap,isplay)dag = pipeline.CondorDAG(‘mydag.dag’)datafind_job = pipeline.LSCDataFindJob()inspiral_job = inspiral.InspiralJob()
for seg in data: df = pipeline.LSCDataFindNode() df.set_start(seg.start()) df.set_end(seg.end()) for chunk in seg: insp = inspiral.InspiralNode() insp.set_start(chunk.start()) insp.set_end(chunk.end()) insp.add_parent(df)
dag.write()
LIGO-G040107-00-Z LSC Meeting March 2004
Putting It All Together
data = pipeline.ScienceData()data.read(‘segwizard.txt’,2048)data.make_chunks(length,overlap,isplay)dag = pipeline.CondorDAG(‘mydag.dag’)datafind_job = pipeline.LSCDataFindJob()inspiral_job = inspiral.InspiralJob()
for seg in data: df = pipeline.LSCDataFindNode() df.set_start(seg.start()) df.set_end(seg.end()) for chunk in seg: insp = inspiral.InspiralNode() insp.set_start(chunk.start()) insp.set_end(chunk.end()) insp.add_parent(df)
dag.write()
LIGO-G040107-00-Z LSC Meeting March 2004
Putting It All Together
data = pipeline.ScienceData()data.read(‘segwizard.txt’,2048)data.make_chunks(length,overlap,isplay)dag = pipeline.CondorDAG(‘mydag.dag’)datafind_job = pipeline.LSCDataFindJob()inspiral_job = inspiral.InspiralJob()
for seg in data: df = pipeline.LSCDataFindNode() df.set_start(seg.start()) df.set_end(seg.end()) for chunk in seg: insp = inspiral.InspiralNode() insp.set_start(chunk.start()) insp.set_end(chunk.end()) insp.add_parent(df)
dag.write()
LIGO-G040107-00-Z LSC Meeting March 2004
S2 Inspiral DAG
LIGO-G040107-00-Z LSC Meeting March 2004
Conclusions
• Use of Condor DAGman has been very successful» Simplifies management of analysis workflow» More time to concentrate on scientific questions
• Infrastructure written in lalapps is simple to use» Python modules are documented in lalapps documentation
• Reusable code» LIGO/TAMA inspiral analysis (Steve Fairhurst)» Stochastic lalapps pipeline (Adam Mercer)
• Fast, simple, efficient!
