Upload
ethel-butler
View
223
Download
0
Embed Size (px)
DESCRIPTION
automating of job submission in Athena/AthenaMC I. motivation, mc simul:5 recon: evgen:50 79 tariq mahmoud hepcg workshop, siegen, need automate job (re)submission
Citation preview
GangaTasksGangaTasksAutomating of job Automating of job submission wit submission wit Athena/AthenaMC in GangaAthena/AthenaMC in Ganga
hepcg workshop, siegen, 13.12.2007hepcg workshop, siegen, 13.12.2007
johannes ebke, johannes elmsheuser, johannes ebke, johannes elmsheuser, tariq tariq mahmoudmahmoud
tariq mahmoud hepcg workshop, siegen, 13122007
automating of job submission in Athena/AthenaMC
outline
I. introduction, motivation
II. concept of tasksIII. some illustration IV. conclusion
automating of job submission in Athena/AthenaMC
I. motivation, mc
simul:5
recon:10
1
1 3 5 97
2 4 6 108
1
evgen:50
7 9
tariq mahmoud hepcg workshop, siegen, 13122007
3 5
3
6
3 5
63
3
6
5
6
3
3
3
need automate job
(re)submission
automating of job submission in Athena/AthenaMC
I. motivation, mc
tariq mahmoud hepcg workshop, siegen, 13122007
1 3 5 972 4 6 108
1
1 9
7
7
7 1 3 5 972 4 6 108
1
1 7 9
36
3 5
633
5
56
566
6
1 3 5 972 4 6 108
1
1 7
36
3 5
63
1 3 5 972 4 6 108
1
1 7 9
3 5
3 5
53
1 3 5 972 4 6 108
1
1 7 9
36
3 5
63756
1 3 5 972 4 6 108
1
9
7
7
3
5
55
1 3 5 972 4 6 108
1
1 7 9
36
3 5
63
automating of job submission in Athena/AthenaMC
I. motivation, mc
1 3 5 972 4 6 108
1
1 7 9
36
3 5
63756
1 3 5 972 4 6 108
1
1 9
7
7
3
5
55
1 3 5 972 4 6 108
1
1 7 9
36
3 5
63
1 3 5 972 4 6 108
1
1 9
7
7
3
5
5
6 7
4
4
4
4need automate job
(re)submission
need job chaining
(interconnection)
tariq mahmoud hepcg workshop, siegen, 13122007
automating of job submission in Athena/AthenaMC
I. motivation, analysis
complete dataset
tariq mahmoud hepcg workshop, siegen, 13122007
A,B,C: f1,f2,f3, …,fn
A,B,C f1,f2,f3A,B,C f4,f5,f6
A,B,C f7,f8,f9A,B,C f7,f8,f9
A,B,C f4,f5,f6
A,B,C f7,f8,f9A,B,C fn-2,fn-1,fn
incomplete datasetA: f1,f2,f3B: f1,f5,f6,f7,f8,f9C: f1,f2,f7,f8,f9,f10
A,C f1,f2B f5,f6
A f3
B,C f7,f8,f9C f10
automating of job submission in Athena/AthenaMC II. concept of tasks
taskfew parameters: task parameters: float, name, number of events etc.
process parameters: datasets, process name, geometry etc.
input/output specifications Athena version
AbstractJobgenerated automatically: name, task, status, necessary, prerequisites, run limit, number of attempts and attempt jobs, …
AbstractJobAbstractJob
jobjob
job Ganga jobs
tariq mahmoud hepcg workshop, siegen, 13122007
automating of job submission in Athena/AthenaMC II. concept of tasks: GangaTasks
MCTask
MCJobMCJobMCJob
AnaTask
AnaJob
Task
AnaJobAnaJob
repository/tools.data
TaskList(GangaObject)
GPI.tasks
tariq mahmoud hepcg workshop, siegen, 13122007
automating of job submission in Athena/AthenaMC III. illustration, resubmission
tariq mahmoud hepcg workshop, siegen, 13122007
automate job (re)submission
need job chaining
(interconnection)
automating of job submission in Athena/AthenaMC
III. illustration: chaining task stepsmc.total_events = 200;
mc.events_per_job={"evgen": 50, "simul": 5, "recon":10}
1 3 5 7 9
1 3 5 7 9
1 3 5 7 9
1 3 5 7 9
1 3 5 972 4 6 108
1 3 5 972 4 6 108
1 3 5 972 4 6 108
1 3 5 972 4 6 108
2 3 41 evgen:i-0
simul:i-j
recon:i-k
tariq mahmoud hepcg workshop, siegen, 13122007
mc.total_events = 200 mc.events_per_job= {"evgen": 50, "simul": 5, "recon":10} mc.float=10
AbstractJob.prerequisites()
tariq mahmoud atlas sw week, cern, 24102007
automating of job submission in Athena/AthenaMC III. illustration: interconnections,
prerequisites
automate job (re)submission
job chaining (interconnection)
the user sets few parameters … and gets
RESULTS
automating of job submission in Athena/AthenaMC
IV. conclusion
GangaTask package: automates job management (small scale production system)works with Ganga Athena and Ganga AthenaMC
(available from Ganga443 onwards) works at cernwiki with tutorial: https://twiki.cern.ch/twiki/bin/view/Atlas/GangaTasksapplicable to any job submission-software on the grid
To do: 1. implementing site-test-code 2. making use of GangaRobot3. act on user suggestionstariq mahmoud hepcg workshop, siegen, 13122007
Configure once – run anywhere