Upload
corin
View
26
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Reinforcement Learning applied to Meta -scheduling in grid environments Bernardo Costa Inês Dutra Marta Mattoso. Outline. Introduction Algorithms Experiments Conclusions and Future work. Introduction Algorithms Experiments Conclusions and Future work. Introduction. Relevance: - PowerPoint PPT Presentation
Citation preview
ISPA 2008 APDCT Workshop 1
Reinforcement Learning applied to Meta-scheduling in grid environments
Bernardo Costa
Inês Dutra
Marta Mattoso
ISPA 2008 APDCT Workshop 2
Outline
Introduction Algorithms Experiments Conclusions and Future work
ISPA 2008 APDCT Workshop 3
Introduction Algorithms Experiments Conclusions and Future work
ISPA 2008 APDCT Workshop 4
Introduction
Relevance: Available grid schedulers usually do not employ a
strategy that may benefit a single or multiple users.
Some strategies employ performance information dependent algorithms (pida).
Most works are simulated.
Difficulty: monitoring information not reliable due to network latency.
ISPA 2008 APDCT Workshop 5
Introduction Algorithms Experiments Conclusions and Future work
ISPA 2008 APDCT Workshop 6
Study of 2 Algorithms (AG) A. Galstyan, K. Czajkowski, and K.
Lerman. Resource allocation in the grid using reinforcement learning. In AAMAS, pages 1314–1315. IEEE, 2004.
(MQD) Y. C. Lee and A. Y. Zomaya. A grid scheduling algorithm for bag-of-tasks applications using multiple queues with duplication. 5th IEEE/ACIS International Conference on Computer and Information Science and 1st IEEE/ACIS International Workshop on Component-Based Software Engineering, Software Architecture and Reuse. ICIS-COMSAR, pages 5–10, 2006.
ISPA 2008 APDCT Workshop 7
What is reinforcement learning?
Machine learning technique used to learn behaviours given a series of temporal events.
Non-supervised learning. Based on the idea of rewards and
punishments.
ISPA 2008 APDCT Workshop 8
Algorithms
AG and MQD use reinforcement learning to associate an efficiency rank to an RMS.
Reinforcement learning native to AG. MQD was modified to use this technique to
estimate computational power of an RMS. AG allocates RMS in a greedy and probabilistic
way. MQD allocates RMS associatively and
deterministically.
ISPA 2008 APDCT Workshop 9
Algorithms
Calculating efficiency: Reward is assigned to RMS that has performance
better than average. Reward can be negative (punishment). RMS may not change its efficiency value.
ISPA 2008 APDCT Workshop 10
Algorithms
Calculating efficiency: parameters: and l is the importance of the time spent executing a
task affects rewarding.
l is a learning parameter
ISPA 2008 APDCT Workshop 11
Algorithms
AG: With high prob, associates job to the best
available RMS, otherwise, selects randomly. MQD:
Groups of jobs sorted according execution time are associated to an RMS. Most efficient executes the heaviest jobs. Initial allocation to estimate RMS´ efficiency
ISPA 2008 APDCT Workshop 12
Algorithm AG
ISPA 2008 APDCT Workshop 13
J1 J3J2 J4 J5
J6 J7 J8 J9
R1E = 0
R2E = 0
R3E = 0
ISPA 2008 APDCT Workshop 14
J4 J5
J6 J7 J8 J9
R1E = 0
R2E = 0,3
R3E = -0,3
ISPA 2008 APDCT Workshop 15
J7 J8 J9
R1E = 0,3
R2E = 0,057
R3E = 0,51
ISPA 2008 APDCT Workshop 16
Algorithm MQD
ISPA 2008 APDCT Workshop 17
J140
J350
J215
J430
J510
J670
J720
J820
J940
R1E = 0
R2E = 0
R3E = 0
ISPA 2008 APDCT Workshop 18
J140
J350
J215
J430
J510
J670
J720
J820
J940
R1E = 0
R2E = 0
R3E = 0
ISPA 2008 APDCT Workshop 19
J140
J350
J430
J670
J820
J940
R1E = 0,3
R2E = -0,3
R3E = 0
ISPA 2008 APDCT Workshop 20
J140
J350
J820
R1E = 0,09
R2E = -0,09
R3E = -0,3
ISPA 2008 APDCT Workshop
Avg per proc
Global Avg
ISPA 2008 APDCT Workshop
ISPA 2008 APDCT Workshop
ISPA 2008 APDCT Workshop
ISPA 2008 APDCT Workshop 25
Introduction Algorithms Experiments Conclusions and Future work
ISPA 2008 APDCT Workshop 26
Experiments
GridbusBroker:No need to install it in other grid sitesOnly requirement: ssh access to a grid
nodeRound-robin scheduler (RR)
Limitations:Does not support job duplication Imposes a limit on the number of active
jobs per RMS
ISPA 2008 APDCT Workshop 27
Experiments
Resources in 6 grid sites:LabIA: 24 (Torque/Maui)LCP: 28 (SGE)Nacad: 16 (PBS PRO)UERJ: 144 (Condor)UFRGS: 4 (Torque)LCC: 44 (Torque)
ISPA 2008 APDCT Workshop 28
Experiments
Objective: study performance of algorithms in a real grid environment.
Application: bag-of-tasks. CPU intensive.
Duration between 3 and 8 minutes.
ISPA 2008 APDCT Workshop 29
Experiments
Evaluation criteria: makespan.
Makespan was normalized with respect to RR
ISPA 2008 APDCT Workshop 30
Experiments
Phase I: Tuning of parameters and l 500 jobs.
Phase II: Performance of re-scheduling. Later load increased to 1000 jobs.
ISPA 2008 APDCT Workshop 31
Experiments
One experiment is a run of consecutive executions of RR, AG and MQD.
A scenario is a set of experiments with fixed parameters.
For each scenario: 15 runs. T-tests to verify statistical difference
beteween AG/MQD e RR, with 95% confidence (the results have a normal distribution).
ISPA 2008 APDCT Workshop
ISPA 2008 APDCT Workshop 33
Experiments (Phase I)
ISPA 2008 APDCT Workshop 34
Experiments (Phase II)
ISPA 2008 APDCT Workshop 35
Introduction Algorithms Experiments Conclusions and Future work
ISPA 2008 APDCT Workshop 36
Conclusions and Future work
Results showed that was possible to achieve optimizations with both AG and MQD wrt RR
Experiments validate MQD simulation results found in the literature.
Reinforcement learning is a promising technique to classify resources in real grid environments.
ISPA 2008 APDCT Workshop 37
Conclusions and Future work
Study the behavior of AG and MQD with other kinds of applications, e.g., data intensive, with dependencies.
ISPA 2008 APDCT Workshop 38
Questions?
ISPA 2008 APDCT Workshop 39
Annex
ISPA 2008 APDCT Workshop 40
Definições
Gerenciador de recursos: sistema que gerencia a submissão e execução de jobs dentro de um domínio específico.
Resource Management System (RMS): sinônimo para gerenciador de recursos.
Batch job scheduler: escalonador típico de um RMS. Ex: SGE, PBS/Torque.
ISPA 2008 APDCT Workshop 41
Definições
Meta-escalonador: um escalonador que não tem acesso direto aos recursos, mas apenas aos RMS que os gerenciam.
Aprendizado por reforço: técnica que induz um agente a tomar decisões por meio de recompensas oferecidas.
Makespan: tempo total gasto por um meta-escalonador para finalizar a execução de um conjunto de jobs a ele designado.
ISPA 2008 APDCT Workshop 42
Definições
Job: aplicativo submetido ao grid por um usuário, executado em geral por um RMS. Exemplos de tipos de jobs: Bag-of-Tasks: jobs que não possuem relação de
dependência ou precedência explícita entre si. Troca de parâmetros (APST): jobs de um mesmo
executável que diferenciam-se por um valor de entrada que varia entre as execuções.
ISPA 2008 APDCT Workshop 43
ISPA 2008 APDCT Workshop 44