A Framework for Divisible Load E-science
Applications in Optical Grids
A thesis
Submitted to the Faculty of Graduate Studies and Research
In Partial Fulfillment of the requirements
For the degree of
Doctor of Philosophy
in
Engineering
University of Regina
By
Mohamed Moustafa Mohamed Ahmed Abouelela
Regina, Saskatchewan
May, 2013
c©Copyright 2013: M. Abouelela
UNIVERSITY OF REGINA
FACULTY OF GRADUATE STUDIES AND RESEARCH
SUPERVISORY AND EXAMINING COMMITTEE
Mohamed Moustafa Mohamed Ahmed Abouelela, candidate for the degree of Doctor of
Philosophy in Electronic Systems Engineering, has presented a thesis titled, A Framework
for Divisible Load E-Science Applications for Optical Grids, in an oral examination held
on April 18, 2013. The following committee members have found the thesis acceptable in
form and content, and that the candidate demonstrated satisfactory knowledge of the subject
material.
External Examiner: *Dr. Diwakar Krishnamurthy, University of Calgary
Supervisor: Dr. Mohamed El-Darieby, Software Systems Engineering
Committee Member: Dr. Craig M. Gelowitz, Software Systems Engineering
Committee Member: Dr. Malek Mouhoub, Department of Computer Science
Committee Member: **Dr. Mehran Mehrandezh, Industrial Systems Engineering
Chair of Defense: Dr. Yuchao Zhu,
Department of Political Science
*teleconference
**Not present at defense
A Framework for Divisible Load E-science Applications in
Optical Grids
Mohamed Abouelela
Submitted for the Degree of Doctor of Philosophy
May 2013
Abstract
E-science applications require discovering, collecting, transferring and processing large
volumes of scientific data. In divisible load e-science applications, data is generated
and stored in geographically distributed repositories (e.g., instruments, sensors, cam-
eras, satellites and storage facilities). The generated data can be divided into in-
dependent subsets to be analysed distributed at many computing locations. Such
applications usually require optical networking for fast and reliable data transfer. In
this thesis, we propose a framework for divisible load applications in optical grids.
Within this framework, schedulers are developed to co-schedule computational and
optical network resources. Moreover, processes to handle divisible load applications
in different architectures including centralized, hierarchical, peer-to-peer and super-
peer are proposed. Fault management techniques are introduced to handle network
faults by considering distinctive characteristics of e-science applications and optical
grids. This research conducts a comprehensive study of different algorithms, tech-
niques and processes that have been introduced within the framework. The results
provide guidelines to build more efficient, scalable and reliable optical grid systems.
The results can serve as a guide to the best choices in selecting the scheduling algo-
rithms, fault management techniques and architectures according to different network
and application parameters.
ii
Acknowledgements
It gives me great pleasure in expressing my gratitude to all those people who have
supported me and had their contributions in making this thesis possible. First and
foremost, I must acknowledge and thank the Almighty God for blessing, protecting
and guiding me throughout this period. I could never have accomplished this without
the faith I have in the Almighty.
I wish to express my deepest and sincere gratitude and appreciation to my super-
visor, Prof. Mohamed El-Darieby, Associate Professor of Software Systems Engineer-
ing department, for his support, guidance and helpful suggestions. His guidance has
served me well and I owe him my heartfelt appreciation.
I am very grateful to Faculty of Graduate Studies and Research, University of
Regina for an FGSR Research Award and Graduate Scholarship.
My family has played always an important role in my studies, so I would like to
express my deep gratitude and appreciation to my parents, my wife and my sister for
their continuous support.
iii
Table of Contents
1 Introduction 1
1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.2 Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.3 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.4 Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.5 Proposed Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.6 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2 Related Work 18
2.1 Optical Grid Projects . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.2 Inter Domain Operability in Optical Grids . . . . . . . . . . . . . . . 21
2.3 Grid Network Architecture . . . . . . . . . . . . . . . . . . . . . . . . 23
2.4 Multi-source Divisible Load Job Scheduling . . . . . . . . . . . . . . . 26
2.5 Network Recovery Mechanisms . . . . . . . . . . . . . . . . . . . . . 28
iv
2.5.1 Recovery Mechanisms in Optical Networks . . . . . . . . . . . 28
2.5.2 Recovery Mechanisms in Optical Grids . . . . . . . . . . . . . 29
3 Framework Overview and Methodology 31
3.1 High Level Framework . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.1.1 Optical Grid System Architecture . . . . . . . . . . . . . . . . 32
3.1.2 Core Components . . . . . . . . . . . . . . . . . . . . . . . . . 37
3.1.3 Multi-source Divisible Load Job Scheduling . . . . . . . . . . 39
3.1.4 Fault Management . . . . . . . . . . . . . . . . . . . . . . . . 41
3.2 Performance Evaluation Methodology . . . . . . . . . . . . . . . . . . 44
3.2.1 Input Parameters . . . . . . . . . . . . . . . . . . . . . . . . . 46
3.2.2 Performance Metrics . . . . . . . . . . . . . . . . . . . . . . . 48
4 Core Components 51
4.1 Resource Information Aggregation and Abstraction . . . . . . . . . . 51
4.1.1 Scalar Abstraction Techniques . . . . . . . . . . . . . . . . . . 52
4.1.2 Time Availability Aggregation Algorithms . . . . . . . . . . . 59
4.2 Load Balancing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
4.2.1 DLA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
4.2.2 NADLA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
4.2.3 GA-LD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
v
4.3 Path Computing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
4.4 Performance Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . 75
4.4.1 Performance of Aggregation and Abstraction Techniques . . . 75
4.4.2 Performance of Load Balancing Algorithms . . . . . . . . . . . 79
4.4.3 Performance of Path Computing Algorithms . . . . . . . . . . 80
5 Scheduling Multi-Source Divisible Load Applications 86
5.1 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
5.2 GA Based Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
5.2.1 GA Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
5.2.2 Fitness Function . . . . . . . . . . . . . . . . . . . . . . . . . 91
5.2.3 Initialization . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
5.3 DLT Based Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . 94
5.4 Iterative Scheduling Algorithm . . . . . . . . . . . . . . . . . . . . . . 96
5.5 Performance Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . 99
5.5.1 Performance of GA Based Algorithm . . . . . . . . . . . . . . 99
5.5.2 DLT Based Algorithm vs. Iterative Scheduling Algorithm . . . 104
6 Scheduling Divisible Loads in Different Architectures 112
6.1 Hierarchical Scheduling: Top Down Approach . . . . . . . . . . . . . 114
6.2 Hierarchical Scheduling: Bottom Up Approach . . . . . . . . . . . . . 116
vi
6.3 P2P Scheduling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
6.3.1 P2P Resource Discovery . . . . . . . . . . . . . . . . . . . . . 120
6.3.2 P2P Resource Allocation . . . . . . . . . . . . . . . . . . . . . 123
6.4 Superpeer Scheduling . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
6.5 Performance Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . 130
6.5.1 Comparison Between Different Architectures . . . . . . . . . . 130
6.5.2 Performance of P2P Scheduling Process . . . . . . . . . . . . . 137
6.5.3 Performance of Superpeer Scheduling Process . . . . . . . . . 158
6.5.4 Performance of Hierarchical Scheduling Process: Top Down Ap-
proach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
7 Fault Management 172
7.1 Grid Based Network Restoration Techniques . . . . . . . . . . . . . . 173
7.1.1 Re-routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
7.1.2 Relocation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
7.1.3 Divisible Load Relocation . . . . . . . . . .
