Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
1
Paper Number 097469 1
High performance computing application to address non-point source pollution at a 2
watershed level 3
Chetan Maringanti, 4
Graduate Student, Department of Agricultural and Biological Engineering, Purdue 5 University, West Lafayette, IN. 47906, email: [email protected] 6
Indrajeet Chaubey, 7
Associate Professor, Department of Agricultural and Biological Engineering and 8 Department of Earth and Atmospheric Sciences, Purdue University, West Lafayette, IN. 9 47906, email: [email protected] 10
11
Abstract. Best management practices (BMPs) have been proven to effectively reduce the 12
Nonpoint source (NPS) pollution loads from agricultural areas. BMP selection and placement 13
problem needs to be addressed for the placement of BMPs in a watershed at a field scale that 14
would achieve maximum pollution reduction subjected to minimum cost increase for the 15
placement of BMPs in the watershed through optimization techniques. The BMP selection 16
problem is linked with a hydrologic and water quality simulation model to estimate the pollution 17
loads at various locations in the watershed. However, the optimization techniques get very 18
complicated as the size of the watershed gets larger as the design variables increase 19
proportionately. Also, the BMP optimization is linked to the watershed level hydrologic and 20
water quality simulation mode and this makes the problem require very high amount of 21
computation time for the simulations to get to a near optimal set of BMP placement in the 22
2
watershed. In this regard, the high performance computing (HPC) helps a great deal by providing 23
a platform for running the simulations on a multitude of processors rather than a single 24
processor. Unix serves as the de-facto operating system for most of the HPC applications as the 25
architecture allows to run simulations that take a long time to run in much lesser time when 26
compared to the Windows OS machines. In the present work we have used the Soil and Water 27
Assessment Tool (SWAT) model in combination with a genetic algorithm (GA) optimization 28
technique NSGA-II to optimally select and place BMPs at a watershed scale. We have tested the 29
application of the optimization algorithm to reduce sediment, nitrogen, and phosphorus from the 30
L’Anguille River Watershed, Arkansas and pesticide from Wildcat Creek Watershed, Indiana. 31
The SWAT model was compiled in Unix and the optimization routine for BMP selection and 32
placement was coded as an objective function in NSGA-II program in C language. The 33
optimization model was compiled in Unix to optimize the various BMPs that can be placed at a 34
field scale, which is represented as a Hydrologic Response Unit ( HRU) in SWAT. 35
Keywords. High performance computing, BMP, optimization, nonpoint source pollution 36
37
38
39
40
41
42
43
3
Introduction 44
Best management practices (BMP) selection and placement problem consists of finding an 45
optimum solution for BMP placement in a watershed that will achieve the two most important 46
objectives of maximum pollution reduction with a minimum net cost increase for the 47
implementation of BMPs. It is required to optimize the two objectives with the variables being 48
the BMPs at every farm level in the watershed. Optimization algorithms such as genetic 49
algorithms are widely used for BMP selection and placement in a watershed. Most of the 50
optimization models developed are linked dynamically with a watershed level simulation model 51
to estimate the pollutant loads at a watershed scale during the optimization [Arabi et al., 2006; 52
Bekele and Nicklow, 2005; Srivastava et al., 2002]. Presence of the dynamic linkage in the BMP 53
selection and placement models restrict their application to relatively smaller watersheds. Also, 54
the number of possible solutions to be searched during optimization increases exponentially with 55
the number of BMPs considered for implementation due to which a vast number of BMPs were 56
not considered. This problem was addressed through development of a novel methodology that 57
utilizes a database, called as BMP tool, to estimate the pollution effectiveness of any given BMP 58
and therefore removes the requirement of a dynamic linkage with the watershed simulation 59
model during the optimization [Gitau et al., 2006; Maringanti, 2008]. Also, the model for 60
optimization can be run for multiple pollutants and for multiple years of consideration 61
independent to each other and this is a slightly computationally intensive job. More information 62
regarding the model and the method used can be obtained from Maringanti et al., [2009]. In 63
order to validate the applicability of the solutions obtained after optimization using the BMP 64
tool, some solutions need to be picked up from the Pareto-optimal front and evaluated through 65
4
the watershed model to check if similar pollution reductions were obtained. The evaluation of 66
these solutions using the watershed model is again a computationally time intensive job. 67
High performance computing (HPC), that consists of super computers and cluster of processors 68
are used to solve problems that are highly computation intensive. Multiple jobs that are required 69
to accomplish a particular task are distributed to make them run in parallel among multiple 70
processors and therefore decreasing the overall wall clock time for execution. HPC has been 71
used extensively in calibrating hydrologic [Confesor and Whittaker, 2007] and environmental 72
modeling [Vrugt et al., 2006]. As the number of processors increase, the overall computation 73
time is reduced in a range of 50% - 80% [Vrugt et al., 2006]. 74
HPC would greatly help reduce the computation time required to run replicates of the 75
optimization algorithm for multiple pollutants and multiple average years of concern. Also, the 76
computation time required to evaluate the watershed model the solutions picked from the BMP 77
tool output would be highly reduced using HPC. 78
79
Objectives 80
The overall goal of this study is to evaluate the performance of HPC based on a Unix based 81
cluster when compared to the performance of a single processor in windows. In order to achieve 82
this overall goal, the following tasks are performed 83
1) The BMP optimization tool based on a genetic algorithm is run for nitrogen, phosphorus, 84
and sediment along with 5 different averages of pollutant loads (5yr, 10yr, 15yr, 20yr, 85
and 25 yr) in Lincoln Lake Watershed, Arkansas. 86
5
2) The validation of the solutions obtained from the optimization of the BMP optimization 87
tool are used to evaluate the pollutant loads using a watershed model to estimate nitrogen, 88
phosphorus, and sediment in L’Anguille River Watershed; and pesticide in Wildcat Creek 89
Watershed. 90
91
Theoretical Background 92
Multi-objective genetic algorithm (MOGA) 93
94
Genetic algorithms are heuristic based search algorithms, based on Darwin’s theory of evolution, 95
that function well in searching optimum solutions when the search space gets large in the 96
variable and objective function domain. As the BMP selection and placement problem requires 97
two objectives (reduction in pollutant loads and reduction in net cost for placement of BMPs) to 98
be minimized simultaneously, it is required to use a multi-objective genetic algorithm (MOGA) 99
for optimization. Non-dominated sorted genetic algorithm (NSGA-II) (Deb, 1999, 2001; Deb et 100
al., 2002) is one such MOGA that can search a large number of variables and objective functions 101
space to find an optimal solution. The overall computation complexity of the algorithm is 102
O(MN2), which usually is O(MN3) for most of the evolutionary techniques (Deb et al., 2002): 103
where O is ‘order of’, M is the number of objective functions, and N is the population size. Non-104
domination sorting, crowding distance, and elite sets are important properties of the NSGA-II 105
algorithm that ensure that the solutions that are obtained during the optimization have a good 106
spread in the objective function space and the optimization is ensured to reach a Pareto-optimum 107
(Figure 1). This also ensures that the algorithm does not undergo a premature convergence. 108
Crossover and mutation are other genetic operations that are important to generate the future 109
6
offspring based on the parental genetic information and create genetic modification to reach 110
convergence respectively. 111
112
Watershed model (Soil and Water Assessment Tool) 113
The watershed model used in this study was Soil and Water Assessment Tool (SWAT) model. 114
The SWAT is a process based distributed parameter watershed scale simulation model designed 115
for use in gauged as well as ungauged basins to simulate long term effects of various watershed 116
management decisions on hydrology and water quality response (Arnold et al., 1998). The 117
SWAT model divides the watershed into subwatersheds or subbasins based on the outlets 118
selected within the watershed by the user and further divides subbasins into land areas, called 119
hydrologic response units (HRUs), based on land use, management, and soil properties. SWAT 120
performs most of the calculations at HRU level, which are then delivered into the stream present 121
in the particular subbasin and then routed to the downstream subbasin. 122
The input data needed by the model are related to watershed physical characteristics, climate, 123
plant growth, reservoir data (if any), and management practices at HRU level. The typical GIS 124
data needed by the model are digital elevation map (DEM), soil, and land use maps. The climatic 125
input data required by SWAT are precipitation, temperature, solar radiation, relative humidity, 126
and wind on a daily scale from multiple climatic gauge locations. Agricultural activities can be 127
given as input to the model by modifying the management files. SWAT simulates the flow, 128
nutrients, sediment, and chemicals at the subbasin or HRU level. Surface runoff is computed 129
using a modification of the SCS curve number technique (Soil Conservation Service, 1972). 130
Modified Universal Soil Loss Equation (MUSLE) (Williams, 1975) is used by the SWAT model 131
to estimate the soil erosion and sediment yield in the watershed. Nitrogen and phosphorus are 132
7
applied through fertilizer, manure or residue application. Bulk of phosphorus transport takes 133
place in the adsorbed form to the sediment whereas nitrogen is highly dissolvable in water and 134
therefore transported mainly through surface/subsurface flow. The important feature in SWAT is 135
that it aids in modeling the various BMPs (structural and management based) by changing 136
appropriate parameters in the input files of the model. This feature is utilized in the development 137
of BMP tool which estimates the effectiveness of BMPs for a particular pollutant reduction. 138
More details about the development of BMP tool can be obtained from [Maringanti, 2008]. 139
High performance computing facility 140
The Steele supercomputing facility at Purdue University consists of a cluster that is made up of 141
893 Dell dual quad-core computer nodes with an estimate peak performance of 60 teraflops. The 142
machines run on Red Hat Linux operating system with various computational packages installed 143
on each machine. Steele supercomputing facility is the largest among the universities located in 144
the Midwest region of the United States and ranks 104 in the TOP500 list of global 145
supercomputers. The parallel simulations of the various model runs were run on the Steele 146
cluster from which our research group owns 4 nodes (32 processors). 147
Study area and data available 148
Lincoln Lake watershed (LLW) is located within the Illinois River Basin located in Northwest 149
Arkansas. The watershed has a total area of 32 km2 with a mixed land use consisting of pasture, 150
forest, urban residential, urban commercial and water representing 35.84%, 48.6%, 11.9%, 151
1.54% and 2.2% of the watershed area, respectively (Figure 1). The watershed contains more 152
several hundreds of chicken houses. The 303 (d) lists Illinois River an impaired water body with 153
phosphorus being the main cause of impairment. 154
8
L’Anguille river watershed (LRW) is located in the Mississippi delta region of east Arkansas 155
(Figure Error! No text of specified style in document..). The watershed covers an area of 2520 156
km2 and drains the entire stretch (157 km) of the L’Anguille River. LRW is predominantly an 157
agricultural watershed; with more than 60% of the watershed in arable land. The main crops 158
grown in the watershed are rice (26%) and soybean (46%) and represent most (~95%) of the 159
agricultural land in the watershed (Error! Reference source not found.). The L’Anguille River 160
was included in the list of impaired water bodies by the Arkansas Department of Environmental 161
Quality (ADEQ) in 1998 (ADEQ, 2002). Excessive sediment and nutrients are the main source 162
of impairment of the river (ADEQ, 2002). 163
Wildcat Creek watershed (WCW) ( 164
Figure Error! No text of specified style in document..) (USGS 8 digit HUC 05120107) with a 165
drainage area of 1956 km2, located in north central Indiana was used for testing the optimal BMP 166
selection and placement. The watershed is predominately agricultural with 74% row crops (36% 167
Soybean, 38% Corn), 21% pasture, and 3% urban area (NASS, 2001). A high pesticide (atrazine) 168
level from the agricultural areas has degraded the water quality of many watersheds in Indiana 169
(Homes et al., 2001). Various NPS pollution reduction projects are being undertaken in the 170
watershed through funds from the clean water act section 319. These funds, to effectively be 171
utilized to reduce NPS pollutants, need to be combined with a BMP optimal selection and 172
placement tool to choose the best economical and ecological alternative. The proposed multi-173
objective optimization methodology can be applied for the selection and placement of BMPs in 174
Wildcat Creek Watershed for pesticide (atrazine) reduction from areas planted with corn. 175
176
177
9
178
Methodology 179
Figure 5 describes the methodology that is adopted during the computation process. Nitrogen, 180
phosphorus, and sediment are the three pollutants are considered for the LRW. The SWAT 181
model that is used as a pollutant load estimator was simulated for 25 years. Five different annual 182
averages (for 5, 10, 15, 20, and 25 years) were considered to study how the annual pollutant 183
loads are changed due to the averaging process. These 5 averages and 3 different pollutants of 184
concern require 15 different BMP selection and placement model runs to obtain the respective 185
Pareto-optimal solutions after the optimization. These 15 model runs were made to run on 186
Windows machines using a Windows compiled executable and on Steele supercomputer using a 187
Unix compiled executable. The multiple jobs that are required for evaluating the optimization 188
model are distributed into multiple processors on Steele to generate the final Pareto-optimal 189
fronts that provide a range of solutions for the two objective functions. 190
Solutions from the final Pareto-optimal front are picked ensuring that the entire range is covered. 191
N, P, and sediment optimization solutions were used from the LRW and pesticide solutions were 192
used from the WCW. The variables that represent the BMPs to be implemented in each of the 193
HRU are used to modify the input SWAT files and simulate the SWAT model to get the 194
pollutant loads considering the in stream routing processes. The SWAT model used in the 195
simulations was compiled in Unix and the runs are made on the Steele for different possible 196
SWAT input configurations corresponding to the implemented BMPs. 197
198
199
10
200
Results and discussion 201
Table 1 illustrates how the SWAT model computation times vary for the various watersheds 202
considered in the study using both Windows and Unix compiled versions. It is observed that the 203
Unix version of SWAT outperforms the Windows counterpart by being more than 20 times 204
computationally efficient. 205
Output from the SWAT model is used to develop the BMP tool and as an input into the NSGA-II 206
and the optimization run is made with a population size of 800 and run for 20,000 generations. 207
The optimization run took more than 12 hours to run on a Windows machine and less than 1 hour 208
to run on a Unix machine. The 15 optimization runs required were distributed into 15 different 209
processors and therefore the entire computation required less than an hour to get the final optimal 210
solutions. If the same analysis was performed on a single Windows machine, it would have 211
taken more than 7 days to complete the optimization. Figure 6 describes the progress of the 212
Pareto-optimal front with the increasing generation number for sediment reduction. It is 213
observed that the optimization results provide a range of solutions for the pollution reduction 214
from the baseline. Similar analysis was performed for phosphorus and nitrogen. 215
Figure 7 presents the progress of the optimization front as the generation number increases for 216
pesticide control in LRW. It is observed that at the end of 5000 generations, the solutions are 217
spread well to provide a range of solutions for the selection and placement of BMPs in the 218
watershed for pesticide control. 8 different solutions spanning the entire spread of the Pareto-219
optimal front at the end of generations 10, 20, 50, 5000 are selected and evaluated using the 220
SWAT model to estimate the pollutant loads. These 32 different model runs were independent 221
11
of each other and therefore were evaluated in 32 different processors. This reduced the 222
computation time from 128 minutes on a single Windows machine to less than a minute when 223
run parallel on Steele. 224
Figure 9 presents the progress of the Pareto-optimal front as the generation number increases and 225
the Pareto-optimal front at the end of the optimization for P, N, and sediment reduction in LRW. 226
It is again observed that the Pareto-optimal front at the end of the final generation provides a 227
range of alternative solutions for selection and placement of BMPs that can be chosen based on 228
the required criteria for pollution reduction or available funds for BMP implementation. Figure 229
10 represents the SWAT simulated pollutant loads with the corresponding costs for the solutions 230
20 solutions picked from the range of the Pareto-optimal fronts (Figure 9) for generations 10, 40, 231
100, 1000, and 5000. The 20 picked solutions from the selected generations were set up to run 232
on 20 different processors. The overall computation time to run these 100 different solutions 233
was less than 5 minutes which would have been at least 16 hours when evaluated on a single 234
CPU running Windows. 235
236
Conclusion 237
It is observed from the study that high performance computing (HPC) helps reducing the overall 238
computation time to simulate multiple tasks by distributing them to run in parallel on multiple 239
processors. The overall computation time to run the optimization model for BMP selection and 240
placement was reduced by more than 90% by performing the runs in parallel on a supercomputer 241
in contrast to a single processor run on Windows. Also, the overall computation time required to 242
run the solutions picked from the simplified BMP tool based optimization technique to run the 243
12
SWAT model was also reduced by more than 90% because of the high performance 244
supercomputing alternative chosen when compared to single machine Windows run. The results 245
obtained from this study are encouraging to perform computationally intensive jobs on a 246
distributed parallel architecture. Future work in this regard could be to use high performance 247
computing develop a dynamically linked BMP selection and placement model that distributes the 248
various processes during a particular generation into multiple slave processors and the outputs 249
from the slave processors is received by the master processor that performs various optimization 250
routines to the variables based on the objective function values and therefore considerably 251
reducing the computation time. 252
253
254
255
13
REFERENCES 256
Arabi, M., R. S. Govindaraju, and M. M. Hantush (2006), Cost-effective allocation of watershed 257 management practices using a genetic algorithm, Water Resources Research, 42(10), 258 W10429. 259
260
Bekele, E. G., and J. W. Nicklow (2005), Multiobjective management of ecosystem services by 261 integrative watershed modeling and evolutionary algorithms, Water Resources Research, 262 41(10), 10406. 263
264
Confesor, R., and G. Whittaker (2007), Automatic calibration of hydrologic models with multi-265 objective evolutionary algorithm and Pareto optimization, Journal of the American Water 266 Resources Association, 43(4), 981-989. 267
268
Gitau, M. W., T. L. Veith, W. J. Gburek, and A. R. Jarret (2006), Watershed-level BMP 269 selection and placement in the Town Brook watershed, NY, Journal of the American 270 Water Resources Association, 42(6), 1565-1581. 271
272
Maringanti, C. (2008), Development of a multi-objective optimization tool for selection and 273 placement of BMPs for nonpoint source pollution control, M.S. Thesis. Purdue 274 University, 157. 275
276
Maringanti, C., I. Chaubey, and J. Popp (2009), Development of a Multi-Objective Optimization 277 Tool for the Selection and Placement of BMPs for Nonpoint Source Pollution Control, 278 Water Resources Research, (Accepted). 279
280
Srivastava, P., J. M. Hamlett, P. D. Robillard, and R. L. Day (2002), Watershed optimization of 281 best management practices using AnnAGNPS and a genetic algorithm, Water resources 282 research, 38(3), 1021. 283
284
Vrugt, J. A., B. Ó Nualláin, B. A. Robinson, W. Bouten, S. C. Dekker, and P. M. Sloot (2006), 285 Application of parallel computing to stochastic parameter estimation in environmental 286 models, Computers & Geosciences, 32(8), 1139-1155, doi:10.1016/j.cageo.2005.10.015. 287
288
289 290
14
291
Table 1. Computation time on Windows and Unix machines for various SWAT model 292 simulations used in the study 293
294
Watershed Years of simulation No. of HRUs Computation time
Windows Unix
L’Anguille River Watershed
18 433 10 mins 30 secs
Wildcat Creek Watershed
7 403 4 mins 15 secs
Lincoln Lake Watershed
28 1461 30 mins 3 mins
295 296
15
297 298
Figure 1. Pareto-optimum front progress during multi-objective optimization 299
300
Objective function 1 (0,0)
16
301
302
Figure 2. Land use distribution of Lincoln Lake Watershed (Source: Chiang et al, 2008). 303
304
305 306
17
307
308 Figure Error! No text of specified style in document.. Location of L’Anguille River Watershed 309 in Arkansas and the location of USGS stream gauging stations in the watershed 310
311
312 313
• Colt 07047942
• Palestine 07047950
18
` 314
Figure Error! No text of specified style in document.. Location of Wildcat Creek Watershed 315
in Indiana and the observed gauge locations 316
317 318
19
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
Figure 5. Flow chart that shows the methodology used in the study to distribute the 339 computationally intensive jobs (dashed arrows) to multiple processors. 340
341
SWAT BMP tool
Genetic Algorithm (NSGA-II)
BMP selection and placement tool
Nitrogen Phosphorus Sediment
5yr average pollutant load
10yr average pollutant load
15yr average pollutant load
20yr average pollutant load
25yr average pollutant load
Output Pareto-optimal front of solutions
Pick solutions
Run SWAT
Wildcat Creek and L’Anguille River watersheds
342
343
344
Figure 6. Change in the objective function as the number of generation increases for 5 different 345
annual averages considered for sediment346
15 year
the objective function as the number of generation increases for 5 different
annual averages considered for sediment in LLW.
25 year
5 year
5 year
20
the objective function as the number of generation increases for 5 different
20 year
10 year
21
347
348 Figure 7. Progress of the Pareto-optimal front during optimization of the model for pesticide control in LRW 349 350
22
351
Figure 8. Performance of the solutions obtained from the BMP tool when implemented at the 352
watershed level using the SWAT model to simulate the pesticide loads 353
354
0.4 0.5 0.6 0.7 0.8 0.9 1 1.1
0
20
40
60
80
100
120
140
↓Baseline
Avg. Pesticide Concentration (µg/L)
Net
Co
st (
$/h
a)
Generation 10Generation 20Generation 50Generation 5000
23
355
Figure 9. Pareto-optimal fronts during the optimization (Left, the numbers indicate the 356 generation number for the provided spread) and after the final generation (Right) for Sediment, 357
Phosphorus, and Nitrogen reduction 358
359
360
24
Figure 10. Simulation of the solutions obtained during optimization using the SWAT model for 361
estimating the (a) sediment, (b) phosphorus, and (c) nitrogen loads at Palestine gage station in 362
the watershed 363
364
b
c