Abstract— An artificial intelligence strategy for the localization and monitoring of pollution-caused events caused by contamination in a lake, of a system consisting of an Autonomous Surface Vehicle (ASV) and a network of wireless beacons is proposed and evaluated. Particularly the event of algae bloom is considered. For this purpose, the path planning of the ASV is calculated at different phases with the help of an evolutionary algorithm. The main novelty of the proposed strategy is that it follows an intelligent online learning approach. Therefore, the ASV learns from the environment and makes decisions depending on the collected data. The proposed path planning is based on the adaption of the travelling salesman problem with constraints, using the beacons as geo-localization references and information support infrastructure. The approach consists of different phases that balance the exploration of the lake for searching new events and the exploitation of already discovered ones. A suitable configuration of the fitness function allows an efficient balance between exploration and intensification. Simulation results show that the level of coverage achieved are at least 85% for a situation where up to two dynamic algae blooms occurred at different locations in the lake.

Index Terms—Environmental Monitoring; Autonomous Surface Vehicles; Genetic Algorithms; Autonomous Intelligent Systems; Travelling Salesman Problem; Distributed System.

I. INTRODUCTION

pacarai Lake is the greatest inland body of freshwater in Paraguay, with an approximate area of 90 km2 and

located 25 kilometers east from the Paraguay capital, Asuncion (Fig. 1). It is the main source of water supplying for three towns located at the shore of the lake, such as San Bernardino, Aregua and Ypakarai. It is also a tourist attraction, particularly in the summer months, where people visit the beaches and different aquatic sports are practiced. Its importance is also related to the natural life developed in the wetlands of the basin surrounding of the lake. Moreover, the agriculture activities, which are one of the main activities of the country, have also influenced the lake. However, it has suffered from the pollution from the cities next to it and other disposals that arrive at the lake through water streams.

Y

Back in the 1980’s, a study indicated the excess of phosphorus and nitrogen in the lake, which are harmful substances for human life. In this condition, the lake is said to be in eutrophication [1]. Therefore, it facilitates the appearance of algae. A particular case known as blue-green algae is produced by cyanobacterias, which is shown in (Fig. 2). Some types of cyanobacterias are responsible for producing toxins,

or cyanotoxins, which can cause damage to animals and humans. In 2012, a blue-green algae bloom occurred where Anabaena Microsystis cyanobacteria was detected, which causes a hepatoxin. Even though the levels are below danger conditions, more attention has started to be paid to the lake. The reason for this is that the hepatoxin algae bloom also produces a strong and unpleasant smell in the water and makes it unattractive for bathing and water sport activities. Since then, many actions have been carried out, like frequently monitoring the condition of the lake. So far, these actions have been performed manually at specific points of the lake. In addition, three fixed monitoring stations have been installed for permanent monitoring [2].Nevertheless, a more general evaluation of the lake is still required. This paper proposes the use of an Autonomous Surface Vehicle (ASV) that travels throughout the lake and takes samples as it moves. Therefore, the movements of the ASV should be selected to intelligently cover the lake. An area of the Ypacarai lake is said to be covered if the ASV goes through such area and takes a sample of water. Therefore, the monitoring task can be described as a coverage problem. Suh coverage problem can be transformed into a path planning optimization since the distance travelled by the ASV will be highly correlated with the area covered as long as redundancy is avoided in the selected routes. The path of the ASV can be modeled as travelling salesman problem (TSP) but including constraints such as invalid routes. To model the cities in the TSP, a number of beacons are placed at the shore of the lake. The TSP is a traditional mathematical combinatorial problem that has already been solved in the literature efficiently by using evolutionary algorithms such as genetic algorithms (GA) [3]. This work goes a step forward to a simple coverage of the lake and seeks an adaptive strategy for locating and monitoring new algae blooms. Since the prediction of the appearance and dynamics of the Algae bloom is a difficult task [4], this work considers two aspects for designing an adequate approach to monitor the algae bloom in a lake. First, there are several uncontrolled factors [5], such as temperature and eutrophicated state, that contribute the appearance of it. Therefore, it is difficult to predict the exact moment and place of this occurring. Second, the algae bloom is a dynamic phenomenon that modifies its characteristics (size) in time. Considering these two aspects, the proposed strategy consists of two phases, the exploration phase and the intensification phase. At each phase, an adequate path planning for the ASV is found to achieve the

INTELLIGENT ONLINE LEARNING STRATEGY FOR AN AUTONOMOUS SURFACE VEHICLE IN LAKE ENVIRONMENTS USING EVOLUTIONARY COMPUTATION

Mario Arzamendia, Daniel G. Reina, Sergio L. Toral, Derlis Gregor, Eleana Asimakopoulou, Nik Bessis

1

objective of the phase. More precisely, in the exploration phase, the path planning is aimed at finding one or several algae blooms in the lake. Once the bloom is located, the intensification phase aims at a more detailed monitoring in the region. The strategy divides the monitoring campaign in time frames, and at each frame a specific phase (exploration or intensification) is selected according to the available information from the conditions of the lake. After finding the path, the ASV travels the selected path to collect new information. After finishing the path, a new frame begins, and the type of phase is selected according to the updated information. This process is repeated until the final user stops it. The important idea behind this strategy is that the ASV adjusts their operation by learning the variations that occur in the environment.The paper is structured as follows. Section 2 presents some relevant related works about using genetic algorithms as a path planning technique for robots. Section 3, describes in more details the proposed strategy. Section 4 presents the simulation experiments and the obtained results. Finally, Section 5 concludes this study.

Figure 1. Ypakarai Lake

Figure 2. Example of cyanobacteria appearance

II.RELATED WORKIn this section, the latest research about the application of ASV for environmental purposes is evaluated. Later, an overview about the use of genetic algorithm in path planning is given. At the end, a comparison is made between the

contributions of previous work and the work which is presented here. ASVs have been used in different applications in aquatic environments [6][7]. For example, Dunbabin and Grindham [6] have used two types of robot-based boats for monitoring greenhouse gas emission, like methane, from lakes, reservoirs and wetlands. The first one is a catamaran based type used as a standalone ASV and the second one a custom-designed ASV for cooperative tasks of robot teams. Idris et al. [7] have developed a catamaran type ASV to perform automatic bathymetry measurements in a lake and data telemetry. Perhaps one of the most similar works to the one presented in this paper is from Pradalier et al. [8], which developed an ASV for monitoring cyanobacteria in Lake Zurich. This lake is similar in size to Ypakarai Lake (66.8 km2), but with a larger depth (136 m maximum and a mean of 49.9 m). Because of this depth the ASV includes a winch to submerge a sensor probe for vertical sampling. The monitored cyanobacteria are the Planktothrix rubescens; a different type from the Microsystis Aeroginusa found at Ypakarai Lake. But both cyanobacterias produce the same hepatoxin called microsystin. Another approach for detecting cyanobacteria with the phyaconin sensor is the use of image recognition, as shown in Romero-Vivas [9]. They propose a method to detect Anabaena sp (another type of cyanobacteria), in the sea floor in shallow waters (less than 1 m). The detection is performed from video recorded attached to an ASV with an image stabilization mechanism. The advantage of this technique is that this is a non-invasive method.Autonomous navigation consists of four essential requirements: perception, localization, cognition and path planning, and motion control. Heuristics have been increasingly used in the research domain of path planning [10]. In general, a path planning algorithm determines an obstacle free route that a robot should travel from a starting point to a destination point. The navigation is composed of two levels, a global planner that calculates a first path between origin and destination, and a local planner, which is used to adapt to the dynamic conditions of the environment [11]. Using heuristics in path planning has its advantages, such as finding a relatively good solution in a reasonably period. Some techniques belonging to this group are the Artificial Neural Networks (ANN), Fuzzy Logic (FL) and Natured Inspired Methods (NIM). Genetic Algorithms (GA) belongs to this last category, together with other popular techniques such as Particle Swarm Optimization (PSO) or Ant Colony Optimization (ACO). The GA is a well-studied technique since the 1960’s that has been applied in a variety of scenarios. The main idea behind this technique is that it mimics the evolution behavior in nature, which is the “survival of the fittest”. This expression means that from an initial population of individuals, representing solutions to a given problem, a population evolves through generations producing better individuals or with much higher quality than individuals in the initial population (fittest). Recent work attempts to improve the performance of GA in path planning by proposing new types of approaches (e.g., solution representation, initial population selection) or operators (parent selection, crossover and mutation). In

2

Alnasser et al. [12], the authors propose an adequate representation of the solution in a grid map environment, together with a smart crossover operator and a fitness function that considers different criteria for robot motion. For example, the minimization of the number of turns is considered because it can reduce the energy consumption of the robot. Also, GA parameter adjustment is proposed to avoid premature convergence, as shown in the work of Karami et Hasanzadeh [13], in which they adapt the selective pressure of the selection operator for 2D robot planning. The selective pressure is adjusted according to the standard deviation of the fitness function of the latest population. GA parameter adjustment is surveyed in [14], and it is stated that considering these algorithms are applied to a variety of problems, and the performance depends much of these parameters. They present a classification according to a new conceptual model that subdivides the process of adapting parameter values into four steps. Among the conclusions it can be mentioned that the number of works using adjustment in mutation, crossover and population has increased linearly, while other adjustments as parent selection, replacement representation still receive little attention.The previous works has considered static environments. However, a more realistic approach is to consider a dynamic environment. A dynamic environment considers that new information about the environment is updated as the robot

moves. In [15], the authors use the concept of visible space to find the path between two positions in a grid map with obstacles. Visible space means that the next waypoint in a path is selected from a set of feasible points, excluding the obstacles. To do this, first they create an initial feasible population and later they propose novel mutation operators that correct infeasible paths. The proposed approach can cope with obstacles that have not been considered initially.Other scenario where GAs has been applied is with the classical TSP. For example, in [16], a GA algorithm is used to find a sequence of charging stations for a fleet of electrical vehicles (EV). A combination of TSP with path planning is shown in [17] where it is studied for the case of an Unmanned Air Vehicle that attempts to visit certain points in the map while avoiding certain forbidden zones, which represent for example, an enemy’s radar.

In this work the same combination of TSP and path planning for autonomous vehicle is applied. But differently from [17], instead of forbidden zones there are regions of interest (ROI) where there are places that the ASV should traverse. An additional difference is that here there is an adjustment of the objective of the mission that it is reflected in the adjustment of the GA parameters. Table I summarizes a comparison between some characteristics of this work and the related work.

TABLE I. COMPARISON OF WORK ABOUT GA AND PATH PLANNING

Ref. Approach Constrained Dynamic Environment Contribution

[12] Grid map Yes No Solution representation Crossover operator Fitness function

[13] Grid map Yes No Selection operator[15] Grid map Yes Yes Solution representation

Mutation operator[16] TSP Yes No Solution representation

Crossover operator Initial population generation

[17] TSP Yes No Unfeasible path recovery

Proposed approach

TSP + Grid map Yes Yes Adaptive solution representation Adaptive fitness function

III. PROBLEM STATEMENT AND PROPOSED APPROACH

This section has been divided into two parts. First the lake environment is described as well as the TSP modeling with constraints. Also, a description of the ASV and its operation to locate algae bloom in the lake is included. Second, the proposed two-phase strategy is described in detail.

A. Covered problemYpakarai Lake is located approximately between latitudes

25˚ 15’ and 25˚ 22’ (14 kilometers) south; and between

longitudes 57˚ 17’ and 57˚ 23’ west (10 kilometers). The largest distance between two points at the shore is about 14 kilometers. The area that should be covered is around 70 square kilometers. Its average depth is about 1.31 meters with a maximum value of 3 meters. Being a relatively shallow lake facilitates the task of characterizing its conditions, because it can be considered a homogenous mass of water

3

(a) (b)Figure 3. (a) Beacon distribution and (b) numbering convention

Around the lake, a network of beacons is to be deployed at the shore as data collecting points from the ASV, as shown in Fig. 3 (a). For achieving the covered task with a single drone, the network of n beacons will serve as waypoints to be visited by the ASV at least once and it is defined as eq. (1) and shown in Fig. 3(b)

B= {b0 , b1 , b2 , …,bn−1 } (1)

where n=60 and b i=( x i , y i ) is beacon i with coordinates (xi,yi)The distance between two consecutive beacons is less than 1 kilometer. This distance is the expected wireless transmission range of the transceiver installed at the beacons. Notice that this is a normal transmission range for wireless devices based on Zigbee technology in a line-of-sight (LOS) scenario [18].This covered problem is clearly analogous to the TSP and the number of possible solutions is very large, since (n-1)!/2 combinations are possible. Therefore, this number of combinations is prohibitive for a brute force algorithm that evaluates all possible solutions. It is important to point out that the TSP belongs to a class of combinatorial problems known as NP-hard problems [19]. There are different ways to represent it mathematically but looking at it as a permutation problem, considering Pn as the collection of all permutations of the set{1,2 , …, n } and n= 60, the TSP is the searching of π=( π (1 ) , π (2 ) , …, π (n ) ) in Pn such that the fitness the following function

cπ (n ) π (1)+∑i=1

n−1

cπ (i ) π ( i+1 ) (2)

is minimized, where c is the cost or distance between two cities, which are beacons in the proposed approach. This is known as the tour length [20]. The conventional TSP problem looks for the minimization of the distance travelled, but in this case the maximization of the distance travelled is searched. Particularly, the target is to maximize the sampling area, which is directly related to the distance travelled, but also to minimize redundant data (route intersections of the path). Furthermore, this is a constrained case because due to the shape of the lake there are some routes between two beacons that are unfeasible, which are the routes that at some point travel outside the lake. This route will be called invalid routes from now on.In our problem, the beacons are not for reference only but also for data exchange between the beacons and the ASV. This data exchange might be the collection of the measurements of the ASV (e.g. phycocianin, dissolved oxygen, PH, water temperature, etc.), which will be forwarded through a multi-hop route to the gateway or base station and vice versa. In Fig. 3(b) it is indicated as a green square. A candidate for the base station is beacon b48 (W 57 22’ 21” and S 25 18’ 9”), which is located at the town of Aregua (green square in Fig. 3(b)) where a better infrastructure is available to set up a control center. Another possible alternative is beacon b18 (W 57 17’ 51” and S 25 18’45”), which is located at the town of San Bernardino. In order to identify each beacon, they are assigned a number between 0 and 59. Fig. 3 (b) shows a numbering convention that will be used throughout the paper. In Fig. 2 (b), the coordinates are translated from degrees, minutes and seconds to meters, considering 1 second = 30 meters. One unit in both axes of Fig. 2(b) is equivalent to 1 meter. The origin of this graph is conventional and taken at point 25˚ 22’ 21” S and 57˚ 22’ 57” W, as shown as a red pin in Fig. 3 (a).

4

As the ASV travels visiting the beacons, it will take measurements of the quality of water and detect the presence of Algae Bloom. The blue-green algae can be detected by evaluating the levels of phycocianin, which is a pigment contained in the cyanobacteria. This pigment absorbs the light at 620 nm (red-orange) and fluorescence at 650 nm. There are many commercially available sensors for this purpose offered by different manufacturers. The sensors measure the number of cells/mL (cells per milliliter). The WHO (World Health Organization) has defined three levels of health threat due to exposure of cyanobacteria. Above 20,000 cells/mL is considered a moderate probability of adverse health effects, and over 100,000 cells/mL the probability is high [21].

In addition to the phycocianin sensors, the ASV can be equipped with other sensors for water quality evaluation such as temperature, PH, dissolved oxygen, turbidity conductivity and oxidation reduction potential. The data from these sensors can be used in future studies to find any relationship pattern with the algae bloom.

B. The proposed approach: Intelligent online learning strategy based on reconfigurable GA parameters

This section presents the details of the proposed strategy. The main objective of the strategy is to determine the path of the ASV according to different situations encountered during the monitoring of the lake. The main tool used for this is a Genetic Algorithm (GA) [22]. The problem is modeled as a Travelling Salesmen Problem (TSP) adjusting it according to different conditions in the different phases of the strategy. A diagram of the proposed strategy is shown in Fig. 4. It consists of two phases such as exploration and intensification phases. Phase 1 is the exploration phase, in which the ASV movement is chosen to maximize the coverage of the lake. Notice that at this point, no algae bloom has been detected. If an area with a presence of the algae bloom is detected, the strategy moves to the phase 2, in which the ASV movement focuses more on the region with the blooms presence. The ASV stays in this phase until it is noticed that the appearance of the algae bloom is stabilized (not growing or static). If so, the ASV returns to the phase 1, to start exploring again other areas that have been left out during the intensification phase. The basic block diagram of Fig. 4 is expanded in Fig. 5 with more level of detail.

Figure 4. Diagram

5

Figure 5. Detailed strategy diagram

An important feature of the proposed strategy is that it learns from the dynamic environment and modifies the GA objective according to the purpose of the current phase of the approach (exploration or intensification). During the first phase the ASV collects the data from the lake to learn and infer its state. Then, it retunes the GA objective (fitness function) to collect more data from the polluted region and learn about the environment and its dynamic features. When the ASV detects a loss of knowledge such as the size of the detected event (algae bloom detection indicated by the ASV onboard sensors) has decreased, the ASV adapts again its movements to gather more information about the lake. Notice that this strategy presents a continuous intelligent online learning approach. The strategy begins without any knowledge about the conditions of the lake. Phase 1 will calculate a full exploratory trajectory for the ASV. Since the path planning is modeled as a TSP problem, the ASV visits a set of data collection beacons (nbeacons = 60) distributed uniformly at the shore of the lake. To apply a genetic algorithm to the problem two elements must be defined: a representation of a possible solution or individual and its quality level or fitness function.

1) Solution representationA solution of the problem is the sequence of beacons that the ASV will visit. Fig. 6 illustrates the representation of an individual, being bx the beacon identification. This representation has been used since it is the classical encoding for TSP problem. The only condition for the individual creation is that there cannot be repeated beacons in the list. Consequently, each beacon can only be visited once. Notice that this solution representation is valid for the two phases of the proposed approach.

b0 b1 ….. bn-2 bn-1

Figure 6. Path representation (not necessarily in order).

2) Exploration phaseThe quality of the solution during the exploration phase (sampling area) is represented by the following equation:

fit1=(1−

ninvroutes

nbeacons−1 )∗(SASV∗ ∑i=0

nbeacons−2

dbi bi+1– SASV

2 ∗nintersec)A lake

(3)

, where ninv_routes term refers to the number of invalid routes, i.e., the routes that do not travel entirely in the lake; SASV is the sampling distance, i.e. the radius of the area around a sampling point that is being represented by that sampled measurement; dbi b j

is the distance from beacon i to beacon j; nintersec is the number of route intersections in the solution and Alake is the lake area.The first term in the numerator is a penalizing factor that decreases the fitness value of the individuals that include invalid routes; the second terms indicates a subtraction between the total area sampled and the redundant data at the route intersection points. This value is divided by the Lake area to obtain a relative covered area. In [23], several fitness functions are evaluated to measure the coverage of the ASV in Ypacarai lake. The results in [23] show that (3) is suitable for the present coverage problem. Also in [23], it is shown that the GA outperforms other approaches such as random, and greedy techniques.

In the detailed diagram (Fig. 5), first an initial population of valid individuals is created, and the fitness function is calculated. Then, it enters to the sub-routine of the Genetic

6

Algorithm (GA), which is shown in Fig. 7. The output of this algorithm is the path with the highest fitness function.

Figure 7. Genetic Algorithm diagram

Figure 8. Map grid

To represent the conditions of the lake, after a path has been defined, the map of the lake is divided in a grid with ngrid

squares of size ssquare (Fig. 8). When an algae bloom is detected then its location is recorded (square in the grid). An algae

bloom is detected if the levels exceed the values defined by the end user; for example, if the levels of cells are above 20.000 or 100.000, according to the risk levels.

After visiting all the beacons, a map of the lake it is obtained with all the approximate locations where the evidence of the algae bloom’s presence has been collected. It is important to notice that the ASV can pass through the same grid square more than once. Whenever a significant presence of the algae bloom is detected, then the ASV moves to the next phase. Otherwise, the ASV performs the same routes as before. As a result of phase 1, the lake map is obtained, which includes the grid squares where the algae bloom was detected and the number of times that this event happened. 3) Transition between Exploration and Intensification phases

The transition to phase 2 is performed if there is an area or several areas with a significant presence of the algae bloom. For each one of these events its size and location are stored. With all this information, a bounding box is created around the event (see Fig. 9). Considering that the event is a grid square and given the characteristics of the lake, the bounding box will be delimited by a horizontal stripe defined by the upper and lower side of the event grid square More formally, if there is an estimated square algae blooming (AB) with side sAB and center coordinates (xAB,yAB), the bounding box is the set of

7

grid squares inside the lake with coordinates (xgrid_sq, ygrid_sq) that satisfies the condition

x AB−1.5∗s AB≤ xgri dsq≤ xAB+1.5∗SAB (4)

Figure 9. Region for selecting a sub-tour graph

As a result, the bounding box is an area that covers the width of the lake and its height is three times the size of the estimated AB size. 4) Intensification phaseOnce the bounding boxes are defined, a sub-set of beacons (B’) inside the bounding boxes are selected to intensify the evaluation of the blooms. The beacons are selected from B if xi from beacon bi (i = {1,2, 3,…,n}) satisfies

B'={b ' 0 ,b ' 1 , b '2 , …,b ' m } ,where m<n−1 (5)

Therefore, the TSP is reduced since a lower number of beacons is selected allowing a faster intensification of the detected areas. After selecting the sub-set of beacons, a new trajectory is calculated using the GA so that the number of times that the ASV passes over the squares that registered the presence of algae bloom is maximized. A new fitness function is defined as

fit2= ∑i=0

nbeacons−2

nalgi ,i+1 (6)

where n_algi,i+1 is the number of squares with algae that the ASV has detected when moving from beacon i to beacon i+1. As a result of the phase 2, a more precise evaluation of the area with the algae will be obtained. Next, the size of the algae bloom regions is recalculated again from the results obtained.If the size of any of the algae regions has increased, then the strategy remains at this phase. However, if all the regions have remained the same size or decreased, then the strategy moves again to the phase 1.

IV. SIMULATIONS

This section includes the simulation results obtained. The section has been divided into two parts. First, we describe the simulation environment used to evaluate the proposed approach. Second, we show the simulation results obtained.

A. Simulation environmentThis section presents the parameters used for the conducted simulations. Table II contains the parameters used for modeling the lake environment and the characteristics of the events

TABLE II. ENVIRONMENT PARAMETERS

Parameter Value

Number of beacons 60

Distance between beacons (km)

1 (approximately)

Lake Size (km2 ) 68.72

Number of grids – ngrid 4,200 (60x70)

Square size – Ssquare (km2) 0.04

Square side – Sside (km) 0.2

Sampling distance – SASV (km) 0.02

Algae Bloom 1 size (km2) 1.00 – (Condition 1)

2.96 – (Condition 2)

0.36 – (Condition 3)

Algae Bloom 2 size (km2) 1.96

An event refers to an occurrence of Algae Bloom (AB) area that appears in the lake because of it euthrophication state. For the simulations, it is assumed that the bloom has a square shape, and its size consists of a multiple number of grid squares defined as:

SAB(km2)=nAB∗Ssquare(km2) (7)

where nAB is the number of grid squares and Ssquare is the defined size of the square. Two algae bloom areas will be considered in the simulations, Algae Bloom 1 (AB1) and Algae Bloom 2 (AB2). Algae Bloom 2 will remain static during the simulations, while Algae Bloom 1 will vary its size. The variations will occur uniformly around the center of the bloom, i.e. it will increase or decrease uniformly. Fig. 10 shows the locations of AB1 and AB2, as well as its variation through time. AB1 is located to the upper left side of the lake.

The second group of parameters is shown in Table III and is related to the genetic algorithms used for finding the solution of the path planning. The population size selected is big enough to ensure a high initial exploration. The number of generations used is sufficient for the convergence of the genetic algorithm. A roulette wheel mechanism is used for the selection of the parents. The roulette wheel means that individuals with higher fitness have higher slots in the roulette

8

than individuals with lower fitness, so that their probabilities for reproduction are increased [24]. Then the ordered crossover exploits a property of the path representation, which is that the order of cities (not their positions) is important [20]. As it was mentioned in Section III, the ordered crossover constructs a sub-tour of one parent while preserving the relative order of cities of the other parent. The mutation operator shuffles the attributes (genes) of an individual. Each element is exchanged with another element randomly according to an independent probability inpb (0.05) [25]. Lastly, the elitism rate is the amount of the best individuals that will be preserved for the new generation. This mechanism is used to guarantee that the best individuals are not lost during the evolution procedure due to the probabilistic selection. The 20 % of best individuals passes directly to the next generation.

The problem was simulated in a Debian 7.9 server with the following specifications, an Intel Xeon v3 E5-2603 1.6 GHz CPU and 64 GB RAM memory. The model was implemented in Python version 2.7.13, where the genetic algorithm was implemented with the module Distributed Evolutionary Algorithm for Python (DEAP) version 1.0.2 [25]. The code of the simulator has been made online available in [26].

TABLE III. GENETIC ALGORITHM PARAMETERS

Parameter Value

Number of simulations

10

Population size 100

Number of generations

1,000

Selection Roulette Wheel

Crossover Ordered (OX1)

Cross-over probability Pc

0.8

Mutation Shuffle index (indpb = 0.05)

Mutation probability Pm

0.2

Elitism rate 0.2

Fitness

fit1=(1−

ninvroutes

nbeacons−1 )∗(SASV∗ ∑i=0

nbeacons−2

dbi bi+1– SASV

2 ∗nintersec)A lake

fit2= ∑i=0

nbeacons−2

nalgi ,i+1

Figure 10. Evolution of the Algae Bloom regions over time

B. Simulation resultsThis section presents the performance of the proposed

methodology considering the dynamic on the evolution of the algae bloom regions shown in Fig. 10.

The whole simulation scenario was divided in six-time frames (Table IV). Each time frame is the period that it takes to the ASV to travel one complete path obtained from our

algorithm. The periods can differ considerably because the exploration phase has a broader area to cover than the intensification phase. They were calculated using an estimated average speed of the ASV of 10 km/h, giving a total time of 170 hours to perform the six frames.

9

TABLE IV. FRAME DURATION

Time Frame Duration (Hours) Phase

1 55 Exploration

2 15 Intensification

3 15 Intensification

4 15 Intensification

5 15 Intensification

6 55 Exploration

Table V shows the behavior of the methodology as it tries to adapt to the dynamic of the bloom regions. At the end of each frame, the ASV has travelled the planned path and the sensed data results are available. First, the strategy starts in full exploratory mode and it finds a path that maximizes coverage (Frame 1, Table VII). After travelling this path, the two blooms are located, and the strategy moves to the intensification phase, where a new path with a subset of selected beacons is calculated. The resulting path with its corresponding sampled results is shown in the column Frame 2 of Table VII. Then, the same path is travelled again in to continue monitoring these two regions, but at this point a variation is included (Frame 3, Table VII). The top bloom increases its size, and the resulting sample shows that there is a variation in the size of the AB. The strategy locates these new conditions and the GA is adapted to the new objective. The resulting path will provide also a new sampled set of results (Frame 4, Table VII). In the next step (Frame 5, Table VII), the same path is travelled again, but the top AB reduces its size. Because of this event, the strategy assumes that the AB starts to disappear, and it is possible to evaluate again other areas in the lake, switching to the exploration mode. The GA is executed at this point considering again all the beacons around the lake and the results are shown in the last step (Frame 6, Table VII).

TABLE V. ALGAE BLOOM COVERAGE RESULTS

Frame 1

Frame 2

Frame 3

Frame 4

Frame 5

Frame 6

Conditions of the lake(see Fig. 9)

1 1 2 2 3 3

Phase (see Fig. 3)

1 2 2 2 2 1

GA used? Y Y N Y N Y

Distance Travelled (kilom.)

536.08 141.88 141.88 162.81 162.81 531.65

First Coverage (%)

90.5 93.2 86.2 87.7 89.7 96.5

Redundant Coverage

200.0 125.7 97.7 110.0 101.7 196.5

(%)

The numerical results of the simulations are shown in Table V and Table VI. Table V contains values obtained at the end of each frame. The first row shows the conditions of the lake during the frame, according to the conditions defined in Fig. 10. Then, the second row details the phase of the strategy used during the frame. The third row indicates if the GA was executed in this frame. Finally, the next three rows show the performance of the ASV. The fourth row is the distance, the fifth the percentage of AB that has been covered at least once, and the sixth the percentage of AB that has been covered more than once. It is important to notice that these percentages are evaluated considering the estimated size of the bloom. Table VI presents the statistics related to the best individual after running the simulation multiple times. In Table VI, the first row is the total coverage (first + redundant coverage in Table V) of the simulation with the best solution. Then, the average and the standard deviation are shown in the following rows. Fig. 11 represents the results of running multiple simulations in the form of boxplots.

TABLE VI. SIMULATION STATISTICS

Frame 1

Frame 2

Frame 3

Frame 4

Frame 5

Frame 6

Best Total Coverage (%)First + Redundant

290.5 218.9 183.9 197.7 191,4 293,0

Average (Total Coverage %)

253.8 202.2 165.1 187.7 190.2 266.7

Standard Deviation (Total Coverage %)

37.5 9.4 9.6 7.7 15.9 26.7

Note that in Frame 1, the coverage of the bloom is already high. However, by applying the intensification phase, a better first coverage is achieved but with less distance travelled and less redundant samples. This percentage falls in Frame 3 because the size of the bloom increased. The strategy applied makes the GA to increase again this coverage (Frame 4). In Frame 5 and using the same calculated path planning, the coverage increases because the size of the bloom was reduced. Finally, at Frame 6 the ASV returns to the exploratory phase, increasing the coverage, but at the expense of increasing the distance travelled. However, this requirement is accepted considering that the strategy is covering a wider area.

10

TABLE VII. SIMULATION RESULTS

Frame 1 Frame 2 Frame 3Path Planning

Real conditions of the lake

Sampled result

11

Frame 4 Frame 5 Frame 6Path Planning

Real conditions of the lake

Sampled result

12

Figure 11. Boxplot of multiples simulations

Previous results showed the solutions of the path planning found by the proposed algorithm and how well the solutions achieve the objective of discovering and evaluating the presence of AB in the lake. Next, a closer look about the performance of the GA regarding its capability of finding solutions in the proposed problem is shown. More precisely, the evolution of the GA through the generations is shown in Fig. 11 (a)-(d). These correspond to frames 1, 2, 4 and 6, respectively (phase 1 or 2, depending on the frame). The yellow line represents the best solution of all simulations, the blue line is the average of best solutions of the different simulations and the blue shaded area is the standard deviation.

In the previous simulations, the GA was executed four times for frames 1, 2, 4 and 6. In frames 1 and 6 the fitness functions correspond to equation 3, and it gives the coverage of the lake in the exploratory phase. It is seen in Fig 11 (a) and (d) the best coverage of the lake is near 15% and that in average the best solution starts to converge around the generation 1,000. It is important to mention that the standard deviation remains similar throughout all the generations.

In frames 2 and 4 (Fig. 12 (b) and (c)), the fitness function is given by equation 2 divided by the total estimated sizes of the blooms. That is, the percentage of the estimated AB covered. In this case, the values are the ones shown in Table VI (Best total coverage). This is also the same as the sum of the first coverage plus the redundant coverage values of Table V. The intensification phase results in a 219 % total coverage in frame 2, while in frame 4 in a 198% total coverage. Comparing with the exploration phase, the average converges earlier, since the number of beacons in the solution is smaller. However, the standard deviation is larger, leading to think that there is a greater variance of the best solution from simulation to simulation. In order to balance the computational effort of the algorithms in both cases, it could be run less times in the exploration phase while increasing the number of the generations. On the other hand, in the intensification phase, the number of generations could be reduced and increase the number of simulations. Another way of interpreting this result is that the initial population has greater impact in the intensification phase than in the exploration phase. This is because in the exploration phase the number of constrains is higher than in the intensification phase, meaning that the search space of this last phase is much wider.

(a) (b)

13

(c) (d)

Figure 12. Evolution of the solutions

V. CONCLUSION

An online learning approach to determine the path planning of an ASV for locating and monitoring outbreaks of algae blooms in the Ypacarai Lake in Paraguay is proposed. Considering the unpredictability of the phenomenon and the temporal dynamic behavior, a strategy with an exploratory phase and an intensification phase is proposed. If no algae bloom is located, or the monitored ones have reduced their size, the path is calculated considering an exploratory phase. On the other hand, if new bloom is detected, or the monitored ones have increased their size, the path is calculated using the intensification phase. The proposed approach models the problem as a constrained Travelling Salesman Problem (TSP) in a real scenario (Ypacarai Lake) and it uses a genetic algorithm (GA) that adapts its parameters (solution representation and fitness function) according to the collected data from the lake to find its solution.

The main conclusions of the simulation results are:1. After locating the algae bloom in the exploratory

phase, the intensification phase achieves similar levels of the coverage while reducing the distance travelled and consequently the time to perform the task.

2. If there is a variation of the size, the strategy adapts to the new conditions. At the end the strategy returns to the exploration phase to search new algae bloom, if the monitored ones show signs of vanishing (size reduction).

3. Overall, the strategy keeps levels of at least 85% of the algae bloom, as it is seen in the row First coverage % of Table V.

4. The intensification phase reduces the time of monitoring a region with algae bloom (around 15 hours), however the exploratory phase time is still much larger (55 hours).

The proposed online strategy can be applied to other complex scenarios with unpredictable factors where the use of

autonomous vehicles is suitable, such as relief actions in disaster scenarios [27][28][29], and vehicular ad hoc networks [30], among others. As future work, a feasible improvement of the proposed strategy is to apply the well-known extension of the TSP called Vehicle Routing Problem. In this case, a fleet of vehicles are used to complete the task. Besides the work on developing large autonomous surface vehicles, there are also other works towards the implementation of smaller vehicles with lower cost that would allow the practical implementation of the approach. This field known as Swarm Robotics, [31] takes inspiration mainly from swarms found in nature like ants, bees, etc. Still there is plenty of room for research until reaching a decentralized system which is robust, versatile and scalable.

ACKNOWLEDGMENT

The authors would like to thank Fundación Carolina and its PhD scholarships program.

REFERENCES

[1] Multidisciplinary Technological Research Center (CEMIT). National University of Asuncion (UNA), 2016, October. Technical Report – 12th Sampling Campaign from Ypakarai Lake

[2] Hydroinformatics International Center (CHI), Itaipu Hydroelectric. 2016. Sistema de Monitoreo, Control y Estudios de la Cuenca del Lago Ypacarai [Online]. Available at: http://hidroinformatica.itaipu.gov.py/gestiondecuenca/py/ypacarai/. [Accessed September, 5th., 2017]

[3] S.N. Sivanandam, S.N. Deepa, “Introduction to genetic algorithms” in Science & Business Media, Springer, 2007

[4] L. Wang, X. Wang, X. Jin, J. Xu, |H. Zhang, J. Yu, Q. Sun, C. Gao, L. Wang, “Analysis of algae growth mechanism and water bloom prediction under the effect of multi-affecting factor,” Saudi journal of biological sciences, 24(3), pp.556-562, 2017

[5] J.C. Ho, A.M. Michalak, “Challenges in tracking harmful algal blooms: A synthesis of evidence from Lake Erie,” Journal of Great Lakes Research, 41(2), pp.317-325, 2015

[6] M. Dunbabin and A. Grinham, “Quantifying Spatiotemporal Greenhouse Gas Emissions Using Autonomous Surface Vehicles,” Journal of Field Robotics, 34(1), pp.151-169, 2017

14

[7] M.H.B.M. Idris, M.A.A.B.C. Kamarudin, M.I. Sahalan, Z.B.Z. Abidin and M.M. Rashid, “Design and development of an autonomous surface vessel for inland water depth monitoring,” in Proc. IEEE Computer and Communication Engineering (ICCCE), 2016, pp. 177-182

[8] C. Pradalier, T. Posch, J. Pernthaler and R. Siegwart, “Design and application of a surface vessel for autonomous inland water monitoring,” IEEE Robotics & Automation Magazine, pp.1-9, 2012

[9] E. Romero-Vivas, F.D. Von Borstel, C.J. Pérez-Estrada, D. Torres-Ariño, J.F. Villa-Medina and J. Gutiérrez, “On-water remote monitoring robotic system for estimating the patch coverage of Anabaena sp. filaments in shallow water,” Environmental Science: Processes & Impacts, 17(6), pp.1141-1149, 2015

[10] T.T. Mac, C. Copot, D.T. Tran and R. De Keyser, “Heuristic approaches in robot path planning: A survey,” Robotics and Autonomous Systems, 86, pp.13-28, 2016

[11] R. Arnay, N. Morales, A. Morell, J. Hernandez-Aceituno, D. Perea, J.T. Toledo, A. Hamilton, J.J. Sanchez-Medina and L. Acosta, “Safe and reliable path planning for the autonomous vehicle verdino,” IEEE Intelligent Transportation Systems Magazine, 8(2), pp.22-32, 2016

[12] S. Alnasser and H. Bennaceur, “An efficient Genetic Algorithm for the global robot path planning problem,” in Proc. IEEE Digital Information and Communication Technology and its Applications (DICTAP), 2016, pp. 97-102.

[13] A.H. Karami and M. Hasanzadeh, “An adaptive genetic algorithm for robot motion planning in 2D complex environments,” Computers & Electrical Engineering, 43, pp.317-329, 2015

[14] A. Aleti and I. Moser, “A Systematic Literature Review of Adaptive Parameter Control Methods for Evolutionary Algorithms,” ACM Computing Surveys (CSUR), 49(3), p.56, 2016

[15] X. Zhang, Y. Zhao, N. Deng and K. Guo, “Dynamic path planning algorithm for a mobile robot based on visible space and an improved genetic algorithm,” International Journal of Advanced Robotic Systems, 13(3), p.91, 2016

[16] F. Alesiani and N. Maslekar “Optimization of charging stops for fleet of electric vehicles: A genetic approach,” IEEE Intelligent transportation systems magazine, 6(3), pp.10-21, 2014

[17] M. Cakir, “2D path planning of UAVs with genetic algorithm in a constrained environment,” in Proc. IEEE Modeling, Simulation, and Applied Optimization (ICMSAO), 2015, pp. 1-5

[18] Digi International Inc., Digi Xbee S2C 802.15.4 RF Modules, Datasheet, 2017

[19] D.S. Johnson and L.A. McGeoch, “The traveling salesman problem: A case study in local optimization,” Local search in combinatorial optimization, 1, pp.215-310, 1997

[20] P. Larranaga, C.M. Kuijpers, R.H. Murga, I. Inza, S. Dizdarevic, “Genetic algorithms for the travelling salesman problem: A review of representations and operators,” Artificial Intelligence Review, 13(2), pp.129-170, 1999

[21] L.C. Backer, W. Carmichael, B. Kirkpatrick, C. Williams, M. Irvin, Y. Zhou, T.B. Johnson, K. Nierenberg, V.R. Hill, S.M. Kieszak, Y.S. Cheng, “Recreational exposure to low concentrations of microcystins during an algal bloom in a small lake,” Marine Drugs, 6(2), pp.389-406, 2008

[22] C. Reeves, Genetic algorithms. Handbook of metaheuristics, pp.55-82, 2003

[23] M. Arzamendia, D. Gregor, D. G. Reina, S. L. Toral, “An Evolutionary Approach to Constrained Path Planning of an Autonomous Surface Vehicle for Maximizing the Covered Area of Ypacarai Lake,” Soft Computing, https://doi.org/10.1007/s00500-017-2895-x

[24] K Sastry, D.E. Goldberg and G. Kendall, “Genetic algorithms,” in Search methodologies, pp. 93-117, Springer US, 2014

[25] DEAP documentation. [Online]. http://deap.readthedocs.org/en/1.0.x/.[Accessed September, 5th., 2017]

[26] Intelligent Online Learning Strategy fon ASV In Lakes Environment Simulator [Online]. https://github.com/Mariuspy/ASVPathPlanningGA . [Accessed September, 5th., 2017]

[27] D. G. Reina, H. Tawfit, S. L. Toral, “Multi-subpopulation Evolutionary Algorithms for Coverage Deployment of UAV-networks,” Ad Hoc Networks, Vol. 68, pp. 16-32. 2017

[28] E. Asimakopoulou, N. Bessis, “Buildings and Crowds: Forming Smart Cities for more Effective Disaster Management,” in Proc. IEEE Innovative Mobile and Internet Services In Ubiquitous Computing (IMIS), 2011, pp. 229-234

[29] E. Asimakopoulou, Advanced ICTs for Disaster Management and Threat Detection: Collaborative and Distributed Frameworks, IGI, 2010

[30] J. Sánchez-García, J. M. García-Campos, D. G. Reina, S. L. Toral, F. Barrero, “On-siteDriverID: A Secure Authentication Scheme based on Spanish eID Cards for Vehicular Ad Hoc Networks,” Future Generation Computer Systems, Vol. 64, pp. 50-60, 2016

[31] L. Bayındır, “A review of swarm robotics tasks,” Neurocomputing, 172, pp.292-321, 2016

Mario Arzamendia was born in Asuncion, Paraguay, in 1977. He received the B.E. in Electrical Engineering from the University of Brasilia (UnB), Brazil in 2002 and the M.Sc. in Electronic Engineering from Mie University, Japan in 2009. Currently he is a researcher at the Laboratory of Distributed Systems (LSD) in the Faculty of Engineering at

the National University of Asuncion (FIUNA), Paraguay; and is working toward his Ph. D. in the Department of Electronic Engineering at the University of Seville, Spain. His main interests include wireless sensor networks, embedded systems applications and metaheuristics optimization.

Daniel Gutiérrez was born in Seville, Spain, in 1983. He received the B.E. degree in electronic engineering and M.S. degree in electronics and telecommunications from the University of Seville, Seville, Spain, in 2009 and 2011 respectively. He obtained the Ph.D. degree in electronic engineering in 2015 by the University of Seville, Seville. Since 2017, he is working at Loyola

Andalucía University. His current research interests include the application of evolutionary computation approaches in wireless multi-hop networks such as ad hoc networks, delay tolerant networks and flying ad hoc networks.

Sergio Toral was born in Rabat, Morocco, in 1972. He received the M.S. and Ph.D. degrees in electrical and electronic engineering from the University of Seville, Spain, in 1995 and 1999, respectively. He is currently a full Professor with the Department of

15

Electronic Engineering, US. His main research interests include ad hoc networks and their routing protocols, deployment of wireless sensor networks, flying ad hoc networks, real-time and distributed systems, intelligent transportation systems, and embedded operating systems. He is actually an author or coauthor of 73 papers in major international peer-reviewed journals (with JCR impact factor) and of over 100 papers in well-established international conferences and workshops.

Derlis Gregor was born in Asuncion, Paraguay, in 1980. He received the Bachelor Degree in Systems Analysis and the Computer Engineering from the American University, Asuncion, Paraguay, in 2007. He received the M.Sc. and Ph.D. Degrees in

Electronic, Signal Processing and Communications from the Higher Technical School of Engineering (ETSI), University of Seville, Spain, in 2009 and 2013, respectively. He is currently a Teacher Research Categorized “Level 1” by the National Council for Science and Technology (CONACYT) and Head of the Department of Computer Science (DCC), located on the Technological Innovation Center (CITEC), Engineering Faculty of the National University of Asuncion (FIUNA), National University of Asuncion (UNA), Paraguay. His research interest focuses on the application of Intelligent Transportation Systems (ITS). Interoperability in Sensor Networks, Embedded Systems and Instrumentation Systems. Distributed Systems applied to Renewable Energy. Development and Management of Open Source software project for Embedded Systems.

Eleana Asimakopoulou received the MA and PhD degrees in the area of Disaster Management from the University Westminster and Loughborough University, UK, in 2009 and 2008 respectively. She is been a lecturer and a visiting researcher in various institutions including the Hellenic National Defence College, Athens, Greece

and an expert evaluator in national and European proposals for funding. Her research is in the area of utilizing disruptive technologies (IoT, grid and cloud computing) for disaster management. She has been involved in a number of projects and has published over 50 works. Dr Asimakopoulou is an editor of a book and an associate editor of a refereed international journal.

Nik Bessis received his BA from the T.E.I. Athens and his MSc and PhD degrees from De Montfort University, UK in 1995 and 2002 respectively. He is currently a full Professor and the Head of the Department of Computer Science at Edge Hill University, UK. He is a fellow of HEA, BCS and a senior member of IEEE. His research is on social graphs for network and big

data analytics as well as developing data push and resource provisioning services in IoT, FI and inter-clouds for a number of settings including disaster management. He is involved in a number of funded research and commercial projects in these areas. Prof Bessis has published over 270 works and won 4 best papers awards. He is an editor of several books and an editor-in-chief of a refereed international journal.

16