Accuracy assessment of MODIS fire products in African Savanna …ir.uz.ac.zw/jspui/bitstream/10646/3461/1/Mpofu_N_Accuracy... · 2019. 10. 16. · African grasslands and savannas

0 | P a g e

University of Zimbabwe

Accuracy assessment of MODIS fire

products in African Savanna woodlands.

A Dissertation submitted to the Department of Geography and

Environmental Science in Partial Fulfilment of the Requirements for

the Master of Science Degree in Geographical Information Science

and Remote Sensing, June 2016

NDUMEZULU T MPOFU

R156317D

i | P a g e

Acknowledgements

My greatest challenge was moving from a social science back ground into a purely scientific

line of thinking yet, I consider it a privilege to have ventured into this pathway. Despite all the

hurdles I had to jump all the way I learnt eventually that, “where there is a will, there surely is

a way”. I therefore start by expressing my sincere gratitude to my family for all the support

they gave during the course of my Master of Science study.

Let me begin by acknowledging the contributions of my thesis supervisor Doctor

Mhosisi Masocha, who helped me realise my potential and develop a scientific, independent,

smart and hardworking mind. To Masocha, I am not able to thank you enough for helping me

realise the scope of my study and perfect its rationale. I thank you most for the times when you

pushed me to work smart and discover things on my own, it would be very ungrateful of me

not to admit the positive difference it made in me. I am grateful I had you for my mentor. You

taught me a lot, from the basic GI science to scientific writing, appreciation and ‘yes!’, I will

always remember to limit the content in my slides when I teach somewhere else or make

presentations out there in the world.

To Henry Ndaimani, your encouragement, motivation and assistance during the course

of my thesis, I appreciate with overwhelming gratitude.

To my classmates, I thank you all for all the contributions you made when things where

a bit hazy on my end. The spirit we had as a team, may it persist even out there in the world.

Finally, chief of all, I thank God for providing for me continually, life, strength, courage and

dedication throughout the course of my study. My faith in you will abide forever. Amen

ii | P a g e

Abstract

Fire poses a continuous threat to forest ecosystems and can dramatically reduce valuable timber

species in forest woodland areas. The occurrence of fire in Baikiaea plurijuga woodlands

warrants the need of an active and timely fire detection system. Rudimentary methods of

detecting fires are still in use in most of Zimbabwe’s forest reserves, yet remote sensing has

played a pivotal role around the globe in detecting and monitoring both active incidences and

post fire burnt areas. Several satellite systems have been validated in different biomes of the

world for both MODIS MOD14A1 and MCD14ML.However, there still remains a gap of

knowledge in the accurate detection of fires in Baikiaea plurijuga woodlands. In this study we

evaluate the accuracy of MODIS fire products using, confusion matrices, kappa statistic, true

skill statistic (TSS), remote sensing and Geographical Information techniques where employed

to assess the accuracy of MODIS MOD14A1 burnt area product and MCD14ML active fire

product in fire detection. This is the first time results of accuracy assessment of fire products

are reported in Baikiaea plurijuga woodlands. In both study sites, course resolution 1 km

MODIS MOD14A1 burnt area fire product has a continually poor index of agreement with

ground data kappa 0 and TSS value is 0. However, for MODIS MCD14ML we found high

kappa and true skill statistic values, showing a high. However, we recorded high kappa and

TSS values for the MODIS MCDML active fire product. These results, are consistent with the

premise that, increase in spatial resolution reduces the sensors ability to detect fires in African

Savanna woodlands.

iii | P a g e

Table of contents

Acknowledgements ..................................................................................................................... i

Abstract ...................................................................................................................................... ii

Table of contents ...................................................................................................................... iii

List of Figures ........................................................................................................................... iv

List of Appendices .................................................................................................................... vi

List of Tables ............................................................................................................................ vi

List of Abbreviations and Acronyms ....................................................................................... vii

Chapter 1: Introduction .............................................................................................................. 1

1.1: Problem statement........................................................................................................... 3

1.2: Objectives ....................................................................................................................... 3

1.3: Hypothesis ...................................................................................................................... 3

1.4: Justification of the study ................................................................................................. 4

Chapter 2: Materials and methods ............................................................................................. 5

2.1: Gwayi State Forest .......................................................................................................... 5

2.1.2: Matusadona National Park ....................................................................................... 7

2.2: Data Collection ............................................................................................................... 8

2.3: Data Analysis ................................................................................................................ 11

2.3.1: Pre-processing and Image analysis ........................................................................ 11

2.3.2: Post Classification...................................................................................................... 12

2.4: Statistics employed ................................................................................................... 12

Chapter 3: Results .................................................................................................................... 15

3.1: Validation of MODIS MOD14A1 in Gwayi State Forest ............................................ 15

3.2 Validation of MODIS MCD14ML ................................................................................ 15

3.3: Validation of MODIS MOD14A1 in Matusadona National Park ................................ 15

3.4: Validation of MODIS MCD14ML ............................................................................... 16

Chapter 4: Discussion .............................................................................................................. 29

Chapter 5: Conclusion.............................................................................................................. 31

Bibliography ............................................................................................................................ 33

Appendices ............................................................................................................................... 38

iv | P a g e

List of Figures

Figure 1: Map showing the spatial distribution of Baikiaea plurijuga woodlands in Zimbabwe

and the location of Gwayi State forest. ...................................................................................... 6

Figure 2: Map showing fire points collected from the Forest Commission database for the years

2000 to 2009 ............................................................................................................................ 10

Figure 3 Landsat 5 false colour composite. ............................................................................. 10

Figure 4 Landsat 5 false colour composite ............................................................................. 11

Figure 5 Map showing agreement between ground fire points and the classified MOD14A1 fire

product for the year 2000. ........................................................................................................ 17







Figure 9 Map showing agreement between ground fire points and the classified MCD14ML

fire product for the year 2000. ................................................................................................. 21







Figure 13: Map showing agreement between ground fire data and the MOD14A1 fire product

for the year 2015 (day 1) .......................................................................................................... 25

v | P a g e


for the year 2015 (day 2) .......................................................................................................... 26

Figure 15: Map showing agreement between ground fire data and the MCD14ML fire product

for the year 2015 (day 1) .......................................................................................................... 27


for the year 2015 ...................................................................................................................... 28

vi | P a g e

List of Appendices

Appendices 1: Error matrix tables for MODIS MOD14A1 in Gwayi Forest .......................... 38

Appendices 2: Error matrix tables for MODIS MCD14ML in Gwayi Forest ......................... 39

Appendices 3: Error matrix tables for MODIS MCD14ML in Matusadona National Park .... 39

Appendices 4: Error matrix tables for MODIS MOD14A1 in Matusadona National Park ..... 39

List of Tables

Table 1: Shows the two satellites validated in this study and their characteristics.................... 9

Table 2: Accuracy statistics for MODIS MOD14A1 and MODIS MCD14ML fire products in

Gwayi State forest in Zimbabwe.............................................................................................. 16


Mutusadona National Park in Zimbabwe ................................................................................ 17

vii | P a g e

List of Abbreviations and Acronyms

MODIS – Moderate Resolution Spectroradiometer

FIRMS – Fire Information for Resource Management

TSS – True Skill Statistic

ASTER – Advanced Spaceborne Thermal Emission and Reflection Radiometer

NOAA – National Oceanic and Atmospheric Administration

1 | P a g e

Chapter 1: Introduction

Fire has the capacity to break sustainable environments and can dramatically reduce the living

biomass in tropical forests even to the same degree as logging activities (Weber and Flannigan,

1997). Fire can significantly affect biomass composition and content of tropical trees and the

composition of flora, which are not adapted to this disturbance (Martins et al., 2012).

Ecosystems subjected to, frequent and severe fire disturbance can offset this balance resulting

in the ecosystem’s loss of recovery ability and major ecological functions (Held, 2006).

Empirical studies have demonstrated the higher mortality of trees with smaller diameter in

areas which experienced recent burning (Stroppiana et al., 2003) . Whereas an increase in

mortality of larger trees (≥50 cm dbh) has been reported 1-3 years after a fire (Stroppiana et

al., 2003) attributed to both natural and anthropogenic causes. Africa is referred to as the “Fire

Continent” and biomass burning is recognised as an important and extensive function of

African grasslands and savannas. However, 168 million hectares burn annually and savanna

burning accounts for 50% of this total. In African savannas fire is a key component playing a

pivotal ecological role in controlling vegetation patterns (Houghton, 2007).

Satellite based remote sensing observations have been used to generate global-scale burned

area (BA) products providing estimates of the extent and severity of damages (Brink and Eva,

2009). Satellite remote sensing is an essential technology for gathering post-fire-related data in

an affordable and time conserving manner, given the extremely broad spatial expanse and often

limited accessibility of the areas affected by forest fire (Veraverbeke et al., 2012). The

assessment of fire regimes across broad areas is done competently using data obtained from

remote sensors such as Moderate Resolution Spectroradiometer, SPOT, and National Oceanic

and Atmospheric Administration fire products, which are specifically designed for this purpose

2 | P a g e

(Korontzi et al., 2006) . Fire detection is performed using an algorithm that exploits the strong

emission of mid-infrared radiation from fires (Giglio et al., 1999). Burnt area detection is

critical for forest reserves to estimate, in automated manner, the aggregate expanse of the burnt

surface in time and space and is highly demanded for statistical inventories. Moreover, it is a

step towards more detailed analysis such as burn frequency and severity (Giglio et al., 2009).

Several studies, (Padilla et al., 2014) have validated different fire products in various parts and

different biomes of the world. Padilla et al., (2015) for example, compared the accuracies of

six remote sensing global burned area products using stratified random sampling and

estimation at a global scale in 2008. Padilla et al., (2014) carried out a performance study of

the MODIS MCD45A1 in automated burned area detection, in the Brazilian savanna.

Schroeder et al., (2008) validated GOES and MODIS active fire detection products using

ASTER and ETM+ data. A spatio-temporal analysis of fire relapse and magnitude for semi-

arid savanna ecosystems in Southern Africa was also done by (Pricope and Binford, 2012)

using moderate-resolution satellite imagery. In 2000 (Silva et al., 2003) used SPOT-

VEGETATION satellite data to estimate, during the dry seasons areas in Southern Africa. Silva

et al., (2003) carried out a study on burned area mapping in Greece using SPOT-4 HRVIR

images and Object-Based image analysis and suggested that, accurate information relating the

impact of fire environment is a key factor in, quantifying the impact of fires on landscapes,

selecting and prioritizing treatments applied on site, planning and monitoring restoration and

recovery activities and providing baseline information for future monitoring.

It is important to note that, not all satellites are efficient in detecting fire scars due to different

spatial, spectral and temporal resolutions. Empirical studies have established that, fire

comprises a major threat to protected areas and suggested that, a special focus be placed in the

detection of fire scars and rapid alert of fires within forest reserves. This study tested the

3 | P a g e

performance of two MODIS fire products in detecting burnt area scars in Baikiaea plurijuga

woodlands that is, MODIS MOD14A1 and MODIS MCD14ML.

1.1: Problem statement

By measuring thermal anomalies caused by active fires, remote sensing has considerable

potential for mapping burnt areas and fire regimes. However, limited studies have focused in

evaluating the impact of sensor resolution in detecting burnt scars or active fires particularly in

African Savana woodlands. . Because of their hardwood nature, African Savanna woodlands

have been designated as an important economic resource and the incidence and severity of

forest fires appears to have accelerated over the past decade reducing the commercial value of

this timber resource.

1.2: Objectives

1. To test the performance of MODIS MOD14A1 and MCD14ML fire products in the

detection of fire scars within African Savanna woodlands.

2. To assess the effects of spatial resolution and sensor characteristics on ability of remote

sensing products to detect fire in African Savanna woodlands

1.3: Hypothesis

Increase in spatial resolution decreases the sensors ability to detect fire scars in African

Savanna woodlands.

4 | P a g e

1.4: Justification of the study

Baikiaea plurijuga or teak woodlands, confined to the western and north western parts of the

country, on Kalahari sands, extending over one million hectares; contain the most

commercially exploitable indigenous timber species. Their average productivity ranges

between 150-200 m3/ha with a mean annual increment of 0.6 to 0.7 m3/ha (Chihambakwe,

1987). Because of their economic value, about 800 000 hectares of the natural baikiaea

woodlands were demarcated and gazetted as forest reserves. However, the incidence and

severity of forest fires appears to have accelerated over the past decade and reduces the

commercial value of timber (CHIHAMBAKWE, 1987)). Hence, accurate detection of fire in

these ecosystems using remote sensing and statistical indices of agreement may contribute

towards development of an operation fire detection system.

5 | P a g e

Chapter 2: Materials and methods

2.1: Gwayi State Forest

In this study accuracy assessment of the two fire products was done in two different study sites

that is, Gwayi State forest and Matusadona National park. Gwayi forest is one of several

designated forest areas meant to preserve biodiversity in its various forms across the country

(Zimbabwe). Gwayi state forest covers an area of about 1430 km2 and is located in the west of

Zimbabwe between latitudes 18̊ 45ˈ and 19̊ 30ˈS and longitudes 27̊ 40ˈ and 28̊ 22ˈE (Figure

3.1). Gwayi forest falls in region IV and V. Natural Region IV is located in the low-lying areas

in the north and south of the country and is characterised by a mean annual rainfall of 450-650

mm, severe dry spells during the rainy season and frequent seasonal droughts. Natural Region

V covers the lowland areas below 900 m above sea level in both the north and south of the

country and the mean annual rainfall in this region is less than 650 mm and is highly erratic.

Gwayi state forest is often referred to as the ‘Kalahari Sand Forest’ (Matose, 2008). This is

because most of the forest is located on deep Kalahari sands, whilst a small portion consists of

varied soils derived from sandstones and grits, with strong influences of Basalt. Considerable

alluvial soil occurs in close proximity to the main river such as Gwayi River (Mudekwe, 2006).

The uneven topography, poor soils, short rainy seasons and droughts that occur make the area

unsuitable for crop production even drought resistant dry-land cropping is subject to risk and

uncertainty. In general, Gwayi forest, as an indigenous forest, is critical in the management and

protection of the fragile Kalahari sand ecosystem. For rivers feeding into Zambezi, Gwayi

forest serves as a an important shield to catchment area protection and biodiversity

conservation, a shelter to wild animals and as a foundation of commercial timber and non-

6 | P a g e

timber forest products such as honey, mushrooms, edible insects and indigenous fruits (Matose,

2008).

The forest area was predominately designated for ensuring a sustained supply of hardwood

timber and revenue from timber concessions by the Forestry Commission (Field and Raupach,

2004). More so, Gwayi forest is particularly valuable for wildlife resource, as it contains the

complete range of ecosystems and an existing valuable wildlife population (Karsenty, 2008).

Grazing leases have gained importance to the Forest Commission's revenue generation system

since the 1990s and after wildlife related revenue and timber concessions, grazing leases are

the third highest source of revenue for the Gwayi Forest (Mudekwe, 2006).

Figure 1: Map showing the spatial distribution of Baikiaea plurijuga woodlands in

Zimbabwe and the location of Gwayi State forest.

7 | P a g e

2.1.2: Matusadona National Park

One of the few existing nature reserves, once considered also, a haven for endangered Black

Rhino and lions in Africa is Matusadona National park located on Lake Kariba’s shore. (Taylor,

1986). West of the Ume River, east of Sanyati river an area of about 338 000 (Purchase and

Du Toit, 2000). The origin of its name dates back to the mid-1900’s and it was named after the

foot-high “Matuzviadonha” Hills, also known as the Zambezi escarpment, located west of the

park. The Open woodlands on the upland, in the rear of the escarpment, are dominated by

Julbernardia globiflora (Purchase and Vhurumuku, 2005). Common also, on the slopes and

edges of the escarpment are acacia, Brachystegia glaucescens, and thick patches of Jesse bush,

Mopane scrub and woodland. Lakeshore marks the entire northern boundary of the park, a

shoreline frequently subjected to irregular variations in water level as a result of fluctuations

in annual rainfall and still experiencing rapid ecological development and change (Purchase

and Du Toit, 2000).

8 | P a g e

Figure 2. Map showing the location of Mutusadona National Park and ground fire points for

the year 2015.

2.2: Data Collection

Ground point data of burnt area scars was collected from the Forestry Commission Database

for the years 2000 to 2009 as shown in Figure 2 below. GPS fire data recorded in Matusadona

National park was provided by Doctor Masocha at University of Zimbabwe Geography and

Environmental Science Department. MODIS MOD14A1 1 km burnt area fire product and

MODIS MCD14ML 375 m active fire product for the years 2000 to 2009 where downloaded.

The selected Moderate Resolution Imaging Spectroradiometer (MODIS), has a I km spatial

resolution and a 12 hour return interval or temporal resolution. The MOD14A1 burnt area

product is based on the bidirectional reflectance distribution function (BRDF) modelling of the

surface, and detection of a persistent change in surface response, and primarily uses visible and

9 | P a g e

shortwave infrared time series data (Roy et al., 2005). Since initiation in April 2001, the Rapid

Response System is providing MODIS fire data and imagery (Justice et al., 2000). The Fire

Information for Resource Management System (FIRMS) MCD14ML product provides data in

a flexible web GIS format (URI.3) at a spatial resolution of 375m and was developed to provide

near real-time data within a 2 to 4hour acquisition time interval (Justice et al., 2010). (Justice

et al., 2002) also stated that, the active fire algorithm uses multiple channels to detect thermal

anomalies on a per pixel basis and the algorithm uses brightness temperatures derived from the

MODIS 4 and 11 Am channels, denoted by T4 and T11, respectively.

Table 1: Sensor characteristics validated in this study.

Satellite Spatial

Resolution

Temporal

resolution

No. of Bands Product

MODIS

MOD14A1

1km 12hr 8 Burnt Area

MODIS

MCD14ML

375m 2-4hrs Active Fire

10 | P a g e

Figure 2: Map showing fire points collected from the Forest Commission database for the

years 2000 to 2009

Figure 3 Landsat 5 false colour composite showing an overlay of ground fire data on burnt

areas for the year 2000 (left tile) and 2005 (right tile) in Gwayi State forest. The black border

marks the boundary of Gwayi State forest, fire in these images appears red and ground fire

points (for the respective years) used for validation appear yellow.

11 | P a g e

Figure 4 Landsat 5 false colour composite showing an overlay of ground fire data on burnt

areas for the year 2006 (left tile) and 2009 (right tile). The black border marks the boundary

of Gwayi State forest, fire in these images appears red and ground fire points (for the

respective years) used for validation appear yellow.

2.3: Data Analysis

2.3.1: Pre-processing and Image analysis

MODIS MOD14A1 1km burnt area fire product and MODIS MCD14ML 375 m fire product

for the years 2000 to 2009 where downloaded from two remote sensing websites,

glovis.usgs.glov and FIRMS Alert. The two Modis fire products were downloaded with a

geographic coordinate system and projected to UTM coordinate system zone 35 South, were

the study site is located. Given the limited data obtained from the ground for cross validation,

only images for the years 2000, 2005, 2006 and 2009 where used to test the performance of the

afore mentioned fire products in detecting fire scars in Baikiaea plurijuga woodlands.

The slicing operator in ILIWIS 3.0 was used to classify the range of values of the MOD14A1

fire product into two classes, fire and none fire output map. A group domain was created

12 | P a g e

beforehand listing the upper value boundaries of the groups and the class names. The classified

maps were imported into Arc GIS 10.1 and cells of the classified map that correspond to the

areas defined by the study area were masked out for further analysis.

2.3.2: Post Classification

We assessed the accuracy of the re-classified images using fire incidence data measured in the

field with a global position system (GPS) handheld receiver. In Gwayi, the location of active

fire scars was measured by forest officers employed by the Forest Commission. In Matusadona,

one of the authors measured location of fire incidence points. For each study site and date, we

overlaid a point map of burnt sites on top of raster layer for active fires obtained from MODIS,

FIRMS and SPOT the databases in Arc Map 10.1. We then used the extraction tool from the

arc tool box spatial analyst package to extract the cell values of each raster that is, the fire

products, based on the set of fire incident data measured in the field. We then used the

frequency tool from the statistics package to create a new table containing unique field values

and the number of occurrence of each unique field value, that is, for every different ground

point and every different predicted value, what the frequency is. Then we calculated error

matrices or pivot tables using results from the frequency tables.

2.4: Statistical Analysis

To summarise the relationship between satellite data and reference test data we used 2x2 error

matrices. From the error matrices overall accuracy, which incorporates the major diagonal and

excludes the omission and commission errors, was calculated summarising the total agreement

or disagreement between the maps (Allouche et al., 2006).

13 | P a g e

We also employed Khat a kappa statistic which is a measure of agreement or accuracy. To

guarantee the accuracy of remote sensing derived data in fire detection, we tested both MODIS

burnt area and active fire products. The kappa statistic derived from error matrices, incorporates

the off-diagonals of the error matrices representing agreement obtained having removed the

proportion of agreement that could be expected to occur by chance. Kappa values Range from

-1 to +1, values > 0.80 present strong agreement, values between 0.4 and 0.8 represent

moderate agreement and Values < 0.4 are indicative of a poor agreement (Balogun and Salami,

2016).

𝐾 =Pr(𝑎) − Pr(𝑒)

1 − Pr(𝑒)

In this study we also used the true skill statistic, This statistic is a simple and intuitive measure

for the performance of distribution models when predictions are expressed as presence–absence

maps (Allouche et al., 2006). It is a skill score expressing the hit rate relative to the false alarm

rate and remains positive as long as H remains greater than F. For a 2x2 confusion matrix TSS

is defined as:

𝑇𝑆𝑆 =𝑎𝑑 − 𝑏𝑐

(𝑎 + 𝑐)(𝑏 + 𝑑)𝑜𝑟𝑆𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦 + 𝑆𝑝𝑒𝑐𝑖𝑓𝑖𝑐𝑖𝑡𝑦 − 1

It takes into account omission and commission errors, and success as random speculation.

Sensitivity is the proportion of observed presences that are predicted as such therefore, it

quantifies omission errors. Specificity is the proportion of observed absences that are predicted

as such therefore, it quantifies commission errors. TSS Values Range from -1 to +1. A value

of +1 indicates a perfect agreement and values less than or equal to 0 indicate a performance

significantly no better than random (Nüchel et al., 2015).

14 | P a g e

Compared to kappa, TSS is not affected by prevalence, size of the validation set. In contrast to

kappa, documented effects of prevalence on TSS can be interpreted as evidence for real

ecological phenomena rather than statistical artefacts (Allouche et al., 2006).

15 | P a g e

Chapter 3: Results

3.1: Validation of MODIS MOD14A1 in Gwayi State Forest

Results from kappa and true skill statistic for the year 2000 to 2009 suggest that, there is a poor

agreement between the MOD14A1 fire product and the ground fire points used as the test set

and, the sensors ability to detect fires is no better than random. All fire pixels have been

commissioned to the non-fire class resulting in low values of both kappa and true skill statistic

(Table 1). Figure 5, 6, 7 and 8 below show also that, there is no improvement in the products

ability to detect fires over the years under study.

3.2 Validation of MODIS MCD14ML

Accuracy statistics for MODIS MCD14ML suggest a strong agreement between the fire

product and ground fire data. With the exception of the year 2000 with low kappa and true skill

statistic values, 2005, 2006 and 2007 gave us values close to one for both kappa and true skill

statistic suggesting a performance far from random in detecting fires in Baikiaea Plurijuga

woodlands (Table 1). However, as shown in figure 10, 11 and 12, there are a few commission

and omission errors reducing the products overall accuracy in fire detection.

3.3: Validation of MODIS MOD14A1 in Matusadona National Park

Kappa and true skill statistic results show that there is a poor agreement between the fire

product and ground fire data collected in Matusadona national park. We found very low values

of kappa and TSS in both days that is, 18 September 2015 and the 20th of September of the

16 | P a g e

same year. Figures 13 and 14 below confirm visually the disagreement between the product

and ground data.

3.4: Validation of MODIS MCD14ML

We found a moderate agreement between the MODIS MCD14ML active fire product and the

ground test data collected in Matusadona National Park. We also observed that, moderately

high values of kappa and the true skill statistic for 20 September 2015 and relatively values for

the 18th of September same year (Figure 15 and 16).


Gwayi State forest in Zimbabwe.

Fire Product Year Overall

accuracy

Kappa Value TSS Value

MOD14A1 2000 0.5 0 0

2005 0.5 0 0

2006 0.5 0 0

2009 0.5 0 0

MCD14ML 2000 0.5 0 0

2005 0.8 0.6 0.6

2006 0.9 0.9 0.9

2009 0.8 0.7 0.7

17 | P a g e


Mutusadona National Park in Zimbabwe

Fire Product Day/Year Overall

Accuracy

Kappa Value TSS Value

MOD14A1 18 September

2015

0.4 0 0

20 September

2015

0.3 0 0

MCD14ML 18 September

2015

0.5 0.3 0.3

20 September

2015

0.7 0.6 0.6

Figure 5 Map showing agreement between ground fire points and the classified MOD14A1

fire product for the year 2000.

Figure 5 illustrates an overlay of ground data onto the MOD14A1 burnt area product for the

year 2000 in Gwayi State forest. The red points are fire data and the black surface is a clip of

18 | P a g e

the MOD14A1 burnt area product clipped using Gwayi State forest boundary. It can be

observed that in this year the product did not detect any burnt areas at least within Gwayi and

the fire data is falling onto none fire pixels. This indicates a poor index of agreement between

ground fire and the burnt area product.


;fire product for the year 2005.

Figure 6 is showing that there is no improvement in agreement between the test set and the

burnt area product from the year 2000. Instead, a poor agreement between the two datasets can

still be observed for the year 2005. No burnt area scars can be seen in the MOD14A1 for this

year.

19 | P a g e



Figure 7 is showing a poor agreement between the test set and the 2006 MOD14A1 burnt area

product. Ground fire data is falling onto non-fire pixels and there is no evidence of fire burnt

areas within Gwayi State Forest.

20 | P a g e



Figure 8 is showing an agreement map between ground data and the 2008 MOD14A1 burnt

area product. It can be seen that the entire test set is falling on none fire pixels. This again

shows that there are a poor agreement between the two data sets.

21 | P a g e



Figure 9 illustrates that no fires were detected in Gwayi State forest by the MCD14Ml fire

product in the year 2000. Appendix 2 shows that there are no fires correctly detected as fire,

six non fire points were correctly predicted as none fire and 6 out of 6 fire points on the

ground were predicted as non-fire. The agreement between the two data sets is poor.

22 | P a g e



Figure 10 illustrates an overlay of ground fire data onto MCD14ML fire points for the year

2005. Yellow points represent ground fire data and the purple points represent MCD14ML fire

points. It can be observed that, 16 fire points were correctly predicted as fire on the product

data points, 9 fire points on the ground were falsely predicted as non-fire. None of the fires

were ground fire were falsely predicted as none fire on the product and 25 none fire points were

correctly predicted as none fire. The agreement is moderate.

23 | P a g e



Figure 11 illustrates an overlay of ground data for the year 2006 on an MCD14ML fire product.

It can be observed that, 13 fire points were correctly predicted as fire, 3 none fire points on the

ground were predicted as fire and 16 none fire points were correctly predicted as none fire on

the product. The overall agreement between ground data and the active fire product is high.

24 | P a g e



Figure 12 illustrates an agreement of ground data and the MCD14Ml product for the year 2009.

It is evident that, 8 fire points were correctly predicted as fire, none of the ground fires were

falsely predicted as none fire and 12 none fire points on the ground were correctly predicted as

none fire. The agreement is slightly above moderate.

25 | P a g e


for the year 2015 (day 1)

Figure 13 illustrates ground fire points in Matusadona for the 18th of September 2015. It can

be seen that there is only one fire scar falling in Gwayi and none of the fire points are overlaying

onto the scar. This shows a poor agreement of the two data sets. Appendix 4).

26 | P a g e


for the year 2015 (day 2)

Figure 14 illustrates fire data on an MOD14A1 burnt area product for the 20th of September

and no improvement can be observed. None of the fire points overlay with the burnt area

represented by the red point.

27 | P a g e

Figure 15: Map showing agreement between ground fire data and the MCD14ML fire

product for the year 2015 (day 1)

Figure 15 illustrates fire points on an MCD14ML active fire product for the 18th of September

2015 in Matusadona National Park. It is evident that 6 fire points were correctly predicted as

fires, 11 none fire points were correctly predicted as non-fire, 13 non-fire points were falsely

predicted as and none of the fires were incorrectly classified as none fire. The agreement

between the two data sets is poor.

28 | P a g e


for the year 2015

Figure 16 illustrates fire data overlay on an MCD14ML active fire product for the 20th of

September 2015. Observations show an improvement in the agreement of the ground data set

and the active fire product. 10 fire points were correctly classified as fire, 8 none fire points

were falsely classified as fire, none of the fire points were incorrectly classified as none fire

and 11 none fire points were correctly classified as none fire. This again shows a moderate

agreement between the two datasets.

29 | P a g e

Chapter 4: Discussion

Findings in this study reveal that MODIS MOD14A1 burnt area fire product has a poor index

of agreement with ground validation data, suggesting that the performance of the product in

detecting burnt areas is no better than random. This finding is consistent with the hypothesis

that increase in spatial resolution decreases the sensors ability to detect fire scars in forest

ecosystems as small fire patches are masked out (Roy and Boschetti 2009). We found that

during the period between 2000 and 2009 detection accuracy of burnt areas was consistently

poor in Gwayi and this result is consistent with the observation in Matusadona National park

in 2015 a different ecosystem dominated by Julbernardia globiflora and Colophospermum

Mopane. Multiple factors may result in poor detection of burnt area fire scars. For example,

errors of sensor data, lack of sensitivity of the burnt area classification algorithms, a course

spatial resolution and limitations of using a global or continental burnt area product on a

smaller location (Roy et al. 2005), (Padilla et al. 2014). However, these findings contradict the

findings of Padilla et al., (Padilla et al. 2014) whose results showed two biomes with higher

accuracy that is, Boreal forest, and Tropical & Subtropical savannah. These findings also differ

from that of a study carried out in Ethiopia by (Molinario et al. 2014) whose results show 90%

probability of fire detection using MODIS MOD14A1. These findings suggest that using a

course spatial resolution, continental burnt area product at a smaller scale may result in poor

detection of fires in Baikiaea plurijuga woodlands than a high resolution sensor with a more

frequent acquisition return interval. This is further emphasised by similar findings in

Matusadona National Park, a different ecosystem from Gwayi State forest.

However, for MODIS MCD14ML with a higher spatial resolution, we consistently find high

accuracy values showing a strong index of agreement with the ground validation data. These

findings suggest that, MODIS MCD14ML active fire product is more accurate in detecting

30 | P a g e

active fires in Baikiaea plurijuga woodlands and its performance is far from random. The

validation results of this study demonstrate that a higher spatial resolution and a more frequent

return interval are important sensor characteristics to explain a higher detection of fires in both

study sites, which harbour Baikiaea plurijuga woodlands. That moderate resolution fire

products are better capable to detect fire in different ecosystems has been demonstrated in

different biomes of the world (Schroeder et al. 2008) Amazonia, (Morisette et al. 2005) South

Africa and (Csiszar et al. 2006) in Northern Eurasia. Consequently, improved sensor

characteristics are more likely to produce relatively frequent omission errors while commission

errors are comparatively rare and active fire data is closely associated to real burnt areas, with

very low commission errors (Hantson et al. 2013).

Relating satellite data on burnt areas can be helpful in understanding the link between fire and

Baikiaea plurijuga ecosystems in the Kalahari sands. This study compared the performance of

two different MODIS fire products using two validation methods that is, kappa and true skill

statistic to account for method limitations in accuracy assessment. Moreover, a comparison of

results from Gwayi and Matusadona reserves using two different data sets ensured us a

conclusive assessment of the two products without data set bias. This type of data or

information is crucial and will assist ecologists to understand a key component of fire regimes

within Baikiaea plurijuga ecosystems which are of economic value in Zimbabwe. Therefore,

eliminating any method limitation was critical. MODIS burnt area products have been validated

in various portions and diverse ecosystems of the globe (Giglio et al. 1999). A number of

publications have been made on the validation of MODIS MOD14 using ASTER, twice in the

Brazilian Amazonia (Morisette et al. 2005);(Schroeder et al. 2008), and once in Siberian boreal

forests (Giglio et al. 2006) and one for southern Africa (Morisette et al. 2005) which discussed

tropical grasslands and savanna biomes. Our findings are novel in view of that, to our

31 | P a g e

knowledge no validation effort has been attempted with regard to Baikiaea plurijuga

woodlands, suggesting that MODIS MCD14ML active fire product may be used to detect and

monitor fire incidences in Baikiaea plurijuga woodlands.

The major limitation to this study is the size of the validation set for all the years which is

relatively small. However, to account for the validation set size, we employed the True Skill

Statistic which is not affected by the size of the validation set and assessed the same products

using a different validation set in a different study site. The availability of data only to the year

2009 limited us to the MODIS MOD14A1 product data archive, however in this perspective

we highlight the MODIS collection 5 MCD45A1 and the new MODIS collection 6 MCD64A1

product, based on the integration of optical and thermal data, (Giglio et al. 2009), which is

anticipated to substantially improve burned area detection specially in dense vegetated areas.

Further investigation may also focus on what effect burned area size has on the sensors ability

to detect post fire scars as this information would be of great interest for burned area algorithm

developers and end-users.

‘

32 | P a g e

Chapter 5: Conclusion

To our knowledge, this study provided the first satellite accuracy assessment evidence that an

increase in spatial resolution may decrease the sensors ability to detect forest fires in Baikiaea

plurijuga woodlands. Increasing the sensors spatial resolution may be necessary to improve

the sensors ability to detect fires in these woodlands, since Baikiaea plurijuga is a hardwood

of economic value, necessitating sustainable conservation of the timber species. This study

concluded that, the global scale burnt area fire product MODIS MOD14A1 is not suitable for

fire detection in Baikiaea plurijuga woodlands, as observed in our findings where kappa and

TSS values constantly remained at 0 implying a poor agreement and a performance no better

than random in both study sites and comparing the four years of assessment in Gwayi. We

suggested that, the poor performance may be a result of the sensors course spatial resolution

and a poor return interval (temporal resolution). However, MODIS MCD14ML active fire

product had high indices of agreement with ground data in both study sites and overall in all

the years under study. Thus, we concluded that the product may suitable for use as an

operational system to detect and map burned areas in an accurate and timely fashion in Baikiaea

plurijuga woodlands.

33 | P a g e

Bibliography

Allouche, O., Tsoar, A. and Kadmon, R. (2006) 'Assessing the accuracy of species distribution

models: prevalence, kappa and the true skill statistic (TSS)', Journal of applied ecology,

43(6), pp. 1223-1232.

Balogun, B. O. and Salami, A. T. (2016) 'Biofuel Production Induced Land-use Land-cover

Change in Selected Geopolitical Zones of Nigeria', Journal of Sustainable

Development, 9(3), pp. 6.

Brink, A. B. and Eva, H. D. (2009) 'Monitoring 25 years of land cover change dynamics in

Africa: A sample based remote sensing approach', Applied Geography, 29(4), pp. 501-

512.

CHIHAMBAKWE, M. 'Policies and strategies for resource management in Zimbabwe'. SACC

Natural Resource Workshop, Blantyre, Malawi.

Csiszar, I. A., Morisette, J. T. and Giglio, L. (2006) 'Validation of active fire detection from

moderate-resolution satellite sensors: the MODIS example in northern Eurasia',

Geoscience and Remote Sensing, IEEE Transactions on, 44(7), pp. 1757-1764.

Field, C. B. and Raupach, M. R. (2004) The global carbon cycle: integrating humans, climate,

and the natural world. Island Press.

Giglio, L., Csiszar, I. and Justice, C. O. (2006) 'Global distribution and seasonality of active

fires as observed with the Terra and Aqua Moderate Resolution Imaging

Spectroradiometer (MODIS) sensors', Journal of Geophysical Research:

Biogeosciences, 111(G2).

Giglio, L., Kendall, J. and Justice, C. (1999) 'Evaluation of global fire detection algorithms

using simulated AVHRR infrared data', International Journal of Remote Sensing,

20(10), pp. 1947-1985.

34 | P a g e

Giglio, L., Loboda, T., Roy, D. P., Quayle, B. and Justice, C. O. (2009) 'An active-fire based

burned area mapping algorithm for the MODIS sensor', Remote Sensing of

Environment, 113(2), pp. 408-420.

Hantson, S., Padilla, M., Corti, D. and Chuvieco, E. (2013) 'Strengths and weaknesses of

MODIS hotspots to characterize global fire occurrence', Remote Sensing of

Environment, 131, pp. 152-159.

Held, A. (2006) 'Forestry Department'.

Houghton, R. (2007) 'Balancing the global carbon budget', Annu. Rev. Earth Planet. Sci., 35,

pp. 313-347.

Justice, C., Giglio, L., Korontzi, S., Owens, J., Morisette, J., Roy, D., Descloitres, J., Alleaume,

S., Petitcolin, F. and Kaufman, Y. (2002) 'The MODIS fire products', Remote Sensing

of Environment, 83(1), pp. 244-262.

Justice, C., Wolfe, W., El-Saleous, N., Descloitres, J., Vermote, E., Roy, D., Owens, J. and

Masuoka, E. (2000) 'The availability and status of MODIS Land products', The Earth

Observer, 12(6), pp. 10-18.

Justice, C. O., Giglio, L., Roy, D., Boschetti, L., Csiszar, I., Davies, D., Korontzi, S., Schroeder,

W., O’Neal, K. and Morisette, J. (2010) 'MODIS-derived global fire products', Land

Remote Sensing and Global Environmental Change: Springer, pp. 661-679.

Karsenty, A. (2008) 'The architecture of proposed REDD schemes after Bali: facing critical

choices', International Forestry Review, 10(3), pp. 443-457.

Korontzi, S., McCarty, J., Loboda, T., Kumar, S. and Justice, C. (2006) 'Global distribution of

agricultural fires in croplands from 3 years of Moderate Resolution Imaging

Spectroradiometer (MODIS) data', Global Biogeochemical Cycles, 20(2).

35 | P a g e

Martins, F. d. S. R. V., Xaud, H. A. M., dos Santos, J. R. and Galvao, L. S. (2012) 'Effects of

fire on above-ground forest biomass in the northern Brazilian Amazon', Journal of

Tropical Ecology, 28(06), pp. 591-601.

Matose, F. (2008) 'Institutional configurations around forest reserves in Zimbabwe: the

challenge of nested institutions for resource management', Local Environment, 13(5),

pp. 393-404.

Molinario, G., Davies, D. K., Schroeder, W. and Justice, C. O. (2014) 'Characterizing the

spatio-temporal fire regime in Ethiopia using the MODIS-active fire product: a

replicable methodology for country-level fire reporting', African Geographical Review,

33(2), pp. 99-123.

Morisette, J. T., Giglio, L., Csiszar, I., Setzer, A., Schroeder, W., Morton, D. and Justice, C. O.

(2005) 'Validation of MODIS active fire detection products derived from two

algorithms', Earth Interactions, 9(9), pp. 1-25.

Mudekwe, J. 'Demography and dynamics of commonly harvested woody species in a protected

indigenous forest in Zimbabwe'. Multiple use management of natural forests and

woodlands: Policy refinements and scientific progress. Proceedings of Natural Forests

and Savanna Woodlands Symposium IV, Port Elizabeth, South Africa, 15-18.

Nüchel, J., Bøcher, P. K. and Svenning, J.-C. 'Present and potential distribution of Snub-nosed

Monkey'. International Biogeography Society: 7th Biennial Conference.

Padilla, M., Stehman, S. V. and Chuvieco, E. (2014) 'Validation of the 2008 MODIS-MCD45

global burned area product using stratified random sampling', Remote Sensing of

Environment, 144, pp. 187-196.

Padilla, M., Stehman, S. V., Ramo, R., Corti, D., Hantson, S., Oliva, P., Alonso-Canas, I.,

Bradley, A. V., Tansey, K. and Mota, B. (2015) 'Comparing the accuracies of remote

36 | P a g e

sensing global burned area products using stratified random sampling and estimation',

Remote sensing of environment, 160, pp. 114-121.

Pricope, N. G. and Binford, M. W. (2012) 'A spatio-temporal analysis of fire recurrence and

extent for semi-arid savanna ecosystems in southern Africa using moderate-resolution

satellite imagery', Journal of environmental management, 100, pp. 72-85.

Purchase, G. and Du Toit, J. (2000) 'The use of space and prey by cheetahs in Matusadona

National Park, Zimbabwe', South African Journal of Wildlife Research, 30(4).

Purchase, G. and Vhurumuku, G. (2005) 'Evaluation of a wild-wild translocation of cheetah

(Acinonyx jubatus) from private land to Matusadona National Park, Zimbabwe (1994-

2005)', Zambezi Society, Bulawayo.

Roy, D., Jin, Y., Lewis, P. and Justice, C. (2005) 'Prototyping a global algorithm for systematic

fire-affected area mapping using MODIS time series data', Remote sensing of

environment, 97(2), pp. 137-162.

Roy, D. P. and Boschetti, L. (2009) 'Southern Africa validation of the MODIS, L3JRC, and

GlobCarbon burned-area products', Geoscience and Remote Sensing, IEEE

Transactions on, 47(4), pp. 1032-1044.

Schroeder, W., Prins, E., Giglio, L., Csiszar, I., Schmidt, C., Morisette, J. and Morton, D.

(2008) 'Validation of GOES and MODIS active fire detection products using ASTER

and ETM+ data', Remote Sensing of Environment, 112(5), pp. 2711-2726.

Silva, J., Pereira, J., Cabral, A. I., Sá, A. C., Vasconcelos, M. J., Mota, B. and Grégoire, J. M.

(2003) 'An estimate of the area burned in southern Africa during the 2000 dry season

using SPOT‐VEGETATION satellite data', Journal of Geophysical Research:

Atmospheres, 108(D13).

37 | P a g e

Stroppiana, D., Grégoire, J.-M. and Pereira, J. M. (2003) 'The use of SPOT VEGETATION

data in a classification tree approach for burnt area mapping in Australian savanna',

International journal of remote sensing, 24(10), pp. 2131-2151.

Taylor, R. (1986) 'The unsuccessful introduction of white rhinoceros to Matusadona National

Park, Kariba', Pachyderm, 6, pp. 14-15.

Veraverbeke, S., Gitas, I., Katagis, T., Polychronaki, A., Somers, B. and Goossens, R. (2012)

'Assessing post-fire vegetation recovery using red–near infrared vegetation indices:

Accounting for background and vegetation variability', ISPRS Journal of

Photogrammetry and Remote Sensing, 68, pp. 28-39.

Weber, M. G. and Flannigan, M. D. (1997) 'Canadian boreal forest ecosystem structure and

function in a changing climate: impact on fire regimes', Environmental Reviews, 5(3-

4), pp. 145-166.

38 | P a g e

Appendices

2000 2005

Predict Predict

0 1 Total 0 1 Total

Ground data 0 6 0 6

Ground

data 0 25 0 25

1 6 0 6 Total 1 25 0 25

Total 12 0 12 Total 50 0 50

Appendices 1: Error matrix tables for MODIS MOD14A1 in Gwayi Forest

2000 2005

Predict Predict

1 0 Total 1 0 Total

Ground data 1 0 6 6

Ground

data 1 16 0 16

0 0 6 6 0 9 25 34

Total 0 12 12 Total 25 25 50

2006 2009

Predict Predict

0 1 Total 0 1 Total

Ground data 0 16 0 16

Ground

data 0 12 0 12

Total 1 16 0 16 1 12 0 12

Total 32 0 32 Total 24 0 24

39 | P a g e

2006 2009

Predict Predict

1 0 Total 1 0 Total


Ground

data 1 8 0 1

0 3 16 19 0 4 12 11

Total 16 16 32 Total 12 12 24

Appendices 2: Error matrix tables for MODIS MCD14ML in Gwayi Forest

2015/09/18 2015/09/20

Predict Predict

1 0 Total 1 0 Total

Ground data 1 6 0 6

Ground

data 1 10 0 10

0 13 11 24 0 8 11 19

Total 19 11 30 Total 18 11 29

Appendices 3: Error matrix tables for MODIS MCD14ML in Matusadona National Park

2015/09/18 2015/09/20

Predict Predict

1 0 Total 1 0 Total


Ground

data 1 0 0 0

0 0 0 0 0 18 11 29

Total 19 11 30 Total 18 11 29

Appendices 4: Error matrix tables for MODIS MOD14A1 in Matusadona National Park

Documents

Accuracy assessment of MODIS fire products in African Savanna …ir.uz.ac.zw/jspui/bitstream/10646/3461/1/Mpofu_N_Accuracy... · 2019. 10. 16. · African grasslands and savannas