Accuracy assessment of Remote Sensing Data

ACCURACY ASSESSMENTACCURACY ASSESSMENT

ACCURACY ASSESSMENT

– Assess accuracy of a remote sensing output is one of the most important steps in any classification exercise!!

– Without an accuracy assessment the output or

results is of little value

Goals:

Assess how well a classification worked

Understand how to interpret the usefulness of someone else’s classification

Nature of Classification

1) Class definition problems

2) Inappropriate class labels

3) Mislabeling of classes

Accuracy Assessment

Overview

Collect reference data: “ground truth”

Determination of class types at specific locations

Compare reference to classified map

Does class type on classified map = class type determined from reference data?

Reference Data

Some possible sources

Aerial photo interpretation

Ground truth with GPS

GIS layers

Reference Data

Issue 1: Choosing reference source

Make sure you can actually extract from the reference source the information that you need for the classification scheme

I.e. Aerial photos may not be good reference data if your classification scheme distinguishes four species of grass. You may need GPS’d ground data.

Reference data

Issue 2: Determining size of reference plots

Match spatial scale of reference plots and remotely-sensed data

I.e. GPS’d ground plots 5 meters on a side may not be useful if remotely-sensed cells are 1km on a side. You may need aerial photos or even other satellite images.

Sampling Methods

Simple Random Sampling:observations are randomly placed.

Stratified Random Samplingminimum number of observationsare randomly placed in eachcategory.

Sampling Methods

Systematic SamplingObservations are placed at equal intervalsaccording to a strategy.

Systematic Non-Aligned Sampling:

a grid provides even distribution ofrandomly placed observations.

Sampling Methods

Cluster SamplingCluster Sampling

Randomlyplaced “centroids” used as a baseof several nearby observations. The nearby observations can berandomly selected, systematicallyselected, etc...

Error matrix

Summarize using an error matrix

Class types determined from

reference sourcereference source

Class types determined

from classified

image

# Plots Cotton Sugarcane Fodder Totals

Cotton 50 5 2 57

Sugarcane 14 13 0 27

Fodder 3 5 8 16

Totals 67 23 10 100

Total Accuracy

Quantifying accuracy– Total Accuracy: Number of correct plots / total number of

plotsClass types determined from


Class types

determined from

classified map


Cotton 50 5 2 57


Fodder 3 5 8 16

Totals 67 23 10 100

%71100*100

81350 =++=AccuracyTotal

Diagonals represent sites classified correctly according to reference data

Off-diagonals were mis-classified

Total Accuracy

Problem with Total Accuracy:Summary value is an average

Does not reveal if error was evenly distributed between classes or if some classes were really bad and some really good

Therefore, include other forms:User’s accuracyProducer’s accuracy

User’s and producer’s accuracy and types of error

User’s accuracy corresponds to error of commission (inclusion):

Producer’s accuracy corresponds to error of omission (exclusion):

User’s Accuracy

From the perspective of the user of the classified map, how accurate is the map? For a given class, how many of the pixels on the

map are actually what they say they are?– Calculated as:

Number correctly identified in a given map class /

Number claimed to be in that map class

User’s Accuracy



Class types

determined from

classified map


Cotton 50 5 2 57


Fodder 3 5 8 16

Totals 67 23 10 100

%88100*57

50,'

==AccuracyCottonsUser

Example: Cotton

Producer’s Accuracy

From the perspective of the maker of the classified map, how accurate is the map? For a given class in reference plots, how many of

the pixels on the map are labeled correctly?

– Calculated as:

Number correctly identified in ref. plots of a given class /

Number actually in that reference class

Producer’s AccuracyClass types determined from


Class types

determined from

classified image

# Plots Conifer sugarcane Fodder Totals

Cotton 50 5 2 57

sugarcane 14 13 0 27

Fodder 3 5 8 16

Totals 67 23 10 100

%75100*67

50'

==Accuracys,Cottonproducer

Example: Cotton

Summary so far


reference sourcereference sourceUser’s AccuracyClass types

determined from classified

Image


Cotton 50 5 2 57 88%Sugarcane 14 13 0 27 48%Fodder 3 5 8 16 50%

Totals 67 23 10 100

Producer’s Accuracy 75% 57% 80% Total: 71%

Accuracy Assessment in ERDAS Imagine

Make a file in Excel with ONLY the X-Y coordinates for your GPS points collected in field

Now save this new file as a .txt file (Tab delimited)

In ERDAS From the Accuracy Assessment window

go to Edit > Import User-defined Points.

Put GPS co-ordinates file and click ok the window opens as:

Now you can enter your class values for the reference numbers in the Accuracy Assessment table

Go to Edit > Show Class Values and the Class column will be filled in with the values that are the actual values of the pixels located at the X-Y coordinates that you indicated

Finally, you can look at the statistics of your accuracy assessment by first going to Report > Options. Then go to Report > Accuracy Report and the following table will appear

References

• Lillesand and Kiefer, Chapter 7

• Congalton, R. G. and K. Green. 1999. Assessing the accuracy of remotely sensed data: Principles and practices. Lewis Publishers, Boca Raton.

• Congalton, R.G. 1991. A review of assessing the accuracy of classification of remotely sensed data. Remote Sensing of Environment 37:35-46

• ERDAS, (2005). ERDAS 9.1, Field Guide. Geospatial Imaging, LLC Norcross, Georgia.

• Van Neil, T.G. and McVicar, T.R., (2004) Current and potential uses of optical remote sensing in rice based irrigation systems: A review, Australian Journal of Agricultural Research, 55, 155-85, CSIRO

Engineering

Accuracy assessment of Remote Sensing Data