SRTM 30m Digital Elevation Models:

Downloading and processing from seamless.usgs.gov/Run.htm

Nathan A. Toké

01/22/04

Table of Contents

1. Introduction

2. Downloading data sets using seamless

3. Unzipping and organizing seamless files

4. Importing files into ArcMap

5. Projecting grid files

6. Reducing grid file size

7. Merging grid files

8. Using hillshade and color ramp visualization

9. Comparing SRTM and NED data sets in terms of slope tendencies

Step 1 – a common DEM projection

Step 2 – calculating slope files

Step 3 – slope files integerized

Step 4 – clipping slope files to one another

Step 5 – converting raster (grid) to features (shapefile)

Step 6 – exporting slope points as .txt

Step 7 – using Vi to clean .txt files

Step 8 – using Mat Lab scripts to compare the data

Introduction

This document is a tutorial describing how to download and process DEM data

from the USGS web site (seamless.usgs.gov) using the ESRI products: ArcMap,

ArcCatalogue, and ArcToolbox. I downloaded the Shuttle Radar Topography Mission

(SRTM) 30m data for the Rocky Mountain GEON test bed; which extends in latitude

from 31 to 49 degrees north and in longitude from 103 to 115 degrees west. In this

document I will describe how I downloaded, unzipped, and organized these files. Then I

will discuss how to import grid files into ArcMap, reduce file size, re-project, merge,

rename, and hillshade the SRTM 30m grid files. Finally, I will describe how to calculate

slope files from the DEM data and one way in which I compared SRTM slopes to NED

slopes over the same geographical space.

Downloading files from seamless

SRTM 30m, NED, and many other useful GIS datasets are available for download

from seamless.usgs.gov/Run.htm. The data are available by several methods of selection.

The selection options are on the left side of the seamless viewer. I chose to use the define

area by coordinates option because I needed to download a specific area (figure 1).

Figure 1: Define area by coordinates allows you to select a specific area for download (right window).

When selecting an area for download, seamless returns multiple zipped files that

make up the area of selection. The amount and the size of these files vary depending

upon the area you select, so it is best to select an area of consistent size and range, this

way the zipped files will be the same size and their boarders will align. I consistently

selected areas of 1 degree of latitude by 12-degree strips of longitude, from 103 to 115

degrees west (the width of GEON). This returned 8 zipped files for download, each file

had dimensions of 1 degree of latitude by 1.5 degrees of longitude (figure 2). I limited the

selection area to these increments because seamless requests that you pay for the data to

be sent by CD if the selection area is greater than 100mb of zipped data (about 30 square

arc seconds for the SRTM 30m data).

Figure 2: After defining and area by coordinates, seamless breaks the download area into zipped files.

Each file covers a discrete area. Using the same coordinate dimensions for each selection will provide

consistent file dimensions. In this case 8 files (each 1 degree of latitude by 1.5 degrees of longitude)

completed the area of selection that was one degree of latitude by 12 degrees of longitude.

To complete the download click download on the desired zip file. Seamless

opens a pop up window that displays the data retrieval progress. Eventually a file

download window will appear (figure 3), seamless will select a random numerical file

name. At this point it is important to change the file name to one corresponding to the

geographical area of the file, I chose to name files according to their range in latitude and

longitude. It is also important to save the zip files to folders corresponding to that name, I

chose to bin the 8 zipped files for each 12-degree strip of longitude into a folder named

according to that strip’s latitudinal range.

Figure 3: Upon clicking download for one of the desired files (figure 2) a window will appear that shows

you the file retrieval process from seamless (background). Once the file is ready for download the above

option will appear (foreground). Remember to name the files and store them in a systematic way that

makes them easily identifiable with their geographical location.

Unzipping and organizing files

The zip files downloaded from seamless can be unzipped using WinZip or a

similar program. Double click the zipped file and WinZip will unzip it. Then extract each

grid file to a predetermined location, maintaining file organization (keep each file in a

folder corresponding to its geographical location). This is important because the unzipped

files will have a random numerical name. To deal with this problem I stored the files in

folders corresponding to their latitudes. I then renamed each file in ArcCatalogue

according the lower bounding latitude and upper bounding longitude. Example: An

SRTM30m grid file from 31-32 degrees north and 110.5-112 degrees west would be

named srtm_31_112. (*Note: file and folder names should not have spaces since ESRI

scripts do not deal with spaces well) Once unzipped, the grid files should always be

manipulated (renamed, moved, or deleted) using ArcCatalogue because ArcCatalogue

will optimize file organization, keeping track of and protecting changes that may affect

the files’ ability and efficiency for use in the Arc products.

Importing files into ArcMap

Open ArcMap. Under the file menu and select add data, or select the add data

icon on the ArcMap display. In the add data dialogue box, navigate to the folder

containing the grid file to import, select this file and click add. You may add multiple

files to your map. It is useful to add a reference (shape file) layer under the grid files.

Most useful are political boundary maps or maps displaying natural boundaries such as

water bodies. These types of files can be downloaded from sites similar to seamless. I

used a US states shape file from the national atlas digital GIS warehouse:

http://nationalatlas.gov/atlasvue.html

Projecting files

The files you downloaded from seamless may have a different projection than you

would like to use when viewing the files in ArcMap. The usefulness of a projection

depends upon several factors including what you will use it to view, the geographical

location of the file, and the projection of any other files to be used in conjunction with the

file you are adding. To change a file’s projection first open ArcToolbox. In ArcToolbox

open data management tools, under that toolbox open the projections toolbox and double

click on project wizard (coverages, grids) (figure 4). (Do not use define projection

wizard because this option only renames the projection, it does not actually change it).

Figure 4: Opening the Project Wizard in ArcToolbox.

In the project wizard you may choose to project a data set to a specified

coordinate system or you may project the data set to match existing data. For the first file

you should select Project my data to a specified coordinate system and click next

(figure 5a). Now you must select the dataset for which you would like to project a

coordinate system. Click the folder icon (figure 5b) and navigate to the SRTM dataset.

ArcToolbox will list the parameters of the SRTM file’s current projection (SRTM30m

grid files come projected in WGS 84). Click next. The next dialogue box (figure 5c) asks

you what projection you want your data to have. In this example, I would like to project

the file in UTM coordinates. To do this scroll down the projections list (they are in

alphabetical order) and select UTM, click next. In the next dialogue box you must choose

several UTM parameters (figure 5d). Make sure the units = meters and the leave the x

and y shift at zero. You must choose a UTM zone for your SRTM data. This is based

upon the geographical location of your dataset. To select the correct zone find your

dataset location on a UTM zone map, one such map can be found at

http://www.dmap.co.uk/utmworld.htm. Now a datum must be applied to the dataset

(figure 5e). In this case I used NAD 27 – CONUS (North American Dataset 1927,

Continental US). Scroll down and select which datum you desire and click next. You

must now specify an output dataset (figure 5f). When specifying an output file make sure

to save the file as a grid type and maintain organization by naming the file

geographically. Click next, if the projection information you desired matches the

summary click finish. The dataset with its new projection is now complete. If the grid file

is large the final step may take several minutes, be patient!

At this point it is useful to make sure that the projection has been defined

correctly. One way to check this is through ArcCatalogue. Open ArcCatalogue and

navigate to the file that you defined the projection for. Right click on this file and select

properties. Under spatial reference the projection reads Transverse_Mercator because I

selected a UTM projection, the projection should match whatever you defined. It should

not read GCS_WGS_1984, which is the projection that the SRTM data comes with.

To change the projections of your remaining SRTM data you may now choose

project the data set to match existing data in the define projections wizard (figure 6a).

This may be done as long as the datasets are within the same UTM zone, if this is true

click next. Now select a grid to project and click next (figure 6b). Now you must select a

grid that you wish to match the projection to, the coordinate system’s parameters should

appear below and match the projection you desire for your dataset (figure 6c), click next.

You must specify an output dataset (figure 6d). Remember to save the file as grid type

and proceed. Now a summary of your projection input is displayed (figure 6e). If

everything is in order, click finish and a new grid file with this projection will be created.

Continue this process to define the new projection for each of your data sets remembering

to define the correct zone for each of the datasets. Periodical checking of projections in

ArcCatalogue is a good idea.

a) b)

c) d)

e) f)

g)

Figure 5 (a-g): Steps to follow in defining a coordinate system interactively.

a) b)

c) d)

e) Figure 6 (a-e): Steps to follow in projecting a grid to match existing data.

Reducing SRTM30m Grid File Size

Each point of the SRTM30m grid files is assigned an elevation value (in meters)

with many insignificant digits (figure 7). The presence of these extraneous decimal places

over the entire grid file increases each grid file by several orders of magnitude. To reduce

the file size the grids can be manipulated in raster calculator to produce copies consisting

of only integer point values. This maintains precision at the 1-meter level, which is still

more precise than the 10-meter vertical confidence interval for the SRTM data.

Figure 7: SRTM_48_115 grid file, projected in ArcMap. Notice the highlighted layer’s range of

elevation values. The SRTM data is assigned too many insignificant digits. This increases the file’s size by

several orders of magnitude. I fixed this disk space problem by using raster calculator and the integer

function to produce grid file copies without any decimal places. The range in confidence of the SRTM data

is 10 vertical meters.

In ArcMap, add an SRTM30m grid file: example srtm_49_115 (figure 7). This 1-

degree of latitude by 1.5-degree longitude file is roughly 80mb in size (with pyramids

drawn). Under the “Tools” menu, open “extensions”. Make sure that the spatial analyst

tool is available (figure 8a). Then open the spatial analyst toolbar: under the “View”

menu, select toolbars and then spatial analyst to open the toolbar (figure 8b).

a) b)

Figure 8: Steps for opening the spatial analyst toolbar: a) turning on spatial analyst, b) opening the toolbar.

Under the spatial analyst toolbar (figure 9), select “Options”, at the bottom of the

menu. In the “General” tab of “Options” Pick a working directory where you want to

store files created by spatial analyst (figure 10a). Select the “Extent” tab and pick

“Intersection of Inputs” since you will be using raster calculator to make a copy of each

grid value (figure 10b), click ok.

Figure 9: The spatial analyst toolbar contains the functions shown in the above menu.

a) b)Figure 10: Choose a working directory for files created in spatial analyst/raster calculator to be stored (a)

and set the analysis extent as intersection of inputs (b).

Now open the “Raster Calculator” under the spatial analyst toolbar (figure 9).

Type newsmallfile = int([oldbigfile]) into the raster calculator (make sure your new file

name corresponds to its geographical location, there is one space on each side of the

equals sign, and that there are parenthesis and brackets as shown) and click evaluate

(figure 11). ArcMap will then build a table and evaluate the solution. Once this process is

done a new grid file should appear. The output file should be exactly the same as the

input file less many megabytes in size. This can be assured by subtracting the input file

from the output file using raster calculator and the following equation: Difference =

[oldbigfile] - [newsmallfile]. The difference file will produce a grid from the values

produced by subtracting each of corresponding points of the two grids; all points should

be zero if everything has been entered correctly. The integer truncated grid files are about

20mb, one-fourth the size of the original SRTM30m files. Because there is no difference

between the grid files and since the vertical error in the SRTM 30m data is greater than

the 1-meter integer function truncation used to reduce the grid size, we can be confident

that reducing the grid size is a valuable task for the interest of data management.

Figure 11: “Raster Calculator” interface with an example for how to reduce grid file size for the SRTM

30m data.

Merging SRTM30m grid files

When working with larger regions covered by the SRTM data you will want to

merge grid files. This can be done in raster calculator. Before merging it will be useful to

reduce the file size as described above because it will take less time for raster calculator

to evaluate the merge equation and you will be able to merge more files. Merging files is

limited by your computer system’s memory (if you have 2Gb of memory then you will be

able to merge only as many SRTM grids that add up to less than 2Gb).

With two files added to ArcMap open “Options” under the spatial analyst toolbar:

under the “General” tab choose a location for the merged output file (figure 10a) and

under extent (figure 10b) choose union of inputs since you are merging two files which

do not overlap. Now open “Raster Calculator” under the spatial analyst toolbar (figures 9

and 11). Type in the following expression, replacing the hypothetical file names with

geographically significant ones and evaluate: fileAC = merge([fileAB] , [fileBC]). The

merged files will be continuous because the SRTM data were downloaded using the

selection by coordinates option.

Hill shade and color ramp visualization

The visual appeal of the digital elevation models can be greatly improved by

applying a stretched color ramp to highlight regions of elevation differences and using

the “Hillshade” function to visualize changes in relief. The “Hillshade” function can be

found under “Surface Analysis” in the spatial analyst toolbar (figure 9). Enter the desired

parameters for the azimuth and altitude and select an output raster file, then click ok

(figure 12).

Figure 12: Parameters for creating a Hillshade grid file.

Hillshade files are excellent for viewing drainages and patterns of relief (slopes)

over a region (figure 13a), but they do not provide information about the magnitude of

elevation differences over those regions. DEMs provide this information. To make a

really nice figure where both elevations and the patterns of relief can be seen one must

overlay these two maps. Hillshading looks the best if it is done over a DEM in UTM

projection. Hillshading does not work well with units of decimal degrees because as you

move away from the equator the degrees of latitude and longitude are not equivalent and

because the x/y coordinates are much longer than the z units. This results in small

differences in relief appearing as vast hill shadowing effects. In order to see the DEM

through a Hillshade map (or vise versa) one must make the top layer (the Hillshade)

transparent. This can be done by right clicking on the top layer, selecting properties and

under properties selecting the display tab. Now make the topmost layer transparent,

somewhere between 40 and 70% transparent should work well. Now you would be able

to see the information provided in both layers at once if they did not both have the same

color scale. To fix this problem change the color ramp of the DEM layer. This option

is also found under the layer’s properties, this time in within the symbology tab. Make

sure that the values are shown as stretched and then pick a color ramp which displays

elevation differences well. It is best to tinker with this until you find a color scheme that

brings out the regions of elevation best for you (figure12b). Now both the regions of

elevation as well as the patterns of relief are visible (figure 12c).

Figure 12a: An example of a Hillshade map created from SRTM30m data. Notice that you can see the

patterns of relief very well. Drainages can be distinguished and other topographic features are evident.

Figure 12b: A color ramped DEM clearly shows regions of high elevation and low lands based upon color

schemes. White are mountain tops and pale greens represent the low lands.

Figure 12c: A Hillshade map made 40% transparent with a color ramped DEM underneath allows hill

slopes, drainages, and other features of relief to be seen along with the different regions of elevation.

Many other manipulations are available in the Arc products that allow for

improved visualization and can be used to study the geomorphology, hydrology, and

tectonic features of a region using digital elevation models and other GIS data.

Comparing SRTM and NED data sets in terms of slope tendencies

An interesting comparison of the SRTM 30m DEM data to the NED 30m data is

to analyze them in terms of their differences in slope. This comparison reveals

differences between the two data sets answering questions such as: how do SRTM slopes

in regions of high relief compare to NED slopes of high relief? This analysis is important

because it is important to know the tendencies of each data set over different regions of

relief. Do SRTM data sets provide reliable data over regions of high relief? What about

the NED data? As long as this grid point by grid point analysis is done over the exact

same geographical area this analysis is possible. The following steps outline this process.

Refer to the accompanying SRTM and NED comparison report for analysis. File

organization is important, especially with the number of files created for each slope

comparison

Step 1: Project SRTM and NED into a common projection. UTM works well

for this analysis. Follow the projection steps from earlier in this report.

Step 2: In ArcMap, calculate slope files for both of these UTM projected files.

To do this first open the UTM files, then select surface analysis from the spatial analyst

toolbar, and choose the slope option. The output measurement should be in degrees, z

factor = 1, cell size should equal the cell size of the input file, and remember to specify an

output file.

Step 3: The slope files must be integerized and multiplied by 1000 in Raster

calculator (Figure 13) (will divide by 1000 in Mat lab). Remember to select the extent of

calculation and the working directory options under the spatial analyst toolbar.

Fig 13: Sample integer calculation.

Step 4: For point-by-point analysis in Mat Lab the slope files need to have

exactly the same geographical extent, they must have the same number of rows, columns,

and the same cell size. It is likely that the SRTM and NED files you are working with do

not meet these requirements because the SRTM has many areas of data voids also; the

files may differ in their number of rows or columns (check under the file’s general

properties). Voids, extra rows, and columns can be removed by clipping the files to one

another using the extent option and raster calculator under the spatial analyst toolbar of

ArcMap. First load the SRTM and NED integerized slope files. Next, it is important to

set the spatial analyst extent to intersection of inputs, remember to specify your work

directory as well (file management). Now use raster calculator to created clipped files for

both the SRTM and NED slope files using the following general equations:

Equation 1 will replicate the location of SRTM data voids on the NED file so the two

can be compared point-by-point in Mat Lab.

NEDclippedslope = Con( SRTMslope >= 0, NEDslope)

Equation 2 will remove any extra columns/rows on the SRTM data making the NED and

SRTM clipped slope files equal in extent.

SRTMclippedslope = Con(NEDslope >= 0, SRTMslope)

Step 5: The clipped grid files must now be converted to point files. Under spatial

analyst toolbar select the convert: raster to feature tool. In the dialogue box specify one

of the clipped slope files as the input raster, select the output geometry type as point, and

specify a location for the slope point file (Figure 14). Repeat the process for the other

clipped file (NED or SRTM). Do not let the point files load (un click them from visible

extent) because they are very large and will use up CPU memory.

Figure 14: Converting raster to features.

Step 6: Point files must now be exported to .txt files for use in Mat Lab. Right

click the point files and open the attribute table. Under options, select export. Export all

records. Make sure to select the output file type as .txt. This process will take a very long

time. Typically these text files are more than 100mb. So be prepared to wait.

Step 7: Cleaning the .txt slope files. The exported text file contains extraneous

information in the first line as well as commas that Mat Lab cannot deal with, so they

must be cleaned. This can be done using the text-editing environment of VI. Open the

point file in VI through SSH Secure Shell Client by typing vi(space)filename. To do this

you will need to navigate to between folders in secure shell, to do this use the following

commands in bold:

1. Print working directory (show which folder you are in) = pwd

2. To move up one folder = cd(space)..

3. To list the files within the working directory = ls

4. To open a folder (remember not to follow the folder name with the file extension)

= cd(space)foldername

Then, once the file is loaded in vi, perform the following operations (enter what is typed

in bold)

1. Delete the first line that contains erroneous entries = dd

2. Replace commas with spaces = :1,$s/,/(space)/g

3. Finish = wq

Complete this process for both files. This process can also be accomplished in Microsoft

Word using search and replace, but this would be much more time consuming because it

takes many repeated commands due to the MS program’s memory limits.

Step 8: Use Mat Lab to compare the NED and SRTM slope data, it is useful to

create scripts for easy repetition and analysis alteration.