An Introduction to Python for GIS Specialists and Data Managers

11

An Introduction to Pythonfor GIS Specialists and Data

Managers

An Introduction to Pythonfor GIS Specialists and Data

Managers

Jacob MorganBrent Frakes

National Park ServiceFort Collins, CO

April, 2008

Jacob MorganBrent Frakes

National Park ServiceFort Collins, CO

April, 2008

22

Session GoalsSession Goals

• Overview of Python

• See power of the language

• Understand basic syntax

• References to other sources

• Too little time to really cover everything

• Overview of Python

• See power of the language

• Understand basic syntax

• References to other sources

• Too little time to really cover everything

33

SessionsSessions

• One: Basic Python

• Two: Using the ESRI Geoprocessor

• Three: Formatting Data and Python Scripting

• Each one hour

• Examples and practice

• One: Basic Python

• Two: Using the ESRI Geoprocessor

• Three: Formatting Data and Python Scripting

• Each one hour

• Examples and practice

44

Session 1: OverviewSession 1: Overview

• Scripts and the Debugger

• Commenting

• Data Types

• Operators

• Decisions and Loops

• File Handling

• Modules, Functions and Classes

• Scripts and the Debugger

• Commenting

• Data Types

• Operators

• Decisions and Loops

• File Handling

• Modules, Functions and Classes

55

Scripts and the DebuggerScripts and the Debugger• General Setup and Notes

– All code is text-based (*.py extension)– Once compiled, a new file will have a *.pyc extension.– Python is VERY picky about indentation– Python is case sensitive

• IDLE – Free– GUI interface for development– Compiler included with ArcGIS– Provides debugging but doesn't allow for breakpoints

• Pythonwin – Free– GUI interface for development– Good debugger with breakpoints– http://sourceforge.net/projects/pywin32/

• General Setup and Notes– All code is text-based (*.py extension)– Once compiled, a new file will have a *.pyc extension.– Python is VERY picky about indentation– Python is case sensitive

• IDLE – Free– GUI interface for development– Compiler included with ArcGIS– Provides debugging but doesn't allow for breakpoints

• Pythonwin – Free– GUI interface for development– Good debugger with breakpoints– http://sourceforge.net/projects/pywin32/

66

Scripts and the DebuggerScripts and the Debugger

• Exercise– type at the prompt >>'hello world'– type at the prompt >>print 'hello world'– Run a script with the following code:

• print 'hello world'

• Exercise– type at the prompt >>'hello world'– type at the prompt >>print 'hello world'– Run a script with the following code:

• print 'hello world'

77

CommentingCommenting

• Why?– Make your code understandable to you and others – How to use code– Why code was written– Debugging (comment out parts of script)

• Ways to Comment– Single Line: use the pound (#) sign before the comment text.

# single line commented text

– Multiple Lines: use triple quotes before and after a text block.“ “ “ These two lines of textAre being commented out “ “ “

• Why?– Make your code understandable to you and others – How to use code– Why code was written– Debugging (comment out parts of script)

• Ways to Comment– Single Line: use the pound (#) sign before the comment text.

# single line commented text

– Multiple Lines: use triple quotes before and after a text block.“ “ “ These two lines of textAre being commented out “ “ “

88

ExerciseExercise

• Add both types of comments to your script• Add both types of comments to your script

99

Data TypesData Types

• Strings

• Numeric

• Lists

• Tuples

• Strings

• Numeric

• Lists

• Tuples

1010

StringsStrings

• An ordered collection of characters used to represent text-based information

• Common String Literals and Operations– strNull = ' ' #empty string– str1 = "metadata" #double-quotes– str2 = '.xml' #single quotes– str3 = str1 + str2 #concatenate– strNull*100 #repeat– str2.upper() #string method call– str1[0:2] #indexing– str1 = str1+str2 #strings can't be modified

but can be reassigned

• An ordered collection of characters used to represent text-based information

• Common String Literals and Operations– strNull = ' ' #empty string– str1 = "metadata" #double-quotes– str2 = '.xml' #single quotes– str3 = str1 + str2 #concatenate– strNull*100 #repeat– str2.upper() #string method call– str1[0:2] #indexing– str1 = str1+str2 #strings can't be modified

but can be reassigned

1111

StringsStrings

• Special Strings– \n #newline– \t #horizontal tab– \\ #backslash

• Special Strings– \n #newline– \t #horizontal tab– \\ #backslash

1212

StringsStrings

– Type the following>>>a='Hello'>>>b = 'World'>>> a+b>>> (a+b)*10>>> a[0:1]>>> a[-4:-1]>>>a.upper()>>>a.lower()– Add two strings– Try using the two other string methods

– Type the following>>>a='Hello'>>>b = 'World'>>> a+b>>> (a+b)*10>>> a[0:1]>>> a[-4:-1]>>>a.upper()>>>a.lower()– Add two strings– Try using the two other string methods

1313

Numeric TypesNumeric Types

• Four types:– Integers

• Numeric type providing a container for whole numbers. Plain integers are precise up to 32 bits (dependant on platform and compiler).

– Floating Point Numbers • Numbers those with a decimal component. These are

called floating point since the decimal point can be positioned anywhere in the digit string by a signed integer exponent-

– Long Integers– Complex Numbers

• Four types:– Integers

• Numeric type providing a container for whole numbers. Plain integers are precise up to 32 bits (dependant on platform and compiler).

– Floating Point Numbers • Numbers those with a decimal component. These are

called floating point since the decimal point can be positioned anywhere in the digit string by a signed integer exponent-

– Long Integers– Complex Numbers

1414

Numeric OperatorsNumeric Operators

Operation Result

x + y sum of x and y

x - y difference of x and y

x * y product of x and y

x / y quotient of x and y

x // y (floored) quotient of x and y

x % y remainder of x / y

-x x negated

+x x unchanged

abs(x) absolute value or magnitude of x

int(x) x converted to integer

long(x) x converted to long integer

float(x) x converted to floating point

complex(re,im) a complex number with real part re, imaginary part im. im defaults to zero.

c.conjugate() conjugate of the complex number c

divmod(x, y) the pair (x // y, x % y)

pow(x, y) x to the power y

x ** y x to the power y

1515

Numeric ConsiderationsNumeric Considerations

• Integer calculations yield integers (including division!)

• Floating point calculations yield floating points

• The constructors int(), long(), float(), and complex() can be used to produce numbers of a specific type. However, converting numbers between types may result in rounding or truncating digits.

• Integer calculations yield integers (including division!)

• Floating point calculations yield floating points

• The constructors int(), long(), float(), and complex() can be used to produce numbers of a specific type. However, converting numbers between types may result in rounding or truncating digits.

1616

Numeric Types - ExercisesNumeric Types - Exercises

>> 1 # 1 >>a= 2 #set value to 2>>b= 3>>b+a>>b/a #still an integer>>value = "10" #not an integer>>c = 1.27 #floating point>>c+b>>b/c>>d = int(c) #rounding

>> 1 # 1 >>a= 2 #set value to 2>>b= 3>>b+a>>b/a #still an integer>>value = "10" #not an integer>>c = 1.27 #floating point>>c+b>>b/c>>d = int(c) #rounding

1717

ListsLists

• An ordered collections of arbitrary objects

• Fully indexed

• Variable in length, heterogenous, and arbitrarily nestable

• Can be altered

• Always in brackets

• Aka Arrays!

• An ordered collections of arbitrary objects

• Fully indexed

• Variable in length, heterogenous, and arbitrarily nestable

• Can be altered

• Always in brackets

• Aka Arrays!

1818

List - ExamplesList - Examples

d = [] # empty list

d = [1,2,3] #list of three integers

d = [1,2,"3"] #two integers and a string

d[0] #first element

d = [[1,2, 3],[3,4, 5]] #3 by 3 array

len(d) # 2

len(d[0]) #3

d = [] # empty list

d = [1,2,3] #list of three integers

d = [1,2,"3"] #two integers and a string

d[0] #first element

d = [[1,2, 3],[3,4, 5]] #3 by 3 array

len(d) # 2

len(d[0]) #3

1919

Lists - ExercisesLists - Exercises

>>L1 = [1,2,3,4,5,6]>>len(L1)>> L1 [0]>> L1.pop() #remove last element from stack>> L1 #6 is removed>> L1 [0:2] #index 0-2>> L1 [-1] #index from end of list>>L2 = [5,6,7]>> L1.append(L2) #insert a new list>> L1 >> L1.extend(L2) #extend the current list>> L1

>>L1 = [1,2,3,4,5,6]>>len(L1)>> L1 [0]>> L1.pop() #remove last element from stack>> L1 #6 is removed>> L1 [0:2] #index 0-2>> L1 [-1] #index from end of list>>L2 = [5,6,7]>> L1.append(L2) #insert a new list>> L1 >> L1.extend(L2) #extend the current list>> L1

2020

OperatorsOperators

• Operators compute a value when presented with a string or number

• Common Operators include:– x=y #assign– x==y #true or false– x < y #True or false– x<=y #True or false– x>y #True or false– x*y, x/y, x+y, x-y #Basic math– x[1] #index– x[1:10] #slice– x or y #y evaluated only if x is false– x and y #y is evaluated only if x is true

• Operators compute a value when presented with a string or number

• Common Operators include:– x=y #assign– x==y #true or false– x < y #True or false– x<=y #True or false– x>y #True or false– x*y, x/y, x+y, x-y #Basic math– x[1] #index– x[1:10] #slice– x or y #y evaluated only if x is false– x and y #y is evaluated only if x is true

2121

Operators - ExercisesOperators - Exercises

• >>>x = 5

• >>>y = 6

• >>> x < y

• >>> x > y

• >>> x < y or y > 7

• >>> x < y and y > 7

• >>>x = 5

• >>>y = 6

• >>> x < y

• >>> x > y

• >>> x < y or y > 7

• >>> x < y and y > 7

2222

Controlling FlowControlling Flow

• Decisions/Conditionals are statements that determine if a series of conditions is true.

• Most common– if– for– while

• Decisions/Conditionals are statements that determine if a series of conditions is true.

• Most common– if– for– while

2323

Decisions/ConditionalsDecisions/Conditionals

if condition:action(s)

elif condition:action(s)

else condition:action(s)

if condition:action(s)

elif condition:action(s)

else condition:action(s)

2424

“if” Condition Example“if” Condition Example

x = 15

if x > 5 and x < 10 :

print “The value is between 5 and 10”

else:

print “The value is not between 5 and 10”

x = 15

if x > 5 and x < 10 :

print “The value is between 5 and 10”

else:

print “The value is not between 5 and 10”

2525

“if” Condition - Exercise“if” Condition - Exercise

>>>x = 7

>>>if x > 5 and x < 10 :

… print “The value is between 5 and 10”

>>>x = 7

>>>if x > 5 and x < 10 :

… print “The value is between 5 and 10”

2626

“for” loop“for” loop

• Supports repeated execution of a statement, or block of statements, controlled by an iterable express

for target in iterable:statement(s)

else: #optional on normal termination

statement(s)

• Supports repeated execution of a statement, or block of statements, controlled by an iterable express

for target in iterable:statement(s)

else: #optional on normal termination

statement(s)

2727

“for” Examples“for” Examples

for x in range (1,30,2):print x

for letter in “hello world”:print letter

infile = open (“dataset.txt”,”r”)for line in infile:

print line

for x in range (1,30,2):print x

for letter in “hello world”:print letter

infile = open (“dataset.txt”,”r”)for line in infile:

print line

2828

For ExercisesFor Exercises

>>>for x in range(1,20):… if x%2 == 0:… print x, “ is even”

>>>import glob>>>files = glob.glob(“*.*”)>>>for file in files:… print file

>>>for x in range(1,20):… if x%2 == 0:… print x, “ is even”

>>>import glob>>>files = glob.glob(“*.*”)>>>for file in files:… print file

2929

“while” loop“while” loop

• allows looping to occur only while a specified condition is true. WHILE evaluates the condition each time the loop completes and terminates the loop when the condition evaluates to false

while expression:statement(s)

else: #optional for normal termination

statement(s)

• allows looping to occur only while a specified condition is true. WHILE evaluates the condition each time the loop completes and terminates the loop when the condition evaluates to false

while expression:statement(s)

else: #optional for normal termination

statement(s)

3030

While ExamplesWhile Examples

x = 100

while x>0:

x = x/2

print x

x = 100

while x>0:

x = x/2

print x

3131

While ExerciseWhile Exercise

>>>x = 20

>>>y = 0

>>>while x>=y:

… print y

… y += 2

>>>x = 20

>>>y = 0

>>>while x>=y:

… print y

… y += 2

3232

File HandlingFile Handling• Ways to open, read, write, and close files

File2read = open (filename, mode = 'r')

filename - string containing filename and extension (e.g., 'metadata.xml')file modes - indicates how to read or write the file

'r' - file must already exist and is read only'w' - creates a new file for writing. Overwrites any existing filename'a' - Write only mode. If file already exists, information is appended.'r+' - File must already exist and is opened for both reading and writing.'w+' - New file is created. Open for reading and writing.'a+' - File open for reading and writing. If file already exists, data is appended.

• Ways to open, read, write, and close files

File2read = open (filename, mode = 'r')

filename - string containing filename and extension (e.g., 'metadata.xml')file modes - indicates how to read or write the file

'r' - file must already exist and is read only'w' - creates a new file for writing. Overwrites any existing filename'a' - Write only mode. If file already exists, information is appended.'r+' - File must already exist and is opened for both reading and writing.'w+' - New file is created. Open for reading and writing.'a+' - File open for reading and writing. If file already exists, data is appended.

3333

Useful File MethodsUseful File Methods

• file.read() - reads up to the end of file and returns it as a string.

• file.readline() - reads one line from file (up to the '\n') and returns a string

• file.seek(pos, how = 0) - Move to a position in the file. 'How' is the reference point for moving (0 = start of file, 1 = current position; 2 = end of file).

• file.write() - Writes a string to a file

• file.writelines(line) - writes a line to a file

• file.close() - # must be closed for it to be unlocked by Windows

• file.read() - reads up to the end of file and returns it as a string.

• file.readline() - reads one line from file (up to the '\n') and returns a string

• file.seek(pos, how = 0) - Move to a position in the file. 'How' is the reference point for moving (0 = start of file, 1 = current position; 2 = end of file).

• file.write() - Writes a string to a file

• file.writelines(line) - writes a line to a file

• file.close() - # must be closed for it to be unlocked by Windows

3434

File ExamplesFile Examples• infile = open(“d.txt”, 'r')• data = infile.read() #the string data is assigned the file

contents• outfile = open(“e.txt”,'w') # assign outfile to the file object for

writing• outfile.write(data) # write the contents to the file

• infile.close()• outfile.close()

• infile = open(“d.txt”, 'r')• data = infile.read() #the string data is assigned the file

contents• outfile = open(“e.txt”,'w') # assign outfile to the file object for

writing• outfile.write(data) # write the contents to the file

• infile.close()• outfile.close()

3535

File ExercisesFile Exercises

>>>filename = “C:\\temp\\test.txt”>>>outfile = open (filename ,’w’)>>>outfile.write(“hello world”)>>>outfile.close()>>>infile = open(filename ,’r’)>>>data = infile.read()>>>print filename, “-”, data>>>infile.close()

>>>filename = “C:\\temp\\test.txt”>>>outfile = open (filename ,’w’)>>>outfile.write(“hello world”)>>>outfile.close()>>>infile = open(filename ,’r’)>>>data = infile.read()>>>print filename, “-”, data>>>infile.close()

3636

Modules, Functions, and ClassesModules, Functions, and Classes

• Module– Single Python file– Provides definitions of functions, variables or

classes– Corresponds to a specific thema (e.g., read,

edit, and write zipfiles)– Composed of functions and/or classes.

Functions can be stand-alone or are part of a class. Whether a function or class, both accomplish tasks.

• Module– Single Python file– Provides definitions of functions, variables or

classes– Corresponds to a specific thema (e.g., read,

edit, and write zipfiles)– Composed of functions and/or classes.

Functions can be stand-alone or are part of a class. Whether a function or class, both accomplish tasks.

3737

Standard ModulesStandard Modules• Supported by

Python - integral to many applications

• Index can be found at: http://docs.python.org/lib/modindex.html

• Importing is simple:

• >>import module

• (e.g., import os)

• Supported by Python - integral to many applications

• Index can be found at: http://docs.python.org/lib/modindex.html

• Importing is simple:

• >>import module

• (e.g., import os)

3838

Accessing FunctionsAccessing Functions

import module

module.function(arguments…)

Examples

import osos.getcwd() #get current directory

import globglob.glob(“*.py”) #get a list of all files with the .py extension

import module

module.function(arguments…)

Examples

import osos.getcwd() #get current directory

import globglob.glob(“*.py”) #get a list of all files with the .py extension

3939

Instantiating ClassesInstantiating Classes

import module

instance = module.class(arguments)

instance.method()

ExamplesImport gzip #import gzip module

Zipfile = gzip.GzipFile(“datafile.zip”) #instantiate GzipFile class

files = Zipfile.read() #Read zipfile and assign it to the files string

import module

instance = module.class(arguments)

instance.method()

ExamplesImport gzip #import gzip module

Zipfile = gzip.GzipFile(“datafile.zip”) #instantiate GzipFile class

files = Zipfile.read() #Read zipfile and assign it to the files string

4040

Getting Documentation About Modules

Getting Documentation About Modules

dir() – List of available functions and methodsmodule.__doc__ - module documentationmodule.function.__doc__ - function/method

documentation

Exampleimport osdir(os)os.rename.__doc__

dir() – List of available functions and methodsmodule.__doc__ - module documentationmodule.function.__doc__ - function/method

documentation

Exampleimport osdir(os)os.rename.__doc__

4141

Third Party ModulesThird Party Modules

• Many others have developed modules and posted them to the Python Package Index (http://pypi.python.org/pypi/)

• Many others have developed modules and posted them to the Python Package Index (http://pypi.python.org/pypi/)

4242

ExercisesExercises

>>>import os

>>>dir(os)

>>>os.rename.__doc__

>>>import shutil

>>>shutil.__doc__

>>>dir(shutil)

>>>import os

>>>dir(os)

>>>os.rename.__doc__

>>>import shutil

>>>shutil.__doc__

>>>dir(shutil)