Automating Processes Using Python & Task Scheduler

Automating Processes Using ModelBuilder, Python & Windows Task SchedulerKelly Klose, GISP - GIS Manager - City of Mansfield

23rd Annual SCAUG Conference April 11, 2013

Using Python & Task Scheduler

Using Python to run geoprocessing tasks outside of ArcGIS can speed workflows. Tasks requiring simple user input such as query variables for data extraction may be run as standalone applications, without the need to load data into ArcGIS.Windows built-in Task Scheduler may be used to execute Python scripts for processes requiring simple user input, as well as those that can be executed without user input.Data can be compiled and placed on local network drives or compressed and sent to recipients via FTP or email.Checks of data may be performed, allowing users to be notified when others have made updates to data.Various tasks may be automated through the use of Python and Task Scheduler, either together or separately.Using Python & Task Scheduler

With Scheduled Tasks, users do not have to remember to perform tasks, since the tasks will be executedat the required time, each time.

The potential for user error is reduced, as the scheduled task will perform the same actions each time it runs,without missing a particular step in the task process, which could cause an error.

Tasks that require heavy user input can be the focus of staff during work hours, and tasks that maybe automated can be executed after hours, when PC and network use is low, i.e. evening or early morning.Results of script processes are ready when users arrive for work.

Mansfield is in three counties, so there are three appraisal districts from which to gather property ownership and tax data.Each has a different schedule for data availability, so run Scheduled Task several days each month,to check for files on FTP sites. If found on FTP, and not on local server, download files.When all files are on local server, run script to compile all data in database for use in GIS.

Request data from appraisal districts and other agencies automatically, using a script.

Compile data and place on FTP, then send email to recipient with link to data, all via script.

Scripts that send email or check FTP will have user credentials included, or can be writtento prompt user for credentials, each time, but then are not truly automated.

Tasks can be set to run as soon as possible after missing a scheduled time, i.e. PC turned off.Using Python & Task Scheduler

Examples:Transfer data between agencies Federal, State, County, COG, Appraisal Districts, other cities, etc.Extract data and place on FTP server or send via email.Download data from agency websites or FTP servers.Run reports on dynamic data - daily, weekly, monthly.Perform radius analysis, summary statistics, incident reports.Query data for unique value counts.Open URLs containing dynamic textual or tabular data.Populate fields in feature class data tables.Fill field values based on calculations or standard values.Transfer attribute values between feature classes.Check network locations for new or modified data.Read directory contents for new or modified files.Using ModelBuilder to Create a Python Script - Demonstration

Using Scheduled Tasks WIN XP

Open the Scheduled Tasks folder by clicking Start > Accessories > System Tools > Scheduled TasksUsing Scheduled Tasks WIN XP

Open the Scheduled Tasks folder by clicking Start > Control Panel > Scheduled TasksUsing Scheduled Tasks WIN XP

Using Scheduled Tasks WIN XP

The Scheduled Tasks FolderTo add a task, click on Add Scheduled Task.Using Scheduled Tasks WIN XP

Using Scheduled Tasks WIN XP

Using Scheduled Tasks WIN XP

Using Scheduled Tasks WIN XP

Using Scheduled Tasks WIN XP

Using Scheduled Tasks WIN XP

Using Scheduled Tasks WIN XP

Using Scheduled Tasks WIN XP

Using Scheduled Tasks WIN XP

Drag-N-Drop script into Scheduled TasksUsing Scheduled Tasks WIN XP

Open a Web Page

Enter path to browserexecutable and add URL as parameter.Tasks are stored as .job files.Using Scheduled Tasks WIN XP

To open Scheduled Tasks, click Start, click All Programs, point to Accessories, point to System Tools, and then click Scheduled Tasks.

To schedule a new task:

1. Double-click Add Scheduled Task to start the Scheduled Task Wizard, and then click Next in the first dialog box.

2. The next dialog box displays a list of programs that are installed on your computer, either as part of the Windows XP operating system, or as a result of software installation.Use one of the following procedures:If the program that you want to run is listed, click the program, and then click Next.If you want to run a program, script, or document that is not listed, click Browse, click the folder and file that you want to schedule, and then click Open.

3. Type a name for the task, and then choose one of the following options:Daily, Weekly, Monthly, One time only, When my computer starts (before a user logs on), When I log on (only after the current user logs on)

4. Click Next, specify the information about the day and time to run the task, and then click Next.Note that the information about the day and time to run the task vary depending on the selection that you made in the previous wizard dialog box. For example, if you chose Weekly, you must indicate the day of the week, the time, and if the task should run every week, every 2 weeks, every 3 weeks, and so on.

5. Type the name and password of the user who is associated with this task. Make sure that you choose a user with sufficient permissions to run the program. By default, the wizard selects the name of the user who is currently logged on.

6. Click Next, and then click Finish after you verify the choices that you have made.

Using Advanced Options in Scheduled Tasks

If you want to change the configuration of the task, click Open in the advanced properties for the task before you click Finish.After you click Finish, the Properties dialog box opens for the task.On the Schedule tab, you can change any of the scheduling options that you chose in the wizard, and you can also change the task configuration so that the task does not run too long, does not run if the computer is running on batteries (for laptops), and to specify whether or not the computer should be idle for the task to run.

NOTE: You can open the Properties dialog box for the task at any time if you open Scheduled Tasks, right-click the task, and then click Properties.

You cannot schedule a task so that it repeats in an interval less than one day; however, you can do this in the Properties dialog box:

1. Click the Schedule tab, and then click Advanced.

2. Click to select the Repeat task check box, and then specify the number of minutes or hours in which you want the taskto be repeated.Using Task Scheduler WIN 7

Create a Basic TaskUsing Task Scheduler WIN 7

Using Task Scheduler WIN 7

Using Task Scheduler WIN 7

Using Task Scheduler WIN 7

Using Task Scheduler WIN 7

Using Task Scheduler WIN 7

Using Task Scheduler WIN 7

Create a TaskUsing Task Scheduler WIN 7

Using Task Scheduler WIN 7

Using Task Scheduler WIN 7

Using Task Scheduler WIN 7

Using Task Scheduler WIN 7

Using Task Scheduler WIN 7

Using Task Scheduler WIN 7

Using Task Scheduler WIN 7

Using Task Scheduler WIN 7

Using Task Scheduler WIN 7

Using Task Scheduler WIN 7

Using Task Scheduler WIN 7

Import a TaskUsing Task Scheduler WIN 7

Tasks may be Imported and Exported. They are stored as XML files.Using Task Scheduler WIN 7

Review Task InformationUsing Task Scheduler WIN 7

Review Task InformationUsing Task Scheduler WIN 7

Review Task TriggersUsing Task Scheduler WIN 7

Review Task ActionsUsing Task Scheduler WIN 7

Review Task ConditionsUsing Task Scheduler WIN 7

Review Task SettingsUsing Task Scheduler WIN 7

Review Task HistoryTasks may be organized into separate folders.Using Task Scheduler WIN 7

Review Task HistoryTasks may be organized into separate folders.

Using Task Scheduler WIN 7

In addition to running tasks on scheduled times or specified intervals, Task Scheduler 2.0 also supports calendar and event-based triggers, such as starting a task when a particular event is logged to the event log, or when a combination of events has occurred. Also, several tasks that are triggered by the same event can be configured to run either simultaneously or in a pre-determined chained sequence of a series of actions, instead of having to create multiple scheduled tasks. Tasks can also be configured to run based on system status such as being idle for a pre-configured amount of time, on startup, logoff, or only during or for a specified time. XPath expressions can be used to filter events from the Windows Event Log. Tasks can also be delayed for a specified time after the triggering event has occurred, or repeat until some other event occurs. Actions that need to be done if a task fails can also be configured.

The actions that can be taken in response to triggers, both event-based as well as time-based, not only include launching applications but also take a number of custom actions. Task Scheduler includes a number of actions built-in, spanning a number of applications; including send an e-mail, show a message box, or fire a COM handler when it is triggered. Custom actions can also be specified using the Task Scheduler API. Task Scheduler keeps a history log of all execution details of all the tasks.

Send an EmailThat task is usable in limited circumstances where a local Windows based SMTP server is available and security credentials are already established.

I use the Task Scheduler to call a python script that sends email messages.

Automated Python Scripts

Python script to check a feature class for features meeting specific criteria.Automated Python Scripts

Python script to check a folder for new data files.

Contents of comm_subdiv_20121108.csv file:Hot_Comm,UNQ_IDX:\Construction-Plans\Commercial\Bergers Reserve-Blk 1 Lot 3.pdf,33380Automated Python Scripts

Python script message box that appears when the required tax data fileshave been downloaded from FTP site and are found on the local server.Automated Python Scripts

Python script that runs only on specific days. Calls another script.The Scheduled Task runs on the first day of each quarter of the year.Automated Python Scripts

Python script to open URL. Used to check plotter paper status and usage meter.Automated Python Scripts

Python script to open URL. Used to check plotter paper status and usage meter.

Automated Python Scripts

Change file extension of Python script to .pyw to disable the console window.On Windows systems, there is no notion of an executable mode. The Python installer automatically associates .py files with python.exe so that a double-click on a Python file will run it as a script.

The extension can also be .pyw, in that case, the console window that normally appears is suppressed.

Source: http://docs.python.org/2/tutorial/interpreter.htmlResources:

Task Scheduler:

http://support.microsoft.com/kb/308569

http://support.microsoft.com/kb/308671http://msdn.microsoft.com/en-us/library/windows/desktop/aa383614(v=vs.85).aspx

http://windows.microsoft.com/en-US/windows7/schedule-a-task

Python:

ArcGIS Desktop Help Search tab keyword python

http://resources.arcgis.com/en/communities/python/

http:// www.python.org

Various books and online forums provide training and example scripts

Programming coursework from TCC, ESRI, or other facilities

Automating Processes Using ModelBuilder, Python & Windows Task SchedulerKelly Klose, GISP - GIS Manager - City of Mansfieldkelly.klose@mansfield-tx.gov

23rd Annual SCAUG Conference April 11, 2013