Python and FME REST

Rob GastonWeb Developer; Farallon Geographics, Inc.

April 9, 2013

San Francisco EAS

EAS is an open source system for maintaining address data

At the City and County of San Francisco, FME is being used to ETL parcel and street data used by EAS

EAS uses FME Server REST API to check the status of ETL jobs before staging new data

EAS, Python and FME

EAS makes calls to FME Server REST API via automation scripts written in Python

Python is used in EAS both on the backend of the web application and for automation

To support automation of calls to FME Server REST services, we built a Python class (FmeServerHelper)

Checking for an FME Workspace with Python

Without FmeServerHelper With FmeServerHelper

fme_server_helper.py

FmeServerHelper API

Get the code: https://code.google.com/p/eas/source/browse/trunk/automation/src/fme_server_helper.py

doesJobExist - Checks to see if FME workspace exists on this host

isJobRunning - Checks to see if FME workspace is currently running on this host

isJobQueued - Checks to see if FME workspace is currently queued to run on this host

getLatestCompletedJob - Gets the data from FME Server for the last job from a given workspace

getJobId - Gets the string id from a FME Server job

getJobStatus - Gets the string job status from a FME Server job

FmeServerHelper API Documentation

Get the code: https://code.google.com/p/eas/source/browse/trunk/automation/src/fme_server_helper.py

doesJobExist(jobDict) : BooleanChecks to see if FME workspace exists on this hostPARAMETERS:jobDict: Dict - a dictionary containing the workspace and repository namesRETURNS: Boolean - the existence state of this workspace on this host

isJobRunning(jobDict) : BooleanChecks to see if FME workspace is currently running on this hostPARAMETERS:jobDict: Dict - a dictionary containing the workspace and repository namesRETURNS: Boolean - the running state of this workspace on this host

isJobQueued(jobDict) : BooleanChecks to see if FME workspace is currently queued to run on this hostPARAMETERS:jobDict: Dict - a dictionary containing the workspace and repository namesRETURNS: Boolean - the queued state of this workspace on this host

getLatestCompletedJob(jobDict) : DictGets the data from FME Server for the last job from a given workspacePARAMETERS:jobDict: Dict - a dictionary containing the workspace and repository namesRETURNS: Dict - a dictionary containing job data from FME Server

getJobId(job) : StringGets the string id from a FME Server jobPARAMETERS:job: Dict - a dictionary containing job data from FME ServerRETURNS: String - the job id

getJobStatus(job) : StringGets the string job status from a FME Server jobPARAMETERS:job: Dict - a dictionary containing job data from FME serverRETURNS: String - the job status

What’s Next…?

Following the principle of DTSTTCPW (do the simplest thing that could possibly work), FmeServerHelper only currently supports the use cases required by EAS

Since EAS is open source, anyone with the time and inclination to create a more complete API for working with FME Server can use fme_server_helper.py as a jumping off point

So… get to work and have fun!

Thank You!

Questions?

For more information: Rob Gaston, rgaston(at)fargeo(dot)com Farallon Geographics, Inc. http://www.fargeo.com/ https://code.google.com/p/eas/