CONNECT. TRANSFORM. AUTOMATE.

Managing Data Synchronization Jerrod Stutzman GIS Platform Engineer – Devon Energy

Project Goal

CONNECT. TRANSFORM. AUTOMATE.

!  Create a centralized spatial system for storing and displaying SCADA (supervisory control and data acquisition) data

!  View data on map via desktop and mobile !  Google-like search and performance

Challenges

!  Need to see consistent data on desktop web app and custom iPhone app

!  Location information from SCADA database is inconsistent and unreliable – need to join with reliable data for mapping

!  Need to show on map along with other Devon assets (wells, pipelines, leases, etc) !  ArcSDE is authoritative spatial database

!  Won’t be used if it isn’t fast!

Solution

!  Spatial Reasoning System (SRS) !  OpenGeo Suite (POSTGIS data served via

GeoServer using GeoWebCache) !  Apache SOLR search platform !  Internally developed web and mobile apps !  FME & FME Server for data creation and

synchronization !  Custom code for certain integrations

Why OpenSource

!  PERFORMANCE !  Biggest pain point historically is our well dataset.

Over 4 million features + 3 geometries (surface location, bottom location, directional path)

!  POSTGIS + GeoServer render and cache this dataset quite fast

!  POSTGIS can store all 3 geometries on one row! !  Reduced database complexity !  Low cost

!  FME is used within the SRS to create and sync data for wells, pipelines, leases, seismic, facilities, etc

!  FME Server handles scheduling and failure reporting (email)

SRS Integrations Diagram

Selected Workbench Example 1 EXCEL Spreadsheet to POSTGIS

Selected Workbench Example 1 EXCEL Spreadsheet to POSTGIS

!  SCADA Radio Tower data is maintained in Excel !  Updated quarterly. Full dataset update !  GPS Location and Range need to be shown

!  Need to store 1 to 3 geometries for each tower

Selected Workbench Example 1 EXCEL Spreadsheet to POSTGIS

Selected Workbench Example 2 Non-spatial Oracle to POSTGIS

Selected Workbench Example 2 Non-Spatial Oracle to POSTGIS

!  Non-spatial well production data from Oracle DB is synchronized with POSTGIS

!  Workflow:

Retrieve last run time from

POSTGIS Query

ORACLE for changes

Query POSTGIS for

matches Compare to determine

INSERT/UPDATE

Determine DELETES

Update POSTGIS

DB triggers update SOLR

Selected Workbench Example 2 Non-Spatial Oracle to POSTGIS

Selected Workbench Example 3 Seismic Data from Oracle DB

Selected Workbench Example 3 Seismic Data from Oracle DB

!  Seismic data can be represented spatially as lines or polygons !  This is stored in the seismic DB as a coordinate list

of vertices (numbers only, non-spatial DB) !  This data needs to be ‘spatialized’ and stored in a

spatial database – synced nightly (incremental updates)

!  Complexity Reduction – store all data (2 different geometry types) in same table

Selected Workbench Example 3 Seismic Data from Oracle DB

Result

Future

!  Ability to do an “UPSERT” DB transaction would greatly simplify some of the logic required to determine DB transaction! !  ARCSDE30 Writer actually had this capability, if it

did not find the record to update, it would insert !  Need to revise old workbenches to use best

practices and gain efficiencies !  Scaling SRS to handle more data and users

Thank You!

!  Questions?

!  For more information: !  Jerrod Stutzman !  Jerrod.Stutzman@dvn.com !  Devon Energy Corporation

CONNECT. TRANSFORM. AUTOMATE.