Embed Size (px)
DESCRIPTION
DCP Midstream, a gas gathering and processing company, has over 60,000 miles of pipelines which are maintained in an enterprise GIS solution. The GIS database has over 158,000 linear pipeline segments in ten non-contiguous States. To import the pipelines into the "One Call, Call Before You Dig" software where only polygons are accepted, a query needed to run to capture only the company operated pipelines and to create a buffer. This process coulid not be handled by the GIS software itself, but FME came to the rescue. In this session, we will explore how FME was used to address this challenge.
Citation preview
Using FME to Overcome General GIS Software Limitations
Alicia Foose, DCP Midstream
DCP Midstream Overview
DCP Midstream, LLC, a 50-50 joint venture between Spectra Energy and ConocoPhillips, is headquartered in Denver, Colorado.
The Company leads the midstream segment as one of the nation’s largest natural gas gatherers and processors in the United States.
DCP Midstream is the largest natural gas liquids (NGLs) producers in the nation.
DCP Midstream Overview
The Company owns or operates 58 plants, 10 fractionating facilities, and approximately 60,000 miles of gathering and transmission pipeline with connections to approximately 38,000 active receipt points.
Visit https://www.dcpmidstream.com for more details.
What Does This Look Like
Close Up Look
GIS Environment
Oracle database ESRI SDE Pipeline Open Database Standard (PODS)
4.02 database model http://pods.org/
The volume and complexity of data can create challenges for GIS analysis.
A Recent Project
DCP Midstream went through an evaluation process to find a software solution to manage the One Call (Call before you dig) process.
One of the solutions only accepted polygon features. Because all of the pipelines in the PODS database are polylines features a buffer needed to be created for the pilot.
To keep the comparisons similar we decided to buffer the pipeline by one foot.
Only the location of the pipelines were of interest.
Remember The 60,000 Miles
Volume Of Data
We were only interested in the pipelines we operated so a query was necessary.
The pipeline layer being used has 164,535 polylines in the database.
SQL> select count(*) from PODS.REGULATORY_SEGMENT;
COUNT(*) ---------- 164535
Laptop processing capacity along with memory limits can become an issue when buffering this volume of data.
Creating The Buffer In FME
The buffer was created in FME because It is easy to set up It can run in the background It doesn’t seem to use as many resources It tends to run faster on my environment It has an aggregate feature It can filter attributes
Creating The Buffer In FME
The goal was to: Query for only DCP Midstream Operated
pipelines. Simplify the data by eliminating most of the
columns. I chose to keep Region because there are only 10
regions (Regions have a logical geographical area)
Buffer the pipelines by 1 foot. Aggregate the data. Export the Polygon feature to an ESRI shape
file format.
Query For DCP Operated
The data was queried directly from the SDE connection in FME – this filters the data on the fly.
The 164,535 rows were reduced by 2,829 to total 161,706 records to buffer.
Transformers Used
The AttributeKeeper was used to reduce the number of columns from 51 down to 6 keeping only REGION_NAME from the SDE layer. Because only the location of a pipeline was required, the associated attributes were not needed.
Transformers Used
The Reprojector was used to project the data from NAD 83 to a projection with a unit of measure in feet.
US48-DUKE was chosen because the projection was created for the continental US and has relatively little overall distortion.
Transformers Used
The Bufferer was used to buffer by 1 foot.
Transformers Used
The Aggregator was used to aggregate the data using the REGION_NAME to group by.
Aggregating the data reduced the number of records from 161,706 to 10.
Transformers Used
The Reprojector was used to project the data back to NAD83.
Finally, the destination dataset was set to a shape file format. A visualizer was used so the output could be viewed right away.
A dissolver transformer was not used because the aggregate combined all of the polygons into Regions and the overlaps were not a concern for the end use.
FME Workspace
Final Results
Total Features Written 10 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Translation was SUCCESSFUL with 0 warning(s) (10 feature(s)/5295013 coordinate(s) output) FME Session Duration: 7 minutes 9.2 seconds. (CPU: 141.9s user, 10.6s system) END - ProcessID: 1480, peak process memory usage: 208120 kB, current process memory usage: 53868 kB.
A Side By Side Comparison
A 1 foot buffer was run in Arc Info using the same query on the same layer.
Dissolve by field - REGION_NAME was selected because it is the closest option to the FME Aggregate .
A Side By Side Comparison
Executing (Buffer_2): Buffer PODS.REGULATORY_SEGMENT \\Server\1_ft_Buffer.shp "1 Feet" FULL ROUND ALL #
Start Time: Thu Mar 11 08:29:01 2010 Dissolving... Output feature 0 cannot be dissolved into other inputs because of memory
limitations Output feature 1 cannot be dissolved into other inputs because of memory
limitations … … Output feature 15 cannot be dissolved into other inputs because of memory
limitations Executed (Buffer_2) successfully. End Time: Thu Mar 11 11:12:32 2010 (Elapsed Time: 2 hours 43 minutes 31
seconds)
A Side By Side Comparison
The FME translation ran in 7 minutes 9.2 seconds with no memory errors.
The Arc Info Buffer wizard ran in 2 hours 43 minutes and 31 seconds with memory limitations errors.
The results from either process were acceptable.
Annual Tax Project
Benjamin Franklin once said that “In this world nothing is certain but death and taxes”.
So lets talk about taxes, specifically property taxes. You might be asking yourself what on earth does FME have to do with property taxes. Well here is your answer-
Each year companies with tangible assets pay property taxes. Pipelines are not excluded.
The Challenge
Every State has unique taxing districts by which they collect and distribute property taxes.
Tax districts can change from year to year although most remain the same.
Population shifts and demographics are the most common cause of tax boundary changes.
DCP Midstream operates primarily in 17 States so tax boundary maintenance is a fairly large undertaking.
Tax Project
Each year the GIS department provides the Tax department with a report of how many feet of each pipeline is in what tax district by install year, diameter and so on.
The first step, for States with electronic data, is to download the current tax boundary files and update the SDE layer with the changes.
The SDE Layer has to be topologically clean. Neighboring states do not tend to use the
exact same state line. This creates gaps and overlaps which are ugly to clean up particularly along rivers.
2008 Oklahoma Tax Districts
2009 Oklahoma Tax Districts
Oklahoma Tax Districts
Who can tell me what changed? Going once Going twice Going three time How are you going to find out? FME has a transformer named Matcher which
detects both geometry and attribute changes from two files.
Lets See What Changed
The 2008 Oklahoma tax districts are added as one source.
The 2009 Oklahoma tax districts are added as another source.
Both are run through the Matcher as input. The Not_Matched features are output to a
visualizer so they can be looked at. The Not_Matched features are output to a
shape file to be used in ArcMap for updating the SDE layer.
FME Workspace
This Is What Changed
Why So Many Changes – River Correction
A Closer Look
Thank You!
Questions?
For more information: Alicia Foose [email protected] DCP Midstream
https://www.dcpmidstream.com http://pods.org/
