Upload
safe-software
View
553
Download
1
Tags:
Embed Size (px)
Citation preview
On the Menu:
Appetizer: 2010 Municipal Census Salad Course: Business Licensing Main Course: Fire Services 9-1-1 Dispatch Dessert: Questions? (you’re all on a diet)
Appetizer: 2010 Municipal Census
The challenge for the kitchen: City-conducted municipal census All residences within City boundaries First on-line census No previous spatial census-base layer available No spatial address point layer available No single address database available All residences require a secure PIN number All residences will be mailed their PIN number A complete residential address listing is required
Desired Outputs
Primary and secondary PIN numbers for each address
A mail merge database including postal codes and PIN numbers
A list of addresses requiring in person PIN letter delivery
A list of addresses requiring institutional enumeration
A final GIS census polygon landbase including assigned PIN numbers
Finding the Right Recipe
The goals for the PIN numbers: No less than 5 characters long No more than 8 characters long Primary and secondary PINs No duplicate PINs Repeatable method No confusing characters (e.g. 0 vs O)
Trial 1: Using Existing Info
The City of Airdrie did it so why can’t we? Available Info:
Street names Legallot ID numbers Centroid X/Y coordinates Neighbourhood names
Strip them down, mash them together, in short make an unproductive mess…
Sampling Ingredients
1. Input Landbase
2. Fetch Centroid Coodinates
3. Trim off decimal places
4. Replace decimal
5. Generate random 9-digit number
6. Generate single random number
7. Extract first letters of street names
And It Tastes Like?
Ummm…best not to be sampled: Generated duplicates Complicated at best Seemed like it would take more work than it
was worth to get it to work Like Plan B would be better: why use existing
ingredients when you can just create some tasty new ones?
Plan B: Just Random
Use individually generated random numbers that match the ASCII character designations for capital letters A to Z.
Stick with just letters instead of mixing in numbers
Generate 7 individual letters Combine in order for the initial 7-digit PIN Recombine to create an 8-digit PIN Little chance of generating duplicates as
there are now 26 choices, instead of 10, for each character
Appetizer: The Model
1. GenerateRandomLetters
2. Create first PIN
3. Check for duplicates
4. Create second PIN5. GIS Output
Appetizer: The Model (more yummies)
6. Addressesmust be sorted
In ascending order of:• Enumeration Area• Enumeration District• Street• House number• Suite number
10. Get rid of offending commas
7. Address mergefile for Admin staff(Access table)
8. Addresses forhand delivery ofPIN letters(Access table)
9. Addresses forCity staff enumeration(Access table)
11. Blank databasefor census contractor
(CSV file)
Appetizer in Review
We went with simple instead of fancy for PIN number generation.
We chose to optimize our development time. We had difficulties in joining postal codes to
address points in FME so ended up doing that in ArcGIS as pre-processing.
Overall development time was approximately 1.5 days, including initial sandbox time.
Very happy with final result.
Salad Course:Mapping Business Licenses
The challenge for the kitchen: To map the home-based business licenses The license database is not spatial The license database is address-based We don’t have a spatial point address layer We do have a census landbase Address format structure does not match
between the two databases We wish to have an automated process for
producing updated business license datasets
Desired Outputs
GIS point layer of home-based businesses Business trade name Address (full and parsed) License number License category/subcategory NAICS code NAICS code full description NAICS sector NAICS class
Salad Course: The Model
Create addressjoin string
Create address points;Match to BL data
Merge with NAICSlookup table
Salad Course in Review
Overall development time was approximately 5 days, including initial sandbox time.
Very happy with final result. Have already updated the data with the latest monthly data dump.
Challenges: Understanding the tools Chaining multiple models together Parameterizing the model for flexibility
Main Course:Fire Services 9-1-1 Dispatch
The challenge for the kitchen: 9-1-1 calls come in from Telus Telus feed has specifically formatted addresses 9-1-1 addresses must be structured to match
Telus feed data as matching is automated 9-1-1 system requires address point dataset All non-relevant addresses are to be stripped Each address must have a unique ID, repeatable
through each data load Addresses cannot be duplicated The City does not have a spatial address dataset
Available Ingredients
Parcel Polygons
Townhouse Polygons
Apartment/CondoSuites
Additional AddressPoints
Desired Outputs
GIS address file Shapefile format Point feature type Containing only addresses that will be
referenced by Telus Containing only those fields required by CriSys Formatted to match Telus feed
Challenge for the Kitchen
Must be repeatable Must be automated as much as possible Must be able to easily QC for duplicate IDs
and Addresses Must be fast Must be flexible to deal with changing inputs
Building the Crust
1. Convert polygons to points2. Filter out non-relevant addresses3. Filter out Null addresses/IDs – multiple methods required
Building the Crust
4. Convert UPPER CASE street names to Sentence Case5. Fix McKenney Avenue (inner capital letter)6. Adjust spelling of key road names to match CriSys/Telus
Building the Crust
7. Checking for duplicate IDs8. Filtering out or dealing with duplicate IDs9. Filter out specific addresses for replacement
Building the Crust
10.Converting LegallotID value to CriSys ID11. Automated removal of duplicate addresses12.Output QC datasets for parcel addresses and duplicates
Making the Filling
1. Convert townhouse polygons to points2. Change UPPER CASE street names to Sentence Case3. Fix McKenney Ave case
Making the Filling
4. Filter for addresses with/without suite numbers5. Create unique ID numbers from LegallotID, suite and house
numbers6. Deal with two specific problem properties
7. Output QC dataset
Making the Topping
1. Converting multi-story suite polygons to points2. Change UPPER CASE street names to Sentence Case3. Fix McKenney Ave case
Making the Topping
4. Create final ID string from LegallotID, house number and suite number (when applicable)
5. Output QC dataset
Putting It All Together
13.Adding in the extra points14.Streaming all the outputs into
a single final address point file
Main Course in Review
This was actually our first (and most complex) FME project
This is the second iteration of the model Taking the course helped in refining the
model (thanks!) It’s a live model and is expected to have to
constantly evolve to deal with data oddities The model is currently pretty robust Challenges were mostly related to discovering
and dealing with data oddities
Recap and Thoughts
We LOVE FME! The software is intuitive It’s also powerful and complicated It definitely meets our needs We see it playing a key role in data integration We use it from soup to nuts – everything from
simple field rearranging to complex translation and integration models
We see it being useful for more than just spatial information
Thank You!
Questions?
For more information: Tammy Kobliuk [email protected] City of St Albert
Enter other resources