Open Addresses Symposium: 8th August 2014

Welcome & HousekeepingJeni Tennison

Today's objectives

1. Learn about the Open Addresses project2. Contribute to Discovery Phase

• help us understand requirements• help us explore feasible approaches

3. Connect with each other

Agenda

10:00 – 11:25 Background11:35 – 12:45 Requirements12:45 – 13:30 LUNCH13:30 – 14:50 Issues & Approaches15:00 – 15:45 Workshop15:45 – 16:00 Wrap-up

Who's here?

Ground rules

• Chatham House (except presentations)• you can report something was said, not who

said it

• Save your questions for discussion time• Own your opinion & listen to others'• There are no wrong ideas

• we are exploring this territory

Housekeeping

• Toilets• No fire alarm• Wifi

• Network: nyquist-guest• Username: visitor.wireless• Password: 2ndavenue

• Hashtag #openaddresses

KEYNOTEExperiences from DenmarkMorten Lind

MINISTRY FOR HOUSING, URBAN AND RURAL AFFAIRS

Addresses and Address DataExperiences from Denmark


Introduction


Ministry for Housing, Urban and Rural AffairsAddress: Gammel Mønt 4, 1117 Copenhagen

Ministry’s Logo

The Ministry

Minister Carsten Hansen


ODI-UK, 08-08-2014 DK Addresses and Address Data 12

Disclaimer

The basic facts are approved

… but personal statements are mine



”Mr. and Mrs. Dursley of number four, Privet Drive was proud to say that they were perfectly normal …”

J.K. Rowlands: ”Harry Potter and the Philosophers Stone” (Chapter 1, p. 1)



Guru



Christopher Corbin, Address Data, PSI, Open data

Michael Nicholson, Intelligent Addressing, EURADIN

Rob Walker, ISO Address Standard



Bob Barr


Concept of Addressing


A

B“Structured information that allows the unambiguous determination of an object for purposes of identification and location”

From ISO 19160 DIS







Addresses are important



Bratislava, August 1968(Nordfoto, DK)

… without addressesnavigation is difficult



CC Paul Townsend(Flickr: brizzle born and bred)

… so road signs were painted out in the UK during WWII

https://www.flickr.com/photos/brizzlebornandbred/5313127093/in/photostream/



… and should be easy to understand



… initiative launched 2009 by the Universal Postal Union, UPU



“As a network, addresses allow individuals to be connected to everyday life,

have a legal identity, participate in the democratic process, be part of the

society, as well as the formal economy, receive public and private services

and participate in the information and communication age.

Governments and public and private services also benefit from the address

network by using it to optimize the reach of policies, communicate with

individuals and support goods and service delivery.

Addresses are the underlying thread connecting these different actors and

their activities, effectively functioning as a network of networks.”

Universal Postal Union, UPU



Delivery ServicesDelivery ServicesCommunication networksCommunication networks

Vehicle navigationVehicle navigation

Civil registrationCivil registration

MarketingMarketingStatisticsStatistics

Location planningLocation planning

Yellow pages, mappingYellow pages, mapping

Social servicesSocial servicesTax, assessmentTax, assessment

Insurance/finacingInsurance/finacing

Property marketProperty market

TransportTransport

Customer relationsCustomer relations

EmergencyEmergency

Health careHealth care

Geo enabled applicationsGeo enabled applications

ADDR3A3A


A Brief History– of addressing in Denmark


… 1960 Addresses covers urban areas only


1967 Post code system introduced

1970-73 Addresses extended to rural areas



1978-80: Standard Address Format

Jennifer RedBlue Street 14A, 1.tv4990 Grenville C

Mun.C Street AddNo Floor Door

413 0915 14A 1 tv



1995-2001: Geocoding and Harmonization

Basic method: Collect addresses from 2-3 property base registers and combine them with the “house-number layer” in the first generation of digital maps.

Store the resulting data in the public Building and Dwelling Register

MapKRR

PropertyData Registers

BBRBuilding- and DwellingRegister

MunicipalTechnical Base Maps



1995-2001: Geocoding and Harmonization Mun StrC AdNo X-coord. Y-coord. Arc. --- ---- ---- -------,-- -------,-- ----- 619 4605 _17A 248.510,45 153.345,08 175,0

The initiative was a volunteer municipal project supported by the government and the local government association by standards, access to public registers and project coordination








Conceptual Issues

The Concept of Addresses and Address Data



From “Address as an Attribute”

Population Register

Tax

Property Register

Building Register

Health Care

Register of Business Entities

Utilities



… which encourages Inconsistency

Property register

Orchards Lane 2

DB1Utility data base:

Old Kings Road 88

DB1



Towards “Address as an Object”

Population Register

Tax

Property Register

Building Register

Customermanagement

Business Entities

Utilities

Address Register



Addresses as a Common Asset for Society

22

20

Mill Road

Before

2

20

Mill Road

Oak Lane

After

0a3f507f-c5df-32b8-e044-0003ba298018



Addresses as a Common Asset for Society

Address System

Concept of Addressesas a common, public

asset and resource

Person(CPR, health,social, Tax …)

BusinessLegal entities production units…

Building, dwelling, utilities, property …



2000-2001: Supported by Legal Framework

Public and Common Private RoadLegislation

§PopulationRegisterLegislation

§BuildingRegisterLegislation

§

Until 2001

Building Register ActSect. 3a-3g

§ 2001: All legal regulation of addressing transferred to the Act on Building and Dwelling Registration(Last revision 2012)

Statutory orderon road names and Addresses

§ 2003: Detailed regulation on rules for addressing(Latest revision May 2014)

From 2001 - …


Address Data Agreement

The 2002 agreement to provide address data as “Open data”, without fee or license restrictions


2000-2002 Deadlock SituationSituation:• State of the art address data, 97 % with HQ geo codes• Very large potential for use; Could be joined with data from other public

base registries• Large user demand from: emergency, police, county administration,

transport sector, health …But:• Data owned by 275 individual municipalities; no common license

agreement or data distributionTherefore: • Almost no re-use of data outside municipalities; but competing private

sector collections/datasetsODI-UK, 08-08-2014DK Addresses and Address Data 43


ODI-UK, 08-08-2014 44

Ministry of Finance comes to rescue

http://www.google.com/url?sa=i&rct=j&q=&esrc=s&frm=1&source=images&cd=&cad=rja&docid=HeP5Wc6hJuUN_M&tbnid=3MNtwK-vGDQHmM:&ved=0CAUQjRw&url=http://www.telegraph.co.uk/news/8349709/The-Yes-Minister-guides-for-civil-servants.html&ei=nwmVUsKdMcXhywPzvYHYCA&bvm=bv.57155469,d.bGE&psig=AFQjCNGCOnY7sKg6ALwsW5vfHrlKzkSKIQ&ust=1385585356894437



2002 …: Free of Charge Data Agreement Prepared by Danish e-Government Taskforce in

Ministry of Finance

Purpose: Increase use of address data including geo codes

Covers full public and private sector use and add-on valueing

Payment only of marginal costs of distribution

No license fee or any other payment based on IPR

Re-distribution to 3rd parties without payment

Economic compensation to municipalities; also for maintenance


Drivers behind the agreement

– and the Evaluation of the Socio-Economic Impact


Public Transport Services


”When we started our first ’Journey Planner project’ we had to manage individual agreements with every single municipality on the use of their address data – in the long run this was not a feasible solution.

Without the ”free-of-charge data agreement our project would have been almost impossible”

Ulla Skjelbo, Project Manager”Rejseplanen”/Journey Planner 1999-2000


Public Safety and Crime Detection


”Access to the official geo coded addresses proved to be one of the most important milestones in the implementation of GIS in the police force” Ole Jacobsen,

National Investigation Center, Danish National Police


Disease Monitoring and Control


Legionella

Examples showing monitoring Legionella infections over a period of 2 decades and detection of the transmission source of a recent Campylobacter outbreak.

”Geo coded address data is a crucial tool to the national monitoring of infectious diseases and to the analysis for detection of transmission source of outbreaks.” Ivan Bæhr, GIS specialist,

Statens Serum Institut – SSI(Danish Center for Disease Control)

Campylobacter


Cancer Research


“In studies of environmental pollutants and human health, the assessment of individual exposure to pollutants are a major challenge.

Without geo coded addresses our recent estimations of radon in the homes of children with leukemia and air pollution at the residences of lung cancer patients, had not been possible.”

Ole Raaschou-Nielsen. Institute of Cancer Epidemiology,

Danish Cancer Society


Postal Delivery Services


Address based planning of mail delivery (courtesy: Post Denmark)

“Post Denmark assesses that the new route planning tool ’TOR’ will reduce delivery costs by 7,5 millions EUR every year. This corresponds to 3,3% of the overall time used for delivery”

(’TOR’ is based on geo coded address data)

Lars Kristensen, Post Denmark in ”Fyens Stiftstidende” 03-12-2004


Emergency Dispatch Services


“When people call 1-1-2, the first thing to know is where the accident happened.

Address and street name is the most important tools to confirm the location – in order for us to dispatch the right ambulance.” Erling Larsen,

Danish 112 Centers’ Secretariat(112 was one of the first address data users)


Commercial use


“In the first year after the Free-of-Charge agreement, everybody thought they could make simple business just by re-selling the original address data.

It very quickly became clear, though, that the opportunities for business was in value-adding and development of new, smart products based on the addresses as a common reference.”

Martin Glarvig, Managing Director, Geomatic DK


Assessing the Impact

Estimated Socio-Economic Benefits 2005-2009 of the Free-of-Charge Agreement



2010: Assessment of Benefits 2005-2010

2000 2005 2010 2015

Good

Better

Best

Average

Start ’Zero’

Today

Impact of agreementHow shall we assess the benefits of the increased address data re-use?



Chain of Address Data Re-use

ADDR

‘Public Data Server’ (PDS)

2nd level

Licensed Data Distributors

3rd 5th4th …

End users

Data distribution from a common hub managed by

DECA (www.ois.dk)

Mun

icip

aliti

es

Re-use and re-re-use

http://www.ois.dk/



Increase in Re-use 2002-2009Use of address data sets

25

1.250

0

200

400

600

800

1.000

1.200

1.400

2001 2009

Da

ta s

ets

dis

trib

ute

d/y

ea

r

2002 2009

Number of Address data sets distributed or updated to 3rd parties in 2002 and 2009(of the 1250, approximately 300 was ’full cover Denmark’

70%

20%

10%

Private sector

State, region

Municipality



Result: Assessment of BenefitsDirect, measurable economic benefits from re-use of public address data 2005-2009: ~ 63 mill. EUR (~471 mill. DKK)

Assessed benefits in 2010:~ 14 mill. EUR (~105 mill. DKK)

Cost of data agreement 2003-2009:~ 2.6 mill. EUR (~20 mill. DKK)

~ In 2010: 0.2 mill. EUR (~1.5 mill. DKK)



Results Confirmed by EU Commission

Pricing of Public Sector Information Study (POPSIS) selected the Danish provision of address data as one of the case studies.

The Study used another method, but the conclusion, that the Free-of-Charge agreement on address data had a significant positive socio-economic impact, was the same.

EU DG InfSoc, October 2011


OSM Paris, 04-04-2014 DK Addresses and Address Data 60

Potential Cost-benefit of a Danish Address

1 1000 5.000.000Address authority Industry Products

The municipal address authority allocates and registers an address or a road name

1000 application developers or data producers integrates the address or road name in their product or service

5 mill. users now have access to the new address or road name in their it-system, GPS, smartphone or tablet

Few costHuge benefit


Present time

Initiatives, opportunities and challenges



We are in a New World> 50 % of all Households! (1,25 mill.)

Antal husstande med GPS navigation og smartphone(Kilde: Danmarks Statistik)

0%

10%

20%

30%

40%

50%

60%

70%

2006 2007 2008 2009 2010 2011 2012

GPS navigation

Smartphone

*) Add to this number of GPS, smartphones and tablets used in private businesses and public sector

http://www.google.com/url?sa=i&rct=j&q=Smattphone%20map&source=images&cd=&cad=rja&docid=eFIy8SBB4yyIzM&tbnid=SWYgLmJN6azjqM:&ved=0CAUQjRw&url=http://pearson4loyalty.com/2013/08/with-smartphone-geolocation-and-trust-local-beats-long-distance/&ei=B90jUp3TFqi70QXK5oGwBw&bvm=bv.51495398,d.d2k&psig=AFQjCNEniIDbyCKR2ldNqXO7EiqJ5-oymQ&ust=1378168394008396



Where Emergency Response is Data-driven

500 times every day, an ambulance is dispatched to an incident where the response time is critical – i.e. few minutes delay could cause death or another serious situation



On the Political Agenda

TV2/Fyn 8. oktober 2012 ”Unfortunately there are many places without an accurate address”





The Address Program


DK Addresses and Address Data 67

The data quality – E.g. timeliness, accuracy, coverage,

harmonization, standardization

The back end infrastructure – E.g. base register systems, coherence with

other basic data sets etc.

The services– E.g. by reliable, high-effect, open services for

mapping, online access and download

… and everything within the concept of Open Data and Open Services

The Address Program shall Improve

ODI-UK, 08-08-2014


External Interest in Improvement *)

• Public sector – at all levels:– Waste of time in management of in-formal, local address registers– Waste of time to sort out data inconsistency from different sources

• Business – Utilities, LBS service providers, transport sector:– Higher accuracy, better address coverage and improved timeliness– Improved data services, reduced costs of acquisition

• Citizens:– Are worried if GPS and emergency service will not find them– Are troubled when services are poor because of bad addressing

*) Based on several stakeholder reports and feed back from users and citizensDK Addresses and Address Data 68ODI-UK, 08-08-2014



National TV-spot from May 2014

http://youtu.be/iOtBG8O1nd4

http://www.youtube.com/watch?v=ViL2EIOEgx4


Benefits for Society

Expected and experienced socio-economic benefits of the Address Program – and examples


Benefits of the Address Program

Improved Data Quality

and timeliness

Improved infrastructure

Systematic Re-use

IT-Infrastructure:(Implementation + operation)

New address registerData Distribution/Services

Improved Data Quality:(implementation costs)

Extra FTE’s in municipalities and MBBL

Address authority tasks:(operational costs)

Extra FTE’s in municipalities and MBBL

Benefits from digital self-service, electronic forms, applications, reporting

Reduced cost of new it-applications and systems

Reduced cost and time of present it and address data management

Other business process benefits In total

~30 mill EUR/yr from 2015

~ 5 mill EUR(one time) +1 mill EUR/yr

~ 85 FTE(one time)

~ 40 FTEevery year



The Address Program Business Case

2012 2016 2020

EvenBetter

Almostperfect?

Good

Today’Zero’

(Do nothing)

Future situation

Impact of Address program

Estimated annual net benefit: 30 mill. EUR/yr



DK Addresses and Address Data 73

• Because our address data are open they are widely in use for many applications

• Because our data are in use and open, users see the mutual benefits in reporting errors

• Because we receive error reports, we are able to improve our address data

Note

ODI-UK, 08-08-2014


Any government wants to create new jobs, private innovation and entrepreneurship – and tax revenue


MINISTRY FOR HOUSING, URBAN AND RURAL AFFAIRSData is the

new Gold– Open data is

a great opportunity for

pushing our economy

Neelie Kroes, Vice President EC


http://www.google.com/url?sa=i&rct=j&q=&esrc=s&frm=1&source=images&cd=&cad=rja&docid=4q7Otq9bZuTkpM&tbnid=GXurupkDETU79M:&ved=0CAUQjRw&url=http://www.webpronews.com/neelie-kroes-discusses-european-net-neutrality-2012-05&ei=bEGVUsvUN8ulyAOa0YCYDA&bvm=bv.57155469,d.bGE&psig=AFQjCNFG8N0EaxyckN0_k6U5PFYYYjpJqA&ust=1385599576915867


Joys Law

"No matter who you are, most of the smartest

people work for someone else.”

Bill Joy, the co-founder of Sun Microsystems


http://www.google.com/url?sa=i&rct=j&q=&esrc=s&frm=1&source=images&cd=&cad=rja&docid=Z8XwxGq5SK-0FM&tbnid=GFc8Gwp5R3BoRM:&ved=0CAUQjRw&url=http://hackingthefuture.blogspot.com/2009/02/bill-joy.html&ei=T0eVUvHaJ-eCyAP-xoGQBA&bvm=bv.57155469,d.bGQ&psig=AFQjCNEvctFFRZ0mG8-8bMyZaUNSiO5H0Q&ust=1385601096142416



Open Address Data: Business Opportunities

http://www.atkins.dk/

http://grontmij.dk/DK/

http://www.septima.dk/

http://www.kmd.dk/da




Contact Information• Morten Lind, Ministry for Housing, Urban and Rural Affairs – Denmark, [email protected] – Twitter: @mortlin

• The Ministry (MBBL): www.mbbl.dk

• Danish Geodata Agency: www.gst.dk

• Danish Basic Data Program: http://uk.fm.dk/publications/2012/good-basic-data-for-everyone/

• The Address Program: www.adresseprogrammet.dk

• Danish Address Website: www.danmarksadresser.dk

• Address data services: www.aws.dk

• The value of Danish address data: http://danmarksadresser.dk/file/389579/Value_Assessment_Danish_Address_Data_UK_2010-07-07.pdf

• Twitter: Danmarks Adresser - @DKAdresser Address data services: @AWS_Suiten

mailto:[email protected]

http://www.mbbl.dk/

http://www.gst.dk/

http://uk.fm.dk/publications/2012/good-basic-data-for-everyone/

http://www.adresseprogrammet.dk/

http://www.danmarksadresser.dk/

http://www.aws.dk/

http://danmarksadresser.dk/file/389579/Value_Assessment_Danish_Address_Data_UK_2010-07-07.pdf

An Open National Address GazetteerHugh Neffendorf

An Open National Address Gazetteer

Open Addresses Symposium Vision for an

Open Address Database

8 August 2014Hugh Neffendorf

BIS feasibility reviewPrompted by ODUG paper to Data

Strategy BoardRequest from MinistersBIS review working with data owners,

representative bodies and government officials

Independent report by KatalysisReport and comments publishedhttps://www.gov.uk/government/publications/an-open-national-address-gazetteer

Now with BIS and Cabinet Office

https://www.gov.uk/government/publications/an-open-national-address-gazetteer



Why Open Addressing? Critical element of our identity for personal

and business reasons Open Data is changing attitudes and

behaviours Addresses are a key core reference and would

be a big win for Open Data The potential for innovation stimulated

growth is the prizeBut, it was recognised that: Costs may be high to government Ownership is complex Compelling and quantifiable benefits case is

elusive

Definitive or Open?Definitive addressing is the priority goalMany users will pay reasonable prices for

good dataBut many are deterred by licensing

and/or priceOpen can support definitive – more usage,

more feedback

Can we achieve both?

Establishing evidence Does Open mean growth? Little hard evidence on Open Data

Tracking often stops when no licensing APPSI seminar was inconclusive

Some good cases of ‘Open’ growth Population census PSMA OS OpenData Companies House

Considerable need for (good) evidence Open Addresses should have monitoring

The address system today

Usage sector Open View

Benefit H/M/L

Open Lite OK?

Notes

Academics ✓✓✓ H Y Better quality research from more accurate data

Individual citizens -- M Y/N Indirect benefit from improved identity authentication and service delivery; better e-products

Devolved administrations

✓✓ H N Scotland strongly in favour; NI interested; Wales?

Land Registry ✓ L N No great consequences. Can see wider benefits

Local government ✓✓ H N Good for citizen engagement. Worried about any resource impacts

Mail and delivery industry

-- L N Some benefit from better quality addresses. Concerned to maintain quality

Market research ✓✓ M Y Will save money on buying addresses; more consistent research

Not for Profit Enterprises

✓✓ M Y Better quality use of data and data exchange

Open data community

✓✓✓ H Y Open Addresses is one of their most important requirements

PAF Solutions Providers

✓✓ M N Generally positive views from larger resellers. Smaller ones may feel threatened. Quality is a concern

Private sector generally

✓✓ H Y Considerable saving and simpler licensing will free up activity

Public sector generally

✓✓ M Y/N Already well covered by PSMA/PSL but can see merits in wider extension

SME market ✓✓✓ H Y Considerable value for software developers

Statistics users ✓✓ M Y Interested in wider data linkage benefits

Vehicle navigation ✓ L Y Some gain from reliable benchmark updates

Valuation Office ✓ L N No great consequences. Can see wider benefits

Web search providers

✓✓✓ H Y Will open considerable growth opportunities

PAF Advisory Board

-- n/a n/a Individual members in favour. Some caution

APPSI ✓✓ n/a n/a Says there will be major benefit

Review conclusions Seven main options

Totally Open Evolving status quo Extended bulk purchases New charging models Addresses as an Open online service Freemium (Free ‘Lite’ products) Commissioned ‘basic’ Open product - recommended

• Also considering scope for efficiencies Hub operation Shared intelligence Less duplication

Address Hub vision

Open AddressesProject VisionJeni Tennison

Wider context

• Data culture is changing• long term, inexorable move to

open

• We have seen this before• software, music, books• changes in technical delivery• changes in cultural expectations

What can we learn?

• Mixed economy persists• customer base for closed shrinks

• Shift from products to services• new opportunities to make

money

• Existing providers can adapt• but it can be hard

"The sale of the PAF with the Royal Mail was a mistake. Public access to public sector data must never be sold or given away again. This type of information, like census information and many other data sets, is very expensive to collect and collate into useable form, but it also has huge potential value to the economy and society as a whole if it is kept as an open, public good."Bernard Jenkin, Chair of Public Administration Select

Committee

“Open data is data that can be freely used, reused and redistributed by anyone – subject only, at most, to the requirement to attribute and share alike.”

Opendefinition.org

Open is not equivalent to free

opendata

freeservice

Hypothesis 1: closed address files are monopoly information assets, embedded in huge amounts of public data, but which cannot be reproduced to a usable quality

Hypothesis 2: it is possible to build & maintain a sustainable open address database better suited to today's requirements using modern, collaborative approaches to data management

Open Addresses Project• Discovery Phase funded from

Release of Data Fund• administered by Open Data User

Group within Cabinet Office

• Legal feasibility• Technical feasibility• Sustainability feasibility

Open Addresses Vision

• Meeting the expectations of the modern information economy

• Collaboratively maintained to benefit everyone

• Providing plenty of scope for value-added products & services that avoid lock-in

Modern requirements

• Addresses are not just for posting mail• Other requirements:

• validation & auto-completion• geocoding for route finding• associating people with areas• classification of addresses for targeting

interventions• linking datasets together

Modern maintenance

• Through collaboration• professional / expert

engagement• the wider crowd

• Supplemented by• targeted (funded) activity

• Addresses are well suited for this

Added value products

openaddresses subsetadd detail

Where we are

• Legal feasibility• received legal opinion• doing due diligence on key open

datasets

• Technical feasibility• data integration & inference• architecture for service provision

• Sustainability feasibility• customer needs & business model

What next

• Pending approval• Alpha phase – Oct-Nov

• incorporating Open Addresses Ltd

• building minimum viable product

• Beta phase – Dec-Mar• developing operational service

• Independent operation – Mar

Unique opportunity

• Green field development• do not need to be limited by

legacy

• You can help this succeed• contribute today• collaborate in the future

• You can benefit from its success• reduce costs, build new products

Questions?

Requirements Introduction

Requirements Carousel

• Six tables• you will visit four

• Go to a table of your choice• listen to the presentation• ask questions & discuss

• When whistle blows, move on

Requirements from:

Table 1

BBC Chris Henden

Table 2

GDS Paul Downey

Table 3

Direct Marketing Assoc.

Tim Drye

Table 4

ONS Alistair Calder

Table 5

LandInform Andrew Harrison

Table 6

KnowWhere Consulting

Steven Feldman

BREAKreturning at 11:35



Table 1

BBC Chris Henden

Table 2

GDS Paul Downey

Table 3

Direct Marketing Assoc.

Tim Drye

Table 4

ONS Alistair Calder

Table 5

LandInform Andrew Harrison

Table 6

KnowWhere Consulting

Steven Feldman

LUNCHreturning at 13:30

Meeting the ChallengesIntroduction

Meeting the Challenges

• Lots of things to work out• green(ish) field development

• Focus on five challenges• hear a talk from an expert• hear a little of our thoughts• discuss in the workshop• fill in your worksheets!

Meeting the ChallengesTracking provenanceFrancis Davey

Tracking provenance

• Three types of sources• data project sources & processes• data contributed by third parties• data inferred from data already in database

• Need to record each source• including licence made available for reuse• including information about processing

Due Diligence

• Need certainty for reusers• Where did addresses

originate?• in initial database• through subsequent additions

• Is there any validation?• if so against what?

Recording Provenance

Calculating Certainty

• More or less certainty about data

• Supporting information• using known localities / streets• other addresses on same street• location of contributor

• Bayesian network?

Meeting the ChallengesSourcing addressesJohn Murray

Open Addresses SymposiumMeeting the Challenges

Address inference from open data

John Murray

Sources of Addresses• Land Registry• Companies House• National Social Housing Register (NROSH)• NHS – GP surgeries, hospitals etc• Lists of schools• Government department asset lists• Scottish gazetteers (are they really open?)8 August 2014 Open Addresses Symposium 122

Sources of Spatial Information• Ordnance Survey:

– Codepoint Open– OS Locator– OS Gazetteer– Street view– Named places, settlement seeds, DLUA boundaries, parishes.

• ONS– ONSPD Postcode directory– Built up areas.– Census boundaries.

• Land Registry– Cadestral Polygons (dispute about whether they are open)

• DfT– National Public Transport Gazetteer.

8 August 2014 Open Addresses Symposium 123

Proposal• Build a street and places gazetteer, to which address points (PAON and

SAON) may be attached.• Use spatial data to verify veracity of loaded data from open sources.• Apply confidence score to each record based on:

– Spatial integrity– Frequency of appearance within and across sources.

• Towns and localities inferred by filling gaps.• Street layout analysis:

– Position of buildings by pixel analysis.– Postcode to numbering: e.g. odds and evens


Pixel Analysis• Overlay vector streets and postcode centroids on OS

StreetView• Use in conjunction with OS locator for context and extent.• Analyse pixel colour within buffer either side of road to

estimate buildings extent.• Can be used to:

– Ensure veracity of other data– Infill missing properties– More accurately assign streets to postcodes


Pixel Analysis


Adding Land Registry


Maximising Available Data• Using ONSPD, correcting postcodes where there is

an unambiguous coordinate match from a terminated postcode to new one.

• Accounts for 50% of retired codes.• Correcting misspellings by reference to

dictionaries using lexical analysis.• Reference earlier versions of the data.8 August 2014 Open Addresses Symposium 128

Source Audits• Land Registry – Good quality, kept up to date,

few errors. Covers England and Wales.• Companies House – Data quality issues,

particularly older companies. Covers UK.• NROSH – Variable quality. Covers England


Prototype• Contains all current GB postcodes.• Streets added where possible.• Localities added where possible.• Corrects retired postcodes where possible.• Shows nearest postcodes if not.• Built from 4 sources, with gaps filled by inference.


Initial ResultsOS_Locator LandReg Companies NROSH Count Percent

0 0 0 1 5,042 0.29%0 0 1 0 18,990 1.09%0 0 1 1 606 0.03%0 1 0 0 111,553 6.40%0 1 0 1 25,514 1.46%0 1 1 0 20,842 1.20%0 1 1 1 5,574 0.32%1 0 0 0 227,971 13.07%1 0 0 1 41,773 2.40%1 0 1 0 115,449 6.62%1 0 1 1 7,065 0.41%1 1 0 0 381,917 21.90%1 1 0 1 166,608 9.55%1 1 1 0 348,669 19.99%1 1 1 1 122,299 7.01%

Unmatched 144,218 8.27% 1,744,090 100.00%


Weaknesses• Lack of addresses for Scotland• Inference not always accurate due to:

– Non-vehicular streets– Streets in close proximity– Not all addresses have a street– Address elements not unique at postcode sector

• Questions about openness of some data8 August 2014 Open Addresses Symposium 132

Conclusion• More study needed on veracity to:

– Understand issues in data.– Ensure integrity of database.– Make more accurate assumptions.

• Crowdsourcing:– Same methods could be used to ensure veracity.– Could be offered a free/low cost service to SMEs

• Lobbying for more data to be made open.8 August 2014 Open Addresses Symposium 133

Questions?

Test drive the prototype.

Building Gazetteer

• Existing open data• from OS (& Royal Mail)• from ONS

• Areas & indicative points• OpenStreetMap

• would require share-alike licence

Getting Addresses

• General sources• Land Registry price point data• Companies House addresses

• Specific lists of• NHS locations• schools• public buildings

• Inference

Meeting the ChallengesStructuring addressesRob Walker

The BS7666 Approach to Structuring Data

Rob WalkerBSI IST/36 Geographic information

138Rob Walker 2014-08-08

139

What is BS 7666?• A standard for gazetteers of geographic objects• Defines a system of spatial references based

upon identifiable real-world locations• Specifies gazetteers of commonly used location

types for spatial referencing• Provides a structure for creating addresses

Rob Walker 2014-08-08

Overall purpose of BS 7666• For defining common geo-referencing systems for

entities within scope• Provides a standard way for identifying and defining

geographic entities• Provides a way of sharing and accessing information

about the geographic entities• Permits the creation of local datasets or gazetteers• Enables the creation of national datasets or gazetteers

140Rob Walker 2014-08-08

Multi-part standard• Part 0 – General model for gazetteers and spatial

referencing• Part 1 – Specification for a street gazetteer• Part 2 – Specification for a land and property

gazetteer• Part 5 – Specification for a delivery point gazetteer

141Rob Walker 2014-08-08

142

Gazetteer• Record of locations of particular type or types

with sufficient information to find and identify each uniquely

• Does not contain detailed (attribute) data about the objects themselves


Characteristics of a geographic object

GeographicObject

Position Identification Classificationreference pointextentdescriptive location

namereference number

object typeusage classification

Addressname/numberstreetlocalitytowncountry

143Rob Walker 2014-08-08

144

Address• Identifies a real-world location (addressable object)• Type of Land and Property Identifier (LPI)• Provides a hierarchical structure of location types• Uses commonly understood names• Not routing instructions • Different from a unique key


145

Property

Address

StreetLocality

Town

Admin Area

Property address model


Scope - entitiesAddressable object

Land and property identifier

Public right of way

StreetBasic land

and property unit

Address

incorporates

covers

references

has

ofidentifies

identified by

MandatoryOptionalOptional/mandatory

One-OneOne-Many

referenced by

146Rob Walker 2014-08-08

147

Structure of Land and Property Gazetteer (Part 2)

• Scope• Definition of terms• Conceptual schema for land and property • Conformance levels • Gazetteer requirements• BLPU identification, description and referencing• Coordinate referencing• Annex – Description of extents• Annex – Data Quality Report


Issues beyond the Standard• Guidelines for creation of data• Property numbering and street naming• Data quality• Verification of conformance to specification• Scope creep• Ownership and access rights to data

Rob Walker 2014-08-08 148

Summary• BS 7666 is a specification for gazetteers of geographic objects• Particularly for Basic Land and Property Units (BLPUs)• Applicable not just to occupied property• Addressing system based on streets• Address structure of:

– Secondary addressable object – Primary addressable object– Street– Locality– Town– Administrative Area

149Rob Walker 2014-08-08

What is an Address?

• What should the scope be?• Delivery points?• Buildings?• Households?• Rooms?

Addressable Locations

room

house

street

locality

locality

authority

Meeting the ChallengesMeasuring qualityBob Barr

Never mind the quality feel the width?

Quality measures for a UK Open Address File

Prof Robert Barr OBEManchester Geomatics and The University of Liverpool

Expert member APPSI and ODUG

The SLA Forum - 8th August 2014 - ODI

Private!

Secrecy and Quality

• Proprietary data sets are seldom available for comprehensive quality assurance

• Where data has been made available for cross matching, results supressed

• Internal quality control measures neither documented nor auditable• Customers seldom in a position to asses overall quality or fitness for

purpose• As quality cannot be audited tested or checked there is little incentive

to improve it – will not increase sales only costs

Quality

Domain

• What is being addressed?• Postal delivery points• Dwellings• Taxable hereditaments• Buildings• Infrastructure

• e.g:• Electricity sub-stations• Cashpoints

Completeness

• When % completeness is claimed, how independent is the measure of the total number of addressable objects in the domain

• Duplication can exaggerate completeness while missing addresses• Most likely to be achieved if:

• Address required for certain activities and it is generated if missing• Public enabled and encouraged to identify missing addresses

Currency

• When is an address added and how soon does it appear in the public file

• When is an address deleted / archived • Most likely to be achieved if:

• Address required for certain activities and it is generated if missing• Public enabled and encouraged to identify missing addresses

Positional Accuracy

• Does any geocode (Lat.Long or OSGR) correctly identify the addressed object?

• Are numbered properties correctly identified, order on street and side of street?

• Do locational attributes of address lie within the correct bounded areas?

• Are any objects ambiguously addressed with multiple different addresses?

• Are any objects that should be addressed missing?

Attribute Accuracy

• Address attributes such as residential non-residential flags need to be correct

• Other address attributes such as the status of the position of the address also need checking

Address Content Accuracy

• Is the right and correct information stored in each address field in the database

• Does the address comply with published specifications for the file• Does the address comply with relevant standards e.g. BS7666

Open

Open Address Quality

• By releasing address intelligence on an open platform quality can be checked and corrections made or suggested

• This works for OpenStreet Map• Providing Open Access on a record by record basis from closed data sets, on a

single platform,would at least allow the evidence from each source to be assessed

• Open Addressing will not create a high quality by itself, however if the major maintainers and providers of address data made their data more open quality could and would be independently assessed and willingness to correct could be checked

Conclusions

• Addressing in the UK is NOT:• Definitive• Authoritative• A Public Good• Of measurable quality• Fit for many specific purposes• Used as widely as it could be• Maintained cost effectively

WHY?

Conclusions

BECAUSE:• Since 1979 the primary motivation for maintaining national address

files has been to make a profit or surplus through trading (often only loosely related to costs)

• BIS (parts), The Treasury, Royal Mail PLC and the Public Data Group appear to have won the argument in government.

• This is likely to be subject to a wide range of continuing challenges….• One of which will be an Open Address File whose quality will be

publicly checked and openly assessed

That’s It!Robert Barr

[email protected]

Quality metrics

• Number of addresses• Coverage

• what baseline can we use?• estimate against census stats?• check against Open Street Map?• against Gazetteer?

• Who can measure & how often?

Meeting the ChallengesOrganising effortJerry Clough

Collaboration

• Why contributors contribute?• self-interest• subversion• fun

• What can we do to capitalise on these?

Correct Your Address

• Let people control their address• new builds• house names• devotion to locality

• Let look-ups be up to date• easing online sales means more

customers

Sustainability

• Core database open & free• Some lookups open & free• What can we charge for?

• service-level agreement?• unlimited lookups?

• Sponsorship opportunities?

BREAKreturning at 15:00

Meeting the ChallengesWorkshop

Meeting the Challenges WorkshopTable 1 Tracking provenance

Table 2 Sourcing addresses

Table 3 Structuring addresses

Table 4 Measuring quality

Table 5 Organizing effort

Wrap UpJeni Tennison

Next Steps

• We will write this up• we hope you do too!• tweet on #openaddresses

• Discovery Phase documentation• mid September

• Alpha & beta approval• during August

Get Involved

• Hiring for Open Addresses• project lead• independent Board members• https://theodi.org/jobs

• Complete the survey• tell us how you want to

contribute

https://theodi.org/jobs

Data & Analytics

Open Addresses Symposium: 8th August 2014