Ordnance Survey – open source at the heart of the enterprise

Ian James

Information Systems Chief Architect, Ordnance Survey

21st September 2013

Always a technology organisation

• Founded 1791, military roots, national mapping agency, national remit

• Public sector but with a commercial mandate

• Technology has always been both a tool and an enabler for change

o Surveying

o Cartography

o Printing

o IT

Production IT software at Ordnance Survey

• Initial experiments using IT to support map making started in late 1960s

• First digital product made available in 1971 (1:125,000 coastline)

• Increasing ramping up of digitisation (1970s – 1990s)

• Move from automating map production to digital data (1990s onwards)

• In the early years much of the “GI” software we used was bespoke

• Over time increasing use made of proprietary off-the-shelf software products (as capability aligned with our needs)

• In the last 10 years, open-source options have developed and matured (as capability has improved)

BespokeProprietary

Open source

1970 Mid-1980s Mid-2000s Today

A short history of open source at Ordnance SurveyApache, Tomcat, J2EE & MySQL for initial web systems

2002

Linux operating system for web-facing applications

2004

Linux operating system for internal production systems

2006

Using Open Layers for OS OpenSpace

2008

PostGIS, GeoNetwork, GeoServer, INSPIRE

2010

PostGIS, Web Services Consolidation

2012 2013

Magento, Solr for Map Shop, Apache Jena for linked data

So where is Ordnance Survey using open source GI?

• Commercial OGC Web Map Service

• WMS and WMTS-like

• Implemented using GeoServer / PostGIS / Astun Loader

OS OnDemand

• API for embedding OS mapping in web sites

• Implemented using OpenLayers / MySQL

• Will be migrating to OnDemand infrastructure

OS OpenSpace / OpenSpace Pro

12,300 Developers2,500 websites

21 enterprise customers12 partners

2,000,000,000 maps served last year

99.9% SLAc100% Actual Uptime

• Mapping search and preview tools in data.gov.uk

• Catalogue publishing services to EU

• INSPIRE conformant open-source WMS

• OpenLayers / GeoServer / MySQL / GeoNetwork

UK Location / INSPIRE / data.gov.uk

• Publishing platform for open linked data

• API access, various output formats, browser-based view and search

• Totally open-source implementation

Linked data service (data.ordnancesurvey.co.uk)

…but not just GI software…

• Consumer map shop and leisure portal

• In-house development and hosting

• Based on Magento, using MySql, Solr and Varnish http accelerator

Online Map Shop

• All software development now uses Agile methodologies

• Tooling to support this is predominantly open-source

• Includes Maven (build automation), Jenkins (Continuous integration), Subversion (version control) and others

Software development tooling

• Fedora Commons – digital content management for our products

• GlusterFS – High-performance network storage file system

…and currently looking at…

Some common factors in these implementations

• Primarily outward facing (public or customer) applications / services

• Mostly read-only applications (with periodic updates)• Online Map Shop a recent departure / exception

• Hosted on a “cloud” platform;• Externally, at Amazon, or;

• Internally, on our own cloud-like infrastructure

How did we get here?

• Initial adoption of open source was not planned:

• More of an organic process with roots in various research / low-key / “off-piste” developments

• Adopted in these areas primarily for expedient reasons – cost / ease of getting started / below the corporate IT radar

• Previously we have shied away from open source for production services (particularly paid-for services)

• But…

• Functionality the products were becoming capable

• We were increasingly comfortable with the open source culture and support options

Why open source GI? Why now?

• Allowed cost-effective cloud-based deployment:

• traditional licensing models didn’t work

• costs of commercial products quickly become prohibitive

• Cost savings (but it’s not free)

• The software products (in the areas we are using them) are :

• mature enough

• functional enough

• supported

• There is also a government mandate to do this…

UK Government Policy

• Government ICT Strategy“Where appropriate, government will procure open source solutions”

• Open source, open standards and re-use: government action plan“For all relevant software procurements across government, open source solutions will be considered fairly against proprietary solutions based on value for money (VFM) and total cost of ownership”

• Government Service Design Manual“…where there is no significant overall cost difference between open and non-open source products that fulfil minimum and essential capabilities, open source will be selected on the basis of its inherent flexibility”

Bias towards open

source

Part of a bigger picture for Ordnance Survey

…Source

…Standards

…Data

…Access

Open source first

• Based on our experiences, we have adopted an “open source first” policy for new software

• When looking for new software we will evaluate open source packages ahead of proprietary ones

• We will adopt an open source solution if:

• The licence is suitable for our needs

• The project is well supported with a lively and responsive user base

• The documentation is good

• We have the appropriate skills in-house or training is readily available

• We can support it (or we can buy support)

• It works equally well as available proprietary software.

• If not, we will go through standard selection / procurement processes

Giving something back

• Contributing to open source developments:

• data.gov.uk map-based search code released as open source (under BSD license)

• Small features and bug-fixing to GeoNetwork

• Paid for INSPIRE-conformance work on GeoServer

• We would like to do more of this:

• Self-interest – we want to ensure the products continue to develop and support what we and others want to do with it

• Altruism – there is a moral imperative to contribute in-kind for something we’re getting for free

• Ultimately we have to balance making a contribution with the pressures of delivering business/revenue-focussed projects

Open source isn’t everything

• We have a large investment (skills / development / licences) in proprietary technologies that the business relies on…

…we’re not going to throw that away overnight

• We don’t – yet – have confidence that open source solutions can always provide…

…enterprise-class reliability and scalability

…some of the niche capability that we require

(but that confidence is growing)

• So for the foreseeable future it will be a mixed economy

…but open source will have an increasing role for us

Some things we have learned…

• Start with little, interesting projects. Don’t dive in head-first with a big open-source strategy. Try stuff out.

• Open source benefits from a more proactive, inquisitive developer

• Skills need to be broadened – technical and cultural/mindset changes

• “Open source is free” …is a myth – costs are in different places

• Need to be rigorous about licences

But we need support?

• Just because it’s open source doesn’t mean you’re on your own……it’s just the support model is different

• Do it yourself (but be aware of the effort required), OR

• Buy it…

…increasingly seeing support eco-systems emerging, both from vendors (e.g. Red Hat) and from dedicated organisations (e.g. OSGeo, Boundless)

…a definite sign that open source is becoming mainstream

Open Source allows us to deliver more effectively

Open Source is mainstream

Open Source is here to stay at Ordnance Survey