How to scale 1000s of API Integrations and not lose your mind

How to scale 1000s of API integrations and not lose your mind Andrew LilesGlobal Technical Partner, DigitasLBi

1

George TaylorTechnical Architect, DigitasLBi

The views expressed in this presentation are those of the presenter, and not necessarily those of Apigee Corporation or the presenter’s employer.

2

Agenda

3

Context

1. How to handle 1000s of integration configurations

2. How to govern the continual evolution of 10 APIs, publicising contracts such that automated tools can enforce the contracts

3. Emulating integrations during development

©2016 Apigee. All Rights Reserved.

4

Consumer Vehicle websites

Web

sites

& a

pps

Inte

grat

ions

Lots of websites

5 ©2016 Apigee. All Rights Reserved.

Brand 1 Brand 2 Brand 3 Brand 4 Brand 5

UK FR DE BR TH AU NZ US IE CA RU ZA CN NL JP PL ES etc

Development QA UAT Staging Production Hot Fix

2000 sites



3300 "integrations

Web

site

s In

tegr

atio

ns

Lots of websites







3300 "integrations

2000 sites Using Apigee

Treated as Apigee apps

Web

site

s In

tegr

atio

ns

But its really a CMS, commerce websites & apps







Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Brand 1 / UK Integration

implementation

Brand 1 / US Integration

implementation

Brand 2 / CN Integration

implementation

Switch based on "brand/market key

•  In Apigee we chose this organisation, but still the base app requires up to a 100 settings.

•  We needed a more sophisticated way to maintain these Custom Attributes

Implementation in Apigee


Dev Testing Production Organisation

Brand 2

UK FR DE BR TH

UAT Hot Fix Staging Environment

Developers

DeveloperApps

Brand 3 Brand 1

Apigee terminology

•  We’ve implemented the same hierarchy in properties files (under Git)

•  We derive the App’s Custom Attributes through: –  Inheritance from the parent level "

è gives us defaults –  Use of Templates "

è allows us to enforce integrity: •  Must have attributes •  Option attributes •  Deny any other attributes

–  Assurance using Unit Tests

Implementation in Apigee


Dev Testing Production Organisation

Brand 2

UK FR DE BR TH

UAT Hot Fix Staging Environment

Developers

DeveloperApps

Brand 3 Brand 1

Apigee terminology

Agenda


Context




•  10 APIs –  3 major versions of some –  more new ones to come

•  57 REST resources, always JSON •  Any major change implies work for

consumers & producers

Consumer stakeholders: •  Developers of core websites •  Developers of affiliated websites •  Developers of mobile apps Producer stakeholders •  Brand/Country IT teams to which the APIs

connect

API Governance

11

APIs Stakeholders

©2016 Apigee. All Rights Reserved.

API Governance


New/changed "Feature

API "Change Request

Review by API "Governance Board

Publish"Contract

MuleSoft’s API Designer

API Governance



API "Change Request

Review by API "Governance Board

Publish"Contract

•  RAML •  JSON with "

JSON Schema

API Governance



API "Change Request Review Publish


JSON Schema •  All reviewed in Git "

(Atlassian Stash)

API Governance



API "Change Request Review Publish


JSON Schema •  All reviewed in Git "

(Atlassian Stash) •  Unit Test ensures "

RAML and JSON "example matches "schema

API Governance


•  When the API implementation is built, automated tests (using Cucumber) not only assert the values, but ensure that the responses adhere to the contract

Inte

grat

ions

API contract

Agenda


Context




Web

site

s In

tegr

atio

ns

Emulating integrations







Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Brand 1 / UK Integration

implementation

Brand 1 / US Integration

implementation

Brand 2 / CN Integration

implementation Mock Service



Component

Component

Component

Component

Component

Component

Mock Service

•  Design goal: no programming required to use, just construct JSON and name the file appropriately

•  For GET requests Mock Service converts a Resource into file path and returns that for the request. If not found, a 404

•  POST writes a file to the file server, so a test engineer can validate the POST

•  PUT, PATCH, DELETE & HEAD all supported

•  Custom handlers used for special cases, e.g. login



Component

Component

Component

Component

Component

Component

Mock Service

Thank you

Andrew LilesGlobal Technical Partner, DigitasLBiGeorge TaylorTechnical Architect, DigitasLBi

Technology

How to scale 1000s of API Integrations and not lose your mind