Backwards CompatibilityThe Science and the Art

Agenda

Where does backwards compatibility matter, and why?

Ways a ReST service can evolve. Strategies for evolving a service in a backwards

compatible fashion. Database backwards compatibility. Best practices.

So you want to deploy a new version

The new version has a different API than the old one. Typically, web service Also: java library Also: schema design

Just upgrade the clients?

May not be able to upgrade all clients at the same time.

Creates tightly coupled releases. Cannot do partial rollbacks.

Not possible for zero-downtime deploys.

Versioning?

/myService/v1 → /myService/v2

Backwards Compatibility

A change to a service is backwards compatible if old clients can continue to call the new service without problems.

Wire Compatibility

Data format not changed in ways that cause parsing errors when old clients make calls.

Semantic Compatibility

When old clients make calls, stuff still works right.

This is the one you want!

Compatibility and deploys

A backwards compatible service change can be rolled out without problems. Zero downtime!

Rolling back other services does not force rolling back your service.

Compatibility and deploys

After (and only after) a service has rolled, can clients upgrade. Or perhaps just change a config flag

If the service needs to roll back, all clients must roll back first.

Ways a ReST service can evolve

Changes to the API For web services, this includes URIs, representations (i.e.

JSON), and HTTP methods Changes to behavior Most evolutions involve both kinds of change

Evolution Strategies

Additional field in request Additional field in response New values in existing request fields Changing field types

Additional field in request

Make it optional (at least at first), with a default value matching old behavior @RequestMapping(value=”/petstore/pets”, method=GET) public getPets() { return petService.getAllPets(); }

@RequestMapping(value=”/petstore/pets”, method=GET) public getPets( @RequestParam(value=“species”, required=false)

Species species) { return species == null ? petService.getAllPets() : petService.getAllPets(species); }

Additional field in request

Can also provide a default value: @RequestMapping(value=”/petstore/pets”, method=GET) public getPets(

@RequestParam(“species”) species)

@RequestMapping(value=”/petstore/pets”, method=GET) public getPets(

@RequestParam(“species”) Species species, @RequestParam( value=”mustBeAdoptable”, defaultValue=”false”)

boolean adoptable)

Additional field in response

Most serialization libraries (including Jackson) can ignore unmapped fields.

New clients can use the field value, while old ones ignore it.

Ensure that the new field does not change semantics of old fields!

New values in a request field

Pretty straightforward, provided that the meaning of pre-existing values does not change.

Example – add a new value to an enum.

public enum Species { DOG, FERRET, CAT }

Narrowing set of allowed request values

Example – No longer allowing searches for Ferrets.

Rarely happens When it does, all clients must upgrade first

before the service can safely rely on the narrowed set of values

Changing type

Of request field: Make sure the old type can be parsed into the new type –

e.g. Integer to Double Of response field:

Make sure new type can be parsed as old value – e.g. Double to Integer

Alternative: add new field with new type, have the old field “forward”

Changing type – forwarding field

{“age”: 12} { “age”: 12, “ageAsDouble”: 12.3 }

New behavior

Changing business logic Should be OK - this is why we do services in the first place!

Performing additional actions (or less actions) If this breaks contract, then a new resource may be

required New failure modes

This is very similar to new values in response fields

Database backwards compatibility

Adding columns, tables, etc is OK...ish They won't always get used! Backwards population may be needed Beware the generic update endpoint!

Removing: First update clients If you know what they are!

Triggers and views can help here.

Safely adding a column (say, “foo”)

Problem: old client calls GET, followed by PUT. Drops foo on the floor

Solution: BEFORE UPDATE trigger IF :new.foo IS NULL THEN :new.foo = :old.foo For inserts, a default value is simpler than a trigger.

Best Practices

Plan ahead for versioning changes Keep your API as narrow as possible. Document and test compatibility requirements. Automated client tests. Keep clients as up to date as possible.

Plan ahead for versioning changes

Always consider versioning issues when making any changes to existing resources. Do this up front, not as an afterthought.

Put care into initial API design. Keep DTOs as simple as possible.

It is very hard to see which fields clients are using.

Keep your API as narrow as possible

You can't break what you never provided. Splunk can show what is called, but not how

the results are used. Narrow APIs often are also better tailored. Avoid generic “update” endpoints if possible Fields are part of the API – only include what

you need!

Document and test compatibility requirements

Make it clear which client versions are expected to be able to talk to which service versions.

Verifying these requirements should be part of any service change.

Automated tests

Write automated tests which exercise the service as a given version of a client would.

Tests for version n should be written before starting version n+1, and ideally before rolling version n to production.

Run old tests against new service version to test backwards compatibility.

Copy and paste can be acceptable here.

Keep clients as up to date as possible

Once all clients have upgraded, any deprecated fields, resources, etc. can be removed from the service.

Maintaining backwards compatibility is expensive, and gets more so with multiple versions.

Cooperation is required.

What if you need to break backwards compatability?

1)Don't!2)See #13)Talk with your colleagues – find a solution.4)There is always a way, and it usually isn't that

hard.

THANK YOU.