Lightening Talk: Auckland Continuous Delivery Meetup [Sept 2014] - Pact: Consumer-Driven Contract Testing (Integrated Tests Are A Scam)

Commercial in Confidence | Assurity Consulting Limited 2014

Pact – Consumer Driven Contract Testing“Integrated Tests Are A Scam”

Pete Sellars – Technical Consultant


Challenge…


Who Writes Integration Test?


Integrated Tests Are A Scam

Integrated tests are a scam—a self-replicating virus that threatens to infect your code base, your project, and your team with endless pain and suffering.


Code Whisperer


Challenge Accepted


Timeline - 2005


Contract Tests – Design By Contract


Timeline - 2006


Contracts Overview

Provider• Closed & Complete• Singular & Authorative• Bounded Stability &

Immutability

Consumer• Open & Incomplete• Multiple & • Non-Authorative• Bounded Stability &

Immutability

Consumer-Driven• Closed & Complete• Singular & • Non-Authorative• Bounded Stability &

Immutability

Consumer-Driven Contracts, A Service Evolution Pattern: Ian Robinson


Consumer-Driven Contract Benefits

…they focus the specification and delivery of service

functionality around key business value drivers.

…give us the fine-grained insight and rapid feedback we require to plan changes and

assess their impact on applications currently in

production

Consumer-Driven Contracts, A Service Evolution Pattern: Ian Robinson


Timeline - 2009


Why Are Integrated Tests A Scam?


Why Are Integrated Tests A Scam

Integrated Tests Are A Scam (2009): J.B. Rainsberger


How to beat the Scam….



Timeline - 2010


Why Are Integrated Tests A Scam?



Why Are Integrated Tests A Scam…

Consider 10 layers with 3 potential branch points

at each layer. Number of code paths: 310 > 59,000.

How about 4 branch points per layer? 410 >

1,000,000. How about 3 branch and 12 layers?

312 > 530,000.

How about 3 branch and 12 layers? 312 > 530,000.

Even if one of your 12 layers has a single code

path, 311 > 177,000.

Even if your 10-layer application has only an

average of 3.5 code paths per layer, 3.510>

275,0001.


Do you feel lucky? Do you?

http://blog.thecodewhisperer.com/2010/10/16/integrated-tests-are-a-scam/#fn6d7a96adc995137445819fd95bd31a74367d9f83


How to beat the Scam…



Timeline - 2010


Sharing Tests!! …

To reduce the chances of unexpected breaks in integration contracts, it's useful to move to a Consumer Driven Contracts

approach. You can facilitate this by letting the supplier team have copies of your

integration contract tests so they can run them as part of their build pipeline.

Integrtation Contract Tests (2010): Martin Fowler

http://www.martinfowler.com/articles/consumerDrivenContracts.html


2010: Writing Consumer-Driven Contracts?


Micro Services


Micro Services


Timeline - 2013


Pact

Testing Interactions with Web Services without Integration Tests In Ruby:

REA Engineering Blog


Pact

Consumer sets up expectations on a mock service provider. Mock

service returns the expected responses.

The requests, and their expected responses, are then written to a "pact“

file

The requests in the pact file are later replayed

against the provider. Actual responses are checked to make sure they match the

expected responses.



Pact

Testing with Pact: Pact Wiki


Pact

Simplifying Micro-Services Testing with Pacts: Ron Holshausen


Pact Features

Pact Wiki


Pact

We now have fast unit-like tests on each side of the

integration point instead of tedious integration tests.

Testing Interactions with Web Services without Integration Tests In Ruby:

REA Engineering Blog


Timeline - 2013


Why Integration Test Are Still A Scam…

Integrated Tests Are A Scam

Doesn’t Provide

Coverage Expected

Encourage Sloppy Coding

Cumulative # Required;

Slow & Brittle


Timeline - 2014


Trial Pact & Pacto

Consumer-Driven Contracts are a testing approach to help service interfaces evolve

with confidence without unknowingly breaking consumers. The similarly named Pact

and Pacto are two new open-source tools which allow testing interactions between

service providers and consumers in isolation against a contract. Both have grown out of

projects which are building RESTful microservices and show great promise.

ThoughtWorks Technology Radar: July 2014

https://github.com/realestate-com-au/pact

http://thoughtworks.github.io/pacto


Do Things Differently?


References

• Blog Posts> Contract Testing, J.B. Rainsberger: http://blog.thecodewhisperer.com/2005/03/02/in-brief-contract-tests/> Integrated Tests Are A Scam, J.B. Rainsberger: http://blog.thecodewhisperer.com/2010/10/16/integrated-tests-are-a-scam/> IntegrationContractTests, Martin Fowler: http://martinfowler.com/bliki/IntegrationContractTest.html> Testing Interactions with WebServices without Integration Tests in Ruby, REA Techblog: http://techblog.realestate.com.au/testing-interactions-with-web-services-without-integration-tests-in-ruby/> Simplifying Micro-Service testing with Pacts, Ron Holshausen: http://dius.com.au/2014/05/19/simplifying-micro-service-testing-with-pacts/

• Articles> Consumer-Driven Contracts – A Service Evolution Pattern, Ian Robinson: http://www.martinfowler.com/articles/consumerDrivenContracts.html> Thoughtworks Technology Radar: http://www.thoughtworks.com/radar> Design By Contract: http://c2.com/cgi/wiki?DesignByContract

• Presentations> Agile 2009 - Integration Tests Are A Scam, J.B. Rainsberger: http://www.infoq.com/presentations/integration-tests-scam> Integration Tests Are A Scam (Slides 2010), J.B. Rainsberger: http://www.slideshare.net/agileee/notes-5407051> DevConFu 2013 – Integrated Tests Are A Scam, J.B. Rainsberger: http://vimeo.com/80533536> Micro-Service Architecture, Fred George: http://www.slideshare.net/fredgeorge/micro-service-architecure> Practical Microserices (YOW 2013), Sam Newman: http://www.slideshare.net/spnewman/practical-microservices-yow-2013> Practical Considerations for Micro-Service Architectures, Sam Newman: http://yow.eventer.com/yow-2013-1080/practical-considerations-for-microservice-architectures-by-sam-newman-1389

• Tools> Pact: https://github.com/realestate-com-au/pact/wiki

− Provider State Example: https://github.com/realestate-com-au/pact/blob/master/example/zoo-app/spec/service_providers/animal_service_client_spec.rb− Generated Documentation Example: https://github.com/realestate-com-au/pact/blob/master/example/zoo-app/doc/pacts/markdown/Zoo%20App%20-%20Animal%20Service.md

> Pacto: http://thoughtworks.github.io/pacto/

http://blog.thecodewhisperer.com/2005/03/02/in-brief-contract-tests/



http://blog.thecodewhisperer.com/2010/10/16/integrated-tests-are-a-scam/



http://martinfowler.com/bliki/IntegrationContractTest.html

http://martinfowler.com/bliki/IntegrationContractTest.html

http://techblog.realestate.com.au/testing-interactions-with-web-services-without-integration-tests-in-ruby/

http://techblog.realestate.com.au/testing-interactions-with-web-services-without-integration-tests-in-ruby/

http://dius.com.au/2014/05/19/simplifying-micro-service-testing-with-pacts/

http://dius.com.au/2014/05/19/simplifying-micro-service-testing-with-pacts/



http://www.thoughtworks.com/radar

http://www.thoughtworks.com/radar

http://c2.com/cgi/wiki?DesignByContract

http://c2.com/cgi/wiki?DesignByContract

http://www.infoq.com/presentations/integration-tests-scam

http://www.infoq.com/presentations/integration-tests-scam

http://www.slideshare.net/agileee/notes-5407051

http://www.slideshare.net/agileee/notes-5407051

http://vimeo.com/80533536

http://vimeo.com/80533536

http://www.slideshare.net/fredgeorge/micro-service-architecure

http://www.slideshare.net/fredgeorge/micro-service-architecure

http://www.slideshare.net/spnewman/practical-microservices-yow-2013

http://www.slideshare.net/spnewman/practical-microservices-yow-2013

http://yow.eventer.com/yow-2013-1080/practical-considerations-for-microservice-architectures-by-sam-newman-1389



https://github.com/realestate-com-au/pact/wiki

https://github.com/realestate-com-au/pact/wiki

https://github.com/realestate-com-au/pact/blob/master/example/zoo-app/spec/service_providers/animal_service_client_spec.rb

https://github.com/realestate-com-au/pact/blob/master/example/zoo-app/spec/service_providers/animal_service_client_spec.rb

https://github.com/realestate-com-au/pact/blob/master/example/zoo-app/doc/pacts/markdown/Zoo%20App%20-%20Animal%20Service.md

https://github.com/realestate-com-au/pact/blob/master/example/zoo-app/doc/pacts/markdown/Zoo%20App%20-%20Animal%20Service.md

http://thoughtworks.github.io/pacto/

http://thoughtworks.github.io/pacto/