Taming the Testing Beast Tuesday, October 23rd 2012

Daniel Doubrovkine (dB.) @dblockdotorg #agiledc 2012

http://art.sy/artwork/francisco-de-goya-saturn-devouring-one-of-his-sons-from-the-series-of-black-paintings-dot

State of Testing: Scary!

Test Automation @

Independent Teams

Hardware Platforms

Vendors

3× automated voting user acceptance tests QA

Test Automation @

Test Code vs. Code 1124×

http://www.sqlite.org/testing.html

Test Automation @ Your Company

Dev : Test 1:1

some automation some people test some software often doesn’t install

QA

Test Automation @ My Company

Testers Formalities 0

100% automation continuous everything 8 developers 8 hours of sleep / day

QA

http://art.sy/artwork/conrad-shawcross-chord

It’s [Relatively] Big

4678 Automated Tests

100% user scenario coverage ~ 800 tests

1h to Run

http://art.sy/artwork/liu-di-animal-regulation-no-2

Technology and Tools

RSpec BDD : it “does X”

Mongo Data Models : ODM with “real” data

Ruby API : HTTP request – response

Integration : Capybara

Technology and Tools : Mongo ODM + Fabricators

context “spline” do

let(:spline) { Fabricate :spline }

it “reticulate! updates reticulated count” do

expect {

spline.reticulate!

}.to change(Spline.reticulated, :count).by(1)

end

end

Technology and Tools : Ruby API context “spline” do

it “create” do

expect {

post “/api/spline?name=test”

response.status.should == 201

}.to change(Spline, :count).by(1)

end

end

Technology and Tools : Integration context “spline” do

let(:spline) { Fabricate :spline }

it “displays a spline” do

visit “/spline/#{spline.id}”

page.should have_css “.spline[id=#{spline.id}]”

end

end

Mixed Bag

High Confidence

Size

Duration

http://art.sy/artwork/liu-di-animal-regulation-no-4

It Quickly Became a Big Problem

http://art.sy/artwork/liu-di-animal-regulation-no-6

Intermittent Failures

Size

Duration

Break Things Up

Multiple Applications

Run Tests in Parallel

Multiple Test Suites

http://artsy.github.com/blog/2012/10/09/how-to-run-rspec-test-suites-in-parallel-with-jenkins-ci-build-flow/

Understand Disable Failing Tests

One Example at a Time

No Hacks Allowed

Wait for Explicit Events

Combine Techniques

Track Failures

http://artsy.github.com/blog/2012/02/03/reliably-testing-asynchronous-ui-w-slash-rspec-and-capybara/

http://www.friendshipismagic.org/forum/members/captain-falcon/albums/miscellaneous/9452-wat/

Test Anxiety High Expectations

Fear of Disappointment

Lack of Control

Inability to Change

http://www.sodahead.com/fun/is-your-school-strict/question-3158781/

Test Automation is Not a ______ Problem Not a Management Problem

Not a Money Problem

Not an External Problem

Not a Consultant’s Problem

Test Automation is a Team Culture Problem Remove Those that Talk About Testing

Hire From Open-Source

Remove the Safety Net

Daniel Doubrovkine (dB.)

@dblockdotorg