Taming the Testing Beast Tuesday, October 23rd 2012
Daniel Doubrovkine (dB.) @dblockdotorg #agiledc 2012
http://art.sy/artwork/francisco-de-goya-saturn-devouring-one-of-his-sons-from-the-series-of-black-paintings-dot
State of Testing: Scary!
Test Automation @
Independent Teams
Hardware Platforms
Vendors
3× automated voting user acceptance tests QA
Test Automation @
Test Code vs. Code 1124×
http://www.sqlite.org/testing.html
Test Automation @ Your Company
Dev : Test 1:1
some automation some people test some software often doesn’t install
QA
Test Automation @ My Company
Testers Formalities 0
100% automation continuous everything 8 developers 8 hours of sleep / day
QA
http://art.sy/artwork/conrad-shawcross-chord
It’s [Relatively] Big
4678 Automated Tests
100% user scenario coverage ~ 800 tests
1h to Run
http://art.sy/artwork/liu-di-animal-regulation-no-2
Technology and Tools
RSpec BDD : it “does X”
Mongo Data Models : ODM with “real” data
Ruby API : HTTP request – response
Integration : Capybara
Technology and Tools : Mongo ODM + Fabricators
context “spline” do
let(:spline) { Fabricate :spline }
it “reticulate! updates reticulated count” do
expect {
spline.reticulate!
}.to change(Spline.reticulated, :count).by(1)
end
end
Technology and Tools : Ruby API context “spline” do
it “create” do
expect {
post “/api/spline?name=test”
response.status.should == 201
}.to change(Spline, :count).by(1)
end
end
Technology and Tools : Integration context “spline” do
let(:spline) { Fabricate :spline }
it “displays a spline” do
visit “/spline/#{spline.id}”
page.should have_css “.spline[id=#{spline.id}]”
end
end
Mixed Bag
High Confidence
Size
Duration
http://art.sy/artwork/liu-di-animal-regulation-no-4
It Quickly Became a Big Problem
http://art.sy/artwork/liu-di-animal-regulation-no-6
Intermittent Failures
Size
Duration
Run Tests in Parallel
Multiple Test Suites
http://artsy.github.com/blog/2012/10/09/how-to-run-rspec-test-suites-in-parallel-with-jenkins-ci-build-flow/
Understand Disable Failing Tests
One Example at a Time
No Hacks Allowed
Wait for Explicit Events
Combine Techniques
Track Failures
http://artsy.github.com/blog/2012/02/03/reliably-testing-asynchronous-ui-w-slash-rspec-and-capybara/
http://www.friendshipismagic.org/forum/members/captain-falcon/albums/miscellaneous/9452-wat/
Test Anxiety High Expectations
Fear of Disappointment
Lack of Control
Inability to Change
http://www.sodahead.com/fun/is-your-school-strict/question-3158781/
Test Automation is Not a ______ Problem Not a Management Problem
Not a Money Problem
Not an External Problem
Not a Consultant’s Problem
Test Automation is a Team Culture Problem Remove Those that Talk About Testing
Hire From Open-Source
Remove the Safety Net
Daniel Doubrovkine (dB.)
@dblockdotorg