Embed Size (px)
DESCRIPTION
As applications grow from single Rails applications to complex systems with multiple, interacting applications & web services, testing becomes more and more difficult. While we can test each application independently, we need to be able to test the full stack. This presentation shows methods, tools and tipps & tricks from testing such a complex application.
Citation preview
Testing of distributed applications
Rails Way Con 2011Jens-Christian Fischer
with Michael Mahlberg & Keith Bingman
@jcfischer
Freitag, 3. Juni 2011
Testing of distributed applications
Rails Way Con 2011Jens-Christian Fischer
with Michael Mahlberg & Keith Bingman
@jcfischer
complex
Freitag, 3. Juni 2011
Agenda
•Background
•What & Why & How of Testing
•Discover the state of your apps
•Make testable apps
Freitag, 3. Juni 2011
why?Freitag, 3. Juni 2011
Freitag, 3. Juni 2011
we move money
Freitag, 3. Juni 2011
What is behind?
1 Rails app
5 Sinatra apps
1 Ruby daemon
1 VOIP Server
1 Speech Synthesizer
1 SQL Database (Postgres)
1 NoSQL Server (Redis) (with Resque)
Freitag, 3. Juni 2011
Architecture
Freitag, 3. Juni 2011
Volcker Trichet
Duisenberg Keynes Greenspan
SmithRedis
Postgres 9
Freeswitch
Acapela
VAAS
Customer Web Merchant WebWidget Customer Voice
Freitag, 3. Juni 2011
Volcker Trichet
Duisenberg Keynes Greenspan
SmithRedis
Postgres 9
Freeswitch
Acapela
VAAS
Customer Web Merchant WebWidget Customer Voice
Freitag, 3. Juni 2011
Volcker Trichet
Duisenberg Keynes Greenspan
SmithRedis
Postgres 9
Freeswitch
Acapela
VAAS
Customer Web Merchant WebWidget Customer Voice
Voice
Front
Back
Database
Freitag, 3. Juni 2011
SmithKeynesGreenspan
Trichet Volcker Duisenberg
Freitag, 3. Juni 2011
Freitag, 3. Juni 2011
Testing
Freitag, 3. Juni 2011
The old days
•Simple web apps
•Click through testing
Freitag, 3. Juni 2011
Royce, W. (1970). Managing the development of large software systems. Proceedings of IEEE Wescon.
Testin
g
Freitag, 3. Juni 2011
Freitag, 3. Juni 2011
Freitag, 3. Juni 2011
Freitag, 3. Juni 2011
In Rails—
everything is better
Freitag, 3. Juni 2011
Automated Tests
Unit
Integration
Frontend tests
Freitag, 3. Juni 2011
Complexity
•Modern applications consist of many pieces
•App Servers, DB, NoSQL, WebServices, Telephony, Speech Synthesizers
•HTTP, Line Protocols, JSON, XML, Corba, RMI
Freitag, 3. Juni 2011
What to testRobert „UncleBob“ Martin:
•Unit Tests
•Component Tests
• Integration Tests
•System Test
•Manual Tests
http://stackoverflow.com/questions/2965483
Freitag, 3. Juni 2011
Unit Tests
•Close to 100% code coverage
•Test Driven Development
•Written by Programmers
Freitag, 3. Juni 2011
Component Tests
•Test ~50% of the System
•Cucumber / Selenium / Fitnesse
•Written by QA & Analysts
•Happy Path & some unhappy paths
Freitag, 3. Juni 2011
Integration Tests
•Cover ~20% of system
•Test assembly of components
•Cucumber / Selenium / Fitnesse
•Written by Architects
Freitag, 3. Juni 2011
System Tests
•Cover 10% of the system
•Test the whole system
•Cucumber / Selenium / Fitnesse
•Written by Architects
Freitag, 3. Juni 2011
Manual Tests
•Exploratory Manual Tests
•Manual but not scripted
•Human ingenuity & creativity
Bach, J. (n.d.). Exploratory testing explained. Online: http://www. satisfice. com/articles/et-article. pdf.
Freitag, 3. Juni 2011
Where do we test?
Freitag, 3. Juni 2011
Testing the edges
Freitag, 3. Juni 2011
Volcker Trichet
Duisenberg Keynes Greenspan
SmithRedis
Postgres 9
Freeswitch
Acapela
VAAS
Customer Web Merchant WebWidget Customer Voice
Freitag, 3. Juni 2011
Volcker Trichet
Duisenberg Keynes Greenspan
SmithRedis
Postgres 9
Freeswitch
Acapela
VAAS
Customer Web Merchant WebWidget Customer Voice
Freitag, 3. Juni 2011
How do we test?
Freitag, 3. Juni 2011
http://blog.objectmentor.com/articles/2009/09/29/ruining-your-test-automation-strategy
Freitag, 3. Juni 2011
Tools of the trade
Selenium
Cucumber
Steak
capybara
webkit
zombie
Freitag, 3. Juni 2011
only cover a minimal part
the web frontend, but our apps are so much
more
Freitag, 3. Juni 2011
Decoupling
Systems diverge
APIs diverge
Freitag, 3. Juni 2011
Web Services
Many services outside our control
Freitag, 3. Juni 2011
Freitag, 3. Juni 2011
Synchronizing
•Tests require complex state across several systems
•Tests involve complex interactions between systems
Freitag, 3. Juni 2011
Questions
Freitag, 3. Juni 2011
Questions to ask
•Development
•Testing
•Production
Freitag, 3. Juni 2011
Development
Is the system I‘m building correct?
Freitag, 3. Juni 2011
Test
Do all the pieces of the system work
together?
Freitag, 3. Juni 2011
Production
Are all parts of my system working?
Freitag, 3. Juni 2011
Development
•Test Driven Development only takes you so far
•Test a single component
•Unit tests in multiple components work, but don‘t „cross the chasm“
Freitag, 3. Juni 2011
Development
•Setting up a complete system for development can be taxing
• pow to the rescue http://pow.cx/
•Foremanhttps://github.com/ddollar/foreman
Freitag, 3. Juni 2011
Testing
• Integration tests need to stress complete application stack
•Does your CI Server handle this?
Freitag, 3. Juni 2011
Outside your app
Freitag, 3. Juni 2011
Volcker Trichet
Duisenberg Keynes Greenspan
SmithRedis
Postgres 9
Freeswitch
Acapela
VAAS
Customer Web Merchant WebWidget Customer Voice
Freitag, 3. Juni 2011
Volcker Trichet
Duisenberg Keynes Greenspan
SmithRedis
Postgres 9
Freeswitch
Acapela
VAAS
Customer Web Merchant WebWidget Customer Voice
Freitag, 3. Juni 2011
Write Mock Services
• If you don‘t control it, mock it
• „Rails Enterprise ready ™ “
Freitag, 3. Juni 2011
Mock Services
•Allows you to code against a „real“ specification
•Allows you to code against failure of said service
•Gives you a stick to beat up the developers of said service („Rails-Enterprise ready“ ™ )
Freitag, 3. Juni 2011
Making it work
Freitag, 3. Juni 2011
Version Control
Repository setup
One Big repository?
One repository for each component?
Tagging / Syncing
Freitag, 3. Juni 2011
Development Servers
Webrick doesn‘t scale
but:
it‘s „easy“ for Rack based apps
pow by 37 Signals
(Mac Only)
Freitag, 3. Juni 2011
Configuration
Configuring half a dozen applications
Again: Synchronisation
Inject configuration (Example Picky)
http://florianhanke.com/picky/
Freitag, 3. Juni 2011
Production
Is my system working correctly?
Probes
•DTrace (OSX, Solaris)
•Monitoring (Scout, NewRelic)
•Your own probes
Freitag, 3. Juni 2011
Freitag, 3. Juni 2011
Freitag, 3. Juni 2011
Car industry
If car manufacturers would build engines the way enterprise IT systems are built, you‘d need to remove the engine from the car and disassemble it completely to find a fault
— @MMahlberg, 2011
Freitag, 3. Juni 2011
OBD• 1996: The OBD-II specification is made mandatory for
all cars sold in the United States.
• 2001: The European Union makes EOBD mandatory for all gasoline (petrol) vehicles sold in the European Union, starting in MY2001 (see European emission standards Directive 98/69/EC [1]).
• 2008: All cars sold in the United States are required to use the ISO 15765-4 [2] signaling standard (a variant of the Controller Area Network (CAN) bus).
http://en.wikipedia.org/wiki/On-board_diagnostics
Freitag, 3. Juni 2011
Solved Problem
Freitag, 3. Juni 2011
Freitag, 3. Juni 2011
http://the-adam.com/adam/rantrave/ibm_360.jpgFreitag, 3. Juni 2011
Diagnosis Channel
•Known from Car / Planes / Mainframes
•Red lights on your dashboard
•Complete diagnosis of system possible
•Complete remote control possible
Freitag, 3. Juni 2011
Rolls Royce
• InFlight Diagnosis of engines remotely
Freitag, 3. Juni 2011
Dashboard
•Look into state of application
•Sanity Checker
Freitag, 3. Juni 2011
Dashboarding
•Extract relevant data from your system
•How many req / s
•How many logins
•Delta / Trends
Freitag, 3. Juni 2011
SanityChecker
•API that allows you to extract the state of your app
•DB there?
•All neighbouring components reachable?
Freitag, 3. Juni 2011
Chaos Monkey
•Random failures
•Better reliability
Freitag, 3. Juni 2011
Build testable apps
•Expose your internals (through APIs)
•Use SOLID design
• Single Responsibility Principle
• Open Closed
• Liskov Substitution Principle
• Inversion of Control
• Dependency Injection
Freitag, 3. Juni 2011
Testable apps
•Obey the abstraction layers
• keep each method at it‘s own level of abstraction
Freitag, 3. Juni 2011
Build probe-able apps
•Measure APIs
$ rails console production
Freitag, 3. Juni 2011
Build controllable apps
•Call functionality of the app
•Create new user
•Create transactions
•Delete stuff
•Do stuff
• ...
Freitag, 3. Juni 2011
Test user / Magic
• In „Enterprise“ systems:
•Test Company
•Test User
•Test Accounts
•Are those in production system?
•Do everything - or not quite?
Freitag, 3. Juni 2011
Use production data in development / test
Freitag, 3. Juni 2011
Wrapping up
Freitag, 3. Juni 2011
What & Why & How of Testing
Discover the state of your apps
Make testable apps
Freitag, 3. Juni 2011
Thank you!
Freitag, 3. Juni 2011
Credits
•http://www.flickr.com/photos/myoldpostcards/4020087797/
•http://www.flickr.com/photos/benncapon/390585495/
• iStockphoto
Freitag, 3. Juni 2011
License
http://creativecommons.org/licenses/by-nc-sa/2.5/ch/Freitag, 3. Juni 2011
