Upload
michael-barker
View
196
Download
1
Tags:
Embed Size (px)
DESCRIPTION
Test Automation presentation for Auckland Software Craftmanship group
Citation preview
7 Years of Test Automation
!
@mikeb2701
Assumptions
• Testing is important
• Automating testing is important
Types of tests• Static Analysis
• Unit Testing
• Integration Testing
• Acceptance Testing!
• Performance Testing
• Testing in Live
• Exploratory Testing
The acceptance test suite as a whole both verifies that the application delivers the business value and guards against regressions or defects that break preexisting functions of the application.
— Humble, Farley, Continuous Delivery, 2010, Chapter 8.
An acceptance test may only drive and assert behaviour in the system through interfaces available to the system’s users.
Writing Acceptance Tests
@RunWith(AcceptanceTestRunner.class)@Multithreaded@AcceptanceTest(id = 29, flavour = {Flavour.PLACE_ORDER, Flavour.API})public class PlaceOrderAcceptanceTest extends DslTestCase {! @Before public void beforeEveryTest() {! adminAPI.createInstrument("name: instrument”, "orderQuantityIncrement: 0.1”,…); registrationAPI.createUser("user", "balance: 100000"); mtfMemberAPI.createMtfMemberAndFixSession("marketMaker"); publicAPI.login("user"); mtfFixAPI.login(“marketMaker"); } // end! @Test public void shouldRejectOrderIfNotLoggedIn() { publicAPI.logout(); publicAPI.placeOrder(“instrument", "side: buy", "quantity: 1.2”, "timeInForce: ImmediateOrCancel”, "unauthorised: true");!
@RunWith(AcceptanceTestRunner.class)@Multithreaded@AcceptanceTest(id = 29, flavour = {Flavour.PLACE_ORDER, Flavour.API})public class PlaceOrderAcceptanceTest extends DslTestCase {! @Before public void beforeEveryTest() {! adminAPI.createInstrument("name: instrument”, "orderQuantityIncrement: 0.1”,…); registrationAPI.createUser("user", "balance: 100000"); mtfMemberAPI.createMtfMemberAndFixSession("marketMaker"); publicAPI.login("user"); mtfFixAPI.login(“marketMaker"); } // end! @Test public void shouldRejectOrderIfNotLoggedIn() { publicAPI.logout(); publicAPI.placeOrder(“instrument", "side: buy", "quantity: 1.2”, "timeInForce: ImmediateOrCancel”, "unauthorised: true");!
@RunWith(AcceptanceTestRunner.class)@Multithreaded@AcceptanceTest(id = 29, flavour = {Flavour.PLACE_ORDER, Flavour.API})public class PlaceOrderAcceptanceTest extends DslTestCase {! @Before public void beforeEveryTest() {! adminAPI.createInstrument("name: instrument”, "orderQuantityIncrement: 0.1”,…); registrationAPI.createUser("user", "balance: 100000"); mtfMemberAPI.createMtfMemberAndFixSession("marketMaker"); publicAPI.login("user"); mtfFixAPI.login(“marketMaker"); } // end! @Test public void shouldRejectOrderIfNotLoggedIn() { publicAPI.logout(); publicAPI.placeOrder(“instrument", "side: buy", "quantity: 1.2”, "timeInForce: ImmediateOrCancel”, "unauthorised: true");!
@RunWith(AcceptanceTestRunner.class)@Multithreaded@AcceptanceTest(id = 29, flavour = {Flavour.PLACE_ORDER, Flavour.API})public class PlaceOrderAcceptanceTest extends DslTestCase {! @Before public void beforeEveryTest() {! adminAPI.createInstrument("name: instrument”, "orderQuantityIncrement: 0.1”,…); registrationAPI.createUser("user", "balance: 100000"); mtfMemberAPI.createMtfMemberAndFixSession("marketMaker"); publicAPI.login("user"); mtfFixAPI.login(“marketMaker"); } // end! @Test public void shouldRejectOrderIfNotLoggedIn() { publicAPI.logout(); publicAPI.placeOrder(“instrument", "side: buy", "quantity: 1.2”, "timeInForce: ImmediateOrCancel”, "unauthorised: true");!
@RunWith(AcceptanceTestRunner.class)@Multithreaded@AcceptanceTest(id = 29, flavour = {Flavour.PLACE_ORDER, Flavour.API})public class PlaceOrderAcceptanceTest extends DslTestCase {! @Before public void beforeEveryTest() {! adminAPI.createInstrument("name: instrument”, "orderQuantityIncrement: 0.1”,…); registrationAPI.createUser("user", "balance: 100000"); mtfMemberAPI.createMtfMemberAndFixSession("marketMaker"); publicAPI.login("user"); mtfFixAPI.login(“marketMaker"); } // end! @Test public void shouldRejectOrderIfNotLoggedIn() { publicAPI.logout(); publicAPI.placeOrder(“instrument", "side: buy", "quantity: 1.2”, "timeInForce: ImmediateOrCancel”, "unauthorised: true");!
DSL
PublicAPI
TradingUI
AdminAPI
Test
MtfMemberAPI
LoginPanelDriver
Selenium
BalancesPanelDriver
InstrumentPanelDriver
JavaAPIDriver
AdminAPIDriver
MtfFixDriver
XML/HTTP
QuickFIX
Drivers
Time Travel
ServiceService
ServiceService
Service
TimeMachine Test
Time Travel To
Time Delta Get CurrentTime
@RunWith(AcceptanceTestRunner.class)@AcceptanceTest(id = 5194, flavour = {Flavour.PLACE_ORDER, Flavour.API})@Multithreaded(tardis = true)public class CancelAndReplaceAcceptanceTest extends DslTestCase{ @Before public void setUp() { dsl.forceToUtcTimeZone(); dsl.enableTimeMachine();! dsl.createTimePoint("name: origin”, "value: <next weekday>"); dsl.createTimePoint("name: marketOpen", "value: origin plus 1 weekdayCalendarOpenOffset”);! dsl.waitUntil("marketOpen");
Pro-Tip:
Isolate as much as possible
Acceptance Testing (Running)
RTAParallel
Sequential
Sequential
Sequential
RTA
RTA
RTA
Scotty RomeroServer
Jenkins
Sources of Intermittency
• Bad code (under test)
• Bad test
• Environmental
Just Fix It!
Acceptance Test DB
Wrap-Up
•Write them well •Run them often •Keep them green
Thank you
Q&A