Product Array project Testing Strategy and Best Practice

Outline

• Testing strategy of Product Array project• Introduction of the project• Best Practice in Functional Testing

• Testing Design pattern application• Introduction BDD

• BDD Conception

• BDD example

• Plan for project next phase

Why need functional tests to show the safety network

• Endeca has already implemented partially search. The project needs to keep old function work and make new functions.

• A lot of java code in JSP and JavaScript in project. The project needs to avoid old function broken by new functions.

• No integration tests for Endeca upgrade

Project Functions development process

Functional Testing strategy in Product Array project

• Selenium and HTMLUnit are selected as testing tools

• HTMLUnit: fast and rich HTML parsing API

• Selenium: real browser based web UI testing

Testing strategy of Product Array project

• Select HTMLUnit as a favor testing tool, especially for only back end generated content testing

• For richer UI with javascript code, choose Selenium as testing tool

• Don’t mix HTMLUnit and Selenium in one test case

Build long term regression test base and practice in continuous integration

Separate testing cases as three groups:

• Not dependent on product data set. Can be used in Continuous Integration server for long term regression testing.

• Dependent on data set containing special product id or category. Suitable for short term automatic testing.

• Semi-automatic test cases. Need to manually set up DNA environment, and then they can be executed, not suitable for continuous Integration.

Challenge

• As project growing, test cases expand so fast

• How to make test case easy maintainable, extendable, understandable is a challenge

• How to make test cases great cover BA’s specification

• How to make test case independent of changeable data base content

Test Design Patterns

• Fixture Setup and Teardown Pattern

• Page Object Pattern• Data Driven Test

Fixture Setup/Teardown Pattern

• The fixture setup logic is to call the appropriate methods or behavior on the System Under Test (SUT) and put them into the starting state.

• As long as our references to the fixture are held in variables that go out of scope, we can count on Garbage-Collected Teardown to do all the work for us.

Page Object pattern

• Page Object offering the "services" that a page offers rather than exposing the details and mechanics of the page.

• The public methods represent the services that the page offers

• Try not to expose the internals of the page

Data Driven pattern

• Store the information needed for tests in a data file and write an interpreter that reads the file and executes the tests.

• The benefits • One test case can repeat multiple similar tests

through navigate the test data file. • This lets behavior we are testing driven by

configuration data and not hard-coded.

Continue Integration

Function tests running on cruise

Behavior Driven Development

• Test driven development is bridging the gap between development and testing

• Behavior driven development goes further to close the gap between BA’s specification and user acceptance test case. Let test cases correctly cover the user’s requirements and designer’s purpose.

BDD new requirement

• First BDD require BA and developer and QA more tightly communication

• require BA write specification follow BDD standard

• Require test case more close specification

• The best situation:

specification is an executable specification

BDD implementation

• BDD in your preferred testing framework

• Jbehave is a java based BDD framework

• BDD great improve test code readability and maintenance

• Lift functional test code from just testing a function to illustrate the whole user story behavior

BDD practice

• Test case scenario

• GIVEN WHEN THEN specification

• For function testing side, after a few test fixture and test runner patterns adopted, the front end test case are much close to specification GIVEN WHEN THEN style

Best Practice in Functional Testing

• Design for testing

• Use test friendly ID and class name, avoid using fragile Xpath expression

• Centralize Endeca or Blue martini parameters as possible

• Use test design pattern, separate test case expression and test verification function

Best Practice in Functional Testing

• Adopt JBehave style let test case close specification and cover specification story scenario

• Use JBehave style to make test case easy maintenance

• Improve specification story scenario coverage through choosing right testing methods

Business requirement Comparison

Highs

Allow customers to limit results by color and size.

Mediums

….

Allow for sorting by:

.....

Lows

.

When customer filters by color, display product images that match the select parent color. E.G. If the user selects blue, display the colorized images that map to the parent color of blue.

All the acceptance criteria below also apply to Sale pages.

# Given

User has performed a search/refine and no ‘Refine by color’ values are applied

# When

User views the ‘Refine by Color’ option

# Then

The ‘Refine by color’ option will have values based on search/refined product array being displayed

Note: This is existing and values should remain as they work now. Only UI is changing for this.

Code style comparison between before and after migrating to BDD

@Test

public void searchCareers() throws Exception {

selenium.open(EndecaTestConstants.SEARCH_PATH_FOR_ROOT_SEARCH);

selenium.waitForPageToLoad("30000");

selenium.type("SearchString", "career");

selenium.click("EndecaSearchButton");

selenium.waitForPageToLoad("45000");

String strSpan = selenium

.getText("//html//body//table//tbody//tr[6]//td//table//tbody//tr[1]//td[2]//p[1]//span");

assertThat("EndecaSearch career page diappear", strSpan,

containsString("SAKS FIFTH AVENUE CAREERS"));

}

@Test

public void verifyAllColorAvailableForSearchItemsAfterKeywordSearch() {

//Given

pageHelper.load(ENTRY_PAGE);

//When

pageHelper.searchFor("pants");

//Then

pageHelper.verifyRefinements(BEIGE, BLACK, BLUE, BROWN, GOLD, GREEN, GREY, MULTICOLOR, NUDE, ORANGE, PINK,

PURPLE, RED, SILVER, TAN, WHITE, YELLOW);

}