28
Functional Testing with the Java Stack Test Runner

Functional Testing with the Java Stack Test Runner

Embed Size (px)

DESCRIPTION

Functional Testing with the Java Stack Test Runner. Disclaimer. This is a training not a presentation. Please be prepared to: Learn Ask questions Participate in the lab Prerequisites: Knowledge of Maven (required) Java Stack ALM Training (recommended). What we’ll cover…. - PowerPoint PPT Presentation

Citation preview

Page 1: Functional Testing with the Java Stack Test Runner

Functional Testingwith the

Java Stack Test Runner

Page 2: Functional Testing with the Java Stack Test Runner

DisclaimerThis is a training not a presentation.

Please be prepared to: Learn Ask questions Participate in the lab

Prerequisites:

Knowledge of Maven (required) Java Stack ALM Training (recommended)

Page 3: Functional Testing with the Java Stack Test Runner

What we’ll cover…In this training, we will discuss

What is functional testing?

How is it unique from other forms of testing?

What is the Java Stack Test Runner?

How does it integrate with Maven and AnthillPro?

How do I invoke Test Runner from my IDE?

Where do I find my functional test results?

Page 4: Functional Testing with the Java Stack Test Runner

What we won’t cover…Topics that we will not have time to discuss are:

Testing frameworks and APIs• i.e. TestNG, Selenium, Spock, etc.Testing methodologiesWriting testsTest case management tools

Page 5: Functional Testing with the Java Stack Test Runner

First, some background…

Page 6: Functional Testing with the Java Stack Test Runner

Stages of TestingWhat When

1. Unit Testing After code compilation

2. Integration testing After the build artifact is packaged.

3. Functional testing* After the build artifact is deployed and started.

4. Load testing After deployment to staging environment

5 End-user testing After code is “feature complete”.

* We will discuss functional testing and how it differs from unit and integration testing.

Page 7: Functional Testing with the Java Stack Test Runner

Unit Testing Scope• Narrow (single class or method)

Purpose• Validates small units of code

Responsible Party• Code developer

Failure:• Indicates code is broken• Typically fails the build

* Has direct dependencies on application code

Page 8: Functional Testing with the Java Stack Test Runner

Integration Testing Scope• Medium (spans two integrated systems)

Purpose• Validate code that integrates with another system.

(e.g. a database or web service) Responsible Party• Code developer.

Failure:• Indicates failure of systems to communicate as

expected.• Does not typically fail the build.

* Has direct dependencies on application code.

Page 9: Functional Testing with the Java Stack Test Runner

Functional Testing Scope• Broad (end-to-end feature testing)

Purpose• Validate functional requirements and/or user stories

Responsible Party• QA resource or feature developer

Failure:• Indicates failure to meet user story requirements• Results recorded in test case management and/or bug

tracker

* Has little or no direct dependencies on application code.

Page 10: Functional Testing with the Java Stack Test Runner

Functional Testingand

Application Life-cycle Management

(ALM)

Page 11: Functional Testing with the Java Stack Test Runner

ALM Build ToolsMaven

• Project definition

• Dependency management

• Plug-in configuration and management

• Build profiles

AnthillPro

• Manages build processes Creation of build artifact(s) Deployment to server

environment Functional test execution

• Maintains build life history

• Provides ability to re-execute a process on any build life

Page 12: Functional Testing with the Java Stack Test Runner

ALM Tools: Maven Life-cycles

The default Maven build life-cycle (simplified):

compile test (unit tests) package integration-test (integration tests) install (to local Maven repository) deploy (to remote Maven

repository)

Where do we plug in functional testing?

Page 13: Functional Testing with the Java Stack Test Runner

ALM Tools: Maven Life-cycles

The woes of having “one build to rule them all”:

Functional Testing ≠ Integration TestingConfuses build with server deploymentMust re-build to re-deploy or re-testComplicates build managementLess productive use of build agents

Page 14: Functional Testing with the Java Stack Test Runner

ALM Tools: Maven Life-cycles

The Java Stack ALM build life-cycle (simplified):

alm-db (run database scripts) alm-deploy (deploy application to

server) alm-test (run functional tests) alm-promote (run all previous phases)

This life-cycle is available to the artifact produced by the “alm” module of a Java Stack project.

Page 15: Functional Testing with the Java Stack Test Runner

ALM Tools: Maven Life-cycle

Advantages of a second build life-cycle: Clean separation of configuration:• One configuration for build• One configuration for deployment and/or functional

testing Separate processes:

• One process for building artifact• One process for deployment and/or functional testing

Simplified build management More productive use of build agents

Page 16: Functional Testing with the Java Stack Test Runner

ALM Tools: Test RunnerIn Stack 3.2, as part of its ALM tooling, the Java Stack introduced the “test-bundle” artifact as a part of its suite of testing plug-ins, collectively known as “Test Runner”:

Why another artifact?

Why another tool?

Page 17: Functional Testing with the Java Stack Test Runner

ALM Tools: The Test Bundle

The “test” jar

Not an executable jar

Difficult to execute without help from Maven.

Relied on maven to supply its dependencies.

Difficult to execute tests from past code revisions.

The “test-bundle” jar

An executable jar

Can be executed with or without Maven.

Comes packaged with all its dependencies.

Each test artifact represents a snapshot of the current code revision.

Page 18: Functional Testing with the Java Stack Test Runner

ALM Tools: Test Runner

The “failsafe:test” goal

Supports TestNG

Supports test suites

Supports environment configuration

Produces JUnit and TestNG reports.

Executes tests from the target/classes folder

The “stack-test:functional-test” goal

Supports TestNG

Supports test suites

Supports environment configuration

Produces JUnit and TestNG reports.

Executes “test-bundle” artifact.

Page 19: Functional Testing with the Java Stack Test Runner

ALM Tools: Functional Test Goal

Common configuration for stack-test:functional-test:• testBundle – groupId and artifactId of QA test bundle• testEnv – the test environment properties to load• skipFTs – whether to skip functional tests• suiteXmlResources – TestNG suites to load• includes – test class patterns (e.g. **/*FT.class)• systemPropertyVariables – custom system properties• argLine – JVM arguments (e.g. memory options, debugger,

etc.)

For the additional options, see:• http://code.lds.org/maven-sites/stack/modules/test/

Page 20: Functional Testing with the Java Stack Test Runner

ALM Tools: The Test BundleSimple to upgrade existing QA projects.1. Upgrade the project to Stack 3.2

Then change the QA pom’s packaging type from

to

2. Add QA module as a dependency to the ALM module.

<packaging>jar</packaging> <packaging>test-bundle</packaging>

<dependency> <groupId>${project.groupId}</groupId> <artifactId>example-qa</artifactId> <version>${project.version}</version> <type>test-bundle</type></dependency>

Page 21: Functional Testing with the Java Stack Test Runner

ALM Tools: The Test Bundle3. Bind the functional-test goal to the alm-test phase:

<plugin> <groupId>org.lds.stack.test</groupId> <artifactId>stack-test-maven-plugin</artifactId> <executions> <execution> <id>test</id> <goals><goal>functional-test</goal></goals> <phase>alm-test</phase> <configuration> <testBundle> <groupId>org.lds.training.testrunner.lab1</groupId> <artifactId>testrunner-lab1-qa</artifactId> </testBundle> </configuration> </execution> </executions></plugin>

Page 22: Functional Testing with the Java Stack Test Runner

ALM Tools: The Test Bundle4. Modify build profiles to fit your needs

<profile> <id>test</id> <properties> <skipFTs>false</skipFTs> <testEnv>test</testEnv> </properties></profile>

Page 23: Functional Testing with the Java Stack Test Runner

Lab #1A Simple QA Test Bundle Project

https://tech.lds.org/wiki/Functional_Testing_with_the_Java_Stack_Test_Runner

Page 24: Functional Testing with the Java Stack Test Runner

Lab #2Running Functional Testswith Maven and your IDE

https://tech.lds.org/wiki/Functional_Testing_with_the_Java_Stack_Test_Runner

Page 25: Functional Testing with the Java Stack Test Runner

Lab #3Running Functional Tests

in AnthillProhttps://tech.lds.org/wiki/Functional_Testing_with_the_Java_Stack_Test_Runn

er

Page 26: Functional Testing with the Java Stack Test Runner

CreditsThe Apache Maven Project:

http://maven.apache.org/

Urban Code, for it’s AnthillPro documentation:

http://www.urbancode.com/html/products/anthillpro/

The Dark Lord Sauron, for the quote, “One ring to rule them all”. (The Lord of the Rings, by J.R.R. Tolkien)

Page 27: Functional Testing with the Java Stack Test Runner
Page 28: Functional Testing with the Java Stack Test Runner

<code snippet block>