Behavior Driven Testing with SpecFlow

GTC-NL-FXD Rachid Kherazi

Behavior Driven Testing (BDT)

with MS VS2010 & SpecFlow

Confidential

GTC-NL for internal use

Goal of presentation

• Informative

• Share my knowledge w.r.t. Testing and Test tools

In the scope of “did you know?”

Gathered in my daily train trips

Amstelveen Best

Confidential


Qeustion 1: what are the following two vegetables?

Subject of today is related to some vegetables

Answer 1: Cucumber Gherkin

Question 2: What is the difference between the two?

Answer 2:

Cucumber: Tool, behavior driven development, …

Gherkin: Language, Business Readable, Domain Specific Language,..

Let start with a Quiz

Confidential


Content

• Behavior Driven Development (Development view)

– Agile Iteration and Test Driven Development (TDD)

– What’s all these DD’s? (TFD, TDD, BDD, ATDD and SDD)

– Scenario/Story and Gherkin-syntax

– Executable Specification

– Behavior Driven Development tools

– Live demo with SpecFlow

• Behavior Driven Testing (Test view)

– DDB vs BDT

Confidential


http://www.synerzip.com/agile-method.htm

Introduction: Agile iterations and TTD

Confidential


ATDD

- Acceptance Test Driven

- Different than BDD?

- Start with the acceptance test

- Examples,

(2) Write the

implementation code to

make the test pass. Keep

it simple

(3) Change the code to

remove duplication in code

and to improve the design

while ensuring that all tests

still pass.

(1) Imagine how the new

code should be called and

write the test as if the code

already existed

Confidential


What are the advantages of TDD?

• TDD shortens the programming feedback loop

• TDD provides detailed specification (tests)

• TDD promotes the development of high-quality code

• TDD provides concrete evidence that your software works

• TDD supports evolutionary development

• TDD encourages developers to write code that is easily tested

TDD is used in

Agile development

eXtreme programming

Confidential


ATDD

- Acceptance Test Driven

- Different than BDD?

- Start with the acceptance test

- Examples,

Test First

Development

Refactor

TDD

Start from acceptation test

Focus on behavior of feature

S/BDD

Confidential


What’s all these DD’s?

TFD Test First Development

TDD Test Driven Development

ATDD Acceptance Test Driven Development

BDD Behavior Driven Development

STDD Story Test Driven Development

TTD = Test First Development + Refactor (remove duplicate, improve design)

Micro view

ATDD start for end user acceptance test

Macro view

BDD focus on the behavior of the feature

Based on user story: a couple of examples on how SUT could be used

Story written in a standard syntax: e.g. Given / When / Then

The [Given, When, Then] syntax is called Gherkin

First create test (Failing test) than create implementation (Test passed)

Given [initial context] When [event] Then [an outcome]

Subsequent steps of the same kind can also be specified with the “And “ and “But” keyword

Variations on a Theme

Context

Specification

AAA

User Story

Structure

Establish Initial Context

Given

Arrange

SetContext

Interact with SUT

When

Act

BecauseOf

Check the results

Then

Assert

ItShould

Confidential


Example: Application Under Test

(3) Application displays “Hello Best Tester: Password is Correct”

(If password = “Rachid”)

(1) type password “Rachid”

(2) press Login button

(0) start application

“Say Hello Best Tester” a simple application for demo purpose.

According req. of this tool: “Rachid”, “Ali” and ‘Pieter” are correct passwords for login.

Confidential


Example: Feature file (Scenario’s: Given When Then)

Feature: Log-In in “Best Tester Application”, check correctness of

password

Scenario: Check Login with correct password

Given Application is started

When correct password is typed

And Login button is pressed

Then Login should be successful

And Messagebox should display "Hello Best Tester: Password Correct"

Describe some piece of functionality of the system

Plain readable text

Features contain one or more scenarios

A scenario describes a single acceptance test for a feature

Most features are composed of multiple scenarios

Tool generates a test for each scenario

The name of the test is generated from the scenario title

Confidential


When scenario/story is in place it would be wonderful if there is a

tool that can run this scenario’s to verify that the implementation

actually behaves as specified.

Executable specification that verifies that the implementation behaves as expected, readable by all in the team, and around it

Requirements

/Behavior

In well known

format

e.g.

Given/When/Then

Tool Implementation

(SUT)

Main benefit

Main benefit is not in the tool!

The benefit is that team get together and write down a common,

shared knowledge in a well known format. In one place.

Confidential


Tools

This was the idea of Cucumber tool that first was created in the Ruby-community.

Cucumber can execute scenarios written in the Given/When/Then-syntax.

For .Net framework SpecFlow is known good tool.

Both tools used Gherkin syntax and are free (open source).

Cucumber

Ruby

SpecFlow

.Net

Confidential


Behavior Driven Development in one slide

Feature file

(Scenario’s: Given When Then)

Test class (for Nunit)

Generated by SpecFow tool (.cs)

Stepper class

binding with implementation (.cs)

Implementation

(.dll, .exe)

Nunit dll

Nunit

Test

report

Code of Implementation (.cs)

Visual Studio 2010 + SpecFlow

SW developer

Requirements

User

feedback 1

2

3

4

5

6

7 8

Confidential


Some handy Attributes

Background

With the keyword background you can run some steps before each scenario in the

feature.

Hooks

Hooks are events that is fired in the following order.

And you can “hook in” code here to do useful things:

- Restore a database

- Dump a log on error

- Init variables before steps

Scenario outlines

Data-driven scenarios or scenario templates

Consists of Scenario template specification with data placeholders

Tool generates parameterized test logic for the scenario outline and individual test

method for each example set (using pairwise testing)

Confidential


Some handy Attributes

Tag

Can be used to specify “categories” for Nunit or other test runners

e.g. @RegressionTestSet

@ignore tag is a special tag “ignore test”

# is reserved for comment

#language

The language of the feature files can be either specified globally in the

configuration file or in the header of the feature file using “#language” keyword

different ISO languages are supported

Debugging Tests

SpecFlow Visual Studio integration also supports debugging the execution of the

tests. Just like in the source code files of your project, you can also place

breakpoints in the SpecFlow feature files. Whenever you execute the generated

tests in debug mode, the execution will stop at the specified breakpoints and you

can execute the steps one-by-one using the standard “Step Over” (F10)

command or you can go to the detailed execution of the bindings using the “Step

Into” (F11) command.

Confidential


SpecFlow configuration file

language

Test runner

Stop at error

Confidential


Live demo “SayHello Application”

VS 2010 and SpecFlow Link Nunitdll

C:/Data/Visual Studio 2010/specFlow/FXD_demo/SpecFlow_FXD_demo/SpecFlow_FXD_demo

Confidential


SpecFlow and generated Test Results

SpecFlow tool generates clear and concise report

Report

Confidential


Behavior Driven Development in one slide

Feature file




Stepper class


Implementation

(.dll, .exe)

Nunit dll

Nunit

Test

report

Code of Implementation (.cs)


SW developer

Requirements

User

feedback 1

2

3

4

5

6

7 8

Confidential


Behavior Driven Testing in one slide

Feature file




Stepper class


SUT (FlashLite)

Nunit dll

Bello/Nunit

Test

report


Requirements

Tester SW/FW developer

implementation

code

Development

environment

User

SW/FW Implementation

feedback

4

2

3

1 5

Confidential


http://www.synerzip.com/agile-method.htm

Behavior driven testing

Development domain

Test domain

Confidential


BDD vs BDT

• Behavior Driven Development (BDD)

– Development domain

– White box

– Micro view

– Input = specification (Behavior)

– Output = Implementation

• Behavior Driven Testing (BDT)

– Test domain

– Black box (interface)

– Macro view

– Input = Specification (Behavior)

– Output = Test

Confidential


Requirement Based Testing in one slide

Feature file




Stepper class


SUT (FlashLite)

Nunit dll

Bello/Nunit

Test

report


Requirements

doc

SW/FW developer

implementation

code

Development

environment

User

SW/FW Implementation

feedback

4

3

2

1

5

Spec.doc to Feature file

convertor

(2) = refactor= clean up and improve Tester

To be

developed

Confidential


Learn more about SpecFlow and BDD

http://blog.dannorth.net/introducing-bdd/

http://www.specflow.org/

https://github.com/techtalk/SpecFlow

http://github.com/techtalk/SpecFlow-Examples

http://github.com/aslakhellesoy/cucumber/wiki

http://www.marcusoft.net/search/label/SpecFlow

https://github.com/marcushammarberg/

https://www.marcusoft.net

Free downloadable book




http://www.specflow.org/

https://github.com/techtalk/SpecFlow




http://github.com/aslakhellesoy/cucumber/wiki

http://www.marcusoft.net/search/label/SpecFlow

https://github.com/marcushammarberg/

https://www.marcusoft.net/

Confidential


1, 2, 3

Given

When

Then

[TestMethod] public void Total_Should_Be_1_When_1_Product_Added() { var cart = new ShoppingCart("TEST"); cart.AddItem(new Product("SKU"));

Assert.AreEqual(1, cart.TotalItems); }

Software

Behavior Driven Testing with SpecFlow