Behavior Driven Testing of Web Services

+

Sreedevi VedulaShraddha Suman

Five Problems

Behavior Driven Testing

For Web Services

Five Patterns`

+The five problems

Can’t name my data? The objects are just exploding! Is there a way I can

reuse them? How deep can this get? Unhappy path testing is truly a unhappy one! The cleanup is a problem as always!

+An Example

Inventory Management Application Create categories Create brands Add products to the inventory Search for products View inventory summary

Disclaimer: All applications used in this presentation are fictitious and

have no code or existence outside of this presentation. Any resemblance to client project or internal project is only a co-incidence.

+1. Can’t name my data?The Problem

Given I create the below categories in inventory | category_id | category_name | | 18765 | Music Players | | 18783 | TV |And I create the below brands in inventory | name | brand_id | | Samsung | 195256 | | Philips | 197654 | And I create the below products in Inventory | brand_id | product_id | category_id | model_id | | 197654 | 197654_10987 | 18765 | E4567 | | 195256 | 197654_11234 | 18765 | M7854 | | 197654 | 195256_11123 | 18783 | R4398 | | 195256 | 195256_10621 | 18783 | V4563 |When I filter the products in the inventory | query_param | value | | brand_id | 195256| | category_id | 18783 |Then the below products should be returned | brand_id | product_id | category_id | model_id | | 195256 | 195256_10621 | 18783 | V4563 |

+Difficulties

Uniqueness Violation if the tests re-run Readability Suffers, difficult to review Hard-coded values obscure cause-effect relationships Numbers might collide with previously generated data

+Use Data TemplatesThe Pattern Applied

Given I create the below categories in inventory | category_id | category_name | | <category_id_1>| Music Players | | <category_id_2>| TV |And I create the below brands in inventory | name | brand_id | | Samsung | <brand_id_1>| | Philips | <brand_id_2>| And I create the below products in Inventory | brand_id | product_id | category_id | model_id | | <brand_id_2> | <product_id_1> | <category_id_1> | <valid-model-id> | | <brand_id_1> | <product_id_2> | <category_id_1> | <valid-model-id> | | <brand_id_2> | <product_id_3> | <category_id_2> | <valid-model-id> | | <brand_id_1> | <product_id_4> | <category_id_2> | <valid-model-id> |When I filter the products in the inventory | query_param | value | | brand_id | <brand_id_1> | | category_id | <category_id_2> |Then the below products should be returned | brand_id | product_id | category_id | model_id | | <brand_id_1> | <product_id_4> | <category_id_2> | <valid-model-id> ?!? |

+Plumbing

Create data templates in code and generate relevant data in code for each template encountered.

Maintain a global test data map and store the test data values against the templates. A subsequent request for the same template should

retrieve this value from the map instead of creating new data.

+2. Object Explosion The Problem

Given I create the below categories in inventory | category_id | category_name | | <category_id_1>| Music Players | | <category_id_2>| TV |And I create the below brands in inventory | name | brand_id | | Samsung | <brand_id_1>| | Philips | <brand_id_2>| And I create the below products in Inventory | brand_id | product_id | category_id | model_id | | <brand_id_2> | <product_id_1> | <category_id_1> | <valid-model-id> | | <brand_id_1> | <product_id_2> | <category_id_1> | <valid-model-id> | | <brand_id_2> | <product_id_3> | <category_id_2> | <valid-model-id> | | <brand_id_1> | <product_id_4> | <category_id_2> | <valid-model-id> |When I filter the products in the inventory | query_param | value | | brand_id | <brand_id_1> | | category_id | <category_id_2> |Then the below products should be returned | brand_id | product_id | category_id | model_id | | <brand_id_1> | <product_id_4> | <category_id_2> | <valid-model-id> ?!? |

+Difficulties

No way to reuse objects No way to access some properties of the objects like

object[property]

+Name Your ObjectsThe ref_name case - The Pattern Applied

Given I create the below categories in inventory | category_id | category_name | | <category_id_1>| Music Players | | <category_id_2>| TV |And I create the below brands in inventory | name | brand_id | | Samsung | <brand_id_1>| | Philips | <brand_id_2>| And I create the below products in Inventory | ref_name | brand_id | product_id | category_id | model_id | | <product_1> | <brand_id_2> | <product_id_1> | <category_id_1> | <valid-model-id> | | <product_2> | <brand_id_1> | <product_id_2> | <category_id_1> | <valid-model-id> | | <product_3> | <brand_id_2> | <product_id_3> | <category_id_2> | <valid-model-id> | | <product_4> | <brand_id_1> | <product_id_4> | <category_id_2> | <valid-model-id> |When I filter the products in the inventory | query_param | value | | brand_id | <brand_id_1> | | category_id | <category_id_2> |Then the below products should be returned | ref_name | | product_4 |

+Name Your ObjectsThe Of Case - The Pattern Applied

Given I create the below categories in inventory | ref_name | category_id | category_name | | category_1 | <unique-category-id>| Music Players | | category_2 | <unique-category-id>| TV |And I create the below brands in inventory | name | brand_id | | Samsung | <brand_id_1>| | Philips | <brand_id_2>| And I create the below products in Inventory | ref_name | brand_id | product_id | category_id | model_id | | <product_1> | <brand_id_2> | <product_id_1> | <category_id of category_1> | <valid-model-id> | | <product_2> | <brand_id_1> | <product_id_2> | <category_id of category_1> | <valid-model-id> | | <product_3> | <brand_id_2> | <product_id_3> | <category_id of category_2> | <valid-model-id> | | <product_4> | <brand_id_1> | <product_id_4> | <category_id of category_2> | <valid-model-id> |When I filter the products in the inventory | query_param | value | | brand_id | <brand_id_1> | | category_id | <category_id of category_2> |Then the below products should be returned | ref_name | | product_4 |

+Plumbing

Store the objects in the global test data map with ref_name as the key and object properties as key-value pairs.

Build a separate data template which matches “of” templates, make the template look up the global test data map and lookup the properties.

+3. How deep can this get?The Problem

{ "inventory":

{ "brands": [

{ "brand_id": "18456",

"products": [

{ "model_id": "E1234", "quantity": 5 }, { "model_id": "E7654", "quantity":

10 } ] }

{ "brand_id": "17654",

"products": [

{ "model_id": "E2345", "quantity": 7 }, { "model_id": "E8765", "quantity":

17 } ] }

]

}

}

+Difficulties

Nested Payloads cannot be handled in one step as Gherkin only allows two-dimensional tables.

+Break Down Your PayloadsThe Pattern Applied When I view the inventory summary

Then I should see the below brands listed in the summary| brand_id || <brand_id of brand_1> || <brand_id of brand_2> |

And I should see the below products for brand <brand_1>| model_id | quantity || <model_id_1> | 5 || <model_id_2> | 10 |

And I should see the below products for brand <brand_2>| model_id | quantity || <model_id_3> | 7 || <model_id_4> | 17 |

+Plumbing

Create steps for each nesting level and make assertions at each level in the payload.

+4. Unhappy path testing The Problem

When I create the below products in Inventory | brand_id | product_id | category_id | model_id | | language specific null value| 12345_10987 | 1052 | E4567 |Then I should see the below error message “brand_id cannot be null”

When I create the below products in Inventory | brand_id | product_id | category_id | model_id | | 12345 | An Array | 1052 | E4567 |Then I should see the below error message ”product_id should be an integer”

When I create the below products in Inventory | brand_id | product_id | model_id | | 12345 | 12345_10987 | E4567 |Then I should see the below error message “category_id is a mandatory parameter”

And then repeat the above scenarios for every property

+Difficulties

Cannot specify language specific invalid values like null, None

Challenging to specify empty string in Gherkin Cannot distinguish between different data types Missing properties would require different scenarios for

each property

+Use Scenario Outlines & Negative TemplatesInvalid Values Case - The Pattern Applied

Scenario Outline: Cannot create products with missing propertiesWhen I create the below products in Inventory | brand_id | product_id | category_id | model_id | | <brand_id> | <product_id> | <category_id> | <model_id> |Then I should see the below error message “<error-message>”

Examples: | brand_id | product_id | category_id | model_id | error-message || <null-value> | <product_id_1> | <category_id_1> | <model_id_1>| brand_id cannot be null. || <empty-string> |<product_id_2> | <category_id_1> | <model_id_2>| brand_id cannot be empty. || <brand_id_1> | <negative-int> | <category_id_1> | <model_id_1>| product_id can’t be negative.|

+Use Scenario Outlines & Negative TemplatesInvalid Data Types Case - The Pattern Applied

Scenario Outline: Cannot create products with missing propertiesWhen I create the below products in Inventory | brand_id | product_id | category_id | model_id | | <brand_id> | <product_id> | <category_id> | <model_id> |Then I should see the below error message “<error-message>”

Examples: | brand_id | product_id | category_id | model_id | error-message || <random-array>| <product_id_1> | <category_id_1> | <model_id_1> | brand_id must be integer. || <random-map> |<product_id_2> | <category_id_1> | <model_id_2> | brand_id must be integer. || <brand_id_1> | <random-string>| <category_id_1> | <model_id_1>| product_id must be integer. |

+Use Scenario Outlines & Negative TemplatesMissing Properties Case - The Pattern Applied

Scenario Outline: Cannot create products with missing propertiesWhen I create the below products in Inventory | brand_id | product_id | category_id | model_id | | <brand_id> | <product_id> | <category_id> | <model_id> |Then I should see the below error message “<missing-property> is mandatory.”

Examples: | brand_id | product_id | category_id | model_id | missing-property || <missing> | <product_id_1> | <category_id_1> | <model_id_1> | brand_id || <brand_id_1> | <missing> | <category_id_1> | <model_id_2> | product_id || <brand_id_1> | <product_id_1> | <category_id_1> | <missing> | model_id |

+Plumbing

Define data templates such as <random-array>, <negative-int> etc., to provide sample values of the data type in code.

Remove properties for the <missing> templates in code and submit the payload.

+5. The cleanup problem!The Problem

Background:

When I create the below products in Inventory | brand_id | product_id | category_id | model_id | | <brand_id> | <product_id> | <category_id> | <model_id> |

Scenario:

When I create the below products in Inventory | brand_id | product_id | category_id | model_id | | <brand_id> | <product_id> | <category_id> | <model_id> |

+Difficulties

Background is usually used (tweaked) to create objects once for the entire feature. But, we cannot really know when the object is created. Can we?

+Track Objects and their LifetimeThe Pattern Applied

Background:

When I create the below products in Inventory once for the entire feature | brand_id | product_id | category_id | model_id | | <brand_id> | <product_id> | <category_id> | <model_id> |

Scenario:

When I create the below products in Inventory | brand_id | product_id | category_id | model_id | | <brand_id> | <product_id> | <category_id> | <model_id> |

+Plumbing Build explicit steps for feature level setups and track Object Lifetime

in the steps

@step('I create the below products in inventory once for the entire feature’)def add_products_for_feature(context): if not DataManager.products_created_for_feature: DataManager.products_created_for_feature = True add_products(context, "feature”)

@step('I create the below products in inventory’)def add_products (context, scope=“scenario”): // Create Products

DataManager.track_objects_for_teardown(products, scope)

Delete objects at “feature” scope after the feature execution and delete objects at “scenario” scope after the scenario execution.

+

That is all! Hope you liked it!

+

Thank You!