A High-Powered Version of Behavior-Driven Development

Outline

7. Scenario Specification8. Architecture9. Our Contributions10.Planned Evaluation11.Future Work12.Conclusion

1. Introduction2. Problem Statement3. Proposed Solution4. Target Audience5. Related Work6. User Scenarios

Introduction

• What is Test Driven Development (TDD)?

• Problems with TDD?o Desired "behavior" of an system is specified in terms

of code, not always the right way to capture behavior

o Other stakeholders in Software lifecycle find it hard to be involved in the whole process

Introduction• What is Behaviour Driven Development (BDD)?

o BDD is TDD done right!!!o BDD was conceived by Dan North in 2003o BDD uses natural language to describe the "desired

behavior" of the system, that can be understood by both the developer and the customer

• Demo of an existing BDD application using Cucumber which is a BDD framework predominantly used with Ruby

Problem Statement• In existing BDD frameworks, developer needs to

manually write the glue code which maps "behaviors" to the implementation code

• This mapping involves writing code for interpreting behaviors in terms of test-cases

• Writing this glue code is tedious and error prone, if the written test code does not depict the specified behavior

• Can this intermediate step of writing glue code be automated?

Proposed Solution: SmartBDD

• Automatically generate test code from "behavior" description, eliminating the intermediate step of writing test code

• This allows the direct execution of "behaviors" on application code

• SmartBDD makes BDD available to the developer community, by reducing the learning curve, time and effort

Target Audience• Developers, who want to practice BDD using a

simplified framework

• Students, SmartBDD abstracts the process of magically generating test code, making BDD more accessible, by allowing students to follow a test-first approach towards software development.

• Stakeholders like customers who may have not followed a test-first approach, can use the system to test application code.

Related Work• There are very few published studies on BDD, most of

which take a relatively narrow view of BDD and only treat it as a specific technique of software development

• There are tools which use UML, XML schema etc. as input and generate an template for the source code.

• Various Frameworks that make BDD accessible JBehave Cucumber JDave

User Scenarios

SmartBDD UI - Eclipse Plugin

Scenario: specification

• GIVEN, an initial context• WHEN, occurrence of an event• THEN, expected outcome• Parameters in Quotes, " "• Connectives - And and Not (only in then condition)

Scenario: exampleScenario 1: Account Deposit

Given a bank account with a balance of "100" $

When a customer deposits "20" $

Then the balance of the account should be "120" $

Architecture

Natural Language Processing• Pre-processing:

o Parsing scenario text to computation object-oriented class

o Syntax validation

Natural Language Processing• Language Augmentation

o Prioritizing/weighting words: based on Part of Speech (POS), e.g., verb, noun, adjective

o Filtering out unimportant words, e.g.,"a", "the", "is"o Clarifying language meaning using:

Synonyms Stemming Grammatical structure

o Using: openNLP, StanfordNLP, WordNet, Snowball

Probabilistic Matcher• Assumptions: naming conventions, e.g., camel casing• Scenarios - implementation coding mapping

o Mapping unit: clauseo Clause - class mapping

Name matching POS (e.g.,nouns)

o Clause - constructor mapping Name matching #parameters Given clauses

o Clause - method mapping Name matching, POS, #parameters

Code Information Extractor

• Is responsible for extracting information from the implementation code at run-time for analysis about the class that is loaded, to keep track of all the classes in a project and the necessary details about them.

• We built a wrapper over the Java Reflection library, to extract Class, Constructor, Method, Variables and Parameter information. It is also used to infer the return type and number of arguments for a method.

Code Generation

• Code generator interacts with all the above components, integrates the information from augmentation engine and information extractor and uses probabilistic values from the matcher Engine, to generate the code on-the-fly.

• The code is generated using a sophisticated on-the-fly generator using the CodeModel library.

Code Generation:Natural Language Interpretation

• The code generator makes certain assumptions about the interpretation of the natural language.

• Given condition specifies object creation. You should be able to create multiple objects as required by the scenario.

• The When condition specifies an action which is a call to a particular method.

• The postcondition i.e. Then, contains a sort of assertion to verify whether the case failed or passed.

Demo Time!!! • HTTP Requester example with BDD approach

• Multiple classes in same scenario - Stack and Queue example

Contributions

• Make BDD more accessible to developers, specifically for students by automating the whole process of test-code generation.

• Creating loosely coupled components, which will help in porting SmartBDD to other languages.

• Unique Idea about probabilistic matching, and how a good model will provide insight into code generation ability.

• Simplified UI for development.

Planned Evaluation• Make SmartBDD open-source and try to involve

developers to use the framework and provide feedback.• Deploy SmartBDD in a CS1 course for Java in Fall

2013.• Things to evaluate

o Compare with other BDD tools and evaluate if our tool is making a difference based on student feedback.

o Does the code generated perform the behavior that the student expects?

o Is the tool generally useful? Does it reduce complexity and developer time? Other Metrics.

Future Work• Lots of things still to do!!!• NLP Augmentation engine:

o Our grammar, still requires parameters to be passed in quotes, we want to infer this information from scenarios.

o Make inference about order of parameters that are passed.

o Augment with root words and synonyms.o Make as much inference as possible.

• Code information extractoro extract details from comments, local variables etc.

Future Work• Probabilistic matcher

o tune it for better results with more accurate probability value.

o How to handle same probability value.• Code generation

o Passing objectso Referring return values from previous method calls.o Better assertion capability.o Provide conditional statements, looping constructs. o Acceptance testing involving interaction between

multiple classes.• UI Improvements, usability and effectiveness.

Conclusion• SmartBDD enables automatic code generation from

"behaviors"

• SmartBDD makes BDD accessible and hopefully easy for students to use.

• Though it may not be industry ready, this tool should fit the need for students in CS1 courses to help them follow sound software engineering practices, by using BDD to unit test their code.

Acknowledgements

• We would like to thank, Dr. Stephen H. Edwards Idea Insightful and Timely Feedback

• All students in CS-6604 course who provided us with useful feedback, and had to bear with us!

Questions?

Thank You!!!