Embed Size (px)
Citation preview
Erik Stensland
Abstract
A challenge for QA historically has been to move our testing efforts as deep into the development cycle as possible. Some of our fellow QA‟ers have been fortunate enough to be able to work with development to write Unit tests. However most times, QA is left to test at the UI level with not much time left for items such as Application Security, Unicode and Stability testing. Wouldn‟t it be nice to have a happy medium where most QA organizations with minimal investment could begin testing at a level before the UI but after Unit testing for these areas specifically. Enter FitNesse, FitNesse has been around since 2005 but more recently with the increase in RESTful type services, at least at Pearson eCollege, it is making a big comeback. I will be discussing our approach on how we are using FitNesse and the REST fixture to test standard CRUD functionality but more importantly how we are starting to use FitNesse to bring Application Security, Unicode and Stability testing closer to our development teams.
Topics
Development Cycle
Pearson eCollege Benefits
FitNesse
REST
Types of REST tests
Functional
Integration
App Security
Unicode
Stability
Traditional
Development Cycle
• Service Development
Unit Testing
• UI Development
Functional Testing
• Integration Development
Integration Testing
• Tuning Development
AppSec/Unicode/Stability
Non Traditional Dev
Cycle
Stability Testing
Functional Testing Integration Testing
Application Security
Testing Unicode Testing
Service Development
Pearson eCollege Benefits
Faster development and test time – Increased releases to production. Set an eCollege record in July for number of releases.
30,000+ validations for a brand new product with first line of code developed in April and released to production in mid-July.
Development and SQE both take ownership in the tests.
Development can run the tests while SQE focuses on new tests using Jenkins.
SQE now provides a huge amount of information to the development process quicker then ever before.
Test cases are now self documenting.
Extremely easy barrier to entry. In less then 12 months we have increased from 1 team utilizing FitNesse to ~10 teams generating 10s of thousands of validations.
A quote from our CTO!
Recently our CTO blogged the following:
“Looking at just Web Services on Demand and some new admin services we have over 21,000 automated validations, testing all the capabilities of several hundred web services in all their permutations. This means that we can test every single service we have built to date on these projects with the click of a button and a few minutes of time. Power. Real Power. Further, we are using open source tools to do the testing. Free. Even further, our team is on the cutting edge. We are extending the open source frameworks and sending the code back to the projects. Innovative. Almost no one has done what we are doing before, and those that have are the likes of Google, Amazon & Twitter.”
FitNesse ???
Fully integrated standalone wiki, and acceptance testing
framework.
Tool for enhancing collaboration in software
development.
Compares customer expectations to actual
results.
Invaluable way to have development and
SQE collaborate on complicated
requirements. (i.e. Test Driven Development)
Architecture
SUT
Test Cases
FitNesse
REST Fixture
REST Services
Easy Barrier to Entry
FREE
Easy to setup
No special hardware.
Wiki web server
Simply use your browser to create, edit and execute tests.
Quick Learning curve.
Nuts and Bolts
Global Variables
!define var1{Leonardo Da Vinci}
Local Variables
|let|$var2|body|/root/data/name/text()| |
Test Tables
Input and Expected output.
|!-smartrics.rest.fitnesse.fixture.RestFixture-!|${trainingHost}|
|setBody|!-{"name": "-!${var1}!-"}-!|
|POST|/create|200|||
|let|$var2|bodyregex|!-Object (.*) was-!| |
“REST” web services
Three defined aspects
Base URI: http://localhost:8082/
Internet media type: JSON, XML
Set of Operations: GET, POST,
PUT, DELETE
Representational State Transfer
Style of software architecture for distributed hypermedia.
Scalability, Generality of interfaces, independent
deployment of components.
RestFixture
Custom Fixture that allows developers, quality and/or
product owners to write test tables for REST services
with simplicity in mind. The idea is to write tests that are
self documenting and easy to write and read, without the
need to write Java code.
Open Source- https://github.com/smartrics/RestFixture
Pearson eCollege - https://github.com/tfredrich/rest-
fixture
Types of “REST” tests.
Functional
JSON, XML, Special Characters, Tunneling, Empty fields, Null fields, Authorization, Invalid
Integrated
User Scenarios / Work Flows
App Sec
Cross-site scripting / SQL injection
Unicode
Different languages
Stability
Timing
What is Functional ?
Functional Service testing is the verification that the SUT
works according to specifications and that it handles
erroneous actions and data correctly
Positive
Negative
JSON and XML
Invalid Fields
Tunneling (?_method=PUT)
Special Characters (@#$%^&*)
Functional Example
STANDARD POST
|!-smartrics.rest.fitnesse.fixture.RestFixture-!|${trainingHost}|
|setBody|!-{"name": "erik"}-!|
|POST|/create|200||//status/text()='success'|
STANDARD READ
|!-smartrics.rest.fitnesse.fixture.RestFixture-!|${trainingHost}|
|GET|/read|200|Content-Type : application/json |!-
//status/text()='success'
//code/text()='200'
//message/text()=‟Successful Read‟-!|
What is Integrated ?
Integration Testing is when individual modules are combined
and tested as a group.
User Scenarios
Work Flows
Integrated Example
STANDARD POST
|!-smartrics.rest.fitnesse.fixture.RestFixture-!|${trainingHost}|
|setBody|!-{"name": ”Erik"}-!|
|POST|/create|200||//status/text()='success'|
|let|$var2|bodyregex|!-Object (.*) was-!| |
URLENCODE
|!-org.eclg.fitnesse.tools.StringFixture-!|
|value|=urlencode()|
|%var2%|var2|
READ USING VARIABLE FROM POST
|!-smartrics.rest.fitnesse.fixture.RestFixture-!|${trainingHost}|
|GET|/correct/%var2%|200|Content-Type : application/json |!-
//status/text()='success'
//code/text()='200'
//message/text()='YOU ARE SUCCESSFUL'
-!|
What is AppSec ?
Malicious Code Injections
Penetration Testing
Input Validation
Variable Manipulation
AppSec evaluates the security posture of an application across
the development life cycle, enabling you to identify, eliminate,
and prevent security risks in the applications that drive your
business.
AppSec Example
APP SECURITY
|!-smartrics.rest.fitnesse.fixture.RestFixture-!|${trainingHost}|
|setBody|!-{"name": "<script>alert(document.cookie);</script>"}-!|
|PUT|/appsec|200||//name/text()='alert(document.cookie);'|
What is Unicode ?
Spanish
Russian
Chinese
Unicode testing is testing character sets that define every
character in most of the speaking languages in the world.
Unicode Example SPANISH
|!-smartrics.rest.fitnesse.fixture.RestFixture-!|${trainingHost}|
|setBody|!-{"name": -!"Puedo comer vidrio, no me hace dao"!-}-!|
|PUT|/unicode|200||//name/text()=„Puedo comer vidrio, no me hace
daño‟|
RUSSIAN
|!-smartrics.rest.fitnesse.fixture.RestFixture-!|${trainingHost}|
|setBody|!-{"name": -!"Съешь же ещѐ этих мягких французских булок да
выпей чаю"!-}-!|
|PUT|/unicode|200||//name/text()=„Съешь же ещѐ этих мягких
французских булок да выпей чаю‟|
CHINESE
|!-smartrics.rest.fitnesse.fixture.RestFixture-!|${trainingHost}|
|setBody|!-{"name": -!"我能吞下玻璃而不伤身体"!-}-!|
|PUT|/unicode|200||//name/text()=„我能吞下玻璃而不伤身体‟|
What is Stability ?
Stability testing checks to see if the service can continuously
perform with in an expected time range.
Timing
Stability Example
STABILITY
|!-smartrics.rest.fitnesse.fixture.RestFixture-!|${trainingHost}|
|setBody|!-{"name": "erik"}-!|
|startTimer|
|PUT|/test|200||//name/text()=„erik‟|
|checkTimer|20|50||
