Embed Size (px)
Citation preview
UI Testing, handling complex UIs at scale
Josh Black :: FEDucation :: March 2nd, 2016
.FED@IBM
Discussion Time A look at the landscape of testing options and techniques for developers.
.FED@IBM
Wikipedia An investigation conducted to provide information about the quality of a product or service under test.
.FED@IBM
TestingHelps developers tell whether the product is working or not.
.FED@IBM
OverviewThe Basics What are the various kinds of tests? How do I know when to use each one?
Writing a Test How can we structure our tests in a way that's clean, readable, and effective?
Real World What kind of tools can I leverage to start testing in my project today?
0Part 0 Setting the Stage
What is a test?
.FED@IBM
Testing When a developer changes a piece of code in a project, how confident am I that sh*t won't break?
.FED@IBM
Testing a Dropdown
class Dropdown { /* ... */ }
const d = new Dropdown({ items: ['Home', 'Cognitive', 'Watson', 'Services'], defaultText: 'Please select an item', target: document.querySelector('.dropdown') });
.FED@IBM
HTML
<ul class="dropdown"> <li class="dropdown__item"> <p id="current">Please select an item</p> <span class="dropdown__arrow"></span> <ul class="dropdown-menu"> <li class="dropdown-menu__item"><a href="#">Home</a></li> <li class="dropdown-menu__item"><a href="#cognitive">Cognitive</a></li> <li class="dropdown-menu__item"><a href="#watson">Watson</a></li> <li class="dropdown-menu__item"><a href="#services">Services</a></li> </ul> </li> </ul>
.FED@IBM
JavaScript
const d = new Dropdown({ items: ['Home', 'Cognitive', 'Watson', 'Services'], defaultText: 'Please select an item', target: document.querySelector('.dropdown') });
dropdown.arrow.addEventListener('click', () => { dropdown.classList.toggle('is-expanded'); });
.FED@IBM
.FED@IBM
.FED@IBM
Testing Allows us to make a trade-off between reliability and speed.
.FED@IBM
Testing a Dropdown
class Dropdown { /* ... */ }
const d = new Dropdown({ items: ['Home', 'Cognitive', 'Watson', 'Services'], defaultText: 'Please select an item', target: document.querySelector('.dropdown') });
.FED@IBM
Testing a Dropdown
describe('Dropdown', () => { it('should ...', () => { // ... }); });
.FED@IBM
Testing a Dropdown
describe('Dropdown', () => { it('should expand the list of options when clicked', () => { // ... }); });
.FED@IBM
Testing a Dropdown
describe('Dropdown', () => { it('should expand the list of options when clicked', () => { // ... });
it('should collapse the list of options when an item is selected', () => { // ... }); });
.FED@IBM
Test-Driven Development
Test Behavior-Driven Development
Add some initial, failing tests
Add Tests
.FED@IBM
Write an initial attempt at making your test pass
Write code
.FED@IBM
Run tests, re-working until they go green
Run Tests
.FED@IBM
Now that you have passing tests, go back and refine
Refactor
.FED@IBM
Unit Tests
Allows us to test small pieces of functionality in isolation.
Integration Tests
End-to-End Tests
Understand and test how a user flows through the system
Test the integration of small pieces of functionality with each other
Testing Basic Types
.FED@IBM
Unit Tests
const addIntegers = (a, b) => a + b;
describe('add', () => { it('should add two integers together', () => { const a = 1; const b = 2;
expect(add(a, b)).toBe(a + b); // true || false }); });
.FED@IBM
Unit Tests
const addIntegers = (a, b) => a + b;
addIntegers(1.5, 'a'); // 1.5a
.FED@IBM
Unit Tests
const addIntegers = (a, b) => { const { isInteger } = Number;
invariant( isInteger(a) && isInteger(b), 'Expected integer arguments, instead got: [%s, %s]', a, b );
return a + b; }
.FED@IBM
Unit Tests
const addIntegers = (a, b) => /* ... */
describe('add', () => { it('should warn if an argument is not an integer', () => { const a = 1; const b = 'a';
expect(add(a, b)).toThrow(); }); });
.FED@IBM
Integration Tests
Scenario
.FED@IBM
Visualization
API Response Validation
Duration
Count Visualization
Scenario
.FED@IBM
Validation
Duration
Count
Scenario
.FED@IBM
Validation
Duration
Count
Testing a Validator
describe('Validator', () => { it('should validate a response from our API', () => { // ... }); });
.FED@IBM
Testing getDuration
describe('getDuration', () => { it('should get a duration value from an API response', () => { // ... }); });
.FED@IBM
Integration Test
import Validator from '../Validator'; import getDuration from '../getDuration';
describe('getResponseDuration', () => { it('should get a duration value from an API response', () => { const response = {}; const validated = Validator(response);
expect(validated.passed).toBe(true); expect(validated.value).toBe(response.result.data);
// ... }); });
.FED@IBM
Integration Test
describe('getResponseDuration', () => { it('should get a duration value from an API response', () => { const response = {}; const validated = Validator(response);
expect(validated.passed).toBe(true); expect(validated.value).toBe(response.result.data);
const durationCount = getDuration(validated.value); expect(durationCount).toBe(300); }); });
.FED@IBM
End-to-End Tests
E2E Focus on real user scenarios, catching bugs before they reach the user.
.FED@IBM
E2E Scenario
.FED@IBM
Hits EnterLogin Form Clicks Form Enter Credentials
Testing Pyramid
.FED@IBM
E2E
Integration
Unit
Testing Pyramid
.FED@IBM
E2E
Integration
Unit
Unit Testing
Allows us to create a fast, reliable feedback loop that isolates errors.
Requirements: Knowledge in areas such as dependency management, mocking, and hermetic testing.
Testing Pyramid
.FED@IBM
E2E
Integration
Unit
Integration Tests
Allows us to verify the behavior of small groups of units
Testing Pyramid
.FED@IBM
E2E
Integration
Unit
End-to-End Tests
Covers a small set of our product features to help catch bugs that may reach customers
Take a Look Just Say No More to End-to-End Tests Covers the practicality of E2E tests, highlighting the advantages of favoring unit/integration tests over E2E.
Move Fast and Don't Break Things Discusses the integration and role of testing in the context of Continuous Integration and Delivery
Link
Link
1Part 1 Writing a test
.FED@IBM
How to actually get started
Setup Test runners, assertions, and spies, oh my!
.FED@IBM
mocha
chai
karma
expect
expect.js
jasmine
sinon
qunitselenium
web driver
ava
jsdom
.FED@IBM
Test Runner
Allows us to write chunks of code to describe what should happen. Handles executing tests for us and returning the result.
Assertion Library
Spies
Sometimes we need to check and verify that what we think will be called is actually called
Enables us to state what we believe to be true, false, and a variety of other states.
Testing Checklist
.FED@IBM
Test Runner
Allows us to write chunks of code to describe what should happen. Handles executing tests for us and returning the result.
Assertion Library
Spies
Sometimes we need to check and verify that what we think will be called is actually called
Enables us to state what we believe to be true, false, and a variety of other states.
Testing Checklist
.FED@IBM
DOM Mocking / Driver
Allows us to verify that what we expect to happen in our product is actually happening
Test Runner
describe('Module', () => { it('should ...', () => { // ... }); });
.FED@IBM
Test Runner
.FED@IBM
Take a Look Mocha Classic JavaScript test framework for JavaScript
Jasmine DOM-less simple JavaScript Testing Framework
https://www.npmjs.com/package/mocha
https://www.npmjs.com/package/jasmine
Karma Spectacular Test Runner for JavaScript https://www.npmjs.com/package/karma
Ava Futuristic Test Runner for JavaScript https://www.npmjs.com/package/ava
Assertions
Assertions Specify what you expect the outcome of a scenario to be.
.FED@IBM
Assertions
describe('Module', () => { it('should ...', () => { const result = Module();
expect(result).toBe(2); }); });
.FED@IBM
Assertions
describe('Module', () => { it('should ...', () => { const result = Module();
expect(result).toEqual({ title: 'FEDucation', description: 'Coolest FED gathering in the world' }); }); });
.FED@IBM
Assertions
describe('Module', () => { it('should ...', () => { const result = Module();
expect(result).toEqual(['some', 'collection']); }); });
.FED@IBM
Assertions
describe('Module', () => { it('should ...', () => { const result = Module();
expect(result).toThrow(); }); });
.FED@IBM
Assertions
describe('Module', () => { it('should ...', () => { const result = Module();
expect(result).to.be.instanceOf(Error); }); });
.FED@IBM
Assertions
it('supports loading multiple keys in one call', async () => { var identityLoader = new DataLoader(keys => Promise.resolve(keys));
var promiseAll = identityLoader.loadMany([ 1, 2 ]); expect(promiseAll).to.be.instanceof(Promise);
var values = await promiseAll; expect(values).to.deep.equal([ 1, 2 ]);
var promiseEmpty = identityLoader.loadMany([]); expect(promiseEmpty).to.be.instanceof(Promise);
var empty = await promiseEmpty; expect(empty).to.deep.equal([]); });
.FED@IBM
Take a Look Expect Write better assertions
Chai BDD/TDD assertion library for node.js and the browser
https://www.npmjs.com/package/expect
https://www.npmjs.com/package/chai
Should Test framework agnostic BDD-style assertions https://www.npmjs.com/package/should
Assert Native assertion library for Node.js
Spies
Spies Stubs out any function and tracks calls to it and all arguments
.FED@IBM
Assertions
const sum = (a, b) => a + b;
describe('A spy', () => { it('should track that the spy was called', () => { spyOn(sum); sum(1, 2); expect(sum).toHaveBeenCalled(); expect(sum).toHaveBeenCalledWith(1, 2); }); });
.FED@IBM
Assertions
describe('Module', () => { it('should track that the spy was called', () => { spyOn(console, 'error'); const result = Module('invalid argument'); expect(console.error).toHaveBeenCalled(); expect(console.error) .toHaveBeenCalledWith('invalid argument'); }); });
.FED@IBM
Isolation
Achieves similar results to mocks, stubs, dummies, fakes, etc for accomplishing isolation
Consistent Interface
Partial Mocking
You as the developer choose what portions of an interface you want to mock
Spies follow the same interface as the functions that they're mocking
SpiesHighlights
.FED@IBM
Take a Look Jasmine DOM-less simple JavaScript Testing Framework
Expect Write better assertions
Sinon JavaScript test spies, stubs and mockshttps://www.npmjs.com/package/sinon
DOM Mocking
End-to-End Tests
Prohibitively Expensive Loading up the browser, and performing a multitude of tests doesn't fit our ideal feedback loop for testing.
Flakiness Race conditions, or other processes that may yield inconsistent results for a test due to using the DOM.
Tooling In the past, we'd have to use complex tool chains like Selenium with Web Driver just to get E2E that don't actually help us, or the user.
Take a Look jsdom A JavaScript implementation of the DOM and HTMl standards
Karma Brings a productive testing environment to developers
https://www.npmjs.com/package/jsdom
https://www.npmjs.com/package/karma
Selenium Web Driver Driving a browser natively as a user would http://docs.seleniumhq.org/projects/webdriver/
DOM Mocking
import jsdom from 'mocha-jsdom'; import { expect } from 'chai';
describe('mocha tests', function () { jsdom()
it('has document', function () { var div = document.createElement('div') expect(div.nodeName).eql('DIV') }); });
.FED@IBM
DOM Mocking
it('should work for DOM events', function () { var div = document.createElement('div'); div.addEventListener('click', () => console.log('FEDs rule!'));
spyOn(console, 'log');
div.click();
expect(console.log).toHaveBeenCalled(); expect(console.log).toHaveBeenCalledWith('FEDs rule!'); });
.FED@IBM
DOM Mocking
import jsdom from 'mocha-jsdom'; import { expect } from 'chai';
const markup = '<html>...</html>';
describe('mocha tests', function () { jsdom(markup)
it('has document', function () { var app = document.querySelector('#app') expect(app.nodeName).eql('DIV') }); });
.FED@IBM
Recap The Basics Overview of Unit, Integration, and End-to-End tests.
The Tools Identify the necessary components that developers need for a robust testing framework.
The Practice How do we go about structuring our tests, and begin writing tests using the techniques/testing frameworks that we've identified0
Google Focus on the user and all else will follow
.FED@IBM
Testing Enables us to move fast and not break things
.FED@IBM
@joshblackfr github.com/joshblack
github.ibm.com/joshblack
Thanks! Any Questions?
