Upload
myron-asher-marsh
View
225
Download
0
Tags:
Embed Size (px)
Citation preview
Agile and the automated testing of accessibility
© 2015 - All Rights Reserved 2
© 2015 - All Rights Reserved 3
Agile Tools
5© 2015 - All Rights Reserved
Shift Left Testing
Catching a defect in the design and requirements phase can reduce cost to fix by up
to 100:1
Software Defect Reduction Top 10 List, Barry Boehm and Victor R. Basili, January 2001
https://www.cs.umd.edu/projects/SoftEng/ESEG/papers/82.78.pdf
© 2015 - All Rights Reserved 6
Rise of Continuous
• Lag between committing code and getting feedback allows damage to magnify– Costs rise– Velocity decreases
• Continuous Integration• Continuous Deployment
http://www.startuplessonslearned.com/2009/06/why-continuous-deployment.html
© 2015 - All Rights Reserved 7
Automation
© 2015 - All Rights Reserved 8
© 2015 - All Rights Reserved 9
Accessibility Testing’s Image
Aim of Software Testing
• Reduce the risk of the existence of a serious problem on a supported configuration to an acceptable level– Define “serious problem”– Define “supported configuration”– Define “acceptable level”
© 2015 - All Rights Reserved 10
© 2015 - All Rights Reserved 11
Android Fragmentation
12© 2015 - All Rights Reserved
Android Fragmentation
We have seen 24,093 distinct devices download our app in the past few months. In our report
last year we saw 18,796. In 2013 we saw 11,868
OpenSignal 2015
© 2015 - All Rights Reserved 13
Combinations
4 operating systems, 2 supported versions5 Assistive technologies4 Browsers, 2 supported versions3 Responsive breakpoints
216 combinations excluding device combinations
© 2015 - All Rights Reserved 14
© 2015 - All Rights Reserved 15
Sparse Testing Matrix
Browser/AT
Chrome KB IE KB
Firefox KB Safari KB Dragon IE NVDA FF Jaws IE VO Safari TB FF Orca FF
Operating
Sytem
Windows
OS X
iOS
Android
Linux
© 2015 - All Rights Reserved 16
© 2015 - All Rights Reserved 17
Accessibility “Acceptance Automation”
18© 2015 - All Rights Reserved
Acceptance Automation
• Comes from the “requirements”– Designers mark up their wireframes and comps– Stories must contain specific accessibility
acceptance criteria• Can go to any level of detail– Test the specific alt attributes on images– Test the DOM order specified in the wireframe– Etc.
© 2015 - All Rights Reserved 19
Open Source Rules Engine
© 2015 - All Rights Reserved 20
aXe Manifesto
• Must have a zero false positive rate• Must be lightweight and fast• Must work in all modern browsers on all
platforms• Must be tested automatically
© 2015 - All Rights Reserved 21
aXe-Core
• Small ~100KB• Fast
– No network connection, no new browser start• Intelligent
– Supports cross domain iframes– Turns document level rules on/off automatically
• Unlimited– No throttling, no server limits
• Reliable– No double-execution or missing CSS problems– No network connectivity problems– No server capacity problems
• Secure– No server access across the wire– No sending data to a server
• Mozilla Public License 2.0– Available via GitHub, npm and Bower
© 2015 - All Rights Reserved 22
aXe Chrome extension
© 2015 - All Rights Reserved 23
aXe Firefox extension
© 2015 - All Rights Reserved 24
aXe extension
• Developers– Use when coding
• Testers– Use during functional testing
• Accessibility Experts– Use for validation
© 2015 - All Rights Reserved 25
aXe API
• Two major calls– axe.getRules() – returns a list of the rules loaded– axe.a11yCheck() – analyze a page or portion of a page
• Getting started– Install package
• bower install –-save-dev axe-core
– Include library• <script src=“(path_to)/axe.js”></script>
– Call function• a11yCheck(target, options, callback);
© 2015 - All Rights Reserved 26
Integrations
• Java Selenium• JavaScript Selenium (axe-webdriverjs)• Grunt (grunt-axe-webdriver)• Cucumber and R-Spec (axe-matchers)• Firefox Dev Tools• Chrome Dev Tools• Ember-aXe (ember-axe)
more…
© 2015 - All Rights Reserved 28
Summary
• Manage your matrix– Optimize for resources and risk
• Leverage automation– Generic automation– Specific automation– Optimize for up-front vs. ongoing effort
© 2015 - All Rights Reserved 29
Questions
• https://addons.mozilla.org/firefox/addon/axe-devtools/
• Chrome: search Google for “chrome axe extension”
© 2015 - All Rights Reserved 30