Embed Size (px)
Citation preview
Plugin Quality Check
Penny Wyatt
JIRA QA Team Lead
Atlassian
Quality Assistance
• Help devs to deliver value to customers
• Testing and testing advice
• Bug prevention
Plugin Quality Process
• Make it work
• Make it lustworthy
• Make it maintainable
• Make it safe
Setting the Quality Bar
• Tradeoff between time, scope and quality
• Mismatched quality expectations - unhappy customers
Make it work
“But I’ve already tested it, it works!”
in the real world
Happy Path Fallacy
• “We’ll only worry about the happy path, for now”
• Only test the actions a normal, reasonable user would
perform.
Happy Path Fallacy
• Quality bar defines what you fix, not what you test.
• OK not to support everything.
• Set customer expectations!
Valid Scenarios
• Sample Plugin
• Purpose-built but not contrived
• Completely unrelated to similar plugins on PAC!
• Only a short list, more detail on CAC
Valid Scenarios
• Different types of data
• Special characters
• Large data sets
• Long strings
Valid Scenarios
• Different browsers
• Layout issues
• Broken functionality
• Behaviour in unsupported browsers
Valid Scenarios
• Different use cases
• Session timeouts
• Deletions/Moves/Edits
• Anonymous access
Make it Lustworthy
User experience
• Guide users towards the actions they should do.
• Avoid empty boxes - use appropriate controls.
• Help them to avoid mistakes.
• Reading documentation should not be required.
Administration experience
• Clear flow for configuration
• Provide helpful error information
• Don’t show stack traces
• Use logging sparingly
Make it Maintainable
Manual and Automated Testing
• Manual testing
• Fast, effective, broad.
• Only tests the current state.
• Automated testing
• Scalable, sustainable.
• Takes time, limited assertions.
Automated Testing
• Good for long-term regression tests
• Optimise for:
• Scenarios most likely to break
• Integration with code out of your control
• Use page objects for UI tests for maintainability
25
Version Numbering
• Versions are constant
• A version number refers to one state of the code only.
• Never re-release a plugin with the same version number.
26
v2.5
v2.5v2.5
Latest?
Yes!
v2.5
Help!
v2.5
???
v2.5
Argh!
!!!!!!!!!
27
28
29
Version Numbering
• One version number means one binary.
• Always bump up the version number.
Make it Safe
How plugins break apps
• XSS
• Inserting user-supplied data into HTML without HTML-
encoding it.
• Allows an attacker to gain control of the victim’s browser.
How plugins break apps
• Insufficient permission checking
• Exposing data to anonymous users
• Not respecting permission schemes
How plugins break apps
• Performance
• Synchronous external requests
• Unbounded memory allocation
How plugins break apps
• Lack of CSS scoping
• Scope every item in the plugin CSS
• Avoid overriding built-in styles
• Lack of JavaScript scoping
(function () {
// code goes here
})();
Final check
Made it work
Made it lustworthy
Made it maintainable
Made it safe
Ship it!
