Reaching the Purple Standard

Reaching The Purple Standard

STPCon – Fall 2014

@2014 Jay Philips & Jean Ann Harrison

Your Presenters

Mobile & Software Test ConsultantEmail: [email protected]: @JA_HarrisonLinkedIn Profile: http://www.linkedin.com/pub/jeanann-harrison/4/b55/865/

Company: Project Realms & TeamQualityProWebsite: www.projectrealms.comwww.teamqualitypro.comEmail: [email protected]: @JayPhilipsLinkedIn Profile: www.linkedin.com/in/jayphilips


Agenda

• Definition of Standard

• Defect types, definitions and solutions

• Documentation

• Bug Reporting

• Tools & Characteristics


The Purple Standard Definition

• The Purple Standard is Prevention, Efficiency, and Learning

• The Purple Standard reaches a level of quality higher than The Gold Standard.

• Purpose: to release a software product with a higher quality standard than the last product released


The Purple Standard

• Achieve true commitment from all stakeholders.• Achieve Prevention of Defects throughout the

software project.• Set up of procedures to limit defects and increase

efficient testing, fixing, reviewing.• Achieve completion of test coverage to meet time

limits of the Software Project schedule.• Review mistakes for resolutions to be

implemented into the software project process to prevent future defects and maintain schedule integrity


Defect Types

There are a variety of defect types but do you know which ones are

which?

Can you name types of defects?


Requirement Defects

• Poorly written requirements can lead to design defects and can delay the project.

• Lack of requirements can lead to lack of design creating functional and performance defects.

• Understood but not documented requirements can lead to misunderstood design, function and use.


Design Defects

• Blockages of functions

• Inter-dependencies are not completely understood

• Variables are mislabeled within components or applications within a system

• Hidden or unknown resource limitations are reached


Performance DefectsTiming Defects

• What is the response from the application to install, to load a page, search for data, perform a function?

• Does the speed or lack of speed meet users’ needs?

• Was the need for speed defined in the requirements, follow a standard or is unknown?

Load/Stress Defects

• Does the application crash because too many users are on the system?

• Crashing occurs due to limited resources like memory or storage?

• Is the CPU load maxed out performing other functions?


Functional Defects

When an application or object within the application does not behave as stated in the business or design requirements.


User Experience Defects

• Look - display

• Feel - accessibility

• Performance

• Installations/Downloads

• Trainability - Intuitive Experience

• Recovery of Unexpected Behavior


System Integration Defects

• More than one application are part of a system to accomplish one task and that task cannot be completed due to a breakdown in connecting the applications.


Process Defects

• How software project activities and tasks are carried out throughout the project can have defects based on imposing “what we’ve always done” without learning from mistakes.

• Lack of processes developed can create a slow down or dead stop in a project.

• Procedures which are ambiguous with no detailed instructions.


Documentation Defects

• Documentation defects can be found when testing any written or pictorial information that describes, defines, specifies, reports, or certifies activities, requirements, procedures, or results.

• If the documentation is poor, non-existent, or wrong, it reflects on the quality of the product and the vendor.


Code Migration/Environment Defects

• Defects can also occur due to how code is migrated from one environment to another.

• Example: Patches need to be installed in order but if they are not could cause functionality issues.

• While you might think your environments are exactly the same that is not always true. Just having a different version of Java can cause environment issues.


Defect Priority

Severity Priority

Severity is associated with standards/functionality.

Priority is associated with scheduling.

Severity refers to the seriousness of the bug on the functionality of the product. Higher effect on the functionality will lead to assignment of higher severity to the bug.

Priority refers to how soon the bug should be fixed.

Generally, the Quality Assurance Engineer decides the severity level.

Priority to fix a bug is decided in consultation with the client/manager.


Does your desk look like this??


DOCUMENTATION ENVIRONMENTS

• Regulated: complying to standards in a particular industry– Purpose of Software, Hardware & Interaction

– Proof of Design, Testing and results• Requirements

• Test Cases, Test Reports,

– Bugs/issues found, fixed, tested, results

• Non-Regulated – Track design & testing coverage

– Record bugs


TESTING DOCUMENTATION

• Test Plan

• Test Cases

• Test Reports

• Test Summary Report

• Test Case Matrix

• Bug Reports

• Release Notes


Documentation Methods

• Written textural word processing document

• Checklist/Spreadsheet document

• Video format document

• Automating


Elements of an Efficient Test Plan

• Test Plan

– Introduction to include Objectives & Team

– Scope of testing

– Assumptions & Risks

– Test Approach including Automation

– Test Environment including Tools, Hardware etc

– Milestones & Schedule / Deliverables@2014 Jay Philips & Jean Ann Harrison

Efficient Tricks – Test Plan

• Keep test plan documentation general and brief

• It’s not necessary to document everything

• Test team – one meeting dedicated to work out plan, helps to train whole team

• Meeting should combine test types developing Test Approach, assign each category to team member for design


Efficient Tricks – Test Case Design• Discussion on pairing up testers in test design

– Testers writing requirements tests

– Highly experienced testers to develop combination tests & train less experienced

– Schedule the pairs to combine efforts on Exploratory Testing session

• Do NOT write out all the steps to perform the test. Write stories regarding test description


Efficient Tricks – Test Execution

• Pair Testing, video tape during test execution

• Combine tests which are related to not only test specific functionality but test inter-dependencies

• Automate overall expected functionality to become more efficient between builds and releases.


Efficient Environments

• Maintain your test environment – test new ones between releases

• Use the testing community for research on tools to help the testing effort

• Create new set ups of hardware where appropriate i.e. mobile labs


Schedules & Deliverables

• Create short term Milestones for Deliverables

• Testers

• Create Milestones for evaluation to maintain documentation and testing

• Add small contingency milestones

• Make deliverables a single task

• Schedule Bug Triage meetings early in schedule


Bug/Issues/Problems Reporting

This image is courtesy of CartoonTester: Andy Gloverhttp://cartoontester.blogspot.com/


Required Elements for a Bug Report?

• Brief one sentence desc

• Full desc to include equipment, software versions

• Steps to reproduce – do not assume any steps

• Expected Results

• Actual results including error messages, screen shots &/or video capturing bug

• Reproducible – how often (finding intermittent, how to reproduce…)


Your Toolbox

Tools

RequirementsManagement

Test Management

Test Execution

Test DesignAutomation & Performance

Defect Management

Utilize the right tools to increase speed to market, improve quality and cost


Elements of efficiency

» Testing assets, manual

and automated, are

stored in a central

repository.

» Standardized testing

tools are utilized to

increase speed to market

and improve productivity

and quality.

» Tooling are integrated to

provide efficiency.

» Test assets are easily

reused to reduce cost.

» Real-time reporting and

metrics are provided for

visibility and decision

making.

» Automatic document

versioning exists to

track changes.


Standard procedures• Requirements Reviews

– Test requirements for ambiguity, omission

– Get testers involved in requirement analysis

• Design/Engineering Reviews – Testers should be involved in design discussions

– Testers can start test design if understanding the design

• Code Reviews– Testers can start to test the system rather than unit or

component level

• Test Design Reviews– Get development to help review if the test coverage is

efficiently covering the scope of the project


The Purple Standard


Thank You• Thank you for attending our session. Please make

sure to fill out the feedback forms.


Image References• Slide 7: Requirement defects

• http://bit.ly/ZMAxOn

• Slide 9 Shopping Cart:

– http://bit.ly/1tIxogl

• Slide 11 User Expérience

– http://bit.ly/10irXYx

• Slide 12 System Integration

– http://bit.ly/1yfqhKq

• Slide 13 Process Defects

– http://bit.ly/1tIppzK

• Slide 14 Documentation Defects

– http://bit.ly/1oqmbPq

• Slide 17 Desk top Papers

– http://bit.ly/1sL0Uyu

• Slide 18 Audit image

– http://bit.ly/1uzCPIR

• Slide 19 Testing Documentation

– http://bit.ly/1uzCHsZ


• Slide 19 Test Case Documentation

– Http://bit.ly/1nxWprG

– Automation Testing

– http://bit.ly/1t8qn7F

• Slide 21 Test Planning

– http://bit.ly/1CQkiwk

• Slide 22 Efficient Tips

– http://bit.ly/1DwyPQd

• Slide 24 Test Execution

– http://bit.ly/1tIpZxE

• Slide 25 Test Environments

– http://bit.ly/ZMClXO

• Slide 33 Feedback

– http://bit.ly/1t8AqJN

http://bit.ly/ZMAxOn

http://bit.ly/1tIxogl

http://bit.ly/10irXYx

http://bit.ly/1yfqhKq

http://bit.ly/1tIppzK

http://bit.ly/1oqmbPq

http://bit.ly/1sL0Uyu

http://bit.ly/1uzCPIR

http://bit.ly/1uzCHsZ

http://bit.ly/1nxWprG

http://bit.ly/1t8qn7F

http://bit.ly/1CQkiwk

http://bit.ly/1DwyPQd

http://bit.ly/1tIpZxE

http://bit.ly/ZMClXO

http://bit.ly/1t8AqJN

Technology

Reaching the Purple Standard