9) SIGISTS13 - Stuart Reid - ISO 29119 The New Standard€¦ · BS 7925-1 BS 7925-2 IEEE 829...

ISO/IEC/IEEE 29119

The New International Software Testing

Standards

Stuart Reid

Testing Solutions Group

117 Houndsditch

London EC3

UKTel: 0207 469 1500Fax: 0207 623 8459

www.testing-solutions.com

©Stuart Reid, 2013

• The purpose of standards

• Motivation for software testing standards

• Development of standards

• Overview of ISO 29119

• Applicability

• Timeline

• Future involvement

What are standards?

– Guidelines documents as they are not compulsory unless mandated by an individual or an organization

– Agreements because they should reflect a certain level of consensus

“Guideline documentation that reflects agreements on products, practices, or operations by nationally or internationallyrecognized industrial, professional, trade associations or governmental bodies”

Why use standards?

• Consumers

– Confidence in compliant products

– Authors provide expertise in standards

• Manufacturers

– Conformance and Marketing

– ‘Safety’ from liability

– Guidelines on production

• But not ‘Best Practice’…

Quality and Standards

What use are standards?

• Basis for:

– Communication – common terminology

– Professional qualifications

– Certification/compliance schemes

– Benchmark of ‘good industry practice’

– Contracts

– Interoperability and consistency……

Standards describe a current ‘body of knowledge’ that provides the basis for a professional discipline

A Case in Point

Motivation for ISO 29119

• Demand for existing 'standards’

• Conflicts in current definitions and processes

• Gaps in the current standards provision

• A Baseline for the Testing Discipline

• Current industry practice is lacking

• Buyers unclear on what is 'good test practice'

Standardization Bodies

ISO/IE

C Softw

are Standards

Published

Maintained

ISO/IEC SC7 – WG26

Quality

Information Technology

Software & Systems

Engineering

WG2 ..................

Software Testing

Representatives of National

Standards Bodies

Programming

Languages

IT Security

Data Management & Interchange

Dependability

ISO/IEC 29119 – Structure

BS 7925-1

BS 7925-2 IEEE 829

Concepts & Vocabulary

Part 1

ProcessAssessment

TestingTechniques

Part 4

Documentation

Part 3Part 2

Processes

Keyword-Driven Testing

Part 5 ISO/IEC 33063

SOFTWARE TESTING CONCEPTS

Scope, Conformance, Normative References

TESTING IN DIFFERENT LIFE CYCLE MODELS

ROLES AND RESPONSIBILITIES IN TESTING

ANNEXES – Metrics, Examples, Bibliography

DEFINITIONS

Part 1: Concepts & Vocabulary

Part 2: Testing Processes

TEST MANAGEMENT PROCESSES

ORGANIZATIONAL TEST PROCESS

DYNAMIC TEST PROCESSES

Instantiating Testing Processes

PLANNING

MONITORING &

CONTROL

COMPLETION

ORGANIZATIONAL TEST

DOCUMENTATION

FEEDBACK ONORGANIZATIONAL TEST DOCUMENTATION

TEST PLAN UPDATES

TESTPLAN

TESTCOMPLETIONREPORT

DYNAMIC TEST

PROCESSES

MANAGEMENT

PROCESSES

TEST PLAN,TEST COMPLETION REPORT,TEST MEASURES

TEST MEASURES

TEST PLAN,CONTROL DIRECTIVES

Test Management Processes

OrganiseTest Plan

Development

Identify & Estimate Risks

Design Test Strategy

Determine Staffing and Scheduling

Document Test Plan

Schedule, Staffing Profile

Test Strategy

Analyzed Risks

Identify Risk Treatment Approaches

Gain Consensus on Test Plan

ApprovedTest Plan

DraftTest Plan

Test Plan Publish

Test Plan

Understand Context

TreatmentApproaches

Test Planning Process

TESTSPECIFICATION

[NO ISSUESNOTICED]

TEST ENVIRONMENT READINESS REPORT

TESTENV’T

REQUIREMENTS

INCIDENTREPORT

TEST DESIGN &

IMPLEMENTATION

ENVIRONMENT

SET-UP

EXECUTION

INCIDENT

REPORTING

(LEVEL) TEST MANAGEMENT PROCESS

[ISSUE NOTICED orRETEST RESULT]

TEST RESULTS

CONTROL DIRECTIVES TEST

MEASURES

(LEVEL)TEST PLAN

Dynamic Test Processes

Test Process Descriptions Example- Organizational Test Process

• Each testing process is described using a standard template (following ISO 24774), for example:

• Purpose

– The purpose of the Organizational Test Process is to develop and maintain organizational test specifications, such as the Test Policy and Organizational Test Strategy.

• Outcomes

– As a result of the successful implementation of the Organizational Test Process:

• The organizational test specification is developed based on stakeholder requirements;

• The organizational test specification is agreed by stakeholders;

• The organizational test specification is published and accessible to stakeholders;

• Conformance of stakeholders with the organizational test specification is managed;

• The organizational test specification is reviewed on a periodic basis;

• Minor updates to the organizational test specification are made as necessary.

• Activities and tasks

– The tester shall implement the following activities and tasks in accordance with applicable organization policies and procedures with respect to the Organizational Test Process.

� Develop Test Specification

• This activity consists of the following tasks:

– Analyze any relevant source documents and the current testing practices within the organization to identify requirements for the organizational test specification.

– etc.

� Gain Consensus on Test Specification

� etc.

• Information items

– The output of the Organizational Test Process is the organizational test specification. The Organizational Test Policy and Organizational Test Strategy are typical examples of organizational test specifications.

Part 3 – Test Documentation

TEST DOCUMENTATION

ANNEXES - EXAMPLES

Test Documentation Types

• Organizational test documentation– Test policy– Test strategy

• Project test documentation – Project test plan– Test project completion report

• Test level documentation– Test plan– Test specification – Test results– Anomaly reports– Level test status report– Test environment report – Test level completion report 22

Part 4 – Test Techniques

TEST COVERAGE MEASUREMENT

ANNEXE – TESTING OF QUALITY CHARACTERISTICS

ANNEXE – SELECTION OF TECHNIQUES

ANNEXE – TEST TECHNIQUE EFFECTIVENESS

TEST DESIGN TECHNIQUES

BLACK BOX WHITE BOX

Are they applicable to you?

• Intended to be a generic standards

• Full vs. tailored conformance

• Small and large organizations

• Agile and Traditional

• Financial, Safety-Critical, Games...

• Following Regulatory Standards?

• Can you afford not to?

Current Status

May 10

May 11

May 12

May 13

Working Draft (WD)Committee Draft (CD)Draft International Standard (DIS)Final Draft International Standard (FDIS)Final International Standard (FIS)

Parts 1, 2 & 3Parts 1, 2 & 3

Part 4Part 4

WDWDCD1CD1

DISDISFDISFDISWDWD

CD1CD1

DISDIS

FDISFDISFIS!!!FIS!!!

CD2CD2CD3CD3

Part 5Part 5

CD4CD4

DIS-2DIS-2

WDWDCDCD

Current status

• Drafts are out for review– Unsurprisingly (for a testing standard) we get plenty of comments (1000s)

• Countries attending meetings include:– Argentina, Australia, Brazil, Canada, China, Colombia, Denmark, Finland, France, Germany, Hong Kong, India, Japan, Korea, Malaysia, New Zealand, Peru, Poland, Russian Federation, South Africa, Spain, Sweden, Turkey, UK, USA

• Last meeting – Montreal, May 2013

• Next meeting – Edinburgh, Nov 201326

What is most likely to stopISO 29119?

• Fear of change

• 'Not invented here' syndrome

• Perceived as competition

• Lack of required use

• Anti-standardization – craftsmen

• No evidence of efficacy

• Cost of use

• Complexity

• Lack of responsiveness

• Lack of inclusiveness…27

Conclusions

• Parts 1, 2 & 3 published September 2013

– My thanks to the editors and reviewers ☺

• International standard will provide practitioners with guidelines for testing that cover all aspects of the life cycle

– Provides a consistent set of definitions, processes, procedures & techniques for software testing

• Will be adopted by IEEE, BSI, ISO and other national standards bodies

Do you want to be involved?

• Join the working group

– representing your national standards body

– 6 day meetings, every 6 months

– contribute between meetings

• Join a WG26 mirror group

– for your national standards body

• Contribute materials

• Review drafts

Any Questions?

Any Volunteers?

Finally…

• sreid@testing-solutions.com

– if you have any questions on the standards

– if you are interested in trialling the standard on a project, reviewing drafts or writing examples

• http://softwaretestingstandard.org/

– WG26 website

• http://www.jtc1-sc7.org/

– access to official documents released by WG 26

9) SIGISTS13 - Stuart Reid - ISO 29119 The New Standard€¦ · BS 7925-1 BS 7925-2 IEEE 829...

Documents

Digital Industries Apprenticeship: Occupational Brief ... · • BS 7925-1 d. code of ethics for accessing private data Advises and supports others on testing processes and procedures

Republic act no 7925

ISO/IEC/IEEE 29119 - · PDF fileMadrid, 26th to 28th May 2014 ISO/IEC/IEEE 29119 The New International Software Testing Standards Stuart Reid ©Stuart Reid, 2015

ISSN 2072-7925 School closures and community ... · 1 School closures and community revitalisation: the case of Óbidos, Portugal ISSN 2072-7925 School closures and community revitalisation:

ISO 29119: Current Status & Future Plans · –based on ISO/IEC/IEEE 29119 standards •ISO 29119 Process Certification –based on •ISO 33063 Test Process Assessment (Process Assessment

ISSN 2072-7925 Well-being at school: does infrastructure ... · PDF filedoes infrastructure matter? ISSN 2072-7925 ... Instructional and Educational Science, ... information communication

Kaspar Loog kaspar.loog@knowit - ut · [ BS 7925-1] 2010-11-22 15 That the system is correct according to the written requirements. o

29119 SAP (170610) ru M.O. MAP II 3250

Robustness Testing of Explanation and Guide · The British Standard (BS) definition of negative testing in BS 7925-1 [BCS 98] is taken from Beizer [Beizer 90] and defines negative

Test management with iso 29119 building up an efficient test process

INTERNATIONAL ISO/IEC/ STANDARD IEEE/FDIS 29119-3 · 2021. 6. 6. · ISO/IEC/IEEE 29119-3 was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology, Subcommittee

ISO/IEC/IEEE 29119: The New International Software · PDF fileISO/IEC 29119 –Structure BS 7925-1 Concepts & Vocabulary Part 1 Part 4 Part 2 Part 3 Confidential - for limited circulation

“The Future of Software Testing” ISO 29119: The New ...€¦ · ISO 29119: The New International Software Testing Standard Stuart Reid, Testing Solutions Group, UK Europe’s

Intermediate Software Testing Syllabus V2 · PDF fileTesting Terms or from BS/ISO/IEC 29119-1:2013 , BS 7925-2:1998 , IEEE 829:2008 , and IEEE 610:1990 as appropriate to the specialist

ISO%2029119%20The%20New%20International%20Software ... · PDF fileFigure3:Timelines!for!ISO/IEC!29119! TheApproachtoTestinginISO’ 29119

Operative Hip Arthroscopy3A978-1-4419-7925-4%2F… · ISBN 978-1-4419-7924-7 ISBN 978-1-4419-7925-4 (eBook) DOI 10.1007/978-1-4419-7925-4 ... with the pathophysiological concept of

Report on the Metrics and Standards for Software Testing ... · Computer Society has given ISO permission to use the BS-7925-1/2 component testing standards as a basis for this part

ISO/IEC/IEEE 29119: The New International Software Testing ...€¦ · ISO/IEC/IEEE 29119: The New International Software Testing Standards Stuart Reid Testing Solutions Group London,

ISO 29119 and Software Testing - now what??

Digital Industries Apprenticeship: Occupational Brief ... · BS 7925-1 d. code of ethics for accessing private data Advises and supports others on testing processes and procedures