A New Model for Testing

@pau

l_ge

rrard

Paul Gerrard paul@gerrardconsulting.com

gerrardconsulting.com

Programme Chair 2014

Agenda

• The Old Ways won't work in the future

• All Testing is based on models

• A New Model for Testing

• Tester Capabilities and Skills

• What next?

• Some Observations (themes that inspired this

work)

Typical tester job spec?

• Skills & Requirements – Thorough C# knowledge including Linq

– Good .Net experience including charting and winforms applications.

– Experience building and maintaining automated test systems

– Ability to interact with the developers to anticipate future needs

– Ability to influence to ensure good test procedures are developed and implemented

• Desirable Skills – Some knowledge of Java for mobile testing

– An interest in photography.

What's happening?

• Stampede to mobile computing

• Big Data

• Internet of (Every)Thing, Pervasive Computing

• Continuous Delivery, DevOps

• (Test) Analytics, Data-Driven Development

• Shift-left, embedded testers, no test team

• Did you really think Agile was the last word?

Three development patterns

Structured

Agile Continuous

Autonomous

There aren't three

patterns

There are many

The old ways won't

work in the future

All Testing is Based on

Models

Examples of models

Models are innate, essential, human

A New Model for

Testing

A Work in Process

ALL Testing is

Exploratory

We explore sources of knowledge ...

... to build test models ...

... that inform our testing.

Forget Logistics (for the time being)

Document or not?

Automated or manual?

Agile v waterfall?

Planned v exploratory?

This technology v that technology?

Judgement, exploring and testing

Testing (the system)

Our model(s) are adequate

Our model(s) are not adequate

Exploring (sources)

Judgement

Creates test models

Uses test models

We explore sources of knowledge to build test models that inform our testing

BTW – Do Developers explore the same way? I think so.

Exploration process

Exploration Definitions specs/stories

People (& you)

Sources

Require-ments

Test Models

Enquiring

Challenging

Sources: People, documents, experience, system under test

Modelling

Test Models: Can be documented

or mental models

Predicting

System under test

Testing process

Testing System

Under Test

Refining

Informing Applying

Interpreting

Test Models

Revise the System

More exploring Reporting

New Model Testing

Paper: http://dev.sp.qa/download/newModel

Consequences of the model

• ALL technologies, methods and logistics

• Aligns with/supports (tentatively): – Waterfall, Agile, Continuous Delivery, TDD/BDD etc.

– The Context-Driven View

– Functional, (Possibly all) Non-Functional types

• A different perspective on: – "Testing v Checking"

– Developer Testing, Embedded Testers, Shift Left

– Test automation

– Skills, capabilities and Certification.

Capabilities

Enquiring, Modelling, Predicting, Challenging

Informing, Applying, Interpreting, Refining

Reporting and Logging

New Model and Skills

• The 'New Model suggests that the core skills

of testers are somewhat different

– Somewhat different to those promoted by

'thought leaders'

– Very different to those promoted by the

certification schemes

• Of course interpersonal skills are, perhaps, the

most critical.

• Analysis, enquiry and elicitation

• Modelling

• Creation of custom models, using

heuristics, guesses, brainstorming,

ideation, creative thinking

• Custom test design techniques

• Comparison of models, value, advantages,

disadvantages, compromises

• Identification, validation and use of

oracles

• Predicate logic and proof

• Hypothesis and inference

• Socratic method

• Rapid Review and Inspection techniques

• Test case design

• Test models and the meaning of coverage

• Testing as controlled experiment

• Observation, Note taking, recording

A very different skillset • Basic data analysis and statistics

• Decision-making with incomplete data

• Computer forensics

• Fault tree analysis

• Failure diagnosis

• Bug advocacy, triage processes and

negotiation

• Meaningful software and test metrics

• Visual presentation of data

• Reporting and presentation skills

• Understanding stakeholders

• Test analytics

• Risk management, risk-based testing and

decision-making

• Critical Thinking

• Interpersonal skills

• Dealing with uncertainty/fallibility

Testing Career Development (speculative)

Foundations

Specialist/Technical

Management

Strategic

Test Strategy Project Intelligence

Test Assurance

Exploration Forensics Interpretation

Scripting/ Programming

Test Automation Technical (Excel, SQL, OS utils etc)

Stakeholder management

Analytics & visualisation

Managing uncertainty

Critical Thinking

ISTQB etc...

Supplier Management

Test Process Management

Methodology

People Skills for Better Business (courtesy of Susan Windsor)

Foundations

Specialist

Management

Strategic

Critical Thinking

Supplier Selection and Management

Stakeholder Identification

Creating Shared Goals

Collaborative Team Working

Managing Cultural Differences

Managing Challenging Conversations

Enhance Your Communication Style

Managing Self Assessment

Define Team Qualities and Competencies

Implementing Assurance

What can we do with this?

• A broader tester skills framework?

– New syllabi

– New skills and training

– A competitive certification scheme? (No, thanks)

• Alignment with BDD, ATDD, TDD...

• Flexibility (agility) to cope with any dev/test

process?

What Next?

The model needs better explanation,

examples, comparisons with existing

approaches

Some Observations

Some of the themes that inspired the

New Model and that it aims to

support

Shift-Left is More

Important than Agile

It's for everyone, not just small teams

Developers write code;

testers test because

developers can’t

No they don’t

Testing doesn’t just

apply to existing

software, at the end

We test systems, not software.

We test early, we test often but it

might not be called testing

Testers Need to Learn

How to Code

Some, perhaps many will have to

Testers don’t own

testing anymore

Be a TestMaster

"Assurance through the lifecycle"

The phase after

development is

REWORK, not Testing

Testing must align with development,

not compete with it or rescue it

A New Model for Testing

THANK YOU!

@pau

l_ge

rrard

Paul Gerrard paul@gerrardconsulting.com

gerrardconsulting.com

Programme Chair 2014