A Tale of Testing the U ntestable - uploads.pnsqc.orguploads.pnsqc.org/2019/slides/Jones-Testing the...

Preview:

Citation preview

A Tale of Testing the UntestableA Tale of Testing the Untestable

Angie Joneshttp://angiejones.tech

http://testautomationu.com@techgirl1908

Senior Developer Advocate, ApplitoolsDirector, Test Automation University

San Francisco, CA, USA

@tech

I needI needa change!a change!

@tech

QA @ TwitterQA @ Twitter

testing as a service@tech

TwitterTwitterAdsAds

@tech

Advertiser canAdvertiser cancreate a campaigncreate a campaignthat targets usersthat targets userswho meet certainwho meet certain

criteria.criteria.

FeatureFeature

@tech

WhatWhatare weare we

@tech

@te

@t

The testThe test

@tech

Challenge #1:Challenge #1:CompanyCompany

lacks testinglacks testingcultureculture

@tech

What aboutWhat aboutintegration?integration?

@tech

There'sThere'sa gapa gap

@tech

Challenge #2:Challenge #2:Company lacksCompany lacksresources forresources for

testingtesting@tech

Challenge #3:Challenge #3:Testing inTesting in

productionproduction@tech

TooTooexpensive!expensive!

@tech

Challenge #4:Challenge #4:TargetingTargetingtest userstest users

@tech

User ID Username Gender Interest1............. ............. F science2............. ............. M art3............. ............. N art4............. ............. U science5............. ............. M science6............. ............. F art7............. ............. F art

@tech

@tech

Challenge #5:Challenge #5:Blocker inBlocker in

targeting liststargeting lists@tech

...then how???...then how???

@tech

Challenge #6:Challenge #6:Serving is notServing is notguaranteedguaranteed

@tech

how does thishow does thisthing work?thing work?

@tech

Challenge #7:Challenge #7:Going lowerGoing lowerin the stackin the stack

@tech

No time!No time!

@techgirl1908

Challenge #8:Challenge #8:Making timeMaking time

for qualityfor quality@tech

NoNohappyhappyendingending

@tech

@tech

The 10 P's of TestabilityThe 10 P's of Testabilityby Rob Meaney

@tech

RetrospectiveRetrospective

@tech

PeoplePhilosophyProductProcessProblemProjectPipelineProductivityProduction IssuesProactivity @tech

The people in our teampossess the mindset,

skillset & knowledge setto do great testing and arealigned in their pursuit of

quality.@tech

The people in our teampossess the mindset,

skillset & knowledge setto do great testing and arealigned in their pursuit of

quality.@tech

The people in our teampossess the mindset,

skillset & knowledge setto do great testing and arealigned in their pursuit of

quality.@tech

PeoplePhilosophyProductProcessProblemProjectPipelineProductivityProduction IssuesProactivity @tech

The philosophy of ourteam encourages wholeteam responsibility for

quality and collaborationacross team roles, thebusiness and with the

customer.@tech

The philosophy of ourteam encourages wholeteam responsibility for

quality and collaborationacross team roles, thebusiness and with the

customer.@tech

The philosophy of ourteam encourages wholeteam responsibility for

quality and collaborationacross team roles, thebusiness and with the

customer.@tech

PeoplePhilosophyProductProcessProblemProjectPipelineProductivityProduction IssuesProactivity @tech

The product isdesigned to facilitate

great exploratorytesting and

@tech

The product isdesigned to facilitate

great exploratorytesting and

@tech

The product isdesigned to facilitate

great exploratorytesting and

@tech

PeoplePhilosophyProductProcessProblemProjectPipelineProductivityProduction IssuesProactivity @techg

The process helps theteam decompose work intosmall testable chunks and

discourages theaccumulation of

testing debt.@techg

The process helps theteam decompose work intosmall testable chunks and

discourages theaccumulation of

testing debt.@techg

The process helps theteam decompose work intosmall testable chunks and

discourages theaccumulation of

testing debt.@techg

PeoplePhilosophyProductProcessProblemProjectPipelineProductivityProduction IssuesProactivity @techg

The team has a deepunderstanding of theproblem the product

solves for their customerand actively identifies and

mitigates risk.@techg

The team has a deepunderstanding of theproblem the product

solves for their customerand actively identifies and

mitigates risk.@techg

The team has a deepunderstanding of theproblem the product

solves for their customerand actively identifies and

mitigates risk.@techg

PeoplePhilosophyProductProcessProblemProjectPipelineProductivityProduction IssuesProactivity @techg

The team is providedthe time, resources,

space and autonomyto focus & do great

@techg

The team is providedthe time, resources,

space and autonomyto focus & do great

@techg

PeoplePhilosophyProductProcessProblemProjectPipelineProductivityProduction IssuesProactivity @techg

The team’s pipelineprovides fast, reliable,

accessible andcomprehensive feedback

on every change as itmoves towards production.

@techg

The team’s pipelineprovides fast, reliable,

accessible andcomprehensive feedback

on every change as itmoves towards production.

@techg

PeoplePhilosophyProductProcessProblemProjectPipelineProductivityProduction IssuesProactivity @techg

For productivity, the teamconsiders and applies the

appropriate blend of testingto facilitate continuousfeedback and unearthimportant problems as

quickly as possible.@techg

For productivity, the teamconsiders and applies the

appropriate blend of testingto facilitate continuousfeedback and unearthimportant problems as

quickly as possible.@techg

For productivity, the teamconsiders and applies the

appropriate blend of testingto facilitate continuousfeedback and unearthimportant problems as

quickly as possible.@techg

PeoplePhilosophyProductProcessProblemProjectPipelineProductivityProduction IssuesProactivity @techg

The team has very fewcustomer impacting

production issues butwhen they do occur theteam can very quickly

detect, debug andremediate the issue.

@techg

The team has very fewcustomer impacting

production issues butwhen they do occur theteam can very quickly

detect, debug andremediate the issue.

@techg

PeoplePhilosophyProductProcessProblemProjectPipelineProductivityProduction IssuesProactivity @techg

The team proactivelyseeks to continuously

improve their testapproach, learn from theirmistakes and experiment

with new tools andtechniques.

@techg

The team proactivelyseeks to continuously

improve their testapproach, learn from theirmistakes and experiment

with new tools andtechniques.

@techg

The team proactivelyseeks to continuously

improve their testapproach, learn from theirmistakes and experiment

with new tools andtechniques.

@techg

PeoplePhilosophyProductProcessProblemProjectPipelineProductivityProduction IssuesProactivity @techg

A Tale of Testing the UntestableA Tale of Testing the Untestable

Angie Joneshttp://angiejones.tech

http://testautomationu.com@techgirl1908

Senior Developer Advocate, ApplitoolsDirector, Test Automation University

San Francisco, CA, USA

Recommended