66
Today’s webinar is presented by Jonathon Wright and he will discuss Testing as a Service - Models. Welcome to the EuroSTAR November Webinar Series This webinar is due to start at 2pm. Make sure you stick around at the end for the Q&A session and continue the conversation with the speaker on Twitter after the show! @esconfs #esconfs www.eurostarconferences.com

EuroSTAR - Testing as a Service Model

Embed Size (px)

Citation preview

Page 1: EuroSTAR - Testing as a Service Model

Today’s webinar is presented by Jonathon Wright and he will discuss Testing as a Service - Models.

Welcome to the EuroSTAR November

Webinar Series

This webinar is due to start at 2pm. Make sure you stick around at the end for the Q&A session and continue the conversation with the speaker on Twitter after the show!

@esconfs

#esconfswww.eurostarconferences.com

Page 2: EuroSTAR - Testing as a Service Model

Testing as a Service - Models

Jonathon Wright, Director, Testing, Quality, AssuranceHitachi Consulting, Global

Jonathon Wright has over 15 years of commercial automation experience with a number of international organisations. Jonathon also contributed to the best-selling book ‘Experiences of Test Automation: Case Studies of Software Test Automation (2012)’ and a number of upcoming books on test automation and

API testing in the cloud. He is an active blogger on ‘Test Automation as a Service’ (TaaaS.net) as well as presenting at various international testing conferences

Gartner (London) / STARWest (California) / Fusion (Sydney) / ANZTB (Melbourne) / BCS SIGIST (London)

@esconfs#esconfs www.eurostarconferences.com

Page 3: EuroSTAR - Testing as a Service Model

Q&A

If you have a question, the control panel on the right has a Questions box. All questions will be answered at the end of the presentation.

The GoToWebinar attendee interface is made up of two parts. The Viewer Window shows the presenter’s screen. The Control Panel is where attendees can interact and ask questions.

The webinar will be recorded and you will be notified via email when it is available.

@esconfs#esconfs www.eurostarconferences.com

Page 4: EuroSTAR - Testing as a Service Model

How can I get the slides?

Slides and all materials will be posted on the EuroSTAR website as well as emailed to you.

@esconfs#esconfs www.eurostarconferences.com

http://www.slideshare.net/jonathon_Wright

Page 5: EuroSTAR - Testing as a Service Model

Join the conversation on Twitter

#esconfs

@esconfs

@Jonathon_Wright

Continue the conversation with the speaker on Twitter after the show!

@esconfs#esconfs www.eurostarconferences.com

Page 6: EuroSTAR - Testing as a Service Model

@jonathon_wright

#esconfs www.eurostarconferences.com

Better TaaS

5th November 2014

[email protected]

Testing as a Service - ModelsJonathon Wright, Director, Testing, Quality, Assurance

Page 7: EuroSTAR - Testing as a Service Model

7

@jonathon_wright

#esconfs www.eurostarconferences.com

Hitachi - a global industry leader.Society Changes, Hitachi Transforms It.

$93.4

Billion947Companies

320,000+Employees

$3.4

Billion

Hitachi ranks 78th on the 2014 FORTUNE Global 500®.

© Hitachi Europe Ltd. 2014. All rights reserved.

Page 8: EuroSTAR - Testing as a Service Model

8

@jonathon_wright

#esconfs www.eurostarconferences.com

Hitachi – in Europe

Information & Telecommunication Solutions Distribution Solutions

Railway Solutions Smart Cities

Service

IT Service, Data Centre Solutions, Consulting

System Solutions

Large-scale systems, Highly reliable systems

Finger Vein Authentication

Healthcare

Hardware, Software

Rolling stock + maintenance

Electrical components

Signaling / Train control systems

Nuclear Power Business

Mining Equipment

Platforms

Smart eco cities

Big Data Infrastructure

Energy Consumption

@jonathon_wright

#esconfs

Water & Natural Resources

Page 9: EuroSTAR - Testing as a Service Model

9

@jonathon_wright

#esconfs www.eurostarconferences.com

Hitachi – Smart Cities - Copenhagen

Effecting the future home of EuroSTAR (2000, 2005, 2010) Copenhagen Cleantech Cluster

City

Admin

Mobility

"Green and Comfortable

Transportation"

"Lower energy consumption"

- City of bikes

- ITS

- Mobility Management

- Copenhagen Connecting

- Buildings

- Transport

- Procurement

- Street Lighting

Energy

"Carbon-neutral city"

- Smart meter by 2020

- Wind turbines

- Biomass in power plant

- Smart Cities

@jonathon_wright

#esconfs

Carbon Neutrality

by 2025

Page 10: EuroSTAR - Testing as a Service Model

10

@jonathon_wright

#esconfs www.eurostarconferences.com

Testing industry is changing forever

Intel© 14nm SoC in 2013

Does ‘Testing as a Function’ still have a future?

Page 11: EuroSTAR - Testing as a Service Model

11

@jonathon_wright

#esconfs www.eurostarconferences.com

Testing as a Function

Legacy Methodologies

Software Development Lifecycle

Hardware Analysis

Testing

Process

MethodologyBusiness

Software Development Lifecycle (SDLC) Model

Page 12: EuroSTAR - Testing as a Service Model

12

@jonathon_wright

#esconfs www.eurostarconferences.com

Testing industry is not ready

Support for business intelligence powered by testing ‘small data’

Hitachi Content Platform Solution

During deployment of

the Hitachi Content

Platform Solution

70TB of satellite

information was

ingested into the

solution platform with

a daily growth rate of

50GB/day.

Business

Intelligence

Platforms

Business

Intelligence

Infrastructure

* http://www.hds.com/assets/pdf/hitachi-storage-solutions-at-work-nasa.pdf

Page 13: EuroSTAR - Testing as a Service Model

13

@jonathon_wright

#esconfs www.eurostarconferences.com

Testing as a Activity

Future Methodologies

Solution Delivery Lifecycle

Application

Portfolio

Business

IntegrationEnterprise

Management

Solution Delivery Lifecycle integration (SDLCi) Model

Page 14: EuroSTAR - Testing as a Service Model

14

@jonathon_wright

#esconfs www.eurostarconferences.com

Testing industry needs new models

Global challenges are not suitable for ‘Testing as a Function’ silos

Complex eco-

infrastructures

intelligent

machine-to-

machine inter-

connections

empowered by

‘Big Data’

* http://www.libelium.com/libelium-smart-world-infographic-smart-cities-internet-of-things/

Page 15: EuroSTAR - Testing as a Service Model

15

@jonathon_wright

#esconfs www.eurostarconferences.com

New Model for Testing*

Model Based Testing (MBT)

*Paul Gerrard, 2014, ‘New Model for Testing’ discussion paper published, 6th August 2014

New Model for Testing

Page 16: EuroSTAR - Testing as a Service Model

16

@jonathon_wright

#esconfs www.eurostarconferences.com

Model Based Design (MBD)

Advanced Modelling Simulation Technologies

Watershed management

Pipe-network Management

Prediction of water pollution

Pipe-network analysis

© Hitachi Europe Ltd. 2014. All rights reserved.

Models can be analysed, visualized, and optimized

Models can provide assurance through, analysis, simulation and prediction

* http://www.computerweekly.com/news/2240233919/Societys-values-moving-from-Mono-to-Koto-says-Hitachi

Page 17: EuroSTAR - Testing as a Service Model

17

@jonathon_wright

#esconfs www.eurostarconferences.com*Jonathon Wright, 2014, ‘Testing as Service’ Models’ discussion paper published, 26th October 2014

Testing as a Service - Models

@jonathon_wright

#esconfsTesting as a Service - Models

https://leanpub.com/taas

Page 18: EuroSTAR - Testing as a Service Model

18

@jonathon_wright

#esconfs www.eurostarconferences.com*Gartner, 2014, ‘Building your Digital Enterprise’ Gartner event, Poland, 15th May 2014

BUILD DIGITAL BLUEPRINT + ROADMAP

Why ‘Digital Enterprise’

‘Digitalised’ businesses are able to better adapt and exploit opportunity

@jonathon_wright

#esconfsTesting in the Digital Enterprise

Page 19: EuroSTAR - Testing as a Service Model

19

@jonathon_wright

#esconfs www.eurostarconferences.com*Jonathon Wright, 2014, ‘Testing as Service’ Models’ discussion paper published, 26th October 2014

Business Delivery Management

Model Based Design (MBD)

Testing as a Service - Models

Business Lifecycle Management

Problem

Business Abstraction Layer

Service Abstraction Layer - Solution Delivery Lifecycle integration (SDLi)

Business Delivery Management - ‘As A Service’ layer

Test Abstraction Layer – Test Solution Delivery

Business Delivery Management - ‘Testing as a Service’ layer

Portfolio Lifecycle Management

Solution Lifecycle Management

Application Lifecycle Management

Test Abstraction Layer – Test Solution Delivery

Solution

Page 20: EuroSTAR - Testing as a Service Model

20

@jonathon_wright

#esconfs www.eurostarconferences.com*Jonathon Wright, 2014, ‘Testing as Service’ Models’ discussion paper published, 26th October 2014

Testing as a Service - Models

• Business Lifecycle Management

– Business on a Page (BoaP) level abstraction or encapsulation representing the core

businesses differentiators such as agility, visibility and flexibility

• Portfolio Lifecycle Management

– number of solutions that are represented by a logical grouping as part of a Business Portfolio

or split by Business Domain or Workstream

• Solution Lifecycle Management

– covers logical groupings of applications that make up the solution landscape

• Application Lifecycle Management

– refers to a single application instance

Business Delivery Management

Business Lifecycle Management

Page 21: EuroSTAR - Testing as a Service Model

21

@jonathon_wright

#esconfs www.eurostarconferences.com*Jonathon Wright, 2014, ‘Testing as Service’ Models’ discussion paper published, 26th October 2014

Business Abstraction Layer

Business Abstraction Layer

Model Based Design (MBD)

Testing as a Service - Models

Business Solution Discovery Business Solution Delivery

Business

on a Page

Business Level

Answers

Business Level

Questions

Business Level

Stories

Business

Requirements

Business Process

Design

Business Process

Modelling

Business Process

Validation

Business Process

Intelligence

Business Process

Scenarios

Business Process

Components

Business Process

Tests

Business Process

Data

Business Process

Transactions

Business User

Stories

Business Level

Keywords

Business Non-

Functional (NFRs)

Business Epic

Themes

Business Definition Layer

Business

Acceptance Criteria

Business Definition

Language

Business

Acceptance Tests

Business Domain

Knowledge

Page 22: EuroSTAR - Testing as a Service Model

22

@jonathon_wright

#esconfs www.eurostarconferences.com

Business Delivery Management

Model Based Design (MBD)

Business Abstraction Layer

Business Solution Discovery Business Solution Delivery

Business Definition Layer

Test Abstraction Layer

Test Solution Discovery Test Solution Delivery

Test Definition Layer

Problem

Business Delivery Management (‘As A Service’ Abstraction Layer)

Service Abstraction Layer - Solution Delivery Lifecycle integration (SDLi)

*Jonathon Wright, 2014, ‘Testing as Service’ Models’ discussion paper published, 26th October 2014

Testing as a Service - Models

Solution

Logistics

Page 23: EuroSTAR - Testing as a Service Model

23

@jonathon_wright

#esconfs www.eurostarconferences.com

Why ‘As A Service’

• Testing as a Service (TaaS)

• Software as a Service (SaaS)

• Platform as a Service (PaaS)

• Infrastructure as a Service (IaaS)

• Hardware as a Service (HaaS)

Testing in the Wild - Autonomous Vehicles

@jonathon_wright

#esconfs *Jonathon Wright, Top Gear Track, Test Results: 1:47.63, http://youtu.be/P8JnQV0nXrc, 31st October 2014

Performance Test Results: 1:47.63

Page 24: EuroSTAR - Testing as a Service Model

24

@jonathon_wright

#esconfs www.eurostarconferences.com*Jonathon Wright, 2014, ‘Testing as Service’ Models’ discussion paper published, 26th October 2014

Problem

Business

Customer

SolutionProblem

Definition

Business

Intelligence

(CIO, COO, CTO)

Solution

Decommission

Social

Intelligence

Business

Innovation

Social

Innovation

‘As A Service’ - Model

Model Based Design (MBD)

Testing as a Service - Models

Business Problem, Idea or Challenge (cost, benefit, time to market)

Business Solution Delivery Quality (cheap, good, fast)

‘As A Service’

CustomerEnd-Users

Page 25: EuroSTAR - Testing as a Service Model

25

@jonathon_wright

#esconfs www.eurostarconferences.com*Jonathon Wright, 2011, ‘Testing as a Service’ presented at BCS SIGIST, London, 5th December 2011

Business Delivery Management

Service Abstraction Layer - Solution Delivery Lifecycle integration (SDLCi)

Testing as a Service - Models

Portfolio Lifecycle Management

Business Lifecycle Management

Solution Lifecycle ManagementBusiness Delivery Management

AdviceProblem

Business Solution

Discovery

Business Solution

Delivery

Page 26: EuroSTAR - Testing as a Service Model

26

@jonathon_wright

#esconfs www.eurostarconferences.com

Why ‘Testing As A Service’

• Test Platform as a Service (TPaaS)

• Test Infrastructure as a Service (TIaaS)

• Test Automation as a Service (TAaaS)

• Functional Testing as a Service (FTaaS)

• Performance Testing as a Service (PTaaS)

• Mobile Testing as a Service (MTaaS)

• Security Testing as a Service (STaaS)

Testing in the Wild – Near Field

Communications (NFC)

• US Census Bureau* world population

counter 7.2 billion people on the planet

• GSMA Intelligence* over 7.2 billion mobile

devices operating in the world right now

• CNET* over 250 million machine-to-machine

connections

* http://www.cnet.com/news/there-are-now-more-gadgets-on-earth-than-people/

Testing as a Service - Models

Page 27: EuroSTAR - Testing as a Service Model

27

@jonathon_wright

#esconfs www.eurostarconferences.com*Jonathon Wright, 2014, ‘Testing as Service’ Models’ discussion paper published, 26th October 2014

Model Based Design

Testing as a Service - Models

Shift Left

Shift Up

Shift Down

Shift Right

Test

in

Dev

Business

Acceptance

Testing

Test

Ops

User

Acceptance

Testing

Social

Enterprise

Solution Delivery Lifecycle integration (SDLi)

Continuous, Integration, Build & Deployment (CIBD)

Business

Enterprise

(CIO, COO, CTO)

Testing as a Service - Model

Page 28: EuroSTAR - Testing as a Service Model

28

@jonathon_wright

#esconfs www.eurostarconferences.com

• Test Platform as a Service (TPaaS)

– Social Enterprise Collaboration

– Actionable Business Insight

– Enterprise Gamification

– Global Testing Marketplace

– All-Channel Customer Experience• Test Infrastructure as a Service (TIaaS)

• Test Automation as a Service (TAaaS)

• Functional Testing as a Service (FTaaS)

• Performance Testing as a Service (PTaaS)

• Mobile Testing as a Service (MTaaS)

• Security Testing as a Service (STaaS)

Business Delivery Management - ‘Testing as a Service’ layer

Testing as a Service - Models

Test Platform as a Service (TPaaS) Model

Page 29: EuroSTAR - Testing as a Service Model

29

@jonathon_wright

#esconfs www.eurostarconferences.com

Social Enterprise Collaboration

• Delivering true business value – Not just driving process efficiency

– Across the SDLCi with cross-functional domain tools

– Providing instant actionable business insight

• Supporting individual, team, organizational, and enterprise collaboration

– Integration with the Global Testing Marketplace (GTM) platform

– Produce high-quality reusable business assets in the cloud

– Capture specialist technical and cross-domain knowledge (i.e. DIDO)

– Game mechanics to encourage collaboration (i.e. Enterprise Gamification)

– Produce design patterns that allow business rules to be codified in business-specific meta-languages (i.e. BPMNv2.2, xPDL, XAML)

• Managing software that gets deployed everywhere

– All-Channel Customer Experience

*Jonathon Wright, 2014, ‘Agile Portfolio Management in the Cloud’ presented at Unicom, London, 20th March 2014

Test Platform as a Service (TPaaS) Model

Embracing Agile Portfolio Management in the Cloud

Page 30: EuroSTAR - Testing as a Service Model

30

@jonathon_wright

#esconfs www.eurostarconferences.com* Huw Price, 2014, Grid-Tools.com, https://www.grid-tools.com/solutions/agile-designer/ , 3rd November, 2014

Business Delivery Value Chain - SDLCi

Requirement <-> Traceability <-> Use Cases <-> Traceability <-> Test Case <-> Traceability <-> Defect

Agile Designer

HP ServiceVirtualization

DataMaker

HP QCHP ALMSelenium

AgileBoards

• Test Cases• Virtual Data• Automation

• Find• Allocate• Synthesize• Data On Demand

• Requirements• Test Cases• Test Data• Defects• Automation

• Story Boards• Work Allocation• Work Tracking• Story Points

• Use Cases• Requirements• Complexity• Test Cases• Test Data• Virtual Data• Automation• Expected Results

• Backlog

Development

Functional

Integration

System Testing

Performance

Acceptance

Developers

Testers

Business

Test Platform as a Service (TPaaS) Model

S

D

L

C

i

Page 31: EuroSTAR - Testing as a Service Model

31

@jonathon_wright

#esconfs www.eurostarconferences.com

Actionable Business Insight powered by ‘Small Data’

Funnel Virtualisation

BusinessInsight

Data

Data

Data

Data

Data

Data

Data

Data

Data

Data

Data

Data

DataData

Data

Data

DataData

* Llyr Jones, 2014, Grid-Tools.com, https://twitter.com/GridTools/status/487264150578790400 , 10th July 2014

Test Platform as a Service (TPaaS) Model

Page 32: EuroSTAR - Testing as a Service Model

32

@jonathon_wright

#esconfs www.eurostarconferences.com

Enterprise Gamification

Service Thinking (Service as a Experience) Business Model

G

A

M

I

F

I

C

AT

I

O

N

E

N

T

E

R

P

R

I

S

E

Shift Left

Shift Up

Shift Down

Shift Right

Test

in

Dev

Business

Acceptance

Testing

Test

Ops

User

Acceptance

Testing

Social

Enterprise

Test Platform as a Service (TPaaS) Model

* http://enterprise-gamification.com/attachments/article/233/Improving%20Software%20Quality%20with%20Gamification%20-%20Full%20Paper.pdf

Business

Enterprise

(CIO, COO, CTO)

Page 33: EuroSTAR - Testing as a Service Model

33

@jonathon_wright

#esconfs www.eurostarconferences.com

Enterprise Gamification

Multi-sided Metrics

Test Platform as a Service (TPaaS) Model

Page 34: EuroSTAR - Testing as a Service Model

34

@jonathon_wright

#esconfs www.eurostarconferences.com

Global Testing Marketplace

Become a Testing Rock Star!

*http://blog.utest.com/2014/11/04/utest-platform-preview-new-dashboard-for-testers-on-paid-projects , 4th Nov 2014

Test Platform as a Service (TPaaS) Model

@jonathon_wright

#esconfs

Page 35: EuroSTAR - Testing as a Service Model

35

@jonathon_wright

#esconfs www.eurostarconferences.com

Global Testing Marketplace

Improved Communication, Collaboration and Mobility

*Jonathon Wright, 2012, ‘Test Automation as a Service’ at Fusion, Sydney, 10th September 2012

Test Platform as a Service (TPaaS) Model

Test©

Page 36: EuroSTAR - Testing as a Service Model

36

@jonathon_wright

#esconfs www.eurostarconferences.com

Global Testing Marketplace

Instant Scalability, Flexibility and Availability

@jonathon_wright

#esconfs *Jonathon Wright, 2014, ‘Solution Lifecycle Management in the Cloud’ at Unicom, London, 27th February 2014

Test Platform as a Service (TPaaS) Model

G

L

O

B

A

L

M

A

R

K

E

T

P

L

A

C

E

Page 37: EuroSTAR - Testing as a Service Model

37

@jonathon_wright

#esconfs www.eurostarconferences.com

All-Channel Customer Experience

The digital journey for transformation or convergence

@jonathon_wright

#esconfs

S

O

C

I

A

L

I

N

T

E

L

L

I

G

E

N

C

E

@jonathon_wright

#esconfs Test Platform as a Service (TPaaS) Model

Page 38: EuroSTAR - Testing as a Service Model

38

@jonathon_wright

#esconfs www.eurostarconferences.com

• Test Platform as a Service (TPaaS)

• Test Infrastructure as a Service (TIaaS)

– H(API)y Testing in the Cloud

– Test First Delivery

– Building bridges in the Cloud

– Cloud 2.0 beyond legacy

• Test Automation as a Service (TAaaS)

• Functional Testing as a Service (FTaaS)

• Performance Testing as a Service (PTaaS)

• Mobile Testing as a Service (MTaaS)

• Security Testing as a Service (STaaS)

Business Delivery Management - ‘Testing as a Service’ layer

Testing as a Service - Models

Test Infrastructure as a Service (TIaaS) Model

Page 39: EuroSTAR - Testing as a Service Model

39

@jonathon_wright

#esconfs www.eurostarconferences.com

Ha(API)y Testing in the Cloud

• Build Bridges in the Cloud

– Use Cloud Maps to Learn, Discover and Model endpoint(s)

– Create Bridges (VNet to VNet) “If you build it, they will test” (IYBITWT)

– Embrace Test Infrastructure as a Service (TIaaS) in the Hybrid Cloud (TQA™ Lab)

• Avoid Legacy Cloud

– Heterogeneous Test Infrastructure as a Service (TIaaS) that are open, flexible and extensible

– Continuous Build, Integration & Delivery in YourTestCloud

– Embrace Community Test Clouds (Collaborate/Share Recipes)

• First Day Delivery

– Enable Day-zero Performance, Penetration & Security Testing as a Service

• Support for powerful Test Automation as a Service (TAaaS)

– UI Testing that is Scalable and Cost-Efficient

– API Testing that is Repeatable, Reliable and Fast

Unlock true delivery innovation

*Jonathon Wright, 2014, ‘Ha(API)y testing in the hybrid-Cloud & beyond’ presented at Gartner ADM, London, May 19th 2014

Test Infrastructure as a Service (TIaaS) Model

Page 40: EuroSTAR - Testing as a Service Model

40

@jonathon_wright

#esconfs www.eurostarconferences.com

Building Bridges in the Cloud

“If you build it, they will test” (IYBITWT)

*Jonathon Wright, 2014, ‘Ha(API)y testing in the hybrid-Cloud & beyond’ presented at Gartner ADM, London, May 19th 2014

Test Infrastructure as a Service (TIaaS) Model

Page 41: EuroSTAR - Testing as a Service Model

41

@jonathon_wright

#esconfs www.eurostarconferences.com

TQA™ Lab

Building Bridges in the Cloud

HP ALM (SaaS)HP PC (SaaS)HP PC

(Controller)

HP PC

(Load Generators)

HP PC

(Load Generators)

Hybrid Cloud

Public Cloud

HP PC

(Load Generators)

Private Cloud (Performance)

HP PC

(MOFW)

Solution Under Test (SUT)Client X

HP CodedFT

Private Cloud (Automation)

Agile Designer

Solution Under Test (SUT)Client X

VN

et-

to-V

Net

HP SV/NV

(SaaS)

HP Agile Manager

(SaaS)

HP StormRunner

(SaaS)

HP ALI (SaaS)

* http://h30499.www3.hp.com/t5/Unified-Functional-Testing/UFT-12-Test-Execution-in-the-Cloud-vs-Local-VMM/td-p/6571480

Cores: 72

RAM: 150gb

Nodes: 144=

SV (Servers) = 50

SV (Designers) = 60

PC (Generators) = 15k

UFT (API) = 100

UFT (Runtime) = 1440

Test Infrastructure as a Service (TIaaS) Model

D

R

D

R

Page 42: EuroSTAR - Testing as a Service Model

42

@jonathon_wright

#esconfs www.eurostarconferences.com

Cloud 2.0

Moving beyond the legacy cloud

Test Infrastructure as a Service (TIaaS) Model

select <IaaS>

select <PaaS>

select <SaaS>

select <TaaS>

select <TaaaS>

Cloud Recipe for Solution Under Test

Cloud Recipe (IFTTT)

CloudSystem Automation

YourTestCloud.com

“Heterogeneous test cloud that are open, flexible and extensible”

Testing Cloud

If <Solution Under Test>

Then <Build Cloud 2.0>

Testing Cloud

*Jonathon Wright, 2014, ‘Ha(API)y testing in the hybrid-Cloud & beyond’ presented at Gartner ADM, London, May 19th 2014

Page 43: EuroSTAR - Testing as a Service Model

43

@jonathon_wright

#esconfs www.eurostarconferences.com

Test First Delivery

Service / Network Virtualisation as part of Test Driven Delivery integration (TDDi)

*Jonathon Wright, 2014, ‘Ha(API)y testing in the hybrid-Cloud & beyond’ presented at Gartner ADM, London, May 19th 2014

Test Infrastructure as a Service (TIaaS) Model

Page 44: EuroSTAR - Testing as a Service Model

44

@jonathon_wright

#esconfs www.eurostarconferences.com

• Test Platform as a Service (TPaaS)

• Test Infrastructure as a Service (TIaaS)

• Test Automation as a Service (TAaaS)

– Test Automation in the Cloud

– AMMi© Assessment Model

– First Day Automation

• Functional Testing as a Service (FTaaS)

• Performance Testing as a Service (PTaaS)

• Mobile Testing as a Service (MTaaS)

• Security Testing as a Service (STaaS)

Test Automation as a Service (TAaaS) Model

Business Delivery Management - ‘Testing as a Service’ layer

Testing as a Service - Models

Page 45: EuroSTAR - Testing as a Service Model

45

@jonathon_wright

#esconfs www.eurostarconferences.comhttp://partners.telerik.com/docs/default-source/Case-Studies/telerik-test-automation-in-the-cloud_automation-development-servicesBB95938E9FB5.pdf

Test Automation as a Service (TAaaS) Model

Test Automation in the Cloud: Case Study

Page 46: EuroSTAR - Testing as a Service Model

46

@jonathon_wright

#esconfs www.eurostarconferences.com

AMMi© Assessment Model

The digital journey toward technical delivery enlightenment

AMMi Phase Target Criteria

Level 0 Accelerating Automated Test Lifecycle Methodology (ATLM)

Level 1 Traditional Framework Driven (Gen 1-5)

Level 2 Managed Processes are planned, performed, measured, and controlled

Level 3 Sustaining Automation process defined and validated against international

standards (ISO-29119)

Level 4 Quantified Predictability of automation process performance

Level 5 Optimising Automation process variation and statistical predictability

* Source: AMMi®(http://www.AMMi.org.uk) + Test Automation Patterns (http://TestAutomationPatterns.wikispaces.com)

Test Automation as a Service (TAaaS) Model

Page 47: EuroSTAR - Testing as a Service Model

47

@jonathon_wright

#esconfs www.eurostarconferences.com

First Day Automation: Test Automation Experiences Book

* Dorothy Graham, ’Experiences of Test Automation’, http://www.dorothygraham.co.uk/automationExperiences/, ISBN-10: 0321754069, 9th January 2012

Test Automation as a Service (TAaaS) Model

Page 48: EuroSTAR - Testing as a Service Model

48

@jonathon_wright

#esconfs www.eurostarconferences.comFunctional Testing as a Service (FTaaS) Model

• Test Platform as a Service (TPaaS)

• Test Infrastructure as a Service (TIaaS)

• Test Automation as a Service (TAaaS)

• Functional Testing as a Service

– Shift everywhere (UP, DOWN, LEFT, RIGHT)

– Continuous, Integration, Build & Delivery

• Performance Testing as a Service (PTaaS)

• Mobile Testing as a Service (MTaaS)

• Security Testing as a Service (STaaS)

Business Delivery Management - ‘Testing as a Service’ layer

Testing as a Service - Models

Page 49: EuroSTAR - Testing as a Service Model

49

@jonathon_wright

#esconfs www.eurostarconferences.com

Shift Everywhere

Test in Business, Test in Dev, Test Ops, Test in the Wild

Functional Testing as a Service (FTaaS) Model

Test in Business Test in Dev Test in the Wild

Page 50: EuroSTAR - Testing as a Service Model

50

@jonathon_wright

#esconfs www.eurostarconferences.com

Functional Testing in UFT 12 and beyond (i.e. CodedFT)

* https://www.packtpub.com/application-development/advanced-qtp-115-test-engineers-cookbook

@jonathon_wright

#esconfsFunctional Testing as a Service (FTaaS) Model

Page 51: EuroSTAR - Testing as a Service Model

51

@jonathon_wright

#esconfs www.eurostarconferences.com

Continuous, Integration, Build & Delivery

Accelerate time to market

@jonathon_wright

#esconfsFunctional Testing as a Service (FTaaS) Model

Page 52: EuroSTAR - Testing as a Service Model

52

@jonathon_wright

#esconfs www.eurostarconferences.com

• Test Platform as a Service (TPaaS)

• Test Infrastructure as a Service (TIaaS)

• Test Automation as a Service (TAaaS)

• Functional Testing as a Service (FTaaS)

• Performance Testing as a Service

– Performance Testing in the Cloud

• Mobile Testing as a Service (MTaaS)

• Security Testing as a Service (STaaS)

Business Delivery Management - ‘Testing as a Service’ layer

Testing as a Service - Models

Performance Testing as a Service (PTaaS) Model

Page 53: EuroSTAR - Testing as a Service Model

53

@jonathon_wright

#esconfs www.eurostarconferences.com

Performance Testing in the Cloud

PTaaS – Microsoft – Visual Studio Team Foundation Service

Performance Testing as a Service (PTaaS) Model

Page 54: EuroSTAR - Testing as a Service Model

54

@jonathon_wright

#esconfs www.eurostarconferences.com

Performance Testing in the Cloud

@jonathon_wright#esconfs Performance Testing as a Service (PTaaS) Model

PTaaS – HP - StormRunner (TrueAPI/TrueClient)

Page 55: EuroSTAR - Testing as a Service Model

55

@jonathon_wright

#esconfs www.eurostarconferences.comMobile Testing as a Service (MTaaS) Model

• Test Platform as a Service (TPaaS)

• Test Infrastructure as a Service (TIaaS)

• Test Automation as a Service (TAaaS)

• Functional Testing as a Service (FTaaS)

• Performance Testing as a Service (PTaaS)

• Mobile Testing as a Service

– Mobile Testing in the Cloud

– Continuous, Mobile, Build & Delivery

– High Volume Mobile (HVMAT)

• Security Testing as a Service (STaaS)

Business Delivery Management - ‘Testing as a Service’ layer

Testing as a Service - Models

Page 56: EuroSTAR - Testing as a Service Model

56

@jonathon_wright

#esconfs www.eurostarconferences.com

Mobile Testing in the Cloud

@jonathon_wright#esconfs

Real Mobile Devices (PerfectoMobile) Test Studio for iOS (Telerik)

* Telerik.com, http://www.telerik.com/mobile-testing, 4th November 2014

Mobile Testing as a Service (MTaaS) Model

Page 57: EuroSTAR - Testing as a Service Model

57

@jonathon_wright

#esconfs www.eurostarconferences.com

Continuous, Mobile, Build & Delivery

MobileCloud™

@jonathon_wright#esconfs

* PerfectoMobile.com, http://www.perfectomobile.com/resources/papers/shift-app-testing-left-paper, 4th November 2014

Performance Testing as a Service (PTaaS) Model

Page 58: EuroSTAR - Testing as a Service Model

58

@jonathon_wright

#esconfs www.eurostarconferences.com

High Volume Mobile Automated Testing (HVMAT)

On demand in the cloud featuring Network Virtualisation

*Jonathon Wright, 2013, ‘Test Automation in the Cloud’ for EuroSTAR, Gothenburg, Sweden, 23rd July 2013

Mobile Testing as a Service (MTaaS) Model

Page 59: EuroSTAR - Testing as a Service Model

59

@jonathon_wright

#esconfs www.eurostarconferences.com

• Test Platform as a Service (TPaaS)

• Test Infrastructure as a Service (TIaaS)

• Test Automation as a Service (TAaaS)

• Functional Testing as a Service (FTaaS)

• Performance Testing as a Service (PTaaS)

• Mobile Testing as a Service (MTaaS)

• Security Testing as a Service

– Security Testing in the Cloud

Business Delivery Management - ‘Testing as a Service’ layer

Testing as a Service - Models

Security Testing as a Service (STaaS) Model

Page 60: EuroSTAR - Testing as a Service Model

60

@jonathon_wright

#esconfs www.eurostarconferences.com

Security Testing in the Cloud

How to prevent epistemic/systemic failure in the cloud from geo-realistic DoS attacks? Application

Security

@jonathon_wright

#esconfs

Infrastructure

Testing

Secure Coding

Mobile Security

Database Security

Information

Security

Our services identify

configuration and

deployment flaws which

may be introduced

when integrating mobile

solutions into an

enterprise environment.

We also perform

manual security testing

and analysis of smart

phone and tablet based

apps. Typically this will

encompass both the

client application and

server side services

supporting the app.

*John Denneny, MD, http://www.pentest.co.uk/whitepapers.html , 5th November 2014

Security Testing as a Service (STaaS) Model

Mobile Security External Infrastructure Security Assessment

Internal Infrastructure Security Assessment

Mobile Application Security Assessment

Firewall Security Assessment

Server Security Assessment

Laptop Security Assessment

Workstation Security Assessment

VOIP Security Assessment

VPN Security Assessment

Wireless Security Assessment

Active Directory Security Assessment

Citrix Security Assessment

Database Security Assessment Service (DSAS)

Page 61: EuroSTAR - Testing as a Service Model

61

@jonathon_wright

#esconfs www.eurostarconferences.com

Testing as a Service – Models - Summary

BusinessAssurance

BusinessQuality

BusinessTesting

Business Assurance provides the visibility and

transparency throughout the solution delivery lifecycle• Digital Enterprise

• Business on a Page

• Business Delivery Management

• Business Intelligence (Actionable Business Insight)

• Business Innovation (Enterprise Gamification)

Business Quality allows enforceable quality across the

entire solution delivery lifecycle owned by everyone• Business Lifecycle Management

• Business Solution Discovery / Delivery

• Business Enterprise Collaboration

• Business Acceptance Testing

Business Testing

• Testing as a Service - Models

• Solution Delivery Lifecycle integration (SLDCi)

• Test First Delivery / First Day Testing

• Continious, Integration, Build & Delivery

• Shift Everywhere (UP/DOWN/LEFT/RIGHT)

• All Channel Customer Experience

Page 62: EuroSTAR - Testing as a Service Model

62

@jonathon_wright

#esconfs www.eurostarconferences.com

Hitachi ConsultingHitachi’s Global Management Consulting & IT Services Business.

$740

MillionFY13 Revenue

6,500experienced &

dedicated

professionals

Better Approach. Better Results. Better Experience.

$10 Billion

results

© Hitachi Europe Ltd. 2014. All rights reserved.

Page 63: EuroSTAR - Testing as a Service Model

@jonathon_wright

#esconfs www.eurostarconferences.com

Page 64: EuroSTAR - Testing as a Service Model

Questions?

@esconfs#esconfs www.eurostarconferences.com

http://testhuddle.com/forums/topic/testing-as-a-service-models/

Page 65: EuroSTAR - Testing as a Service Model

EuroSTAR Webinar Archive

Access almost 70 on-demand software testing webinars by the world's leading testing experts. Topics include….

Agile, Automation, Test Management, Process Improvements, Risk-based Testing, Performance Testing, Test Documentation, Cloud Testing, Session-based Testing, Weekend Testers

Visit the archive!

http://www.eurostarconferences.com/community/member/webinar-archive

@esconfs#esconfs www.eurostarconferences.com

Page 66: EuroSTAR - Testing as a Service Model

The EuroSTAR Community.Are you a member?

Did you know we have an online resource area with almost …

600 software testing presentations, 70 topical webinars, over 40 videos and podcasts and 25 expert eBooks

Do you know that membership is completely free and you can access these premium

testing resources anytime?

Join today! http://www.eurostarconferences.com/community/

member

@esconfs#esconfs www.eurostarconferences.com