20091203 Presentatie Eurostar V02

Bring requirements to life with a paper SIT

- Simulated System Integration Test -

3 december 2009 | Henri Haarmans / Arjan Steltenpool

2

Goal and program

GoalAfter the presentation you should understand how the Simulated SIT helps you in determining defects in the design stage of your SOA landscape.

Program Company and IT-environment; Problems of a SOA-environment; Simulated System Integration Test; Back at the office ...

3

Our roles within Nationale-Nederlanden

Henri Haarmans >25 years experience in IT with

Nationale-Nederlanden; Several roles within IT; For Group Pension, e-Business,

Mortgages and Life Retail; Currently:

Lead Analyst for Life Retail;

Facilitator.

Arjan Steltenpool >10 years experience in IT with

Logica (CMG); Test Analyst, Test Manager,

Trainer; Logica is the preferred supplier for

testing at Nationale-Nederlanden.

4

Nationale-Nederlanden is part of the ING Group

IT Department

Employees: 510 Internal;

220 External from preferred suppliers;

IT change budget for software about € 85 M;

CMM-i level 3.

Nationale-Nederlanden

Employees: about 4250 internal; One of the biggest insurance

companies in the Netherlands; Two locations: The Hague and

Rotterdam; Turnover in 2008 about € 5 Bn; Product lines:

Life Retail;

Pension;

Property& Casualty;

Income;

Mortgages.

We use the Business Domain Model (BDM) for the representation of our organization.

5

The BDM contains all our business processes and each domain is a component with a particular business knowledge

1. Relationship Management

3. Performance Management & Corporate Support

14 Provider Management

141 Provider Selection

11 Customer (Intermediary and End-Client) Management

111 Customer Insight &

Segmentation

114 Prospecting & Campaign

Management

112 Customer Interaction

Policy

142 Sourcing & Contracting

34 Corporate Support

32 Performance Management

321 Customer & Channel

Performance

322 Financial Performance

342 Knowledge Mgt Prod/Claims

343 Communic.Intern / Ext.

344 Human Resources

345 Finance &Accounting

347 Architecture Mgt

IntermediarySales ForceEnd-client

Industry Software(e.g. VA

package, Rolls, RDW)

Internet


Printing at POS

Internet

346 ORM &Fraude

113 Customer InteractionExecution

143 Provider Performance

323 Operational Management

2. ServiceDeliveryChannels

341 Legal & Compliance

2. OutputDelivery

Channels

348 Quality Management

13 Sales & Distribution Management

131 Sales Force Support

133 Intermediary Agreement

132 Intermediary Support

31 Process Support

311 Workflow & Navigation

33 Accounts Receivable Payable

331 Collection & Disbursement

21 Input messages

211 Paper Data Capture

213 Voice

214 Web

212 Electronic

215 Email

22 Output messages

221 Publishing (Paper)

223 Voice

224 Web

222 Electronic

225 Email

8. Banking4. LifeRetail

6. Property&Casualty

7. Income5. Pensions

Change Cockpit

1. Productspecifications

2. Textsfor quotations

& policies

3. Establishing standards

4. Simulationcalculation

7. Monitoring

8. Determining actual results

9. EV calculations

10. Adjustmentor removal

5. Profittesting

6. Productimplementation

Change Cockpit



& policies



7. Monitoring


9. EV calculations


5. Profittesting


Change Cockpit



& policies

3. Establishing standards 4. Simulation

calculation

7. Monitoring


9. EV calculations


5. Profittesting


Change Cockpit



& policies


calculation

7. Monitoring


9. EV calculations


5. Profittesting


Change Cockpit



& policies


calculation

7. Monitoring


9. EV calculations


5. Profittesting


Change Cockpit



& policies


calculation

7. Monitoring


9. EV calculations


5. Profittesting


Change Cockpit



& policies


calculation

7. Monitoring


9. EV calculations


5. Profittesting


Change Cockpit



& policies


calculation

7. Monitoring


9. EV calculations


5. Profittesting


Change Cockpit



& policies


calculation

7. Monitoring


9. EV calculations


5. Profittesting


Change Cockpit



& policies


calculation

7. Monitoring


9. EV calculations


5. Profittesting


Conditional: accommodation, infrastructure and security

12 Product Cockpit

121 Product Developmentmanagement

122 Product & Rules Building

312 Archiving

Product lifecycle Life Retail:

management and processing

Product lifecycle Property & Casualty:

management, processing and claim handling

Product lifecycle Income:


Product lifecycle Pensions:


Product lifecycle Mortgage:


61. Relationship Management

3. Performance Management & Corporate Support

14 Provider Management

141 Provider Selection

11 Customer (Intermediary and End-Client) Management

111 Customer Insight &

Segmentation

114 Prospecting & Campaign

Management

112 Customer Interaction

Policy

142 Sourcing & Contracting

34 Corporate Support

32 Performance Management

321 Customer & Channel

Performance

322 Financial Performance

342 Knowledge Mgt Prod/Claims

343 Communic.Intern / Ext.

344 Human Resources

345 Finance &Accounting

347 Architecture Mgt


Industry Software(e.g. VA

package, Rolls, RDW)

Internet


Printing at POS

Internet

346 ORM &Fraude

113 Customer InteractionExecution

143 Provider Performance

323 Operational Management

2. ServiceDeliveryChannels

341 Legal & Compliance

2. OutputDelivery

Channels

348 Quality Management

13 Sales & Distribution Management

131 Sales Force Support

133 Intermediary Agreement

132 Intermediary Support

31 Process Support

311 Workflow & Navigation

33 Accounts Receivable Payable

331 Collection & Disbursement

21 Input messages

211 Paper Data Capture

213 Voice

214 Web

212 Electronic

215 Email

22 Output messages

221 Publishing (Paper)

223 Voice

224 Web

222 Electronic

225 Email

8. Banking4. LifeRetail

6. Property&Casualty

7. Income5. Pensions

Change Cockpit



& policies



7. Monitoring


9. EV calculations


5. Profittesting


Change Cockpit



& policies



7. Monitoring


9. EV calculations


5. Profittesting


Change Cockpit



& policies


calculation

7. Monitoring


9. EV calculations


5. Profittesting


Change Cockpit



& policies


calculation

7. Monitoring


9. EV calculations


5. Profittesting


Change Cockpit



& policies


calculation

7. Monitoring


9. EV calculations


5. Profittesting


Change Cockpit



& policies


calculation

7. Monitoring


9. EV calculations


5. Profittesting


Change Cockpit



& policies


calculation

7. Monitoring


9. EV calculations


5. Profittesting


Change Cockpit



& policies


calculation

7. Monitoring


9. EV calculations


5. Profittesting


Change Cockpit



& policies


calculation

7. Monitoring


9. EV calculations


5. Profittesting


Change Cockpit



& policies


calculation

7. Monitoring


9. EV calculations


5. Profittesting


Conditional: accommodation, infrastructure and security

12 Product Cockpit

121 Product Developmentmanagement

122 Product & Rules Building

312 Archiving

Product lifecycle Life Retail:


Product lifecycle Property & Casualty:

management, processing and claim handling

Product lifecycle Income:


Product lifecycle Pensions:


Product lifecycle Mortgage:


For example: a new product for Life Retail results in changes in all the 15 green domains

7

The BDM is implemented in a real Service Oriented Architecture (SOA)

Characterization: Distributed requirement and software development of the applications; Communications between domains goes via an Enterprise Service Bus

(ESB).

Complexity: We have various hardware platforms; We use various programming languages; We use in-house built software and vendor applications; Transition from the middle-ware in the ESB to Tibco.

Consequence of a SOA environment:

Defects in the connectivity between applications arise during the System Integration Test (SIT) for the first time.

8

Goal and program



9

In a traditional environment the V-model is a good model for the development and verification of software

Paul E. Brook 1986

A Quality Management System with quality gates ( )and baseline audits provide extra certainty.

ExpectationWish

Customer Requirements

System Requirements

Functional and technical Design

Unit Built

Unit Test

System Test (Functional Acceptance Test)

User Acceptance Test

System Integration Test

Regression Test

Production AcceptanceTest

Test design

One team is developing all the

software!

Unit Design

10

A Quality Management System gives no extra certainty in a SOA environment

The big challenge is:How can we secure that all applications work together as one process to the customer?

ExpectationWish

Regression Test

Production Acceptance Test

System Requirements

per domain

User Acceptance Test

per domain

In a SOA environment one team develops the Customer Requirements and for each domain different teams develop the System Requirements and the software!

Our solution:

Test the process after developing the system and interface requirements.

System Integration Test

?Customer Requirements

11

Different stakeholders, different concerns

Our solution the Simulated SIT (SSIT) provides also answers to the questions from stakeholders.

Test Manager

Understanding of the application landscape;

Reduction in time to test and fix.

Business representative

Do the software developers

understand my requirements?

Software developer

Do I understand the Customer Requirements correctly?

Lead Analyst

Does my solution Work?

Project and Business Management

Certainty of due date and quality of the system.

12

Goal and program


Program Company and IT-environment: Problems of a SOA-environment; Simulated System Integration Test:

Goal;

Preparation;

The test itself;

Roles;

Follow-up;

Benefits;

Back at the office ...

13

Main goal of the SSIT: Find defects in the process flow as soon as possible

A SSIT is: A simulation of the process between the main applications:

One analyst and one business representative per domain simulate the application;

Manually filling in Service Request Forms on paper;

Manually delivering these forms between domains to represent the IT-interfacing;

There is no face-to-face contact between the domains; Databases and screens are on paper; Defects and questions are filled in by the people who are performing the test.

We don’t test: The functions of an application.

Good preparation and facilitation is essential for a successful SSIT.

14

A good preparation is the basis for success

When should a SSIT be executed? After definition of the System and Interface Requirements, the Technical Application

Architecture and Technical Interface Architecture and the Business Use Cases; Before starting to build the applications.

Precondition of a SSIT: More than one Business Use Case or one complex Business Use Case; More than one application; More than one Business Domain.

Test cases made by the Lead Analyst and Test Analyst: Usually 15 - 20 test scenario’s are sufficient; A test scenario maps 1-1 with the Business Use Cases; For every Business Use Case more than one test case; Database information for each application.

The preparation for a SSIT depends on complexity, but shouldn’t be more than 40 hrs.

The Facilitator of the

SSIT plays the role of

the devil’s advocate.

15

The testing day in a nutshell (1 of 3)

Kick-off with all the participants in a central meeting: Project urgency presented by the Project Manager;

Rules of the road presented by the Facilitator;

Explanation of the working of the SSIT presented by the Facilitator.

16


4 test sessions during one day:

Testing is done by filling in Service Request Forms, the templates, logbook etc.;

After one hour, a session ends with an evaluation in every ‘break-out’.Service Request Form:

Between and

Output

Input

Purpose

Service

Testcasenr.:

Questionform

Testcase:

Application:

Name:

Question:

Answer:

Defect form

Testcase:

Application:

Name:

Defect:

Solution:

Action Holder:

17


A central meeting after every test session for discussing the defects, discussing the questions and optimizing the test process;

Between the test sessions: exercises for getting back the energy;

At the end of the day:

Conclusions of the test cases and actions for solving the defects;

Action for outstanding questions;

Evaluation of the day; Afterwards:

The facilitator makes a SSIT test report including defects and questions.

18

Visualization of the simulation

Table 6

Table 1

Table 7

Table 2

Table 5

Table 3

Business Rules

Back Office

Calculation Box

Collection & Disbursement

Front Office

Output

On every table…

Facilitator

Test Manager

Information AnalystLead Analyst

‘Knowledge bank’

19

The shopping list for the SSIT (1 of 2)

A room which can be divided in separate break-outs;

Magnetic white boards (not essential but very convenient);

A facilitator for: The process of the SSIT;

Exercises for bringing back the energy;

Interim evaluations for: Solving problems;

Evaluation of the test scenario’s during the

day.

20

The shopping list for the SSIT (2 of 2)

For every business domain: A break-out;

An in-box and an out-box;

Instructions for the SSIT;

Templates for: Reporting defects;

Questions;

The evaluation;

Screen prints for the online applications;

A log book;

Database content;

Service request forms to other applications.

21

All participants play their own role during the SSIT

Lead AnalystWalking encyclopedia

Business PeopleMonitors that the right business rules are used.

Information Analyst

Processor of the Service Request Forms.

FacilitatorControls the process, initiates a test case, facilitates the evaluation etc.

Test ManagerPostman between the domains

22

Beware: Follow up after a SSIT is crucial

Detailed review on field-level of the services between two applications;

Defects have to be solved; Questions have to be

answered; Test scenarios can be input

for the real SIT; Baseline for working

procedure.

23

Some results from SSIT executed in the last year

A new product for Life Retail: 20 Defects during the SSIT:

6 Missing and wrong services in the TIA;

3 Defects in data model;

3 Defects in the manual processing;

2 misinterpretations of the CR;

6 Defects in the services;

Expected delay when not found: > 3 months;

Online Transaction Module: 17 Defects during the SSIT:

3 Missing services in TIA;

7 Defects in the control function;

1 Defect in the data of a service;

4 Defects in business rules;

2 Defects in the manual processing;

Expected delay when not found: > 6 weeks.

24

Benefits: One day hard work during a SSIT results in answers to the concerns of the stakeholders, and also in:

Detecting logical errors in the process: Interfaces; Services; Order of services.

Detecting differences between the data models in different domains; Participants:

Get the same view of the objective of the project; Get understanding of the importance of the project; Knowing each other promotes efficient communication during testing.

Business: Check on the right process; Information about and understanding of the complexity of the test; Input for the Knowledge Bank and guidelines.

SIT: Test cases of the SSIT are the base for test cases in the SIT. Quality of the SIT becomes higher.

Test Manager: Gets a better understanding of the process to test.

Less or no delay in the delivery of the software.

Defects can be discussed immediately

25

Goal and program



26

Back at the office …

Do you recognize the problem? Painful SITs due to lack of communication between business domains in previous project phases?

Convince project/program management for the necessity of a SSIT; Keep in mind:

The Information Analyst is responsible for all requirements, review activities including the SSIT, but a Test Manager can help facilitate the SSIT;

So …

Thank you for your [email protected]

[email protected]

Documents

20091203 Presentatie Eurostar V02