Agile Testing of Data Warehouse System in

a Non-agile Environment Swiss Testing Day 2015

Yury Chernov

10 March 2015

Classification: public

10 March 2015, public Page

Data Warehouse

2

Typical Requirements to DWH:

Make information easily acceptable – the contents must be

understandable

Present information consistently – the data must be credible

(data quality)

Adapt system to change – user needs, business conditions,

technology are all subject to change

Present information in a timely way

Secure and protect data

Business community should accept the system

DWH is a de-normalised, business-centric

database or set of databases in which the

data from different sources is collected and

stored in one uniform and consistent form

10 March 2015, public Page

DWH Characteristics

3

Attribute Database

OLTP (Online Transaction Processing)

DWH

OLAP (Online Analytical Processing)

Source of data

Operational data from original sources Consolidation data from various OLTP Databases

and files

Purpose of data Execution of fundamental business tasks Planning, problem solving, decision support

Type of data Snapshot of ongoing business processes Multi-dimensional views of business activities

Users Technical Staff Managers, Executives

Inserts & Updates Short and fast inserts/updates initiated by end

users/applications

Periodic long-running batch jobs refresh the data

Queries Standardized, simple and short queries Complex queries involving aggregations

Processing Speed Fast Batch jobs and complex queries that may take

hours

Emphasis Insert, update, delete Select-Retrieval

Space Requirements Relatively small Large due to aggregated structures and history data

Database Design Highly normalized Typically de-normalized

10 March 2015, public Page

DWH in Swiss Exchange Landscape:

Two Roles of DWH as a Testee

4

Role 1: A complicated component for business end-users

Role 2: An important downstream point in E2E testing

SWX DWH is not a database of corporate information

(like SAP), but a storage of trading and reference

data and a set of applications (Billing, Spread

calculation etc.)

10 March 2015, public Page

DWH Architecture

5

Source Stage Store Mart Front-end

Data sources ETL layer (extract, transform, load) Complete data User access layer User access tools

Analytical Data Process Data Operational Data

10 March 2015, public Page

DWH Testing

6

Test objects

Data transformations

Data integration

Data quality

Performance & scalability

User-acceptance

Tools

Quality Center

PCMS

Test Automation Framework

Data profiling scripts

PowerCenter

«Testers are people who realize that things

can be different»

Jerry Weinberg,

“the grandfather of Agile Programming”

10 March 2015, public Page

Agile Testing

7

1. Customer Satisfaction

2. Flexibility

3. Short Iterations

4. Daily Teamwork

5. Motivation

6. Face to Face Communication

7. Working software

8. Sustainability

9. Technical Excellence

10. Simplicity

11. Self-organisation

12. Regular Self-reflexion

Agile Practices

Agile Principles

Agile Values

1. Individuals and interactions over processes and tools

2. Working software over comprehensive documentation

3. Customer collaboration over contract negotiation

4. Responding to change over following a plan

An agile tester: “a professional tester who

embraces change, collaborates well with

both technical and business people, and

understands the concept of using test to

document requirements and to drive

development.”

Lisa Crispin & Janet Gregory in “Agile Testing”

10 March 2015, public Page

DWH Agile Testing

8

Swiss Exchange Landscape

Other Components DWH

Detailed functional specifications No clear boundaries between Verification

(are we building the product right?) and

Validation (are we building the right

product?) – requirements are often not

formalised

Testing includes component, integration,

system & E2E phases, all in special separate

environments

DWH is being tested directly in full-blown

E2E environment

Components are developed across system

projects

DWH is part of system projects and has

additionally ad-hoc deliveries for end-users

10 March 2015, public Page

Practical Lessons Learned

9

Lesson 1: Note everything.

Lesson 2: Maintain your testware permanently.

Lesson 3: Plan enough effort to maintain your test environments.

Lesson 4: Automate as much as possible.

Lesson 5: Your test data should be as representative as possible.

Lesson 6: Your bug tracking system is the best communication mean.

Lesson 7: Pay special attention to the data quality checks.

10 March 2015, public Page

Lesson 1: Note everything

10

Best of all is to note information directly in the test specification.

Your test specification becomes often the most reliable document

about the real system functionality.

Everything should be noted

An agile tester: “a professional tester who embraces

change, collaborates well with both technical and

business people, and understands the concept of

using test to document requirements and to drive

development.”

Lisa Crispin & Janet Gregory in “Agile Testing”

10 March 2015, public Page

Lesson 2: Testware

11

For the agile testing the testware should be always operational ready.

For a project manager the testware upgrade is the lowest priority task in

his project, but you will use this testware in next projects.

Test case should describe what to test, rather than how to test.

Keep the logic of tesware reasonably simple and thus maintainable.

Maintain your testware permanently

10 March 2015, public Page

Lessons 3: Test Environments

12

Test environment should be controlled.

Test environments should be always available.

A running test environment makes 50% of the job.

Plan enough effort to maintain

your test environments

10 March 2015, public Page

Lesson 4: Test Automation

13

With agile testing you will run your scripts much more often, than you

are thinking.

Pay more attention to the data profiling scripts.

Automate as much as possible

SWX:

1. Test Automation Framework for generation of the

trading data.

2. SQL Scripts to compare the content of different DBs.

3. Daily data profiling tasks

10 March 2015, public Page

Lesson 5: Test Data

14

Use a combination of production and synthetic data.

Every problem you saw with your data will once appear in production.

Developers will tend to accuse your test data in all the problems you

see, however in most of the cases they will turn out to be bugs.

Your test data should be as

representative as possible

10 March 2015, public Page

Lesson 6: Bug Tracking System

15

Note every problem in your bug tracking system:

• your will not forget the details

• developers will take it more serious (they don’t like registered problems)

• it can be canceled, if not relevant.

Force (if possible ) developers and business users to register problems in the

bug tracking system.

Avoid using multiple Excel-sheets to register problems.

Your bug tracking system is the

best communication mean

10 March 2015, public Page

Lesson 7: Data Quality

16

Automate data quality checks

Establish data checks scripts that run daily.

Check data quality of the source data files.

Pay special attention to the data

quality checks

"Program testing can be a very effective

way to show the presence of bugs, but is

hopelessly inadequate for showing their

absence."

Dr. E.W.Dijkstra

10 March 2015, public Page

Summary

17

1. Swiss Exchange is mostly being developed and tested with the classical

Waterfall model.

2. DWH has many specifics that lead to agile testing.

3. To properly answer this challenge testers should have a reasonable

common sense based practical approach.

4. Positive and negative experience -> “lessons learned/good practises”

are helping us to successfully test DWH and avoid production problems.

Thank you for your attention.