Test Data Management Concepts and Methods MDUG 2015Microsoft PowerPoint - Test Data Management Concepts and Methods MDUG 2015 10/16/2015 6:08:36 PM

© UBS Hainer GmbH, the IBM Business Partner| 451

UBS HAINER TEST DATA MANAGEMENT

Call 866-464-3724

ESAIGroup.com © UBS Hainer GmbH, the IBM Business Partner

TEST DATA MANAGEMENTCONCEPTS & METHODS



Call 866-464-3724

ESAIGroup.com

Test Planning

• Test Management

EnvironmentManagement

Test Data Management

Test Execution

Reporting

Fault Management



Call 866-464-3724

ESAIGroup.com

• Key requirements

• Automation: Define once, execute regularly

• Create different environments that do not interfere

• Easily add more environments

• Goal: Better QA with reasonable costs



Call 866-464-3724

ESAIGroup.com

• Limitations of Tools for Test Data Management

• Test cases

• Application logic

• What needs to be tested?

• How often a copy is required?



Call 866-464-3724

ESAIGroup.com

• Who needs what?

PRODUCTION REGRESSION INTEGRATION COMPONENT FUNCTION DEVELOPMENTACCEPT



Call 866-464-3724

ESAIGroup.com

• Who needs what?


•• •

•• Amount of Data

•



Call 866-464-3724

ESAIGroup.com

• Who needs what?


•• •

•• Amount of Data

•

Degree of Modification



Call 866-464-3724

ESAIGroup.com

Table CopyingComponent / Integration Tests

Row Level Processing

Unit Tests

Database / Subsystem CloningRelease / System Tests / Preproduction




Call 866-464-3724

ESAIGroup.com




Unit Tests

SUBSYSTEM CLONING | TABLE COPYING | ROW LEVEL PROCESSING




Call 866-464-3724

ESAIGroup.com

Subsystem/Database Cloning




Call 866-464-3724

ESAIGroup.com

• Create a pre-production environment

• Data source for further copy operations

Cloning

ProductionDB2, Oracle, etc.

Pre-ProductionDB2, Oracle, etc.


• Strict separation between prod and test

• Acceptance tests

• RDBMS upgrades



Call 866-464-3724

ESAIGroup.com

Preproduction Clone for DB2, Oracle, SQL Server, etc.

Cloning




Call 866-464-3724

ESAIGroup.com

Time

How does a clone become consistent?

NIGHTSHIFTPOINTIN TIME




Call 866-464-3724

ESAIGroup.com

Time

How does a clone become consistent?

FlashCopy1

SnapShot

EMC TimeFinder

IBM PPRC

HDS ShadowImage

Softek TDMF

Fujitsu Equivalent Copy




Call 866-464-3724

ESAIGroup.com

Time

Transactions




Call 866-464-3724

ESAIGroup.com

Time

Prepare

Rebuild current status

Prepare & (Re)Start




Call 866-464-3724

ESAIGroup.com

• Cloning

•

D

source for other copy operations

• Well defined set of scripts / jobs that read production

• Executed periodically by scheduler

• Strict decoupling of production from preproduction

• Rework after cloning is required






Call 866-464-3724

ESAIGroup.com

• Characteristic of Cloning a source for other copy operations

• Need access to physical hard drives

• No need to stop or start databases

• Hardware assisted copy tools are very beneficial

• DBMS tools can also be used alternatively




Call 866-464-3724

ESAIGroup.com




Unit Tests





Call 866-464-3724

ESAIGroup.com

Table Level Copying




Call 866-464-3724

ESAIGroup.com

Copy

Feeding the other Environments

Production Preproduction

Test Environments Dev

Clone




Call 866-464-3724

ESAIGroup.com

• Wish list for file system level copies:

• Data source for other copy operations

• Specify one schema, copy 1000 tables

1 Database

100 Tables

1000 Indexes




Call 866-464-3724

ESAIGroup.com


• Characteristic Table Copying

• Replace/copy/rename selected objects

• Need authorization to move TS / to use utilities

• Hardware assisted copy tools not applicable

• But optimized for copy of mass data

• Masking/Anonymization feasible



Call 866-464-3724

ESAIGroup.com




Unit Tests





Call 866-464-3724

ESAIGroup.com





Call 866-464-3724

ESAIGroup.com


CopiesCloneProductionDB2, Oracle, etc. Test Environments

Unit Testing

Inserts


FunctionTesting



Call 866-464-3724

ESAIGroup.com

ID INTEGER NOT NULL

FIRSTNAME VARCHAR(256) NOT NULL

LASTNAME VARCHAR(256) NOT NULL

Version 1.0:

Version 2.0:

ID INTEGER NOT NULL

FIRSTNAME VARCHAR(256) NOT NULL

LASTNAME VARCHAR(256) NOT NULL

DATE_OF_BIRTH DATE NOT NULL


Provide data for the next application version



Call 866-464-3724

ESAIGroup.com 28


• Automatic RI check

• All RI dependencies should be brought to test/development



Call 866-464-3724

ESAIGroup.com 29


• Insert Options a source for other copy operations

• Completely replace target

• Append to target tables

• Merge based on primary key



Call 866-464-3724

ESAIGroup.com


• Characteristic of Row Level Processing a source for other copy operations

• RI identification and transfer of relevant data to test system

• Flexible data customization

• Automates refresh of test case data

• Not suitable for mass data

• Developer tool



Call 866-464-3724

ESAIGroup.com

VERSIONING OF TEST DATA

Manage Test Data VersionsMake Tests Reproducible



Call 866-464-3724

ESAIGroup.com

Test

Env.1

32

The Dilemma

Test Case0815

Modifies data

during test execution

• Tests should run under same preconditions

• Tests modify the data




Call 866-464-3724

ESAIGroup.com 33

Test

Env.1

FROZEN TEST DATA

BACKUP BEFORETEST EXECUTION




Call 866-464-3724

ESAIGroup.com 34

Test

Env.1

FROZEN TEST DATA

BACKUP BEFORETEST EXECUTION

Test

Env.2

ON DEMAND




Call 866-464-3724

ESAIGroup.com


• Characteristic of Versioning Management a source for other copy operations

• Availability of data at the push of a button

• Separate test/development from backup procedures

• Reduce error rates when doing acceptance tests

• Fast provisioning of test data instead of finding modified •rows



Call 866-464-3724

ESAIGroup.com

Anonymization/Masking

ANONYMIZATION/MASKING



Call 866-464-3724

ESAIGroup.com


• What is the benefit of Anonymization? a source for other copy operations

• Compliance with legislation

• Preservation of reputation

• Avoid to provide competition with critical data



Call 866-464-3724

ESAIGroup.com

Judicial directives

European Privacy Directive 95/46/EC European Union

PIPEDA, Privacy ACT Canada

HIPAA, PCI DSS USA

IFAI Mexico




Call 866-464-3724

ESAIGroup.com


What must be anonymizedand where?



Call 866-464-3724

ESAIGroup.com


Test Umgebungen

PreproductionDB2, Oracle, etc.

Entwicklung

Anonymized

Area

Test Environment

Development


• Anonymization/Masking in the data landscape

• Strict separation

• Enforce anonymization



Call 866-464-3724

ESAIGroup.com


Test Umgebungen

PreproductionDB2, Oracle, etc.

Entwicklung

Test Environment

Development


• Anonymization/Masking in the data landscape

• Strict separation

• Enforce anonymizationAnonymized

Base Stock



Call 866-464-3724

ESAIGroup.com

• Existing methods

Unload – Modification - Load

While copying

• Fast

• Automized

• Higher safety standard




Call 866-464-3724

ESAIGroup.com


Rewarding of Anonymization/Masking?

• Keep the danger potential low

• Match compliance rules: yes, we anonimyze!

• Improvement of process with every new release



Call 866-464-3724

ESAIGroup.com


• Characteristic of Anonymization/Masking a source for other copy operations

• Anonymization/Masking cannot be right or wrong

• Every application is different, no silver bullet existing

• Risk / Effort / Costs / Effect must be considered realistically

• No definite rules what exactly has to be anonymized exist



Call 866-464-3724

ESAIGroup.com

Thank you for your attention!

Questions?

www.esaigroup.com/doc/TDMOverview.pdf

BPA4DB2 IMPROVE PERFORMANCE AND LOWER COSTS

| 47 © UBS Hainer GmbH, the IBM Business Partner46 Call 866-464-3724

ESAIGroup.com

Contact us to learn more at [email protected] or 866-464-3724www.ESAIGroup.com www.ubs-hainer.com

Inspect-CPU for CICS® APM

TDM & Fast Data Refresh

Solutions

DB2®Performance

Cost Savings,Performance

DB2, CICS z/OS are registered trademarks of IBM. All other trademarks are of their respective owners.

Documents

Test Data Management Concepts and Methods MDUG 2015Microsoft PowerPoint - Test Data Management Concepts and Methods MDUG 2015 10/16/2015 6:08:36 PM