29
built by Bill Hayduk CEO/President RTTS Harry Potter and Data Warehouse Testing Introducing the QuerySurge Data Wizards Martin Henry-Lester Product Specialist RTTS Laura Poggi Marketing Manager RTTS Webinar

Data Warehouse Testing: The QuerySurge Data Wizards

  • Upload
    rtts

  • View
    2.256

  • Download
    5

Embed Size (px)

DESCRIPTION

What Do Harry Potter And Data Warehouse Testing Have In Common? They both use wizardry to get the job done! Introducing the new QuerySurge Data Wizards. See how easy it is to validate your data warehouse data throughout the ETL process with our new Wizards! The Wizards are also perfect for database migrations and upgrades. Why spend time writing SQL queries when you can simply point to your data sources and tell the wizard what tests you'd like to run - fast and easy, no coding necessary! We will walk you through use cases for this new functionality to show you how it can simplify your work and help you achieve efficiency and quality in the data warehouse development cycle.

Citation preview

Page 1: Data Warehouse Testing: The QuerySurge Data Wizards

built by

Bill HaydukCEO/President

RTTS

Harry Potter and Data Warehouse TestingIntroducing the QuerySurge Data Wizards

Martin Henry-LesterProduct Specialist

RTTS

Laura PoggiMarketing Manager

RTTS

Webinar

Page 2: Data Warehouse Testing: The QuerySurge Data Wizards

About FACTSFounded: 1996

Primary Focus:

consulting services, software

Locations: New York, Atlanta, Philly, Phoenix

Geographic region:North America

Customer profile:Fortune 500, > 450 clients

Software:QuerySurge

The Software Quality Experts

RTTS is the leading provider of software quality for critical business systems

Page 3: Data Warehouse Testing: The QuerySurge Data Wizards

built by

The average organization loses $8.2 million annually through poor Data Quality.

- Gartner

46% of companies cite Data Quality as a barrier for adopting Business Intelligence products.

- InformationWeek

Data Quality Best Practices boost revenue by 66%. - Research firm Sirius Decisions

Page 4: Data Warehouse Testing: The QuerySurge Data Wizards

4

About QuerySurge

built by

Page 5: Data Warehouse Testing: The QuerySurge Data Wizards

QuerySurge is the

premier test tool built

to automate Data Warehouse testing

and the ETL Testing Process

What is QuerySurge?

built by

Page 6: Data Warehouse Testing: The QuerySurge Data Wizards

What does

QuerySurge ™do?

built by

QuerySurge finds bad data

• Most firms test < 10% of their data

• Your CxOs rely on BI apps to make strategic decisions

• BI apps sit on top of DWHs that have at best, untested data & at worst, bad data

• QuerySurge tests up to 100% of your data quickly & finds bad data

Page 7: Data Warehouse Testing: The QuerySurge Data Wizards

QuerySurge™ Architecture

built by

Que

rySu

rge

Arch

itect

ure

Target

Sources

Page 8: Data Warehouse Testing: The QuerySurge Data Wizards

8

Architecture-Under-Test

Sources

SQL Q

uery

Da

ta se

t

SQL Query

Data set

Data Set #1

Data Set #2

© 2011 Real-Time Technology Solutions, Inc.

QuerySurge Architecture

User(s)

Web interface

CompareData Sets

QuerySurge™: How it Works

built by

Page 9: Data Warehouse Testing: The QuerySurge Data Wizards

QuerySurge Roles & Uses

Testers - functional testing - regression testing

ETL Developers - unit testing

Data Analysts- review, analyze data - verify mappings and

failures.

Operations teams - monitoring

built by

Page 10: Data Warehouse Testing: The QuerySurge Data Wizards

built by

Design Library Create Query Pairs (source & target SQLs)

10

QuerySurge™ Modules

Scheduling Build groups of Query Pairs Schedule Test Runs

Page 11: Data Warehouse Testing: The QuerySurge Data Wizards

Deep-Dive Reporting Examine and automatically

email test results

Run Dashboard View real-time execution Analyze real-time results

QuerySurge™ Modules

built by

Page 12: Data Warehouse Testing: The QuerySurge Data Wizards

built by

QuerySurge™ Gallery: QueryPair Creation

Source Query

Target Query

Page 13: Data Warehouse Testing: The QuerySurge Data Wizards

built by

QuerySurge™ Gallery: Run Dashboard

QueryPair details

Runtime activity

Result sets

Page 14: Data Warehouse Testing: The QuerySurge Data Wizards

built by

QuerySurge™ Gallery: Summary Report

Duration= 34 sec

720,000 data sets

14,000 data failures

Page 15: Data Warehouse Testing: The QuerySurge Data Wizards

built by

QuerySurge™ Gallery: Detail Report

Individual QueryPair passes

Individual QueryPair fails

Page 16: Data Warehouse Testing: The QuerySurge Data Wizards

Business Problem

built by

In a survey of potential customers we asked:

What are your top

DWH testing challenges?

Page 17: Data Warehouse Testing: The QuerySurge Data Wizards

They need to automate the testing effort

Their Top 6 Responses

They need to speed up the testing cycle

built by

They need to be able to schedule test runs

They need to be able to test across different platforms

They need to share test results with their group

They need to verify more data than they currently do (typically <10%)

Page 18: Data Warehouse Testing: The QuerySurge Data Wizards

automates the testing effort the kickoff, the tests, the comparison, emailing the results

speeds up testing up to 1,000 times faster than manual testing

schedules test runs run now, every Tuesday at 11pm or right after ETL process

tests across different platformsany JDBC-compliant database, DWH, D-Mart, flat file, XML

views & shares results through automated emailing of reports

the QuerySurge solution…

verifies more data verifies upwards of 100% of all data

built by

Page 19: Data Warehouse Testing: The QuerySurge Data Wizards

QuerySurge Value-Add

QuerySurge provides value by either:

in testing data coverage from < 10% to upwards of 100%

in testing time by as much as 1,000 x 

combination of in test coverage while in testing time

19built by

Page 20: Data Warehouse Testing: The QuerySurge Data Wizards

Return on Investment (ROI)

redeployment of head count by 75% because of an increase in coverage

a savings over manual testing (minus queries, manual compares, other)

an increase in better data due to shorter / more thorough testing cycle, saving $millions by preventing key decisions made on bad data. 

20built by

Page 21: Data Warehouse Testing: The QuerySurge Data Wizards

FYI - Training Courses

built by

CoursesFundamentals of QuerySurge 1 dayIntroduction to SQL for QuerySurge 1 day Advanced SQL Techniques for QuerySurge 1 dayData Warehouse Fundamentals 1 day

Delivery Optionsin the Cloud in our Classroom private on-site training

new

Page 22: Data Warehouse Testing: The QuerySurge Data Wizards

built by

Page 23: Data Warehouse Testing: The QuerySurge Data Wizards

QuerySurge™ new release!!!

What do and

QuerySurge™ have in common?

23

They both use to get the job done!!!

Introducing the

built by

Page 24: Data Warehouse Testing: The QuerySurge Data Wizards

24

Fast and Easy. No programming needed.

built by

• Perfect for data warehouse testing, data migrations & upgrades

• Great for verifying that all rows have come over without issue

• Superb at verifying table-to-table transfers

• Ability to graphically create SQL stubs for advanced scripting

Page 25: Data Warehouse Testing: The QuerySurge Data Wizards

Uses:

Verify that the amount of rows match the amount that were expected

built by

Roles:

ETL Developer, Operations, Testers

SQL Skill set / Value added:

beginner: no coding needed

experienced user: saves time

Page 26: Data Warehouse Testing: The QuerySurge Data Wizards

Uses: - Verify data warehouse loads

when no transformation- Verifies data migrations,

upgrades

built by

Roles: ETL Developers, Testers

SQL Skill set / Value added: beginner: no coding neededexperienced user: saves

time

Page 27: Data Warehouse Testing: The QuerySurge Data Wizards

Uses: - Utilize either wizard to create stub

statements for writing SQL query pairs

built by

SQL

SQL

SQL

SQLSQL

SQL

SQL SQL

SQL

SQL

Roles: ETL Developers, Advanced Testers

SQL Skill set / Value added: For experienced users: - No need to look up schema(s) or

table names- Saves tons of time

Page 28: Data Warehouse Testing: The QuerySurge Data Wizards

Visually select source and target database (or DWH)

Wizards’ ease-of-use Select Row Count Wizard or Table Compare Wizard

Select schema(s) from source and targetSelect matching tables in source and target

QuerySurge automatically creates test suite

Now you are ready to run!

Page 29: Data Warehouse Testing: The QuerySurge Data Wizards

Ensuring Data Warehouse Quality

QuerySurgebuilt by

Demo