Upload
others
View
14
Download
1
Embed Size (px)
Citation preview
d
Business Intelligence Testing
401-408, A-Wing, Pride Silicon Plaza, S.B. Road, Shivaji Nagar, Pune -411006,
Maharashtra, INDIA
Email: [email protected] Tel: +91-20-41020202
Global Delivery Centre:
Overview
Modern enterprises are constantly confronted with ever changing market dynamics and
competition. Rapid progression and transformation of the organization is characterized by the
swift and accuracy of the decisions made by the users at every juncture. Organizations rely on
enterprise data warehouse and business intelligence application to seek answers and aid in the
informed decision making.
During development of Data Warehouse high volume of data is transformed, cleansed,
integrated these various types of changes could lead to data corruption or data manipulation.
Therefore, DW testing is a very critical stage in the DW applications
.
BI system is a well-defined amalgamation of data from diverse heterogeneous system. Poor
data quality has diluted the confidence on the systems and hampered business reputation due
to bad decisions. The diversity of the data introduces multi-fold complexities and challenges and
hence a multi-faceted testing of the BI implementation is pivotal for successful adoption.
This document highlight the challenges associated with BI testing and approach for a successful
testing of the BI systems.
Challenges in BI testing
An effective Business Intelligence comprises of enterprise data warehouse at its core. Data
warehouse is a composite and aggregated data store encapsulating data from diverse
databases. The focus here is consolidating and modeling the data to support faster retrieval of
data. With the mounds of data and complex transformations and business logic on the data the
challenges for testing are diverse. Some of the challenges include:
High volume and data diversity
DWH is built by fetching data from heterogeneous sources such as flat files, transactional
systems, excel sheets etc. Different data sources pose different challenges such as data format,
capture and availability. This gets further complicated when basic validations lack in the
application. Hence, QA needs to acquaint with the diverse data sources and ensure data
movement is as per the defined process.
Data Quality and completeness
Data quality and completeness is paramount important in any BI implementation. The challenge
underlies to ensure all the data loaded from the source to target systems is complete, valid and
accurate as per the business requirements.
Data Loading
Data is fed into data warehouse from the source either in totality or incremental fashion as per
the business need. The objective is to ensure the data is regularly updated at regular intervals
at defined time. Testing should ensure that data is fed and the flow is appropriate as per the
defined business rules.
OLAP (Online Analytical Processing)
Data is stored in a multidimensional format categorized by different dimensions as per business
need. Testing of the OLAP should ensure the data in the OLAP cubes is designed as per the
end reporting needs with all the requisite dimensions and measures. The complexity further
increases when the cube design is complex with multiple tables.
Reports
Reports are the final outcome of any BI implementation. The major challenge is to verify the
representation of different reports to ensure that report conveys the right information to
empower users which in turns contributes in effective decision making.
BI Testing Approach
Nitor Test Center of Excellence (TCoE) with its diverse expertise in testing varied BI products
and applications has well-defined approach for testing. This section states our diverse testing
expertise.
ETL Testing
The objective of ETL Testing is to ensure that the data in the warehouse is accurate, consistent,
and complete in each subject area and across each layer. In order to achieve this objective
Nitor TCoE has come up with the testing approach considering the below listed key activities:
End – to- end data flow from source system to staging database, and data loaded in to
target data warehouse
Staging packages (dimension and fact tables’ packages)
Dimensions& Fact Tables used in data warehouse
Data truncation by populating all the data
Parent child relationship between data
Lookups used in ETL packages
Source to Target table mapping as per design documents (check column mappings in all
transformations and while loading the tables)
Error recovery methods
ETL application substitutes default values, ignores invalid data
Auditing is done properly
Slowly changing dimensions, testing of historical data archival
Validate correct processing surrogate keys
Validate connection strings and package configuration files
OLAP Testing
The objective of OLAP testing is to ensure that data from the data warehouse is mapped and
designed correctly in the OLAP Cube. Due consideration is given to following areas while
testing OLAP cubes.
Browse the cube with measures and dimensions
Testing cube performance by measuring the time to process and browse the cube
Verify all the dimension hierarchies in the cube are correctly defined
Validate the data in cube by testing the data back to the staging database
Validate Measure, Aggregations, KPIs ,Calculations used in the cube
Validate data presented in the cube by querying back to the system
Processing options Update/Full
Validate if data from data warehouse is mapped and designed correctly in OLAP cube
as per business needs and specifications
Report Testing
The objective of this testing is to ensure that the data to be presented should be thoroughly
validated and should be in understandable format and following all the standards mentioned in
design documents. The key aspects in the Report testing include.
Validate the reports that are built either from OLAP Cubes or independent data sources
Report’s Drill down, Drill Up and Drill through functionalities
Report’s layout should be as per design document
Validation of data displayed on Reports
Testing report data at lowest granular level
Report navigation to other reports
Report filters
Parameters
Validate data in the report with the backend data
BI/DW Testing
Testing the BI products/application require a pragmatic data centric approach. Effective testing
of the BI testing should encompass following at every stage of testing.
Data completeness 1. Compare record counts between source database data,
staging table data and data loaded to target DW
2. Compare unique values of key fields between source
database, staging database and target DW
3. Validate that no truncation occurs during the ETL process
Data Transformation
1. Validate the data transformation is as per the business logic
2. Create a spreadsheet of input data and expected results and
validate these with the output of ETL package
3. Set up data scenarios that test referential integrity between
tables
Data Quality
Data quality handles with the ability of an ETL system to take care of
any data modification, rejection, duplication and correction.
Meta-Data Testing
1. Table and column names should be as per requirement
document
2. Source and Target column data types should be as per
requirement document
Performance and
Scalability
1. Load the high volume of Production like data to check the
ETL process and check whether ETL process does it in an
expected timeframe
2. Execute joins single or multiple to validate query performance
on large volume databases
3. Validate the OLAP system performance by browsing the
Cube with multiple options
4. Analyzing ETL package loading time with small amount of
data
5. Analyze the maximum users load at peak and off peak time
that are able to access and process the BI reports
Batch execution
Testing
This testing is carried out to verify the data flow when ETL packages
are run in sequential order toload the high volume of Production like
data to check the data flow.
Job Restart Testing
In a production like environment, the ETL jobs fails for numerous
reasons (such as database related failures, connectivity failures
etc.). The jobs can fail half/partly executed. The Test should also
include such scenarios.
Connection Testing
This type of testing includes verifying connection strings from
different data sources to check that data is receiving and loading
from Source systems to Target systems.
Automation Testing
BI testing is a data driven testing and requires exhaustive testing. As the database grows,
manual testing becomes challenging and daunting. Regression test suite helps to minimize the
mundane activities and helps to test on a periodic basis. Test automation can be introduced in
following areas:
Test data generation
Reports testing
Performance testing
Conclusion
BI product/system builds its bedrock on diverse heterogonous systems.Enterprises actively rely
on BI systems to aid in day to day decision making. While comprehensive features and
reporting capabilities have helped in the rapid adoption, poor quality has stranded the adoption
and strangled the investments. A comprehensive and pragmatic testing is vital to avoid fatal
business loss and instill confidence on the system.
About Nitor Test Services
Nitor Test Center of Excellence (TCoE) is an independent test practice with well-defined testing
processes and profound knowledge of tools and techniques to cater to the testing needs of
varied products/applications. Our professional team of test analysts has worked on products of
varying complexity in terms of technology and functionality, projects and solutions across focus
areas such as Business Intelligence, Portals & Collaboration and Enterprise Mobility.
Nitor offers end to end testing services at aid our partners in continuous business growth through our
niche technology focus and diverse domain expertise. Our TCoE offering include
Highlights of Test Center of Excellence (TCoE)
ISTQB Gold Certified test partner
SSAE Soc- II Type-1 accredited organization
Technology and platform agnostic. NitorTCoE is technology /software / tool independent
70+ strong team of certified test engineers
Seven years of profound QA experience
Strong knowledge base with expertise in diverse methodologies and tools
Contact Us
NitorInfotech
401-408, A-Wing, Pride Silicon Plaza
S.B. Road
Pune -411006; Maharashtra, INDIA
Email: [email protected]