Www Rittmanmead Com 2014 01 Obiee Regression Testing an Intr

8/10/2019 Www Rittmanmead Com 2014 01 Obiee Regression Testing an Intr

1/18

pdfcrowd comopen in browser PRO version Are you a developer? Try out the HTML to PDF API

Home About Careers Clients Consulting Training Support Articles Blog

OBIEE Regression Testing An IntroductionJanuary 17th, 2014 by Robin M offatt

In this article Im going to look at ways to test changes that you m ake to OBIEE to ens ure that they dont break existing

functionality. In all but the sim plest IT systems its common for one (planned) action to inadvertently cause another

(unplanned).

What IS Regression Testing?When we make a change to a system we us e functional unit tests to ensure that it does do what it is supposed to do. We

should also m ake sure that the sam e changes dont do what theyre not supposed to, that is, cause functionality already

existing in the system tochange behaviour. If thisdoes happen it is known as a regressionand is something we want to

ensure doesnt happenwithout us knowing. Some examples ofregressionsseen in standard OBIEEdevelopment changes

include:

Reports stop returning data, showing an error instead

Reports start to show the wrong data

Somecombinations of dimensions and facts to no longer show data, or show an error

Search theblog

Recent Posts

Analytics w ith Kibana andElasticsearch through Hadoop

part 3 Visualising the data inKibana

Analytics w ith Kibana and

Elasticsearch through Hadoop part 2 Getting data into

Elasticsearch

Analytics w ith Kibana and

Elasticsearch through Hadoop part 1 Introduction

UKOUG Partner of the YearAw ards

Oracle BI Cloud Service for SaaSApplication Reporting Part 1:Integrating BICS toSalesforce.com using REST APIs

Top PostsOBIEE 11g Security Week :

Managing Application Roles andPolicies, and Managing SecurityMigrations and Deployments

Upgrading OBIEE to 11.1.1.7

OBIEE 11gR1 : Architecture andUse of WebLogic Server

OBIEE 11g Security Week :Connecting to Active Directory,and Obtaining Group Membership
http://www.rittmanmead.com/2010/07/obiee-11gr1-architecture-and-use-of-weblogic-server/http://www.rittmanmead.com/author/robin-moffatt/http://www.rittmanmead.com/2014/01/obiee-regression-testing-an-introduction/http://www.rittmanmead.com/http://www.rittmanmead.com/about/our-teamhttp://www.rittmanmead.com/about/our-teamhttp://www.rittmanmead.com/about/careers/http://www.rittmanmead.com/clients/http://www.rittmanmead.com/consulting/http://www.rittmanmead.com/training/http://www.rittmanmead.com/support/http://www.rittmanmead.com/support/http://www.rittmanmead.com/articles/http://www.rittmanmead.com/articles/http://www.rittmanmead.com/blog/http://www.rittmanmead.com/http://www.rittmanmead.com/http://www.rittmanmead.com/about/our-teamhttp://www.rittmanmead.com/about/careers/http://www.rittmanmead.com/clients/http://www.rittmanmead.com/consulting/http://www.rittmanmead.com/training/http://www.rittmanmead.com/support/http://www.rittmanmead.com/articles/http://www.rittmanmead.com/blog/http://www.rittmanmead.com/http://www.rittmanmead.com/2012/03/obiee-11g-security-week-managing-application-roles-and-policies-and-managing-security-migrations-and-deployments/http://www.rittmanmead.com/2014/11/oracle-bi-cloud-service-for-saas-application-reporting-part-1-integrating-bics-to-salesforce-using-rest-apis/http://www.rittmanmead.com/2014/11/oracle-bi-cloud-service-for-saas-application-reporting-part-1-integrating-bics-to-salesforce-using-rest-apis/http://www.rittmanmead.com/2014/11/ukoug-partner-of-the-year-awards/http://www.rittmanmead.com/2014/11/ukoug-partner-of-the-year-awards/http://www.rittmanmead.com/2014/11/analytics-with-kibana-and-elasticsearch-through-hadoop-part-1-introduction/http://www.rittmanmead.com/2014/11/analytics-with-kibana-and-elasticsearch-through-hadoop-part-1-introduction/http://www.rittmanmead.com/2014/11/analytics-with-kibana-and-elasticsearch-through-hadoop-part-2-getting-data-into-elasticsearch/http://www.rittmanmead.com/2014/11/analytics-with-kibana-and-elasticsearch-through-hadoop-part-2-getting-data-into-elasticsearch/http://www.rittmanmead.com/2014/11/analytics-with-kibana-and-elasticsearch-through-hadoop-part-2-getting-data-into-elasticsearch/http://www.rittmanmead.com/2014/11/analytics-with-kibana-and-elasticsearch-through-hadoop-part-2-getting-data-into-elasticsearch/http://www.rittmanmead.com/2014/11/analytics-with-kibana-and-elasticsearch-through-hadoop-part-3-visualising-the-data-in-kibana/http://www.rittmanmead.com/2014/11/analytics-with-kibana-and-elasticsearch-through-hadoop-part-3-visualising-the-data-in-kibana/http://www.rittmanmead.com/2014/11/analytics-with-kibana-and-elasticsearch-through-hadoop-part-3-visualising-the-data-in-kibana/http://www.rittmanmead.com/http://www.rittmanmead.com/http://www.rittmanmead.com/http://pdfcrowd.com/http://pdfcrowd.com/redirect/?url=http%3a%2f%2fwww.rittmanmead.com%2f2014%2f01%2fobiee-regression-testing-an-introduction%2f&id=ma-141108035645-966c0d38http://pdfcrowd.com/customize/http://pdfcrowd.com/html-to-pdf-api/?ref=pdfhttp://www.rittmanmead.com/http://www.rittmanmead.com/about/our-teamhttp://www.rittmanmead.com/about/careers/http://www.rittmanmead.com/clients/http://www.rittmanmead.com/consulting/http://www.rittmanmead.com/training/http://www.rittmanmead.com/support/http://www.rittmanmead.com/articles/http://www.rittmanmead.com/blog/http://www.rittmanmead.com/http://www.rittmanmead.com/2014/01/obiee-regression-testing-an-introduction/http://www.rittmanmead.com/author/robin-moffatt/http://www.rittmanmead.com/2014/11/analytics-with-kibana-and-elasticsearch-through-hadoop-part-3-visualising-the-data-in-kibana/http://www.rittmanmead.com/2014/11/analytics-with-kibana-and-elasticsearch-through-hadoop-part-2-getting-data-into-elasticsearch/http://www.rittmanmead.com/2014/11/analytics-with-kibana-and-elasticsearch-through-hadoop-part-1-introduction/http://www.rittmanmead.com/2014/11/ukoug-partner-of-the-year-awards/http://www.rittmanmead.com/2014/11/oracle-bi-cloud-service-for-saas-application-reporting-part-1-integrating-bics-to-salesforce-using-rest-apis/http://www.rittmanmead.com/2012/03/obiee-11g-security-week-managing-application-roles-and-policies-and-managing-security-migrations-and-deployments/http://www.rittmanmead.com/2013/04/upgrading-obiee-to-11-1-1-7/http://www.rittmanmead.com/2010/07/obiee-11gr1-architecture-and-use-of-weblogic-server/http://www.rittmanmead.com/2012/03/obiee-11g-security-week-connecting-to-active-directory-and-obtaining-group-membership-from-database-tables/


2/18


Dashboards that reference a particular analysis s top working

As well as these, less com mon system changes can also cause regressions , for example:

An OBIEE vers ion upgrade causes certain types of graph to render in a d ifferent way from the previous version

An OBIEE patch introduces a bug in the front end us er interface

What drives Regression Testing?The requirement for regression testing OBIEE broadly comes from two different types of change:

New binaries that is, an upgrade (or patch) of OBIEE

New application code changes to the RPD, the underlying database schem a, and so on .

These two requirements have the same aim make sure nothing breaks when we m ake the change but differ in ways that

make how we address them important:

1. Frequency: OBIEE may get patched once or twice a year, and upgraded every few years. Com pare this to developmentchanges made to the RPD et al, which users would often like to see happening on a frequent basis (sometimes dailyat the beginning of an implem entation). If these changes are happening with great regularity then (a) we dont want tobe the ones causing the bottleneck because we cant regress ion test them and thus (b) we need to find a repeatableway to perform these tests accurately and quickly.

2. Delta Visibility: When Oracle change the OBIEE code base, we are blind as to what has gone on under the covers.Sure, we know whats changed in the documentation, but as a starting point for what might have broken we can onlyassume everythinghas and test accordingly. Conversely, in a planned development we know exactly what wechangedand we can therefore work out the scope of the necessary testing.

The points in bold above are what I aim to addres s in this article. Regress ion testing OBIEE doesnt have to mean one

technique alone it can be refined based on what we know has changed.

Why Regression Test?If you dont regression testthen you place a wager that youll be ab le to fix any prob lems that arise. As soon as they arise. In

Production. With angry users on the phone. And the project manager screaming b lue murder because their change is getting

blam ed for b reaking everything.

This is a recipe for compounding errors upon errors, not a stable system. Testing, in all flavours, is about gaining

confidenceabout the impact of a proposed change to a system. Functionaltesting reassures us that the change will do what

from Database Tables

Analytics w ith Kibana andElasticsearch through Hadoop -

part 3 - Visualising the data inKibana

Random Posts

Visual Regression Testing ofOBIEE w ith PhantomCSS

Create a Mobile BI App for iOS,Android or BlackBerry w ithin 5Days!

Looking at the ODI12c Hadoop

Demos in the New OracleBigDataLite VM

New s and Updates f rom OracleOpenw orld 2014

Oracle Database 12cR1 - NewFeatures for BI & DW

Tags11gBig Data Appliance

BIPBI Publisherdwem12cEndecaexalyticsextremebigit

goldengatehadoopHiveinit.dinstalllinuxMDS XMLmonitoring

new featuresnqcmdOBIA

obieeodiodi12copatchOracleOracle BI

Applications oracle dataintegratorOracleEndecaOracle EndecaInformation Discoveryow b

performanceReal Time
http://www.rittmanmead.com/2012/03/obiee-11g-security-week-connecting-to-active-directory-and-obtaining-group-membership-from-database-tables/http://pdfcrowd.com/http://pdfcrowd.com/redirect/?url=http%3a%2f%2fwww.rittmanmead.com%2f2014%2f01%2fobiee-regression-testing-an-introduction%2f&id=ma-141108035645-966c0d38http://pdfcrowd.com/customize/http://pdfcrowd.com/html-to-pdf-api/?ref=pdfhttp://www.rittmanmead.com/2012/03/obiee-11g-security-week-connecting-to-active-directory-and-obtaining-group-membership-from-database-tables/http://www.rittmanmead.com/2014/11/analytics-with-kibana-and-elasticsearch-through-hadoop-part-3-visualising-the-data-in-kibana/http://www.rittmanmead.com/2014/05/visual-regression-testing-of-obiee-with-phantomcss/http://www.rittmanmead.com/2013/09/create-a-mobile-bi-app-for-ios-android-or-blackberry-within-5-days/http://www.rittmanmead.com/2014/01/looking-at-the-odi12c-hadoop-demos-in-the-new-oracle-bigdatalite-vm/http://www.rittmanmead.com/2014/10/news-and-updates-from-oracle-openworld-2014/http://www.rittmanmead.com/2013/07/oracle-database-12cr1-new-features-for-bi-dw/http://www.rittmanmead.com/tag/11g/http://www.rittmanmead.com/tag/big-data-appliance/http://www.rittmanmead.com/tag/bip/http://www.rittmanmead.com/tag/bi-publisher/http://www.rittmanmead.com/tag/dw/http://www.rittmanmead.com/tag/em12c/http://www.rittmanmead.com/tag/endeca/http://www.rittmanmead.com/tag/exalytics/http://www.rittmanmead.com/tag/extremebi/http://www.rittmanmead.com/tag/git/http://www.rittmanmead.com/tag/goldengate/http://www.rittmanmead.com/tag/hadoop/http://www.rittmanmead.com/tag/hive/http://www.rittmanmead.com/tag/init-d/http://www.rittmanmead.com/tag/install/http://www.rittmanmead.com/tag/linux-2/http://www.rittmanmead.com/tag/mds-xml/http://www.rittmanmead.com/tag/monitoring/http://www.rittmanmead.com/tag/new-features/http://www.rittmanmead.com/tag/nqcmd/http://www.rittmanmead.com/tag/obia/http://www.rittmanmead.com/tag/obiee/http://www.rittmanmead.com/tag/odi/http://www.rittmanmead.com/tag/odi12c/http://www.rittmanmead.com/tag/opatch/http://www.rittmanmead.com/tag/oracle/http://www.rittmanmead.com/tag/oracle-bi-applications/http://www.rittmanmead.com/tag/oracle-data-integrator-2/http://www.rittmanmead.com/tag/oracle-endeca/http://www.rittmanmead.com/tag/oracle-endeca-information-discovery/http://www.rittmanmead.com/tag/owb/http://www.rittmanmead.com/tag/performance-2/http://www.rittmanmead.com/tag/real-time-decisions-2/http://www.rittmanmead.com/tag/real-time-decisions-2/


3/18


it was designed to do. Performancetesting helps us understand how a system behaves from a response time and capacity

perspective. Regress ion testing gives us the confidence that a change, whilst doing what it ought to, isnt going to affect

something else.

The confidencein what is (and isnt) going to happen when we deploy a change enables us to make these changes more

frequentlyas required by the users. Instead of a long development cycle with a huge number of changes bunched in

together, and one big bang test and releas e, we can take a more rapid, flexible, and responsive approach to development

and release because we have the confidencethat an individual change is going to work.

In addition to confidence in additional releasesto new deployments, a good regression testing framework enables us to have

confidence in making changes to long-standing big ball of mudsystems . So long as we understand the relevant interfaces

points in OBIEE, we can build a pass /fail test framework on top of the mos t complex RPD/schem a.

Targeting Regression Testing EffectivelyRegress ion testing is easy. You pay a troop of monkeys to sit at a set of computers and run every single dashboard, build

every permutation of adhoc report, and if youve jus t upgraded or patched OBIEE, go through the us er interface with a fine

toothed comb. After the appropriate period of several weeks , any differences they find from before your change was made is

a regress ion. Congratulations. All you need to do now is fix the problem and then of course, regress ion test your newchange. So monkeys are one option, but theyre expensive (you should s ee the wholesale market peanut pricethese days),

theyre not infallible (monkeys get distracted by YouTubetoo), and they are slow.

Better than monkeys is automated regression testing, targetedsm artly at the area of OBIEE that has been changed. We will

now take a look at which changes can caus e regress ions in which area, and from that derive a lis t of testing m ethods

appropriate for each type of change m ade.

Regression testing points in the OBIEE stack

To understand howwe can regress ion test OBIEE, let us look at wherea regress ion can be detected. The following diagram

illustrates the reques t/response flow through the components in the OBIEE stack. We can use it to see where regressions

may expose them selves, and thus understand at what points we can cons ider testing for them.

ec s onsreplicationReportService RTDrunReportsampleappscreenscripting

securitystartup testingtrainingXML
http://www.rittmanmead.com/tag/real-time-decisions-2/http://pdfcrowd.com/http://pdfcrowd.com/redirect/?url=http%3a%2f%2fwww.rittmanmead.com%2f2014%2f01%2fobiee-regression-testing-an-introduction%2f&id=ma-141108035645-966c0d38http://pdfcrowd.com/customize/http://pdfcrowd.com/html-to-pdf-api/?ref=pdfhttp://en.wikipedia.org/wiki/Big_ball_of_mud?__hstc=150481449.7a95a279239ec5da11885c9ea7b1e51b.1415437003373.1415437003373.1415437003373.1&__hssc=150481449.1.1415437003374&__hsfp=1770896186http://www.indexmundi.com/commodities/?commodity=peanuts&months=240&__hstc=150481449.7a95a279239ec5da11885c9ea7b1e51b.1415437003373.1415437003373.1415437003373.1&__hssc=150481449.1.1415437003374&__hsfp=1770896186http://www.youtube.com/watch?v=dQw4w9WgXcQ&__hstc=150481449.7a95a279239ec5da11885c9ea7b1e51b.1415437003373.1415437003373.1415437003373.1&__hssc=150481449.1.1415437003374&__hsfp=1770896186http://www.rittmanmead.com/tag/real-time-decisions-2/http://www.rittmanmead.com/tag/replication/http://www.rittmanmead.com/tag/reportservice/http://www.rittmanmead.com/tag/rtd/http://www.rittmanmead.com/tag/runreport/http://www.rittmanmead.com/tag/sampleapp/http://www.rittmanmead.com/tag/screen/http://www.rittmanmead.com/tag/scripting/http://www.rittmanmead.com/tag/security/http://www.rittmanmead.com/tag/startup/http://www.rittmanmead.com/tag/testing-2/http://www.rittmanmead.com/tag/training/http://www.rittmanmead.com/tag/xml/


4/18


Starting from the point of view of the actual end user:

The user interfacemay regress. They may be actual bugs that werent there before, or regress ions in the sens e that

functionality or icons/layout have changed. These changes would typically only come about through software changes

(patching/upgrades).

Regress ions could also occur if you are manipulating the UI through the analysis itself (eg narrative view) and the

behaviour changes , but this type of UI modification is less comm on.

Regress ions caus ed by changes to the underlying data, RPD or analyses are going to manifest thems elves through a

dashboard. This could be in the dataor the presentationof the data (tables, graphs , etc).
http://pdfcrowd.com/http://pdfcrowd.com/redirect/?url=http%3a%2f%2fwww.rittmanmead.com%2f2014%2f01%2fobiee-regression-testing-an-introduction%2f&id=ma-141108035645-966c0d38http://pdfcrowd.com/customize/http://pdfcrowd.com/html-to-pdf-api/?ref=pdf


5/18


Considering a dashboard by its cons tituent parts, an individual analysiscould exhibit differences in its dataor the

presentationof the data

Next to consider is that each analysis sends a Logical SQLrequest to the BI Server. It is not common, but it is pos sible that

a change to the binaries (version upgrade/patch) could introduce a regress ion that caused the Logical SQL to be generated

incorrectly. Specific changes to the RPD can als o cause the Logical SQL generation to change, potentially erroneous ly.


6/18


The Logical SQL that is generated is executed by the BI Server which in turn returns the reques ted logical resultsetdata.

This resu ltset may expose a regress ion in how the BI Server is handling the logical reques t.

A Logical SQL reques t on the BI Server is parsed through the m etadata layer, the RPD, and one or m ore Physical SQL

statements a re sent to the underlying data source(s). An error in the RPD could result in the Physical SQL being generated

incorrectly.


7/18


8/18


Regression testing opportunities

To summarise the previous section, our testing points for regression are as follows.

1. The logical query generated by Presentation Services for an analysis2. The physical query/queries generated by the BI Server to retrieve the data from the data s ource(s)3. The data supplied by the data source to the BI server4. The data supplied by the BI server for an analysis (logical res ultset)


9/18

df d mi b PRO i Are you a developer? Try out the HTML to PDF API

5. User interface, including the dashboard/analysis, taking into account both rendered data and presentation/UI.

Regress ion testing is bas ed around comparing one state (before a planned change) to another (after the planned change).

In considering how we are going to perform our testing, lets take a very sim plistic view on what we need to test:

1. Does it look the same2. Are the numbers the same

Of these two, one is very easy to get a computer to do (and conversely, very laborious to perform manual ly), and the other is

very difficult to explain to a computer (and relatively easy to do manually): -

Telling a computer to fetch some data twice and compare the first result with the second is bread and butterautomation.

Trying to explain to a computer what a page looks like, or what a user interface does is extremely time consuming , and

inevitably specific to the s ingle item in ques tion. Of course, we canprogramm atically compare the underlying code for a

dashboard before and after a change, but the ques tion I pose is whether we should.
http://pdfcrowd.com/http://pdfcrowd.com/redirect/?url=http%3a%2f%2fwww.rittmanmead.com%2f2014%2f01%2fobiee-regression-testing-an-introduction%2f&id=ma-141108035645-966c0d38http://pdfcrowd.com/customize/http://pdfcrowd.com/html-to-pdf-api/?ref=pdfhttp://dictionary.cambridge.org/dictionary/british/bread-and-butter?__hstc=150481449.7a95a279239ec5da11885c9ea7b1e51b.1415437003373.1415437003373.1415437003373.1&__hssc=150481449.1.1415437003374&__hsfp=1770896186


10/18

df di b PRO i Are you a developer? Try out the HTML to PDF API

Computers are blind

The user interface for an OBIEE end user is a web browser, and OBIEE builds its web pages through a set of languages and

protocols that used to be quaintly referred to as Web 2.0. It uses HTML, CSS, XML, and JavaScript, taking plentiful

advantage of as ynchronous page loading and in-flight modifications to the Document Object Model (DOM) too. AJAXis a

term which certainly covers s ome of the magic that goes on. The resulting us er interface is pretty slick with drop down

menus , expanding hierarchy trees, and partial das hboard rendering as data is returned rather than waiting for all analyses to

complete. All of this omits the knockout blow that is Flash, used for rendering all graph objects in OBIEE and the subject of at

notable UI bug in OBIEE 11.1.1.6. The Developer Tools option in modern web browsers gives us a glim pse into what isgoing on under the covers. We can see the number of resources that go into rendering a single page
http://pdfcrowd.com/http://pdfcrowd.com/redirect/?url=http%3a%2f%2fwww.rittmanmead.com%2f2014%2f01%2fobiee-regression-testing-an-introduction%2f&id=ma-141108035645-966c0d38http://pdfcrowd.com/customize/http://pdfcrowd.com/html-to-pdf-api/?ref=pdfhttps://support.oracle.com/epmos/faces/DocContentDisplay?id=1492684.1&__hstc=150481449.7a95a279239ec5da11885c9ea7b1e51b.1415437003373.1415437003373.1415437003373.1&__hssc=150481449.1.1415437003374&__hsfp=1770896186


11/18

df di b PRO i A d l ? T t th HTML t PDF API

and how many layers there are to the object model:


12/18

df di b PRO i A d l ? T t th HTML t PDF API

Getting a computer to interface with all of this, simulating a us er interaction and parsing the response isposs ible with

functional testing tools such as Selenium, Oracle Application Testing Suite, and HPs QuickTest Profess ional. Each of these

tools is capable of simulating a user (often by recording a session as the starting point) and parsing the responses from

OBIEE.


13/18df di b PRO i

Are you a developer? Try out the HTML to PDF API

But, there is a fundamental complication to using these tools . For all the AJAX/CSS/DOM magic to happen, the page that

OBIEE generates is littered with element identifiers (so that the JavaScript code can identify the elem ent to manipulate). For

example , the following table cell has the ID in this particular execution of e_saw_14485_10_1_0_0:

Some of these IDs may change between report executions or sess ions, but either way, cannot be relied on to be consistent.

Therefore, getting our testing tool (such as Selenium) to compare the before/after results to detect a regression becomes a


14/18df di b PRO iAre you a developer? Try out the HTML to PDF API

whole heap m ore tricky. It is pos sible to work out element paths based on their relativeposition within the page rather than

an absoluteID, but that becomes even more page specific and complex to implement. Therefore to compare a before and

after page programatically we have to either

define a particular part of the page aloneto check remains the s ame (and risk chucking the baby out with the bath

water, that is, missing other genuine regressions elsewhere on the page)

or we have to

compile a list of e lements that we expect may change but that we dont count as a regression(i.e. exceptions).

The latter is going to be prone to causing false pos itives (i.e. failing regression tests that arent genuine regress ions)

because it relies on reverse engineering the full Docum ent Object Model of the OBIEE page. All of this is also without even

taking into account software patching and upgrades so far as Oracle are going to be concerned how a page is rendered is

their own business and thus at full liberty to completely change the internal structure of a pageas they desire. Given this

above complication, it becomes clear that building a test agains t a single pageis time consum ing, and it will typically be

specific to that page only. This becomes a problem the greater the scale of the deployment you are trying to test. Hardcoding

the testing for one specific page might be fine, but given more than a handful of pages you risk ending up with a large

inflexible regress ion test code base (that itself may become error prone and need regres sion testing when its changed).

ConclusionSo, we come back to not howwe test the front end but more shouldwe, in everycase? Given a finite amount of time, what

are you going to get mos t benefit from in your regression tests? In the next post I will demonstrate one of the ways you can

get the most bang for your buck when regress ion testing OBIEE, by concentrating your automation efforts on the query part

of the OBIEE stack, and not the front end. Stay tuned!

Many thanks to Gianni Ceresa for his thoughts and assistance on this subject.

Related Posts:

Automated Regression Testing for OBIEE

Visual Regression Testing of OBIEE with PhantomCSS

Incremental refresh of Exalytics aggregates using native BI Server capabilities

Tags: bi server logical sql obiee Presentation Services regression selenium testing
http://pdfcrowd.com/http://pdfcrowd.com/redirect/?url=http%3a%2f%2fwww.rittmanmead.com%2f2014%2f01%2fobiee-regression-testing-an-introduction%2f&id=ma-141108035645-966c0d38http://pdfcrowd.com/customize/http://pdfcrowd.com/html-to-pdf-api/?ref=pdfhttp://www.rittmanmead.com/2014/01/automated-regression-testing-for-obiee/http://www.rittmanmead.com/2014/05/visual-regression-testing-of-obiee-with-phantomcss/http://www.rittmanmead.com/2013/04/incremental-refresh-of-exalytics-aggregates-using-native-bi-server-capabilities/http://www.rittmanmead.com/tag/bi-server/http://www.rittmanmead.com/tag/logical-sql/http://www.rittmanmead.com/tag/obiee/http://www.rittmanmead.com/tag/presentation-services/http://www.rittmanmead.com/tag/regression/http://www.rittmanmead.com/tag/selenium/http://www.rittmanmead.com/tag/testing-2/


15/18df di b PRO iAre you a developer? Try out the HTML to PDF API

Tweet 33 2Like

Tags: bi server, logical sql, obiee, Presentation Services, regression, selenium, testing

Tags: bi server, logical sql, obiee, Presentation Services, regression, selenium, testing

Posted in Oracle BI Suite EE, Testing| 5 Comments

Comments

Gerard NicolasSays:January 18th, 2014 at 12:40 pm

Hi Robin,

Beautiful article with a beautiful start picture. (The baby one where we can see a baby regression tes ting from a proud

father)

I have a couple of remarks. First, the bread and bu tter link does nt work.

The second rem arks is on the regress ion testing definition that you give:

Regress ion testing is bas ed around comparing one state (before a planned change) to another (after the planned

change).

For me, regress ion testing was just to ensure that we didnt have introduce an error by resolving one.

Therefore, running all the unit tests (differences between what you get and what you expect) were su fficient. Why do you

need to compare ? The before change s tate may be wrong.

I stay tuned, promis e.

Good weekend

Robin MoffattSays:January 18th, 2014 at 3:13 pm

Share 10

Hi Ni
http://pdfcrowd.com/redirect/?url=http%3a%2f%2fwww.rittmanmead.com%2f2014%2f01%2fobiee-regression-testing-an-introduction%2f&id=ma-141108035645-966c0d38http://pdfcrowd.com/redirect/?url=http%3a%2f%2fwww.rittmanmead.com%2f2014%2f01%2fobiee-regression-testing-an-introduction%2f&id=ma-141108035645-966c0d38http://pdfcrowd.com/redirect/?url=http%3a%2f%2fwww.rittmanmead.com%2f2014%2f01%2fobiee-regression-testing-an-introduction%2f&id=ma-141108035645-966c0d38http://pdfcrowd.com/redirect/?url=http%3a%2f%2fwww.rittmanmead.com%2f2014%2f01%2fobiee-regression-testing-an-introduction%2f&id=ma-141108035645-966c0d38http://pdfcrowd.com/http://pdfcrowd.com/redirect/?url=http%3a%2f%2fwww.rittmanmead.com%2f2014%2f01%2fobiee-regression-testing-an-introduction%2f&id=ma-141108035645-966c0d38http://pdfcrowd.com/customize/http://pdfcrowd.com/html-to-pdf-api/?ref=pdfhttps://twitter.com/intent/tweet?original_referer=http%3A%2F%2Fwww.rittmanmead.com%2F2014%2F01%2Fobiee-regression-testing-an-introduction%2F&text=OBIEE%20Regression%20Testing%20%E2%80%93%20An%20Introduction&tw_p=tweetbutton&url=http%3A%2F%2Fwww.rittmanmead.com%2F2014%2F01%2Fobiee-regression-testing-an-introduction%2F&via=rittmanmeadhttp://twitter.com/search?q=http%3A%2F%2Fwww.rittmanmead.com%2F2014%2F01%2Fobiee-regression-testing-an-introduction%2Fhttp://www.rittmanmead.com/tag/bi-server/http://www.rittmanmead.com/tag/logical-sql/http://www.rittmanmead.com/tag/obiee/http://www.rittmanmead.com/tag/presentation-services/http://www.rittmanmead.com/tag/regression/http://www.rittmanmead.com/tag/selenium/http://www.rittmanmead.com/tag/testing-2/http://www.rittmanmead.com/tag/bi-server/http://www.rittmanmead.com/tag/logical-sql/http://www.rittmanmead.com/tag/obiee/http://www.rittmanmead.com/tag/presentation-services/http://www.rittmanmead.com/tag/regression/http://www.rittmanmead.com/tag/selenium/http://www.rittmanmead.com/tag/testing-2/http://www.rittmanmead.com/category/oracle-bi-suite-ee/http://www.rittmanmead.com/category/testing/http://gerardnico.com/?__hstc=150481449.7a95a279239ec5da11885c9ea7b1e51b.1415437003373.1415437003373.1415437003373.1&__hssc=150481449.1.1415437003374&__hsfp=1770896186http://www.rittmanmead.com/wp-content/themes/rittman/#comment-57406http://www.rittmanmead.com/http://www.rittmanmead.com/wp-content/themes/rittman/#comment-57411


16/18

Are you a developer? Try out the HTML to PDF API

Hi Nico,

Thanks for the com ments. Ive fixed the link.

Regarding your second comment, if Ive unders tood correctly, youre s aying that regress ion testing could actually be done

by re-running unit tests and any failures would indicate a regress ion? If so, I would agree, but (a) not all *cough* BI

projects have unit tests written for them, (b) unit tests will often not have 100% coverage, and (c) by comparing before and

after states, even if before is wrong, at least it wil l s till be correctly wrong ;-)

Hope that makes s ense, let me know if Ive mis understood your point.

cheers, Robin.

Daniel Says:January 23rd, 2014 at 11:28 am

Hi Robin, spot on. I have used this approach taking advantage of OBIEE web services to test OOTB, custom and s uper

user personal reports.

A few times , we found false pos itives or fake defects where a report works correctly in the upgraded environm ent

(wrong results largely undetected in the bas e environment).

In my experience, the ability to discover data discrepancies before the users while applying a patch or going for anupgrade is priceless.

Looking forward to the 2nd part.

Cheers,

Daniel

Bibin KumarSays:January 23rd, 2014 at 7:09 pm

Robin, excellent article. You have brilliantly explained the nuances of testing BI Applications. Typical customer base of

500 odd reports es timating regress ion testing effort is difficult. I look forward to future articles that involve test

automation and how to m easure a before & after delta.

cheers!

Bibin Kumar
http://pdfcrowd.com/http://pdfcrowd.com/redirect/?url=http%3a%2f%2fwww.rittmanmead.com%2f2014%2f01%2fobiee-regression-testing-an-introduction%2f&id=ma-141108035645-966c0d38http://pdfcrowd.com/customize/http://pdfcrowd.com/html-to-pdf-api/?ref=pdfhttp://www.rittmanmead.com/wp-content/themes/rittman/#comment-57640http://www.linkedin.com/in/bibin?__hstc=150481449.7a95a279239ec5da11885c9ea7b1e51b.1415437003373.1415437003373.1415437003373.1&__hssc=150481449.1.1415437003374&__hsfp=1770896186http://www.rittmanmead.com/wp-content/themes/rittman/#comment-57657


17/18

pdfcrowd.comopen in browser PRO version Are you a developer? Try out the HTML to PDF API

Todd Ryan Says:January 31st, 2014 at 8:14 pm

Very good suggestions/comments. We use Loadrunner to script some of this with each release, helps find issues and

validate performance..

Call us now to talk about your BI project:

+44 (0) 1273 911 268 (UK) or(888) 631-1410 (USA)

or+61 3 9596 7186 (Australia & New Zealand) or

+91 997 256 7970 (India)or+32 280 882 11 (Belgium)

Home

About Us> About us

> About our team

> Contact us

> Our clients

Services> Consulting

> Training

> Support

ConsultingServices> Projects

> Expert Services

> OBIEE 11g

> Sustainability

> On Discoverer?

> Oracle DW

Training> OBIEE

Bootcamp

> OBIEE End-User

> Exalytics

> ODI 11g

Bootcamp

> Oracle BI Apps

Resources> Articles

> Blog

> OBIEE 11g

Blog Authors> Mark Rittma n

> Venkat J

> Peter Scott

> Borkur S

> Mike Vic kers

> Robin Moffatt

> Jon Mead

Rittman Mead Consulting ltd.

Registered Office : Suite B,

First Floor Moore House,

13 Black Lion Street,

Brighton, East Sussex,

BN1 1ND, United Kingdom

Company No. : 6032852

VAT No. : 900 3839 48

Rittman Mead America, Inc.

Registered Office : 4550 North Point Parkway

390 Alpharetta, Georgia 30022, USA

Rittman Mead Oceania Pty Ltd.

Registered Office : 12 Moore Street,

Brighton East,

Victoria, 3187, Australia

Austral ian Company No. : 1 49 458 9 35

Rittman Mead Consulting Pvt Ltd.

Registered Office : Unit 105-106

Regent Prime

Whitefield Main Road

Whitefield
http://pdfcrowd.com/http://pdfcrowd.com/redirect/?url=http%3a%2f%2fwww.rittmanmead.com%2f2014%2f01%2fobiee-regression-testing-an-introduction%2f&id=ma-141108035645-966c0d38http://pdfcrowd.com/customize/http://pdfcrowd.com/html-to-pdf-api/?ref=pdfhttp://www.rittmanmead.com/wp-content/themes/rittman/#comment-58073http://www.rittmanmead.com/http://www.rittmanmead.com/about/http://www.rittmanmead.com/about/our-team/http://www.rittmanmead.com/about/contact-us/http://www.rittmanmead.com/clients/http://www.rittmanmead.com/consultinghttp://www.rittmanmead.com/traininghttp://www.rittmanmead.com/supporthttp://www.rittmanmead.com/consulting/expert-services/http://www.rittmanmead.com/consulting/expert-services/http://www.rittmanmead.com/consulting/obiee-11g-services/http://www.rittmanmead.com/consulting/prepare-build-and-sustain/http://www.rittmanmead.com/consulting/migrating-from-discoverer-to-oracle-bi-suite-enterprise-edition/http://www.rittmanmead.com/consulting/data-warehousing/http://www.rittmanmead.com/training/trn202-oracle-bi-ee-11g-bootcamp/http://www.rittmanmead.com/training/trn-205-oracle-bi-ee-11g-create-reports-dashboards-alerts-and-scorecards/http://www.rittmanmead.com/training/trn-701-exalytics-for-systems-administrators/http://www.rittmanmead.com/training/trn-403-odi-11g-bootcamp/http://www.rittmanmead.com/training/trn-40-oracle-bi-apps-bootcamp/http://www.rittmanmead.com/articleshttp://www.rittmanmead.com/bloghttp://www.rittmanmead.com/obiee11ghttp://www.rittmanmead.com/author/mark-rittman/http://www.rittmanmead.com/author/venkatakrishnan-j/http://www.rittmanmead.com/author/peter-scott/http://www.rittmanmead.com/author/borkur-steingrimsson/http://www.rittmanmead.com/author/mike-vickers/http://www.rittmanmead.com/author/robin-moffatt/http://www.rittmanmead.com/author/jon-mead/


18/18

pdfcrowd.comopen in browser PRO version Are you a developer? Try out the HTML to PDF API

Website Design & Build: tymedia.co.uk

Bangalore

560066

Rittman Mead Belgium

Registered Office : Chausse de Louvain 426

1380 Lasne

Belgium

2010-2011 Rittm an Mead Cons ulting. | Privacy Policy | E: info@rittm anmead.com
http://pdfcrowd.com/http://pdfcrowd.com/redirect/?url=http%3a%2f%2fwww.rittmanmead.com%2f2014%2f01%2fobiee-regression-testing-an-introduction%2f&id=ma-141108035645-966c0d38http://pdfcrowd.com/customize/http://pdfcrowd.com/html-to-pdf-api/?ref=pdfhttp://www.tymedia.co.uk/?__hstc=150481449.7a95a279239ec5da11885c9ea7b1e51b.1415437003373.1415437003373.1415437003373.1&__hssc=150481449.1.1415437003374&__hsfp=1770896186http://www.rittmanmead.com/privacy-policy/mailto:[email protected]

Documents

Www Rittmanmead Com 2014 01 Obiee Regression Testing an Intr