Www Rittmanmead Com 2013 03 Performance and Obiee Test Defin

8/10/2019 Www Rittmanmead Com 2013 03 Performance and Obiee Test Defin

1/26

pdfcrowd comopen in browser PRO version Are you a developer? Try out the HTML to PDF API

Home About Careers Clients Consulting Training Support Articles Blog

Performance and OBIEE part II Test DefineMarch 18th, 2013 by Robin M offatt

Take the first tentative s teps along the yellow brick road of investigating performance and OBIEE with this jum p into the n itty

gritty of how we s hould define a performance test.

An overview of defining a performance testThe first step in asses sing the performance of OBIEE is defining the testwe are going to us e. For troubleshooting a

performance problem in Production, this might be a s ingle report. For performance testing a new system it might be more

complex. The process to define it needs to be the sam e though.

Let us cons ider for a mom ent what it is we want to achieve through a performance test. It should be a validation or possibly

determination of som e or all of the following, which may or may not be formally documented in a project as Non-Functional

Requirements (NFRs):

Report response times

User concurrency levels

Hardware utilisation

It doesn t sound too difficult does it? We can run a report and time how long it takes, and we can run the sam e report multiple

times in parallel and s ee how long it takes then, and around all of that we can gather system OS stats.

This seems to be the level at which some people perceive the complexity of performancetesting; that is, not at all.

But, to put any weight in the above results would presume that the system under test had :

Only one report

No param eters or other interactivity on the report

Every user runs the report at the exact same time

Search theblog

Recent Posts

Analytics w ith Kibana andElasticsearch through Hadoop

part 3 Visualising the data inKibana

Analytics w ith Kibana and

Elasticsearch through Hadoop part 2 Getting data into

Elasticsearch

Analytics w ith Kibana and

Elasticsearch through Hadoop part 1 Introduction

UKOUG Partner of the YearAw ards

Oracle BI Cloud Service for SaaSApplication Reporting Part 1:Integrating BICS toSalesforce.com using REST APIs

Top PostsOBIEE 11g Security Week :

Managing Application Roles andPolicies, and Managing SecurityMigrations and Deployments

Upgrading OBIEE to 11.1.1.7

OBIEE 11gR1 : Architecture andUse of WebLogic Server

OBIEE 11g Security Week :Connecting to Active Directory,and Obtaining Group Membership
http://www.rittmanmead.com/2010/07/obiee-11gr1-architecture-and-use-of-weblogic-server/http://www.rittmanmead.com/2014/11/analytics-with-kibana-and-elasticsearch-through-hadoop-part-1-introduction/http://www.rittmanmead.com/2014/11/analytics-with-kibana-and-elasticsearch-through-hadoop-part-2-getting-data-into-elasticsearch/http://www.rittmanmead.com/2014/11/analytics-with-kibana-and-elasticsearch-through-hadoop-part-3-visualising-the-data-in-kibana/http://www.rittmanmead.com/2013/03/performance-and-obiee-test-define/http://www.rittmanmead.com/2013/03/performance-and-obiee-test-define/http://www.rittmanmead.com/http://www.rittmanmead.com/about/our-teamhttp://www.rittmanmead.com/about/careers/http://www.rittmanmead.com/clients/http://www.rittmanmead.com/consulting/http://www.rittmanmead.com/training/http://www.rittmanmead.com/support/http://www.rittmanmead.com/articles/http://www.rittmanmead.com/blog/http://www.rittmanmead.com/http://www.rittmanmead.com/2012/03/obiee-11g-security-week-managing-application-roles-and-policies-and-managing-security-migrations-and-deployments/http://www.rittmanmead.com/2014/11/oracle-bi-cloud-service-for-saas-application-reporting-part-1-integrating-bics-to-salesforce-using-rest-apis/http://www.rittmanmead.com/2014/11/oracle-bi-cloud-service-for-saas-application-reporting-part-1-integrating-bics-to-salesforce-using-rest-apis/http://www.rittmanmead.com/2014/11/ukoug-partner-of-the-year-awards/http://www.rittmanmead.com/2014/11/ukoug-partner-of-the-year-awards/http://www.rittmanmead.com/2014/11/analytics-with-kibana-and-elasticsearch-through-hadoop-part-1-introduction/http://www.rittmanmead.com/2014/11/analytics-with-kibana-and-elasticsearch-through-hadoop-part-1-introduction/http://www.rittmanmead.com/2014/11/analytics-with-kibana-and-elasticsearch-through-hadoop-part-2-getting-data-into-elasticsearch/http://www.rittmanmead.com/2014/11/analytics-with-kibana-and-elasticsearch-through-hadoop-part-2-getting-data-into-elasticsearch/http://www.rittmanmead.com/2014/11/analytics-with-kibana-and-elasticsearch-through-hadoop-part-2-getting-data-into-elasticsearch/http://www.rittmanmead.com/2014/11/analytics-with-kibana-and-elasticsearch-through-hadoop-part-2-getting-data-into-elasticsearch/http://www.rittmanmead.com/2014/11/analytics-with-kibana-and-elasticsearch-through-hadoop-part-3-visualising-the-data-in-kibana/http://www.rittmanmead.com/2014/11/analytics-with-kibana-and-elasticsearch-through-hadoop-part-3-visualising-the-data-in-kibana/http://www.rittmanmead.com/2014/11/analytics-with-kibana-and-elasticsearch-through-hadoop-part-3-visualising-the-data-in-kibana/http://www.rittmanmead.com/http://www.rittmanmead.com/http://www.rittmanmead.com/http://pdfcrowd.com/http://pdfcrowd.com/redirect/?url=http%3a%2f%2fwww.rittmanmead.com%2f2013%2f03%2fperformance-and-obiee-test-define%2f&id=ma-141108035341-df455196http://pdfcrowd.com/customize/http://pdfcrowd.com/html-to-pdf-api/?ref=pdfhttp://www.rittmanmead.com/http://www.rittmanmead.com/about/our-teamhttp://www.rittmanmead.com/about/careers/http://www.rittmanmead.com/clients/http://www.rittmanmead.com/consulting/http://www.rittmanmead.com/training/http://www.rittmanmead.com/support/http://www.rittmanmead.com/articles/http://www.rittmanmead.com/blog/http://www.rittmanmead.com/http://www.rittmanmead.com/2013/03/performance-and-obiee-test-define/http://www.rittmanmead.com/author/robin-moffatt/http://www.rittmanmead.com/2014/11/analytics-with-kibana-and-elasticsearch-through-hadoop-part-3-visualising-the-data-in-kibana/http://www.rittmanmead.com/2014/11/analytics-with-kibana-and-elasticsearch-through-hadoop-part-2-getting-data-into-elasticsearch/http://www.rittmanmead.com/2014/11/analytics-with-kibana-and-elasticsearch-through-hadoop-part-1-introduction/http://www.rittmanmead.com/2014/11/ukoug-partner-of-the-year-awards/http://www.rittmanmead.com/2014/11/oracle-bi-cloud-service-for-saas-application-reporting-part-1-integrating-bics-to-salesforce-using-rest-apis/http://www.rittmanmead.com/2012/03/obiee-11g-security-week-managing-application-roles-and-policies-and-managing-security-migrations-and-deployments/http://www.rittmanmead.com/2013/04/upgrading-obiee-to-11-1-1-7/http://www.rittmanmead.com/2010/07/obiee-11gr1-architecture-and-use-of-weblogic-server/http://www.rittmanmead.com/2012/03/obiee-11g-security-week-connecting-to-active-directory-and-obtaining-group-membership-from-database-tables/


2/26


How many OBIEE implem entations do you know that match this des cription?

As soon as you accept this p remise, then you have to accept the need to des ign your performance tes t properly if you want

any kind of confidence in the res ults that you get.

Of course, there is the chance [sarcasm]vastly improbable and unheard of I know[/sarcasm] that Performance Testing is

a tick box on a project plan (normally about a week before Go Live), from which there is the expectation that it can deliver the

confidence of an accurate test with the s impli city and brevity of the one-report test I des cribe above. If that is the cas e then I

hope this article will explain why it is a fallacy.

Walk before you runThere are three key things to aim for in the design of a performance tes t:

1. Repeatability. You must be able to rerun the test, so that you can (a) prove bugs/performance problem s have beenfixed, and (b) re-use the test as a base line for future measurem ents

2. Accuracy. This can be difficult with a stack as complex as the OBIEE one, but you must do a ll you can to avoidconflation and mis interpretation of data.

3. Ease of Analysis. If the res ult of your test is unsucces sful, you will have to start analysing the is sue. If your test isunnecess arily big or complex then it will be very difficult and time consum ing to pare it back to find root cause.

Avoid the blunderbus approach. Its tempting (and fun!) to get carried away building a

huge test and run it with a big num ber of users but if it brings the system down then the

only immediately conclusion you can draw is that you can crash the system. To get to any

kind of root cause of the lim itation you will have to run more tests, and you will have to

sim plify the test that you did run to pinpoint the actual iss ue(s).

Build up small tes ts with single users and check things work at this level first. You can

then combine these together into larger tests for load testing. By working this way you get

the added benefit of a better unders tanding of the workings of your load test and system

being tested, as you will have gained this experience from the smal ler tests.

Think of it as as sem bling a cars engine for yourself versus buying a car out of the

showroom . If it breaks (or you want to tune it), youll be in a better place to start if you have

been under the bonnet [hood] already.

from Database Tables

Analytics w ith Kibana andElasticsearch through Hadoop -

part 3 - Visualising the data inKibana

Random Posts

Inside My Home OfficeDevelopment Lab (V MWare, OS X

Server)Photos and PresentationDownloads fr om the RittmanMead BI Forum 2014

Introducing the Updated Oracle /Rittman Mead InformationManagement Reference

Architecture Pt1. - Information

Architecture and the "DataFactory"

Preview ing TimesTen, Endecaand Oracle DW Sessions at the

Brighton BI Forum 2014Goodbye to Stew art...

Tags

11gBig Data Appliance

BIPBI Publisherdwem12cEndecaexalyticsextremebigit

goldengatehadoopHiveinit.dinstalllinuxMDS XMLmonitoring

new featuresnqcmdOBIA

obieeodiodi12copatchOracleOracle BI

Applications oracle data
http://www.rittmanmead.com/2012/03/obiee-11g-security-week-connecting-to-active-directory-and-obtaining-group-membership-from-database-tables/http://pdfcrowd.com/http://pdfcrowd.com/redirect/?url=http%3a%2f%2fwww.rittmanmead.com%2f2013%2f03%2fperformance-and-obiee-test-define%2f&id=ma-141108035341-df455196http://pdfcrowd.com/customize/http://pdfcrowd.com/html-to-pdf-api/?ref=pdfhttp://www.rittmanmead.com/2012/03/obiee-11g-security-week-connecting-to-active-directory-and-obtaining-group-membership-from-database-tables/http://www.rittmanmead.com/2014/11/analytics-with-kibana-and-elasticsearch-through-hadoop-part-3-visualising-the-data-in-kibana/http://www.rittmanmead.com/2013/08/inside-my-home-office-development-lab-vmware-os-x-server/http://www.rittmanmead.com/2014/05/photos-and-presentation-downloads-from-the-rittman-mead-bi-forum-2014/http://www.rittmanmead.com/2014/06/introducing-the-updated-oracle-rittman-mead-information-management-reference-architecture-pt1-information-architecture-and-the-data-factory/http://www.rittmanmead.com/2014/05/previewing-timesten-endeca-and-oracle-dw-sessions-at-the-brighton-bi-forum-2014/http://www.rittmanmead.com/2014/06/goodbye-to-stewart/http://www.rittmanmead.com/tag/11g/http://www.rittmanmead.com/tag/big-data-appliance/http://www.rittmanmead.com/tag/bip/http://www.rittmanmead.com/tag/bi-publisher/http://www.rittmanmead.com/tag/dw/http://www.rittmanmead.com/tag/em12c/http://www.rittmanmead.com/tag/endeca/http://www.rittmanmead.com/tag/exalytics/http://www.rittmanmead.com/tag/extremebi/http://www.rittmanmead.com/tag/git/http://www.rittmanmead.com/tag/goldengate/http://www.rittmanmead.com/tag/hadoop/http://www.rittmanmead.com/tag/hive/http://www.rittmanmead.com/tag/init-d/http://www.rittmanmead.com/tag/install/http://www.rittmanmead.com/tag/linux-2/http://www.rittmanmead.com/tag/mds-xml/http://www.rittmanmead.com/tag/monitoring/http://www.rittmanmead.com/tag/new-features/http://www.rittmanmead.com/tag/nqcmd/http://www.rittmanmead.com/tag/obia/http://www.rittmanmead.com/tag/obiee/http://www.rittmanmead.com/tag/odi/http://www.rittmanmead.com/tag/odi12c/http://www.rittmanmead.com/tag/opatch/http://www.rittmanmead.com/tag/oracle/http://www.rittmanmead.com/tag/oracle-bi-applications/http://www.rittmanmead.com/tag/oracle-data-integrator-2/http://www.rittmanmead.com/tag/oracle-endeca/http://www.rittmanmead.com/tag/oracle-data-integrator-2/


3/26


Define a UserThe first step in defining a good test is to define extremely precisely and clearlywhat it is we are testing. This sections

heading I take from Zed Shaws s uperb article Programmers need to learn statistics. For me it encapsulates in three

words the challenge that we face in des igning a good test. We know we want our system to support more than one us er, and

we probably have a us er concurrency figure (of which more be low) that the system must support. So, how do we define a

user? To quote from the aforementioned article, do we jus t click around a bit, you know, like a user? Maybe we run the

OBIEE-equivilent of SELECT 1 FROM DUALa thousand times sim ultaneously, and then declare our system can support a

thousand concurrent users?

The challenge is that good test or not, test results and system capacity will often be boiled down to a discuss ion of simp le

user numbers. Will the s ystem s upport 100 users? 200 users? And so on.

Modelling the workload of a transactional (OLTP) system is relativelystraightforward in that there is a finite number of

transactions, even if it is a large number. When we consider how to model an OBIEE user in a performance test, we have

additional challenges:

Navigation paths

In an OLTP system such as an online s tore once a user clicks on add to cart we can be pretty sure they will then

either (a) carry on shopping or (b) go to checkout. What about a BI user once theyve run their first report, which will

they run then? Maybe a different report, but quite likely it will be navigation within the existing report (eg drill down,

prompt changes , etc)

Ad hoc

Whilst some reporting will be fixed reports, much business value from BI comes in the analysis and exploration of

data in an ad hoc manner to discover trends etc to give competitive advantage. These ad hoc queries cannot, by

definition, be known in advance, so the performance test definition mus t find a way to address this.

Think about how many types of users you have on your system, and how you would characterise and group them. There will

be different sets of data they access , and varying degrees of interactivity with which they use the tool. Some will be using pre-

built reports with fixed filters, other may be doing extensive ad-hoc analyses , and som ewhere in the middle will be pre-built

reports but with a fair amount of interactivity through functionality such as prom pts and selection s teps.

Define a ReportSo for each of our User types that we define, we need to precisely define the workload for them. This wi ll be m ade up o f the

execution of one or more dashboards /reports. Defining the report that our model users are going to run is not as s imple as it

EndecaOracle EndecaInformation Discoveryow b

performanceReal TimeDecisionsreplicationReportService RTDrunReportsampleappscreenscripting

securitystartup testingtrainingXML
http://www.rittmanmead.com/tag/oracle-data-integrator-2/http://pdfcrowd.com/http://pdfcrowd.com/redirect/?url=http%3a%2f%2fwww.rittmanmead.com%2f2013%2f03%2fperformance-and-obiee-test-define%2f&id=ma-141108035341-df455196http://pdfcrowd.com/customize/http://pdfcrowd.com/html-to-pdf-api/?ref=pdfhttp://zedshaw.com/essays/programmer_stats.html?__hstc=150481449.35d69ee28ad764e69b542078dfc1c354.1415436819737.1415436819737.1415436819737.1&__hssc=150481449.1.1415436819737&__hsfp=1770896186http://www.rittmanmead.com/tag/oracle-data-integrator-2/http://www.rittmanmead.com/tag/oracle-endeca/http://www.rittmanmead.com/tag/oracle-endeca-information-discovery/http://www.rittmanmead.com/tag/owb/http://www.rittmanmead.com/tag/performance-2/http://www.rittmanmead.com/tag/real-time-decisions-2/http://www.rittmanmead.com/tag/replication/http://www.rittmanmead.com/tag/reportservice/http://www.rittmanmead.com/tag/rtd/http://www.rittmanmead.com/tag/runreport/http://www.rittmanmead.com/tag/sampleapp/http://www.rittmanmead.com/tag/screen/http://www.rittmanmead.com/tag/scripting/http://www.rittmanmead.com/tag/security/http://www.rittmanmead.com/tag/startup/http://www.rittmanmead.com/tag/testing-2/http://www.rittmanmead.com/tag/training/http://www.rittmanmead.com/tag/xml/


4/26


may appear. As well as the dis tinction to make between ind ividual reports (Analyses /Answers ) and Dashboards, we als o

need to consider:

Filters

How do we model a report which has six poss ible filters? We cant just call that report A, because each filter

permutation could cause different SQL to be generated by the BI reporting tool and cons equently vastly different

execution plan(s) on the database where the query runs.

Aggregate navigationOne of OBIEEs m any strengths is its aggregate navigation capability, enabling the us er to drill down into data whilst

in the background the tool switches between pre-defined aggregate tables on the database to pick the fastest one to

use. For performance testing we need to make sure that within a report each possib le aggregate is cons idered

separately.

As an example , let us examine a dashboard from SampleApp.
http://pdfcrowd.com/http://pdfcrowd.com/redirect/?url=http%3a%2f%2fwww.rittmanmead.com%2f2013%2f03%2fperformance-and-obiee-test-define%2f&id=ma-141108035341-df455196http://pdfcrowd.com/customize/http://pdfcrowd.com/html-to-pdf-api/?ref=pdfhttp://www.oracle.com/technetwork/middleware/bi-foundation/obiee-samples-167534.html?__hstc=150481449.35d69ee28ad764e69b542078dfc1c354.1415436819737.1415436819737.1415436819737.1&__hssc=150481449.1.1415436819737&__hsfp=1770896186


5/26


At an ini tial glance, we could specify our Report as s imply being Das hboard 3.10. Butthe das hboard is m ade up of

three analyses(plus some s tatic text/image objects)

Product Line Analysis

Ragged Hierarchy
http://pdfcrowd.com/http://pdfcrowd.com/redirect/?url=http%3a%2f%2fwww.rittmanmead.com%2f2013%2f03%2fperformance-and-obiee-test-define%2f&id=ma-141108035341-df455196http://pdfcrowd.com/customize/http://pdfcrowd.com/html-to-pdf-api/?ref=pdfhttp://www.rittmanmead.com/wp-content/uploads/2013/03/sampleapp_3.10_0.png


6/26


Value Based (Parent Child) Hierarchy

We need to consider each of these reports individually. In addition, there are hierarchical columnswhich, like Prompts, must

be included in our test specification. Heres one of the reports from the dashboard in two different states of hierarchicalnavigation.
http://pdfcrowd.com/http://pdfcrowd.com/redirect/?url=http%3a%2f%2fwww.rittmanmead.com%2f2013%2f03%2fperformance-and-obiee-test-define%2f&id=ma-141108035341-df455196http://pdfcrowd.com/customize/http://pdfcrowd.com/html-to-pdf-api/?ref=pdf


7/26


The first version, mostly collapsed, requires 3 queriesagains t the database. The second version requires 21 queries!

Clearly the performance of the two different versions of the report is going to differ.

Depending on the hierarchical column s tates, depending on dashboard prompt values, depending on aggregate drill pathsfollowed, the sam e individual report has the potential to behave quite differently.

Any report or das hboard thats using prom pts needs to have those prom pt values s tated in the test defin ition. The execution

of a query on the database can vary considerably depending on the predicates suppl ied, and it is in part from the

report/dashboard prompts that these predicates come.

Ad-hoc reports

Conspicuous by its absence from the above is how we deal with ad-hoc reporting with OBIEE in our performance tests. This

is the elephant in the roomwhen it comes to performance testing OBIEE, because we cannot go through the sam e

methodical process of cataloging the workload because it is ad hoc. This is discussed in detail later.

User workloadHaving precisely defined the reports which are going to be run, we can s equence them together to represent different types of

users . Clearly this is going to be a generalisation, an estimation, of how a particular group of users is go ing to tendto use

the reports. Unlike an OLTP transaction, where the steps are m uch more predefined and free from poss ible deviations,

users of OBIEE have the flexibility to run any number of reports, drill within existing reports, and s o on.
http://pdfcrowd.com/http://pdfcrowd.com/redirect/?url=http%3a%2f%2fwww.rittmanmead.com%2f2013%2f03%2fperformance-and-obiee-test-define%2f&id=ma-141108035341-df455196http://pdfcrowd.com/customize/http://pdfcrowd.com/html-to-pdf-api/?ref=pdfhttp://en.wikipedia.org/wiki/Elephant_in_the_room?__hstc=150481449.35d69ee28ad764e69b542078dfc1c354.1415436819737.1415436819737.1415436819737.1&__hssc=150481449.1.1415436819737&__hsfp=1770896186http://en.wikipedia.org/wiki/Ad_hoc?__hstc=150481449.35d69ee28ad764e69b542078dfc1c354.1415436819737.1415436819737.1415436819737.1&__hssc=150481449.1.1415436819737&__hsfp=1770896186#Ad_hoc_querying


8/26


Taking the above SampleApp dashboard as an example, one us er (lets call them the Product Analyst) might have the

following test sequence:

1. View Dashboard 3.10in default state2. Drill into hierarchical columns of Product Line Analysis3. View 1.3 Sample Sales - Product Details

Some user types may just run one report and exit; they dont have to have a complex sequence.

You dont need to restrict yourself to one us er type per actual group of users . You could m odel s everal different instances of

each user group in o rder or increase the test coverage and reali sm of different reports running concurrently. Be aware that

the more us er types and tests you have, the longer it will take to build, and the more complex the troubleshooting could be.

Think Time

Users are not computers, which is a shame because it would make reasoning with them a lot easier. It also means that they

do not jus t one run report after another. They run a report, look at the numbers, get a cup of tea, change a prompt value, have

a chat with a colleague, and so on.

As part of your us er workload, plan in s ome think times. Don t spend too long thinking about it (geddit), becaus e remember

this user workload is an approximation of what a certain type of user is going to be do ing.

User concurrency

Once we have designed a s et of individual user workloads, we can bring these together into a

larger test to assess the performance of the system as a whole under load. The aim of load

testing is to assess what happens to the response time of reports, and hardware utilisation, as

the number of concurrent users on the system increases. There are three key things to define in

a load test:

the workload of each user

the different types of user

the number of concurrent users

The first two of these are dis cussed above. When it comes to us er concurrency, we need to be very careful how we derive our


9/26

df d mi b PRO i Are you a developer? Try out the HTML to PDF API

figure, because it has a big impact on the scale of the tests that we run and the discussion of the outcomes.

First, make s ure that you are clear in the definition of concurrency that you us e. These include:

Concurrent user sessions logged into OBIEE, viewing das hboards, formatting reports, and so on.

Concurrently executing queries at the BI Server level, which could be a single user running one report or one

dashboard.

Concurrently executing queries on the database, which could be a single user in OBIEE running a s ingle report or

dashboard requiring more than one SQL request to get its data.

Each of these have their own particular load on the system, but can eas ily differ by orders o f magnitude. Typically a user

concurrency figure will be derived from bums on seats , i.e. number of users . From this, a reckoning is made on how many

will be us ing the system at one time. So unles s its qualified otherwise, I would work with the first definition concurrent

user sessions logged into OBIEE. Butif this is your definition, remem ber it when we come to simu lating users later on

For some reason, user concurrency is often a contentious num ber to define. With the greatest of apologies to Monty Python,

I offer this paraphas ing of a familiar conversation:

yah, my system needs to support 100 concurrent users

100? I d take that as a QUIET DAY for m y system, bwahaha, I need to test at least 1,000 concurrent users!

1,000? You were LUCKY! My system would run a gazilli on concurrent users at three oclock in the morning, work twenty

hours down tpit and still run 5,000 concurrent users be fore having a handful of cold g ravel for its tea

There does s eem to be BSD s yndrome around user concurrency figures, with people always

pitching high num bers. This is counterproductive. If you run a performance test us ing a

completely simulated workloadwith an excess ively high num ber of users as the target then the

margin of error (how accurately your test represents reality) inherent in the s imula tion

approach will be magnified as the user concurrency increases. The net effect of this i s that you

cannot be confident that any problems that do happen will actually happen in real life. Conversly,

if you are taking real workloadand running it for many users, you can hones tly say if it doesnt

work in testing, it wont work in real life.

If you are worried about pitching your user estim ate too low and s o double it or stick a zero on

the end jus t in case, cons ider the following: If you define your test as being a fail if it does not support your high figure of
http://pdfcrowd.com/http://pdfcrowd.com/redirect/?url=http%3a%2f%2fwww.rittmanmead.com%2f2013%2f03%2fperformance-and-obiee-test-define%2f&id=ma-141108035341-df455196http://pdfcrowd.com/customize/http://pdfcrowd.com/html-to-pdf-api/?ref=pdfhttps://www.youtube.com/watch?v=13JK5kChbRw&__hstc=150481449.35d69ee28ad764e69b542078dfc1c354.1415436819737.1415436819737.1415436819737.1&__hssc=150481449.1.1415436819737&__hsfp=1770896186


10/26

df di b PRO i Are you a developer? Try out the HTML to PDF API

concurrent users what then? You have to ask for money, either literally to buy bigger hardware, or in kind, to support the

time to optimis e the system you do have. Its not a great place to be in, and one which may be unnecess ary. A sens ible

approach is the following:

1. Load test to confirm whether the system can support the realistic number of usersin the s hort-term, e.g. go-live and afew months beyond

2. Deferpart of the load testing until after go-live, when you can run a test for medium /long-term user numbers using realuser workload for your tests. The benefit of this is that your second test will be far more accurate because yoursim ulation will not be based on how you thinkyou system will be us ed but how you knowis it being used.

If this approach doesnt suit, then your alternative should be to

1. take a realistic figure to start with for your firsttest2. run a secondperformance test to validate a future growth figure for your user bas e. If you dont have a future growth

figure, then you could consider the second test being to sim ply determine how many users your system cansupport byramping num bers up until things s tart to creak.

The point of doing it this way is that the firsttest as sum ing it gives a pos itive resu lt gives clarity and focus to your overall

results. If the secondtest fails, then at leas t you have the confidence from the first with which to proceed with a decis ion. If

you jus t run one big test then its an all or nothing approach, and if the test fails you have the option of either delaying launch

until a success ful test, or just cross ing your fingers and hoping.

User concurrency numbers for an existing system

If you have an existing s ystem then it is easy to determine your actual us er concurrency through either Usage Tracking(you

do have it enabled, don t you?) or the OBI DMS metrics.

If your user concurrency figure is based on users logged into OBIEE, not necess arily running reports, then the m etric

Oracle BI PS Sessions : Sessions Logged Inshould be appropriate.

For the number of queries (logical SQL) concurrently running on the BI Server, use Us age Tracking (see below), or DMS

metric t.b.c!

The number of concurrent queries on the databas e can either be obtained from your databases monitoring tool, or

through the Oracle BI DB Connection PoolDMS metrics.

To find out more about viewing and capturing OBI metrics over time, see here.

To estimate us er concurrency from Usage Tracking, you can us e the following query. It will analyse Usage Tracking to give
http://pdfcrowd.com/http://pdfcrowd.com/redirect/?url=http%3a%2f%2fwww.rittmanmead.com%2f2013%2f03%2fperformance-and-obiee-test-define%2f&id=ma-141108035341-df455196http://pdfcrowd.com/customize/http://pdfcrowd.com/html-to-pdf-api/?ref=pdfhttp://www.rittmanmead.com/2013/03/performance-and-obiee-execute-and-measure#measure


11/26

df di b PRO i A d l ? T t th HTML t PDF API

an approximate number of concurrent users , but this is based on individual queries run on the BI Server, so it not the sam e

as users simply logged into OBIEE, nor the same as dashboard refreshes(since one dashboard could be many queries):

Estimating user concurrency numbers

If you dont have empirical numbers to use then 10% of your total user base is the rule of thumb often worked to.

User concurrency is multi dimensional

Just as there is no s uch thing as A User, to ensure an accurate performance test user concurrency figures should also be

specified in terms o f :

Typesof concurrent user, for example:

10 Concurrent Finance Power Users

2 Concurrent Finance Ad-Hoc Users

60 Concurrent HR Report Consumers

Periods of system usage, for example :

Monday morning peak

EOD peak

Average

You may only actually test the peak user concurrency model, but its a good idea to have a picture of the s ystem profile,

particularly when analysing the im pact and capacity for ad-hoc usage.

Summary : User and Report Definition

-- Usage Tracking Concurrency usage estimate-- Per minute granularitySELECT TO_CHAR(start_ts,'YYYY-MM-DD HH24:MI')ASstart_ts_min,

COUNT(DISTINCT user_name) ASdistinct_usersFROM s_nq_acct

GROUP BY TO_CHAR(start_ts,'YYYY-MM-DD HH24:MI')ORDER BY1DESC


12/26df di b PRO i A d l ? T t th HTML t PDF API

Remember:

Before you can measure something you really need to lay down a very concrete definition of what youre measuring.

Zed Shaw

Whatever kind of testing you are doing, aim to make it modu lar wherever possible. It will make both analysis and reuse of

test code easier.

Almos t all test defini tions s hould be made up of one or m ore of the following levels :

1. Specific Report or dashboardalong with allprom pt values, hierarchical column settings, etc. This is the base unitofyour performance test.

2. Then take these individual tests, and use them together to create an individual User Workloadmodel for each user.3. Finally, combine the User models to produce an overall Concurrent Userworkload simulation.

See below for a worked example of a sam ple test definition.

Points to remem ber in your test definition:

1. Whilst each dashboard does have its own nam e, a good idea is to use your own ID system to make it easy to refer to aspecific dashboard state. For example , when using prompts there might be two or more versions of the dashboard totest (for different prompt values).Its als o easier to refer to som ething like ID01 in conversation and no tes than it is it Finance Dashboard ProfitabilityEMEA blah blah blah.

2. Dont forget default values in your navigation. If you are testing s pecific dashboard p rompt values, youll need to alsotest the default values first because that is what will get executed when the user first clicks onto the dashboard.

3. As well as testing das hboard response times, factor in the usability of any prompts that the user will interact with. A fastdashboard is not quite so impress ive if the user had to wait a long time for the dashboard prompts to display theavailable values .

4. You need to be writing down nuts and bolts detail here. Resis t the temptation to skimp on the detail or assume youllremem ber you wont, and certainly your colleague who can reuse your testing wont appreciate having to fill in the

blanks to make us e of it.

What do you mean, not all of it has been tested?Before you run off to write down all your dashboards and types of user, consider

the following. Unless you have :

1. A very sm all system2. A lot of time, and/or an infinite number of monkeys
http://pdfcrowd.com/http://pdfcrowd.com/redirect/?url=http%3a%2f%2fwww.rittmanmead.com%2f2013%2f03%2fperformance-and-obiee-test-define%2f&id=ma-141108035341-df455196http://pdfcrowd.com/customize/http://pdfcrowd.com/html-to-pdf-api/?ref=pdfhttp://zedshaw.com/essays/programmer_stats.html?__hstc=150481449.35d69ee28ad764e69b542078dfc1c354.1415436819737.1415436819737.1415436819737.1&__hssc=150481449.1.1415436819737&__hsfp=1770896186


13/26df di b PRO i

Are you a developer? Try out the HTML to PDF API

you are not going to be able to actually test all of your OBIEE system. Fact.

There are too many factors (prom pt values , aggregate paths, ad hoc us age) to be

able to model all plausible (let alone possible) permutations of these.

Designing the scope of a performance tes t needs to take into account the

acceptable riskof things going wrong vs time/budgetavailable vs functionality

required. By functionality, I mean factors such as the breadth of dashboards

delivered, or data made available.

In some s ystems the stability is param ount and therefore the relative riskmus t be minim ised. If this is the case and your

time/budgetis cons trained then therefore the functionalitywill have to be constrained.

If you need to deliver a lot of functionalityand ass uming that time/budgetis fixed, then you will need to accept an

increased riskto the stability of your system.

If you have limi tless time/budgetthen you can deliver lots of functionalityand do large and wide-ranging performance

tests to ensure minimal risk.

The point here is an important one because it is one that needs discus sing with the project owner, or more specifically,whoever will be accountable if things dont go right in Production. For a standard OLTP system it is much easier (relatively) to

design a performance test with a high degree of confidence that mos t [comm on] workloads have been simulated and tested.

However, for a reporting system such as OBIEE there are two reasons why it isnt so easily determined, as dis cussed above

and re-iterated here:

1. Permutations of fixed reporting each dashboard can have multiple prompt dimensions, hierarchical columnexpansions. Users can follow many different drill and navigation paths from any one dashboard.

2. Ad hoc reporting by definition, the report is not known until the user builds it, so we cannot test it beforehand

But, all is not lost. Even if we accept that to test every permutation of report that might run is impos sible, it is still perfectlyrealistic to perform a thoroughperform ance assessm entof a system prior to go-live. The mos t sens ible option is a hybrid

approach of:

Traditional performance testing of fixed reports as described above

Testing predefined ad-hoc reports (if that isnt a paradox)

Holistic and heuris tic evaluation of the system des ign and data model

Resource management to curtail the impact of bad queries

A ti i t i t id tif b d i d d l ith th ( i th T E A th d)
http://pdfcrowd.com/http://pdfcrowd.com/redirect/?url=http%3a%2f%2fwww.rittmanmead.com%2f2013%2f03%2fperformance-and-obiee-test-define%2f&id=ma-141108035341-df455196http://pdfcrowd.com/customize/http://pdfcrowd.com/html-to-pdf-api/?ref=pdfhttp://www.rittmanmead.com/2013/03/performance-and-obiee-introduction#tea


14/26df di b PRO iAre you a developer? Try out the HTML to PDF API

Active moni toring to identify bad queries and deal with them (us ing the T.E.A. method)

Feedback loop : as system usage becomes clearer, refine original performance testing us ing a more accurately

informed m odel (us ing the T.E.A. method)

This may seem a bit of an excuse for a cop-out. If we cant performance test all our system and be confident in the results,

whats the point? Here, I defer to Cary Millsap, a great proponent of the rigorous, s cientific, and rational approach to

performance testing:

How can you possibly test a new application enough to b e confident that youre not going to wreck your production

implementation with performance problems?

You can m odel. And you can test. However, nothing you do will be perfect. It is extremely di fficult to create models and tests in

which youll foresee all your production prob lems in advance of actually encountering those problem s in production.

Some people allow the apparent futility of this ob servation to justify not testing at all. Dont get trapped in that mentality. The

following points are certain:

Youll catch a lot more prob lems if you try to catch them prior to production than if you dont even try.

Youll never catch all your prob lems in pre-production testing. Thats why you need a reliab le and efficient method for

solving the prob lems that leak through your pre-production testing processes.

Somewhere in the m iddle b etween no testing and complete production emulation is the right am ount of testing. The right

amount of testing for aircraft manufacturers is probab ly more than the right amount of testing for companies that sell baseb all

caps. But dont skip performance testing altogether. At the very least, your performance test plan will make you a more

competent diagnostician (and clearer thinker) when it comes time to fix the performance prob lems that will inevitab ly occur

during production operation.

An example OBIEE test definitionHeres an example test definition, put together from SampleAppv207.

What we are going to model is two groups of users . One likes pretty pictures and just runs two chart-based das hboards and

then logs out. The other is interested in numbers and does some filtering on an analysis dashboard.

First, well define the dashboards in use. For brevity, Ive omitted the cons tituent analysis details, but in norm al practice its a

good idea to include them (for future reference, ease of understanding, clarity, etc).

D hb d
http://pdfcrowd.com/http://pdfcrowd.com/redirect/?url=http%3a%2f%2fwww.rittmanmead.com%2f2013%2f03%2fperformance-and-obiee-test-define%2f&id=ma-141108035341-df455196http://pdfcrowd.com/customize/http://pdfcrowd.com/html-to-pdf-api/?ref=pdfhttp://www.rittmanmead.com/2013/03/performance-and-obiee-introduction#teahttp://method-r.com/downloads/doc_details/44-thinking-clearly-about-performance?__hstc=150481449.35d69ee28ad764e69b542078dfc1c354.1415436819737.1415436819737.1415436819737.1&__hssc=150481449.1.1415436819737&__hsfp=1770896186http://www.oracle.com/technetwork/middleware/bi-foundation/obiee-samples-167534.html?__hstc=150481449.35d69ee28ad764e69b542078dfc1c354.1415436819737.1415436819737.1415436819737.1&__hssc=150481449.1.1415436819737&__hsfp=1770896186


15/26df di b PRO iAre you a developer? Try out the HTML to PDF API

Dashboards

Dashboard : ID01

Dashboard: 3.20 Vanilla Visuals

Page: Standard Visuals

Path: /shared/3. Analysis and Dashboards/_portal/3.20 Vanilla Visuals

Dashboard Prompts: None

Analyses:

/shared/3. Analysis and Dashboards/Report Views/Gallery/Examples of Standard Visualizations

Dashboard ID02
http://pdfcrowd.com/http://pdfcrowd.com/redirect/?url=http%3a%2f%2fwww.rittmanmead.com%2f2013%2f03%2fperformance-and-obiee-test-define%2f&id=ma-141108035341-df455196http://pdfcrowd.com/customize/http://pdfcrowd.com/html-to-pdf-api/?ref=pdfhttp://www.rittmanmead.com/wp-content/uploads/2013/03/perf_3.20_1.png


16/26

Are you a developer? Try out the HTML to PDF API

Dashboard : ID02

Dashboard: 3.20 Vanilla Visuals

Page: Trellis-Product Sales

Path: /shared/3. Analysis and Dashboards/_portal/3.20 Vanilla Visuals Trellis-Product Sales

Dashboard Prompts: None

Drill state: Default (NB each chart can be clicked on to drill into detail)

Analyses:

/shared/3. Analysis and Das hboards/Report Views/Trellis/Revenue Breakup

/shared/3. Analysis and Dashboards/Report Views/Trellis/Product Cost and Uni t Price Trend

/shared/3 Analysis and Dashboards/Report Views/Trellis/Brand Cos t and Unit Price Trend
http://pdfcrowd.com/http://pdfcrowd.com/redirect/?url=http%3a%2f%2fwww.rittmanmead.com%2f2013%2f03%2fperformance-and-obiee-test-define%2f&id=ma-141108035341-df455196http://pdfcrowd.com/customize/http://pdfcrowd.com/html-to-pdf-api/?ref=pdfhttp://www.rittmanmead.com/wp-content/uploads/2013/03/perf_3.20_prod_trellis.png


17/26

pdfcrowd.comopen in browser PRO version Are you a developer? Try out the HTML to PDF API

/shared/3. Analysis and Dashboards/Report Views/Trellis/Brand Cos t and Unit Price Trend

Offices Region Hier: All collapsed, except AMERICASexpanded one level

Dashboard : ID03

Dashboard: 1.3 Sample Sales

Page: Brand Analysis

Path: /shared/1. Quick Demos/_portal/1.3 Sample Sales Brand Analysis

Dashboard Prompts:

T05 Per Name Year: Default (2008, 2009, 2010, 2011)

D4 Company: Default (none)

T31 C l W k D f lt (b t 0 d 53)
http://pdfcrowd.com/http://pdfcrowd.com/redirect/?url=http%3a%2f%2fwww.rittmanmead.com%2f2013%2f03%2fperformance-and-obiee-test-define%2f&id=ma-141108035341-df455196http://pdfcrowd.com/customize/http://pdfcrowd.com/html-to-pdf-api/?ref=pdfhttp://www.rittmanmead.com/wp-content/uploads/2013/03/perf_1.3_brand_analysis.png


18/26


T31 Cal Week: Default (between 0 and 53)

Products.Products Hierarchy: Default (none)

Analyses:

/shared/1. Quick Demos/Simple Demo Dashboard/Brand Analysis/Product Line Analysis

/shared/1. Quick Demos/Simple Demo Dashboard/Brand Analysis/Trending Analysis

/shared/1. Quick Demos/Simple Demo Dashboard/Brand Analysis/Gauges

/shared/1. Quick Demos/Simple Demo Dashboard/Brand Analysis/Top Customers Bookings

/shared/1. Quick Demos/Simple Demo Dashboard/Brand Analysis/Daily Revenue Timeline

Dashboard : ID04


19/26


Dashboard: 1.3 Sample Sales

Page: Brand Analysis

Path: /shared/1. Quick Demos/_portal/1.3 Sample Sales Brand Analysis

Dashboard Prompts:

T05 Per Name Year: 2010, 2011

D4 Company: Stockplus Inc.

T31 Cal Week: between 1 and 27

Products.Products Hierarchy: Default (none)

Analyses:


20/26


/shared/1. Quick Demos/Simple Demo Dashboard/Brand Analysis/Product Line Analysis

/shared/1. Quick Demos/Simple Demo Dashboard/Brand Analysis/Trending Analysis

/shared/1. Quick Demos/Simple Demo Dashboard/Brand Analysis/Gauges

/shared/1. Quick Demos/Simple Demo Dashboard/Brand Analysis/Top Customers Bookings

/shared/1. Quick Demos/Simple Demo Dashboard/Brand Analysis/Daily Revenue Timeline

Tip:Use Catalog Manager to generate a report of dashboards and their constituent analyses to make the creation of the

above specification easier. :
http://pdfcrowd.com/http://pdfcrowd.com/redirect/?url=http%3a%2f%2fwww.rittmanmead.com%2f2013%2f03%2fperformance-and-obiee-test-define%2f&id=ma-141108035341-df455196http://pdfcrowd.com/customize/http://pdfcrowd.com/html-to-pdf-api/?ref=pdfhttp://www.rittmanmead.com/wp-content/uploads/2013/03/perf_catman0.pnghttp://www.rittmanmead.com/wp-content/uploads/2013/03/perf_catman1.pnghttp://www.rittmanmead.com/wp-content/uploads/2013/03/perf_catman1.pnghttp://www.rittmanmead.com/wp-content/uploads/2013/03/perf_catman2.png


21/26


User Workload

Having defined the dashboards (ID01 ID04), we define the user types and workloads:

User A

Description: Departm ent X, executive

Workload model:
http://www.rittmanmead.com/wp-content/uploads/2013/03/perf_catman1.pnghttp://www.rittmanmead.com/wp-content/uploads/2013/03/perf_catman1.pnghttp://www.rittmanmead.com/wp-content/uploads/2013/03/perf_catman1.pnghttp://pdfcrowd.com/http://pdfcrowd.com/redirect/?url=http%3a%2f%2fwww.rittmanmead.com%2f2013%2f03%2fperformance-and-obiee-test-define%2f&id=ma-141108035341-df455196http://pdfcrowd.com/customize/http://pdfcrowd.com/html-to-pdf-api/?ref=pdfhttp://www.rittmanmead.com/wp-content/uploads/2013/03/perf_catman2.png


22/26


1. ID01

2. Think time: 60 seconds

3. ID02


5. Logout

User BDescription: Department Y, analyst

Workload model:

1. ID03


3. ID04


5. Logout

Concurrency

Our concurrent user model looks like this. I am defining concurrent usersas active users, that is, executing dashboards or

simulated thinking.

Peak usage

User A : 5 concurrent users

User B : 2 concurrent users

Light usage

User A : 3 concurrent users

User B : 1 concurrent users

Types of testingBefore we can consider the types of test, we need to define what it iswe are tes ting, hence the long section above. Once

youve identified the types of user and s pecific reports they will run, you s hould also look at the type of test you want to run.


23/26


The terminology is frequently used interchangably, so heres my version of it:

Performance testing

With a single user, is the response time acceptable?

Load testing

With concurrent users, is the response time acceptable and s table?

Capacity testing

How does the system behave when concurrent users are sim ulated? do any areas s how signs of reaching capacity

of resource?

Soak testing

How does the system behave if the concurrent user workload is left to run for many hours? For example, are there any

mem ory leaks lurking, or connections which are not closed down properly and will eventually run out?

Stress testing

As we increas e the number of concurrent users beyond that required (and already tested success fully), how far willthe system go until something breaks or response times suffer?

In a high-availability deployment, what happens to response times if we simulate partial stack failures? Does the

system handle this gracefully under load?

These types of test do not have to be mututally exclusive in execution, but in defining your testing make sure you have covered

them all, or explained why they are not necess ary. An example would be performance tes ting a single report change on an

existing s ystem would probably not warrant a full scale s oak test.

In summaryThe output of this phas e of the method should typically be a set of test definitions, not one s ingle one. As already discuss ed,

keeping tests modular and simple is a much better approach than trying to kill all the birds with one boulder.

Footnote: The soft squidgy side of performance testing

Tangent: Enterprise change control processes

Taking the premise dis cussed above that it is impos sible to test every poss ible report execution raises an in teresting

question around change control and the stability of a Production environment. If your company is one where releases m ust

b t i l t t d d i d ff i th bl d f fi tb th i ht t t h thi ti th th


24/26


be triple tested and signed off in the blood of your firstborn the you might want to have this conversation sooner rather than

later. Why? Well if your system has any element of ad-hoc usage then there is a chance that a user will run a query that

disrupts other users on the system. There is always the potential for the perfect storm of it being a critical business period,

high us er traffic, and a rogue ad-hoc query. Without proper understanding of the nature of ad-hoc queries , an alm ighty witch

hunt can ensue, looking for who broke Production.

If this kind of risk is unacceptable then you will have to look into mitigating it with things like resource m anagement and

environment partitioning.

Estimating performance test for project planning

The challenge that the complexity and multiple facets of performance testing gives is finding a way to communicate it

accurately to project managers and business sponsors.

Unlike an es timate for code delivery, which will be days and after which the code will e ither function or not, performance

testing can pretty much take as long as you like, and the kicker is that the tests will only give varying degrees of confidence

that things shouldbe OK in production, but with no guarantees .

Doing the initial analysis on the system is a fixed time, as is building the test rig(s) with which to test it. Once you have donethis, then defining the number of dashboards to test can be done based on the time available. The more dashboards are

tested, the greater certainty can be had that any problems will be uncovered. Other fixed tasks s hould be a review of the

overall des ign, as well as time to properly document the tests undertaken and an analysis of the results obtained. It is a false

economy to skimp on the documentation, as to do so renders almos t useless the results ob tained for future work. Good

documentation of testing undertaken and results found can greatly shorten the time necess ary in the future when revisiting

the work either for a new phas e of development or to troubleshoot performance problems that have arisen.

What next?

Next up is designingthe test that weve just defined read on!

Other articles in this seriesThis article is part of a series on the subject of Performance and OBIEE:

1. Introduction2. Test Define3. Test Design

4. Test Build5 Execute
http://pdfcrowd.com/http://pdfcrowd.com/redirect/?url=http%3a%2f%2fwww.rittmanmead.com%2f2013%2f03%2fperformance-and-obiee-test-define%2f&id=ma-141108035341-df455196http://pdfcrowd.com/customize/http://pdfcrowd.com/html-to-pdf-api/?ref=pdfhttp://www.rittmanmead.com/2013/03/performance-and-obiee-test-designhttp://www.rittmanmead.com/2013/03/performance-and-obiee-test-designhttp://www.rittmanmead.com/2013/03/performance-and-obiee-introductionhttp://www.rittmanmead.com/2013/03/performance-and-obiee-test-definehttp://www.rittmanmead.com/2013/03/performance-and-obiee-test-designhttp://www.rittmanmead.com/2013/03/performance-and-obiee-test-buildhttp://www.rittmanmead.com/2013/03/performance-and-obiee-execute-and-measure


25/26


Tweet 0 2Like

5. Execute6. Analyse7. Optimise8. Summary and FAQ

Comments?Id love yourfeedback. Do you agree wi th this method, or is it a waste of time? What have I overlooked or overemphas ised?

Am I flogging a dead horse?

Because there are several articles in this s eries, and Id like to keep the discus sion in one place, Ive enabled comments on

the summary and FAQpost here, and disabled comments on the others.

Related Posts:

Performance and OBIEE part I Introduction

Performance and OBIEE part VII Optimising OBIEE performance (Tuning)

Performance and OBIEE part VI Analysing results

Posted in Oracle BI Suite EE, Performance| Comments Off

Comments are closed.

Call us now to talk about your BI project:

+44 (0) 1273 911 268 (UK) or(888) 631-1410 (USA)

or+61 3 9596 7186 (Australia & New Zealand) or

+91 997 256 7970 (India)or+32 280 882 11 (Belgium)

Share 2

Rittman Mead Consulting ltd
http://pdfcrowd.com/redirect/?url=http%3a%2f%2fwww.rittmanmead.com%2f2013%2f03%2fperformance-and-obiee-test-define%2f&id=ma-141108035341-df455196http://pdfcrowd.com/redirect/?url=http%3a%2f%2fwww.rittmanmead.com%2f2013%2f03%2fperformance-and-obiee-test-define%2f&id=ma-141108035341-df455196http://pdfcrowd.com/redirect/?url=http%3a%2f%2fwww.rittmanmead.com%2f2013%2f03%2fperformance-and-obiee-test-define%2f&id=ma-141108035341-df455196http://pdfcrowd.com/redirect/?url=http%3a%2f%2fwww.rittmanmead.com%2f2013%2f03%2fperformance-and-obiee-test-define%2f&id=ma-141108035341-df455196http://pdfcrowd.com/http://pdfcrowd.com/redirect/?url=http%3a%2f%2fwww.rittmanmead.com%2f2013%2f03%2fperformance-and-obiee-test-define%2f&id=ma-141108035341-df455196http://pdfcrowd.com/customize/http://pdfcrowd.com/html-to-pdf-api/?ref=pdfhttps://twitter.com/intent/tweet?original_referer=http%3A%2F%2Fwww.rittmanmead.com%2F2013%2F03%2Fperformance-and-obiee-test-define%2F&text=Performance%20and%20OBIEE%20%E2%80%93%20part%20II%20%E2%80%93%20Test%20%E2%80%93%20Define&tw_p=tweetbutton&url=http%3A%2F%2Fwww.rittmanmead.com%2F2013%2F03%2Fperformance-and-obiee-test-define%2F&via=rittmanmeadhttp://twitter.com/search?q=http%3A%2F%2Fwww.rittmanmead.com%2F2013%2F03%2Fperformance-and-obiee-test-define%2Fhttp://www.rittmanmead.com/2013/03/performance-and-obiee-execute-and-measurehttp://www.rittmanmead.com/2013/03/performance-and-obiee-analysing-resultshttp://www.rittmanmead.com/2013/03/optimising-obiee-performance-tuninghttp://www.rittmanmead.com/2013/03/performance-and-obiee-summary-and-faqhttp://www.rittmanmead.com/2013/03/performance-and-obiee-summary-and-faqhttp://www.rittmanmead.com/2013/03/performance-and-obiee-introduction/http://www.rittmanmead.com/2013/03/optimising-obiee-performance-tuning/http://www.rittmanmead.com/2013/03/performance-and-obiee-analysing-results/http://www.rittmanmead.com/category/oracle-bi-suite-ee/http://www.rittmanmead.com/category/performance/http://www.rittmanmead.com/


26/26


Website Design & Build: tymedia.co.uk

Home

About Us> About us

> About our team

> Contact us

> Our clients

Services> Consulting

> Training

> Support

ConsultingServices> Projects

> Expert Services

> OBIEE 11g

> Sustainability

> On Discoverer?

> Oracle DW

Training> OBIEE

Bootcamp

> OBIEE End-User

> Exalytics

> ODI 11g

Bootcamp

> Oracle BI Apps

Resources> Articles

> Blog

> OBIEE 11g

Blog Authors> Mark Rittma n

> Venkat J

> Peter Scott

> Borkur S

> Mike Vic kers

> Robin Moffatt

> Jon Mead

Rittman Mead Consulting ltd.

Registered Office : Suite B,

First Floor Moore House,

13 Black Lion Street,

Brighton, East Sussex,

BN1 1ND, United Kingdom

Company No. : 6032852

VAT No. : 900 3839 48

Rittman Mead America, Inc.

Registered Office : 4550 North Point Parkway

390 Alpharetta, Georgia 30022, USA

Rittman Mead Oceania Pty Ltd.

Registered Office : 12 Moore Street,

Brighton East,

Victoria, 3187, Australia

Austral ian Company No. : 1 49 458 9 35

Rittman Mead Consulting Pvt Ltd.

Registered Office : Unit 105-106

Regent Prime

Whitefield Main Road

Whitefield

Bangalore

560066

Rittman Mead Belgium

Registered Office : Chausse de Louvain 426

1380 Lasne

Belgium

2010-2011 Rittm an Mead Cons ulting. | Privacy Policy | E: info@rittm anmead.com
http://pdfcrowd.com/http://pdfcrowd.com/redirect/?url=http%3a%2f%2fwww.rittmanmead.com%2f2013%2f03%2fperformance-and-obiee-test-define%2f&id=ma-141108035341-df455196http://pdfcrowd.com/customize/http://pdfcrowd.com/html-to-pdf-api/?ref=pdfhttp://www.tymedia.co.uk/?__hstc=150481449.35d69ee28ad764e69b542078dfc1c354.1415436819737.1415436819737.1415436819737.1&__hssc=150481449.1.1415436819737&__hsfp=1770896186http://www.rittmanmead.com/http://www.rittmanmead.com/about/http://www.rittmanmead.com/about/our-team/http://www.rittmanmead.com/about/contact-us/http://www.rittmanmead.com/clients/http://www.rittmanmead.com/consultinghttp://www.rittmanmead.com/traininghttp://www.rittmanmead.com/supporthttp://www.rittmanmead.com/consulting/expert-services/http://www.rittmanmead.com/consulting/expert-services/http://www.rittmanmead.com/consulting/obiee-11g-services/http://www.rittmanmead.com/consulting/prepare-build-and-sustain/http://www.rittmanmead.com/consulting/migrating-from-discoverer-to-oracle-bi-suite-enterprise-edition/http://www.rittmanmead.com/consulting/data-warehousing/http://www.rittmanmead.com/training/trn202-oracle-bi-ee-11g-bootcamp/http://www.rittmanmead.com/training/trn-205-oracle-bi-ee-11g-create-reports-dashboards-alerts-and-scorecards/http://www.rittmanmead.com/training/trn-701-exalytics-for-systems-administrators/http://www.rittmanmead.com/training/trn-403-odi-11g-bootcamp/http://www.rittmanmead.com/training/trn-40-oracle-bi-apps-bootcamp/http://www.rittmanmead.com/articleshttp://www.rittmanmead.com/bloghttp://www.rittmanmead.com/obiee11ghttp://www.rittmanmead.com/author/mark-rittman/http://www.rittmanmead.com/author/venkatakrishnan-j/http://www.rittmanmead.com/author/peter-scott/http://www.rittmanmead.com/author/borkur-steingrimsson/http://www.rittmanmead.com/author/mike-vickers/http://www.rittmanmead.com/author/robin-moffatt/http://www.rittmanmead.com/author/jon-mead/http://www.rittmanmead.com/privacy-policy/mailto:[email protected]

Documents

Www Rittmanmead Com 2013 03 Performance and Obiee Test Defin