- dbmanagement.infodbmanagement.info/Books/MIX/Essbase_Oracle_Essbase_ASO_Efficie… · ... • An Essbase model needs to have

<Insert Picture Here>

Implementing Efficient Essbase ASO ApplicationBuland Chowdhury & Steve LiebermenschSenior Technical Director

Agenda

• Basic Design• Dimension settings and usage• Partitioning • Formulas• MDX Optimization• Tuning Calculation• Aggregation• Materialization • Data Load• Others

The following is intended to outline our general guideline. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions.

Basic Design

• An Essbase model needs to have a focus area or needs to be subject oriented. For example if you are creating a customer analysis application stay focused on elements that are relevant to customer analysis. Don’t introduce additional elements in to the model just because ASO allows it.

• Lack of focus or subject orientation can lead to inter dimensional irrelevance. Always avoid it. Inter dimensional irrelevance occurs when many members of a dimension are irrelevant across other dimensions

Basic Design

• Don’t design an ASO model as if it is a relational database. ASO’s capability has increased significantly but it is still an OLAP application not a Relational Database.

Basic Design

• Theoretically there is no limit on the number of dimensions that an ASO model can have. That should not be an excuse to allow dimension into the model. Don’t introduce a dimension just in case some one needs it. Focus should be relevant to subject area and frequency of use.

• Utilize drill through functionality where elements are not directly tied to subject area but relevant for analysis or less frequently used dimensions.

Dimension settings and usage Compression

• ASO applications can be very large. For this reason, compression becomes very important. Following are the things that can be done to improve compression• Make sure one dimension is selected as a compression dimension• Select a dimension with a higher probability of data intersection with

a lower number of level 0 members• If possible change the member ordering so that a member with

higher data intersection comes first• Round number to 2 decimal points. If data value have more than 2

decimal point values are save as is (no compression)• Ideal compression is achieved if the stored leaf level member count

is evenly divisible by 16

Dimension Settings and Usage Accounts

• Dynamic dimension• Allows for non additive unary operators• Required for time balance functionality• Expense flags are accomplished through UDAs and

member formulae• In the case of large GL type accounts dimensions

should consider storing the dimension


Must be a dynamic dimension due to “-” unary operator on COGS and Total

Expenses


Simplest hierarchy and allows for maximum stored aggregates. Leverages load rule setting

of sign flip on UDA. Requires the user to accept different signage of values.


Preserves original signage and allows for some aggregates to be stored

Dimension Settings and Usage Time

• Can make a good candidate for compression dimension provided no time balance requirements

• Must be stored for time balance feature• Use multi-hierarchy dimension if formulae are

necessary• To-date formulae

• Prior to 9.3.1 best if performed in Time dimension• 9.3.1 and forward, most functional to use analytic dimension


• Use analytic dimension if you need perform time calculations like period to date sums, period over period comparisons

• Use AGGREGATE function if your period to date calculations need to take Time Balance tags into account

ORACLE CONFIDENTIAL


You can write the AGGREGATE function as follows. It some time improves the retrieval performance

YTD: AGGREGATE(PeriodsToDate([Time].Generations(2), [Time].CurrentMember), [Base])

QTD: AGGREGATE(PeriodsToDate([Time].Generations(3), [Time].CurrentMember), [Base])

Where [Base] is the stored member in Analytic dimension called TimeView dimension, where you load data at

Partitioning

• Execution across multiple partitions is performed serially

• Performance will be impacted by volume of data being transferred across the partition

• Query tracking can and should be done against the source model via the partition

• BSO as a source should be used only with forethought

• Formulae on the target model should only be written for members where the calculation crosses partitions

Partitioning

• Target tuning• Perform all calculations possible on the source(s)

Formulas

• Simplify complex formulas using intermediate formulas

• Use shortest possible MDX idioms/expressions• Progressively eliminate cells needing to be processed

for a formula• Avoid CURRENTMEMBER references wherever

possible• Use specific MDX functions/expression patterns that

are internally optimized• Use built-in calculations (Time Balance, Ranking

calculations)

ORACLE CONFIDENTIAL

MDX Optimization Meta-data vs data

Performance difference between the following statements:

IIf(Scenario.CurrentMember = Actual,….)

Or

IIf(Is(Scenario.CurrentMember,Actual),….)

Optimized MDX Functions

• Use the following functions instead of using PARENT, CHILD, LEAD and LAG to navigate a hierarchy

OpeningPeriodClosingPeriodParallelPeriodLastPeriodsPeriodsToDateFirstChildLastChildFirstSiblingLastSiblingMemberRangeRelMemberRange

MDX Optimization

Goal is to calculate a product’s 3 month average Transactions contribution % to [All Merchandise] 3 month average:

MDX Optimization

Round( ( ( ([Time].CurrentMember,[Products].CurrentMember,[Transactions]) +

([Time].CurrentMember.Lag(1),[Products].CurrentMember,[Transactions]) +

([Time].CurrentMember.Lag(2),[Products].CurrentMember,[Transactions]) ) /

(([Time].CurrentMember,Ancestor (Products.CurrentMember, Products.Generations(2)),[Transactions]) +

([Time].CurrentMember.Lag(1),Ancestor (Products.CurrentMember, Products.Generations(2)),[Transactions]) +

([Time].CurrentMember.Lag(2),Ancestor (Products.CurrentMember, Products.Generations(2)),[Transactions]) ) )

,4)

MDX Optimization

Round( ( ( ([Time].CurrentMember,[Products].CurrentMember,[Transactions]) +

([Time].CurrentMember.Lag(1),[Products].CurrentMember,[Transactions]) +

([Time].CurrentMember.Lag(2),[Products].CurrentMember,[Transactions]) ) /

(([Time].CurrentMember,Ancestor (Products.CurrentMember, Products.Generations(2)),[Transactions]) +

([Time].CurrentMember.Lag(1),Ancestor (Products.CurrentMember, Products.Generations(2)),[Transactions]) +


,4)

Remove unnecessary functions. Formatting should be left to the

reporting tool.

MDX Optimization

( ( ([Time].CurrentMember,[Products].CurrentMember,[Transactions]) + ([Time].CurrentMember.Lag(1),[Products].CurrentMember,[Transactions])

+ ([Time].CurrentMember.Lag(2),[Products].CurrentMember,[Transactions])

) / (([Time].CurrentMember,Ancestor (Products.CurrentMember,

Products.Generations(2)),[Transactions]) + ([Time].CurrentMember.Lag(1),Ancestor (Products.CurrentMember,

Products.Generations(2)),[Transactions]) + ([Time].CurrentMember.Lag(2),Ancestor (Products.CurrentMember,

Products.Generations(2)),[Transactions]) ) )Avoid unnecessary references

to CurrentMember. It is assumed without explicitly

using a function

MDX Optimization

( ( ([Time].CurrentMember,[Transactions]) + ([Time].CurrentMember.Lag(1),[Transactions]) + ([Time].CurrentMember.Lag(2), [Transactions]) ) / (([Time].CurrentMember,Ancestor (Products.CurrentMember,

Products.Generations(2)),[Transactions]) + ([Time].CurrentMember.Lag(1),Ancestor

(Products.CurrentMember, Products.Generations(2)),[Transactions]) +


Simplify formulae, no need to use 5 functions where 1 will

do. 3 CurrentMember functions and 2 Lag

functions

MDX Optimization

Sum(LastPeriods(3),Transactions)/ Sum(LastPeriods(3),(Ancestor (Products.CurrentMember,

Products.Generations(2)),Transactions))

Don’t use a function where direct referencing can be

performed. In this model Gen 2 of the Products dimension

will always be the same member

MDX Optimization

Sum(LastPeriods(3),Transactions)/ Sum(LastPeriods(3),([All Merchandise],Transactions))

MDX Optimization

Case When NotIsEmpty(Sum(LastPeriods(3),Transactions)) Then

Sum(LastPeriods(3),Transactions)/ Sum(LastPeriods(3),

([All Merchandise],Transactions))End

Only perform calculation when data to support the math exists

MDX Optimization

Additionally you should break up the complex formula using an additional member:

Create a new member [Last 3 Month Transactions]Assign the formula:

Sum(LastPeriods(3),Transactions)

Then the new % Contribution formula becomes:NonEmptyMember [Last 3 Month Transactions]

[Last 3 Month Transactions] / ([Last 3 Month Transactions],[All Merchandise])

NonEmpty directive, 11.1.1 feature

Tuning Calculation Using NON EMPTY Behavior

• ASO supports association of NON EMPTY behavior properties with formulas in 11.1.x

• Non empty behavior takes advantage of sparse nature of input data to reduce amount of dynamic calculations

• Non empty behavior can be defined by adding the following directives before a formula• NONEMPTYMEMBER <memberlist>• NONEMPTYTUPLE <tuple>

ORACLE CONFIDENTIAL


• The measure [Units] is a loaded value• The measure [Avg. Units/Transaction] is calculated

using formula[Avg. Units/Transaction] = [Units]/[Transactions]

• The value of [Avg. Units/Transaction] is empty if value of [Units] is empty

ORACLE CONFIDENTIAL


• The calculation of [Avg. Units/Transaction] can be skipped if value of [Units] is empty

• We can add non empty behavior for [Avg. Units/Transaction] by rewriting the formula as

[Avg. Units/Transaction] = NONEMPTYMEMBER [Units] [Units]/[Transactions]

• The saving obtained from defining non empty behaviors can be very significant.

• Savings increase with the complexity of the formula and sparseness of the data.

ORACLE CONFIDENTIAL

Examples of NON EMPTY behavior

• [AvgProdTranscation] is empty if [Transactions] for current Product is empty[Measures].[AvgProdTransactions] = NONEMPTYMEMBER [Transactions]

AVG(Product.CurrentMember.Children, [Transactions])

• [YTDSales] is empty if Sales for entire [Year] is empty[Measures].[YTDSales] =NONEMPTYTUPLE (Year,Sales)Sum(PeriodsToDate(Time.Years, Time.CurrentMember), Sales)

ORACLE CONFIDENTIAL

Guidelines for Assigning NON EMPTY Behavior

• If you can identify a member such that a formula is empty if the value of that member is empty use NONEMPTYMEMBER directive with that member

• The members in NONEMPTYMEMBER should be from the same dimension as the formula member to which you apply this directive.

• Use multiple members in NONEMPTYMEMBER if one member is not sufficient

• Use NONEMPTYTUPLE if you need members from multiple dimensions to indicate non empty behavior

ORACLE CONFIDENTIAL

Use LEAVES and NONEMPTYSUBSET functions

• For large dimensions (over 100,000 members) use LEAVES function instead of using DESCENDANTS to get leaf level members.

• LEAVES returns only leaf level members that have non-empty data• Leaves(Products)

• Use NONEMPTYSUBSET to reduce size of your query• NonEmptySubset(Crossjoin(Leaves(Customers),

Leaves(Products), Sales)

ORACLE CONFIDENTIAL

Dynamic Calculations Log Messages

• Application log has some informative messages related to dynamic calculations that can be helpful in tuning• The following message indicates an error in a formula. Fix the

error message or drop these members from outline if they are not required

• Error encountered when loading member [<MemberName>]'s calc string [<FormulaExpression>], ignored

• The following message indicates invalid solve order assignment. Assign correct solve orders for members indicated in the message

• Invalid formula for member [<A>] (reason: depends on a member [<B>] with higher solveorder ) will be ignored during execution

ORACLE CONFIDENTIAL

Dynamic Calculations Log Messages

• The following message indicate that a formula is too complex for Essbase to optimize. Rewrite if possible.

• Formula for member [% of Total] will be executed in [CELL] mode

• The following message indicates that a formula is complex and Essbase couldn’t detect a non empty behavior for it.• The formula for member [MemberName] is Complex. If possible, add

a non-empty directive to optimize for sparse data

ORACLE CONFIDENTIAL

ASO Aggregation

• Use parallel calculation. Set CALCPARALLEL setting to number of CPU available.

• Increase ASO cache size higher than the what you have during data load to accommodate parallel calculation. You can use Maxl to accomplish this.

• Depending on situation a full database aggregation may not be optimal.

• Optimizing aggregations may require an iterative, fine-tuning process.

• Query tracking probably the most efficient way to decide what to aggregate, provided queries reflect the users data need properly; However, if you are knowledgeable about user and their query need you can use User-Defined View Selection where you will be able to select top level, bottom level in secondary hierarchy and many others options. Use hierarchy aggregation settings in the outline editor

Materialization Query hints

• On Dimension/Hierarchy

Materialization

• By default Aggregate Storage Option is parallel threaded with two threads, can use up to 8

• When loading incremental data, aggregate views are also updated

• Materialization scripts can be saved and reused to eliminate regeneration of query tracked results

• The database can be available during materialization without data consistency issues

Materialization Query hints

• Dimensional intersections

ASO Query

• Aggregation and query are related items. Most of the items that we have discussed in previous slides will improve query performance

• A hint can be used to optimize heavily queried level combinations. Avoid creating too many hints, you will lose effectiveness.

Data Load Behavior

• Within a buffer duplicate records are additive irrelevant of load rule settings

• When committing the buffer it adheres to data load rule setting related to overwrite, add to, or subtract from

• Single threaded, can use DLTHREADS cfg settings but thorough testing should be performed

• Materialized views are updated as part of the data load process during an incremental update

Data Load Slice loading

• Is the primary feature enabling Excel based data submission and trickle feeding functionality

• Creates “subcubes” alongside the primary slice of the database

• Dynamic aggregations are performed across the necessary slices to provide query results

• Different materialized views might exist within a slice as compared to the primary slice of the database

Data Load Optimization

• Compression dimension should be across the load file• Consider removing aggregate views prior to an

incremental update of data• Data should be loaded as additive values• Multiple buffers can be used to parallel load the

database. Requires simultaneous Maxl processes to be executed

• Ignore Zeros and Missing values whenever possible, maxl buffer setting parameter

Data Load Optimization

• Don’t load 0 (zero) values if possible• Use ASO cache setting. The number is based on

the number of cells that would be loaded.• While setting ASO cache try not to exceed 25% of

the available physical memory to leave more space for the OS file cache

• For a Database with 1 billion cells or more, the cache size may be set as high as 512 MB or 1 GB if the available memory permits it

• Avoid (if possible) Text substitutions, field splits/joins and white space truncation in rule file

ASO Others

• If possible allocate space for “default” and “temp” table spaces on different physical disk devices where applicable

Documents

- dbmanagement.infodbmanagement.info/Books/MIX/Essbase_Oracle_Essbase_ASO_Efficie… · ... • An Essbase model needs to have