Upload
vanhuong
View
214
Download
0
Embed Size (px)
<Insert Picture Here>
Implementing Efficient Essbase ASO ApplicationBuland Chowdhury & Steve LiebermenschSenior Technical Director
Agenda
• Basic Design• Dimension settings and usage• Partitioning • Formulas• MDX Optimization• Tuning Calculation• Aggregation• Materialization • Data Load• Others
The following is intended to outline our general guideline. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions.
Basic Design
• An Essbase model needs to have a focus area or needs to be subject oriented. For example if you are creating a customer analysis application stay focused on elements that are relevant to customer analysis. Don’t introduce additional elements in to the model just because ASO allows it.
• Lack of focus or subject orientation can lead to inter dimensional irrelevance. Always avoid it. Inter dimensional irrelevance occurs when many members of a dimension are irrelevant across other dimensions
Basic Design
• Don’t design an ASO model as if it is a relational database. ASO’s capability has increased significantly but it is still an OLAP application not a Relational Database.
Basic Design
• Theoretically there is no limit on the number of dimensions that an ASO model can have. That should not be an excuse to allow dimension into the model. Don’t introduce a dimension just in case some one needs it. Focus should be relevant to subject area and frequency of use.
• Utilize drill through functionality where elements are not directly tied to subject area but relevant for analysis or less frequently used dimensions.
Dimension settings and usage Compression
• ASO applications can be very large. For this reason, compression becomes very important. Following are the things that can be done to improve compression• Make sure one dimension is selected as a compression dimension• Select a dimension with a higher probability of data intersection with
a lower number of level 0 members• If possible change the member ordering so that a member with
higher data intersection comes first• Round number to 2 decimal points. If data value have more than 2
decimal point values are save as is (no compression)• Ideal compression is achieved if the stored leaf level member count
is evenly divisible by 16
Dimension Settings and Usage Accounts
• Dynamic dimension• Allows for non additive unary operators• Required for time balance functionality• Expense flags are accomplished through UDAs and
member formulae• In the case of large GL type accounts dimensions
should consider storing the dimension
Dimension Settings and Usage Accounts
Must be a dynamic dimension due to “-” unary operator on COGS and Total
Expenses
Dimension Settings and Usage Accounts
Simplest hierarchy and allows for maximum stored aggregates. Leverages load rule setting
of sign flip on UDA. Requires the user to accept different signage of values.
Dimension Settings and Usage Accounts
Preserves original signage and allows for some aggregates to be stored
Dimension Settings and Usage Time
• Can make a good candidate for compression dimension provided no time balance requirements
• Must be stored for time balance feature• Use multi-hierarchy dimension if formulae are
necessary• To-date formulae
• Prior to 9.3.1 best if performed in Time dimension• 9.3.1 and forward, most functional to use analytic dimension
Dimension Settings and Usage Time
• Use analytic dimension if you need perform time calculations like period to date sums, period over period comparisons
• Use AGGREGATE function if your period to date calculations need to take Time Balance tags into account
ORACLE CONFIDENTIAL
Dimension Settings and Usage Time
You can write the AGGREGATE function as follows. It some time improves the retrieval performance
YTD: AGGREGATE(PeriodsToDate([Time].Generations(2), [Time].CurrentMember), [Base])
QTD: AGGREGATE(PeriodsToDate([Time].Generations(3), [Time].CurrentMember), [Base])
Where [Base] is the stored member in Analytic dimension called TimeView dimension, where you load data at
Partitioning
• Execution across multiple partitions is performed serially
• Performance will be impacted by volume of data being transferred across the partition
• Query tracking can and should be done against the source model via the partition
• BSO as a source should be used only with forethought
• Formulae on the target model should only be written for members where the calculation crosses partitions
Partitioning
• Target tuning• Perform all calculations possible on the source(s)
Formulas
• Simplify complex formulas using intermediate formulas
• Use shortest possible MDX idioms/expressions• Progressively eliminate cells needing to be processed
for a formula• Avoid CURRENTMEMBER references wherever
possible• Use specific MDX functions/expression patterns that
are internally optimized• Use built-in calculations (Time Balance, Ranking
calculations)
ORACLE CONFIDENTIAL
MDX Optimization Meta-data vs data
Performance difference between the following statements:
IIf(Scenario.CurrentMember = Actual,….)
Or
IIf(Is(Scenario.CurrentMember,Actual),….)
Optimized MDX Functions
• Use the following functions instead of using PARENT, CHILD, LEAD and LAG to navigate a hierarchy
OpeningPeriodClosingPeriodParallelPeriodLastPeriodsPeriodsToDateFirstChildLastChildFirstSiblingLastSiblingMemberRangeRelMemberRange
MDX Optimization
Goal is to calculate a product’s 3 month average Transactions contribution % to [All Merchandise] 3 month average:
MDX Optimization
Round( ( ( ([Time].CurrentMember,[Products].CurrentMember,[Transactions]) +
([Time].CurrentMember.Lag(1),[Products].CurrentMember,[Transactions]) +
([Time].CurrentMember.Lag(2),[Products].CurrentMember,[Transactions]) ) /
(([Time].CurrentMember,Ancestor (Products.CurrentMember, Products.Generations(2)),[Transactions]) +
([Time].CurrentMember.Lag(1),Ancestor (Products.CurrentMember, Products.Generations(2)),[Transactions]) +
([Time].CurrentMember.Lag(2),Ancestor (Products.CurrentMember, Products.Generations(2)),[Transactions]) ) )
,4)
MDX Optimization
Round( ( ( ([Time].CurrentMember,[Products].CurrentMember,[Transactions]) +
([Time].CurrentMember.Lag(1),[Products].CurrentMember,[Transactions]) +
([Time].CurrentMember.Lag(2),[Products].CurrentMember,[Transactions]) ) /
(([Time].CurrentMember,Ancestor (Products.CurrentMember, Products.Generations(2)),[Transactions]) +
([Time].CurrentMember.Lag(1),Ancestor (Products.CurrentMember, Products.Generations(2)),[Transactions]) +
([Time].CurrentMember.Lag(2),Ancestor (Products.CurrentMember, Products.Generations(2)),[Transactions]) ) )
,4)
Remove unnecessary functions. Formatting should be left to the
reporting tool.
MDX Optimization
( ( ([Time].CurrentMember,[Products].CurrentMember,[Transactions]) + ([Time].CurrentMember.Lag(1),[Products].CurrentMember,[Transactions])
+ ([Time].CurrentMember.Lag(2),[Products].CurrentMember,[Transactions])
) / (([Time].CurrentMember,Ancestor (Products.CurrentMember,
Products.Generations(2)),[Transactions]) + ([Time].CurrentMember.Lag(1),Ancestor (Products.CurrentMember,
Products.Generations(2)),[Transactions]) + ([Time].CurrentMember.Lag(2),Ancestor (Products.CurrentMember,
Products.Generations(2)),[Transactions]) ) )Avoid unnecessary references
to CurrentMember. It is assumed without explicitly
using a function
MDX Optimization
( ( ([Time].CurrentMember,[Transactions]) + ([Time].CurrentMember.Lag(1),[Transactions]) + ([Time].CurrentMember.Lag(2), [Transactions]) ) / (([Time].CurrentMember,Ancestor (Products.CurrentMember,
Products.Generations(2)),[Transactions]) + ([Time].CurrentMember.Lag(1),Ancestor
(Products.CurrentMember, Products.Generations(2)),[Transactions]) +
([Time].CurrentMember.Lag(2),Ancestor (Products.CurrentMember, Products.Generations(2)),[Transactions]) ) )
Simplify formulae, no need to use 5 functions where 1 will
do. 3 CurrentMember functions and 2 Lag
functions
MDX Optimization
Sum(LastPeriods(3),Transactions)/ Sum(LastPeriods(3),(Ancestor (Products.CurrentMember,
Products.Generations(2)),Transactions))
Don’t use a function where direct referencing can be
performed. In this model Gen 2 of the Products dimension
will always be the same member
MDX Optimization
Sum(LastPeriods(3),Transactions)/ Sum(LastPeriods(3),([All Merchandise],Transactions))
MDX Optimization
Case When NotIsEmpty(Sum(LastPeriods(3),Transactions)) Then
Sum(LastPeriods(3),Transactions)/ Sum(LastPeriods(3),
([All Merchandise],Transactions))End
Only perform calculation when data to support the math exists
MDX Optimization
Additionally you should break up the complex formula using an additional member:
Create a new member [Last 3 Month Transactions]Assign the formula:
Sum(LastPeriods(3),Transactions)
Then the new % Contribution formula becomes:NonEmptyMember [Last 3 Month Transactions]
[Last 3 Month Transactions] / ([Last 3 Month Transactions],[All Merchandise])
NonEmpty directive, 11.1.1 feature
Tuning Calculation Using NON EMPTY Behavior
• ASO supports association of NON EMPTY behavior properties with formulas in 11.1.x
• Non empty behavior takes advantage of sparse nature of input data to reduce amount of dynamic calculations
• Non empty behavior can be defined by adding the following directives before a formula• NONEMPTYMEMBER <memberlist>• NONEMPTYTUPLE <tuple>
ORACLE CONFIDENTIAL
Tuning Calculation Using NON EMPTY Behavior
• The measure [Units] is a loaded value• The measure [Avg. Units/Transaction] is calculated
using formula[Avg. Units/Transaction] = [Units]/[Transactions]
• The value of [Avg. Units/Transaction] is empty if value of [Units] is empty
ORACLE CONFIDENTIAL
Tuning Calculation Using NON EMPTY Behavior
• The calculation of [Avg. Units/Transaction] can be skipped if value of [Units] is empty
• We can add non empty behavior for [Avg. Units/Transaction] by rewriting the formula as
[Avg. Units/Transaction] = NONEMPTYMEMBER [Units] [Units]/[Transactions]
• The saving obtained from defining non empty behaviors can be very significant.
• Savings increase with the complexity of the formula and sparseness of the data.
ORACLE CONFIDENTIAL
Examples of NON EMPTY behavior
• [AvgProdTranscation] is empty if [Transactions] for current Product is empty[Measures].[AvgProdTransactions] = NONEMPTYMEMBER [Transactions]
AVG(Product.CurrentMember.Children, [Transactions])
• [YTDSales] is empty if Sales for entire [Year] is empty[Measures].[YTDSales] =NONEMPTYTUPLE (Year,Sales)Sum(PeriodsToDate(Time.Years, Time.CurrentMember), Sales)
ORACLE CONFIDENTIAL
Guidelines for Assigning NON EMPTY Behavior
• If you can identify a member such that a formula is empty if the value of that member is empty use NONEMPTYMEMBER directive with that member
• The members in NONEMPTYMEMBER should be from the same dimension as the formula member to which you apply this directive.
• Use multiple members in NONEMPTYMEMBER if one member is not sufficient
• Use NONEMPTYTUPLE if you need members from multiple dimensions to indicate non empty behavior
ORACLE CONFIDENTIAL
Use LEAVES and NONEMPTYSUBSET functions
• For large dimensions (over 100,000 members) use LEAVES function instead of using DESCENDANTS to get leaf level members.
• LEAVES returns only leaf level members that have non-empty data• Leaves(Products)
• Use NONEMPTYSUBSET to reduce size of your query• NonEmptySubset(Crossjoin(Leaves(Customers),
Leaves(Products), Sales)
ORACLE CONFIDENTIAL
Dynamic Calculations Log Messages
• Application log has some informative messages related to dynamic calculations that can be helpful in tuning• The following message indicates an error in a formula. Fix the
error message or drop these members from outline if they are not required
• Error encountered when loading member [<MemberName>]'s calc string [<FormulaExpression>], ignored
• The following message indicates invalid solve order assignment. Assign correct solve orders for members indicated in the message
• Invalid formula for member [<A>] (reason: depends on a member [<B>] with higher solveorder ) will be ignored during execution
ORACLE CONFIDENTIAL
Dynamic Calculations Log Messages
• The following message indicate that a formula is too complex for Essbase to optimize. Rewrite if possible.
• Formula for member [% of Total] will be executed in [CELL] mode
• The following message indicates that a formula is complex and Essbase couldn’t detect a non empty behavior for it.• The formula for member [MemberName] is Complex. If possible, add
a non-empty directive to optimize for sparse data
ORACLE CONFIDENTIAL
ASO Aggregation
• Use parallel calculation. Set CALCPARALLEL setting to number of CPU available.
• Increase ASO cache size higher than the what you have during data load to accommodate parallel calculation. You can use Maxl to accomplish this.
• Depending on situation a full database aggregation may not be optimal.
• Optimizing aggregations may require an iterative, fine-tuning process.
• Query tracking probably the most efficient way to decide what to aggregate, provided queries reflect the users data need properly; However, if you are knowledgeable about user and their query need you can use User-Defined View Selection where you will be able to select top level, bottom level in secondary hierarchy and many others options. Use hierarchy aggregation settings in the outline editor
Materialization Query hints
• On Dimension/Hierarchy
Materialization
• By default Aggregate Storage Option is parallel threaded with two threads, can use up to 8
• When loading incremental data, aggregate views are also updated
• Materialization scripts can be saved and reused to eliminate regeneration of query tracked results
• The database can be available during materialization without data consistency issues
Materialization Query hints
• Dimensional intersections
ASO Query
• Aggregation and query are related items. Most of the items that we have discussed in previous slides will improve query performance
• A hint can be used to optimize heavily queried level combinations. Avoid creating too many hints, you will lose effectiveness.
Data Load Behavior
• Within a buffer duplicate records are additive irrelevant of load rule settings
• When committing the buffer it adheres to data load rule setting related to overwrite, add to, or subtract from
• Single threaded, can use DLTHREADS cfg settings but thorough testing should be performed
• Materialized views are updated as part of the data load process during an incremental update
Data Load Slice loading
• Is the primary feature enabling Excel based data submission and trickle feeding functionality
• Creates “subcubes” alongside the primary slice of the database
• Dynamic aggregations are performed across the necessary slices to provide query results
• Different materialized views might exist within a slice as compared to the primary slice of the database
Data Load Optimization
• Compression dimension should be across the load file• Consider removing aggregate views prior to an
incremental update of data• Data should be loaded as additive values• Multiple buffers can be used to parallel load the
database. Requires simultaneous Maxl processes to be executed
• Ignore Zeros and Missing values whenever possible, maxl buffer setting parameter
Data Load Optimization
• Don’t load 0 (zero) values if possible• Use ASO cache setting. The number is based on
the number of cells that would be loaded.• While setting ASO cache try not to exceed 25% of
the available physical memory to leave more space for the OS file cache
• For a Database with 1 billion cells or more, the cache size may be set as high as 512 MB or 1 GB if the available memory permits it
• Avoid (if possible) Text substitutions, field splits/joins and white space truncation in rule file
ASO Others
• If possible allocate space for “default” and “temp” table spaces on different physical disk devices where applicable