NW2008 Tips Tricks EDW v10

8/6/2019 NW2008 Tips Tricks EDW v10

1/48

2008 Wellesley Information Services. All rights reserved.

Tips and tricks for using

SAP NetWeaver BusinessIntelligence 7.0 as your

Enterprise Data Warehouse

Dr. Bjarne Berg


2/48

2

In This Session ...

We will explore 6 large-scale EDW implementations, and see howto apply lessons to your strategy and projects.

Examine the difference between an evolutionary SAP NetWeaverBI data warehouse architecture and a top-driven design method.

Compare the results of using a data mart (bottom-up) approach toan EDW (top down) approach, and determine which approach bestfits your requirements.

Explore the ways in which new SAP NetWeaver BI enhancementscan support real-time data warehousing

We will look at common EDW pitfalls and how to leverage the SAPNetWeaver BI architecture in a large landscape using theCorporate Information Factory (CIF)


3/48

33

What Well Cover

Difference between evolutionary DW

architecture and a design Data marts vs. Data warehouses

Real-time Data warehousing

The many mistakes of EDWs

Successes and failures of six large-scale SAPBI-EDWs

SAP NetWeaver BI architecture & CorporateInfo. Factory (CIF)

Wrap-up


4/48

4

Level of Pre-delivered ContentToolsets &

acceleratorsAnalytical applications

for specific industries

Leve

lofE

mbe d d

ed

Analy

tics

Complex (score cards,

budgeting, planning, KPI)

Interactive Mgmt.

reporting (OLAP, MQE)

Evolution of Data Warehousing

Emerging

(1st generation)

Vertical approach

(2nd generation)

Horizontal approach

(2nd generation)

Integrated analytical

(3rd generation)

Source: Mike Schroeck, David Zinn and Bjarne Berg, Integrated Analytics Getting Increased Value fromEnterprise Resource Planning Systems, Data Management Review, May, 2002;

Adapted: Bjarne Berg How to Manage a BW Project, BW & Portals Conference, 2007, Miami

Emerging

(1st generation)


5/48


6/48

SAPs Technical EDW Architecture

Source: SAP AG

VisualComposer BI

Kit

UDI

XMLASAP

QueryJDBC ODBO

Business Explorer Suite (BEx)

BEx Query Designer

Information Broadcasting

Web

Analyzer

Web

ApplicationDesigner

MS ExcelAdd-in

Report

Designer

BEx Web

DBConnect

BAPI ServiceAPI

File XML/A

BEx AnalyzerBI Pattern

BI Consumer Services

BI Platform

Data Warehouse

Enterprise Portal

Analytic Engine Meta Data Mgr

KM


7/487

SAPs EDW Enablers - Queryoptimization

The SAP BI acceleratormakes query response time50-10,000 faster.

You use process chains to

maintain the HPA engineafter each data load

Both HP and IBM have standard solutionsranging from $32K to $200K+ that can be

installed and tested in as little as 2-4 weeks (+SAP licensing costs)

SAP NW 2004s BI

HPA Engine/Adaptive Computing

Data

Acquisition

InfoCubes

Analytic

Engine

SAP

BW

Any

tool

1.In-memory processing

2.Dictionary-based, smartcompression using integers

3.High parallel data access /horizontal partitioning

4.Column-based data storage

& access/vertical tabledecom osition
http://www.ibm.com/us/


8/488

SAPs EDW Enablers - Remodeling ToolBox

In older BW versions, ifyou forgot to include afield in your infocube,the rework was quitesubstantial and ofteninvolved reloading theinfocube as well.

NW 2004s goes a long way to address the complaintsthat BW is a hard to maintain environment with

Source: SAP AG, RichardBrown, Aug. 2006

In NW2004s you get a new tool to add characteristics

and key figures to your model.


9/489

SAPs EDW Enablers - Central EDW Adm.& Tool reductions

SAP NetWeaver has solutions for acomplete EDW architecture,

including an Administrator Cockpitfor managing the system

In a custom data

warehouse environmentyou need many tools:

- Data loads and transformations

- Scheduling of jobs

- Database management- Data modeling

- Managed query environments

- On-line Analytical Processing tools(OLAP)

- Statistical analysis tools- Data visualization tools

- Formatted reporting tools

- Web presentation tool

- Security administration tool

- EDW administration tool(s)- Others ?

In a SAP data warehouseenvironment you need onetool:

SAP NetWeaver


10/4810

SAPs EDW Enablers - Global Tool Reach

The SAPMessage: BO

and SAPprovides

Alignment,Extension &

Augmentationof two leading,complementary

BI & EIMsolutions

After the SAPs Acquisition of Business Objects, many

have questioned the long-term vision of SAP as theEDW. In Response, SAP published their tool integrationvision in February 2008:

Source: SAP February 2008

SAP EDW E bl L


11/4811

SAPs EDW Enablers - Long-termcommunicated vision

Notice thatSAP Web

ApplicationDesigner isReplaced byXcelsius+ in2009 and a

new tool

calledPioneer willbe launched

that yearalso.

SAP has a long-term commitment to EDW and haspublished their 3-year tool plan so that customers can

plan ahead.

Source: SAP February 2008


12/481212

What Well Cover







Wrap-up


13/48


14/48

14

ODS Vs. Data Warehouse Vs. Data Marts

To Understand the differences between DSO, Data Warehousesand Data Marts we can examine them in terms of usage, modeling

and purpose:

Specific applicationor workgroup focus

Narrow scope

Customized orstand aloneanalysis

Interactive query

Highly summarized

Single subject anddepartmentoriented

Uses dimensional

data modeling

Data Store Objects (DSO) Data Warehouse Data Mart

Acts as source topopulate DW and

marts

Often used foroperationalreporting

Detailed, atomic

data Huge data volumes

Integrated, cleandata

Cross-functional and

cross-departmental Supports data

Provides mgmtreporting

Summarized data

Tuned to optimizequery performance

Multipledepartments or

processes

May act as stagingarea for data marts

Uses dimensionaldata modeling

D t W h V D t M t


15/48

15

Data Warehouse Vs. Data Marts -Implementation Sequence

There are several alternatives for an iterative approachto implementing the various storage structures, based

upon organizational needs.

The various structures can be enterprise ordepartmentally focused.

They can be built first, middle, or last. They can be

stand-alone or combined. The important point is to havea concept of the long term vision of the data warehouseproject and how each type of structure is to be used.

A) ODS first: Start by building an enterprise data warehouse

from a subject area perspective and then gradually movesubsets of data to data marts. This approach may take alonger time to implement.

B) Data mart first: Start by building data marts to get data outto users quickly. This approach may encounter difficulties inintegrating data from an enterprise perspective.

C) Data marts first within the framework or vision of an ODS:

Ad t f b ildi th d t t


16/48

16

Advantages of building the data martsfirst

There is a significant trend in the industry today towardbuilding data marts first, then consolidating backwards

to create the data warehouse and operational datastore. There are several advantages to this approach:

A) Allows faster implementation

The average data mart may take 2-3 months to implement; the

average EDW evolves over many iteration and may take years tomature. Several marts can be started in parallel.

B) Reduces political liability through alignment with aspecific business need.

The mart can deliver value to the organization in a much shorterperiod of time and can focus on a specific business function orproblem. The business sponsors will see faster results and canaffirm their decisions with benefit analysis and feedback. This isimportant to maintaining interest and adequate funding levelsfor the program. This is in contrast to the time and complexity of

Ad t f b ildi th d t t


17/48

17

Advantages of building the data martsfirst (continued)

C) Limits risk while learning how to implement data

warehouse.

Building very large databases of several Terabytes is inherentlycomplex. Backup and recovery systems may require specializedhardware and software. Complex tuning may be necessary to

achieve satisfactory query performance levels.

Identifying and defining data from many different sourcescreates opportunities for users and sponsoring departments todisagree. The ultimate business goals may be overshadowed by

the technical and political difficulties of building the largewarehouse. Starting small with a data mart, experimenting, andusing the implementation as a learning experience, will reducethe risk and may actually result in a higher quality deliverable.

D) Costs less than an EDW.Initially, the economics of smaller scale hardware, software, and

M j Ri k f b ildi th D t M t


18/48

18

Major Risks of building the Data Martsfirst

Data marts do not replace data warehouses.

The data mart is not the next step in data warehouseevolution. It must be planned and implemented as partof the overall architectural vision.

To be effective, you must maintain centralized control of

data distribution to the mart in order to support theenterprises overarching warehouse goals of dataquality, consolidation, and sharing.

Data marts also increase the complexity of the data

warehouse environment with multiple extract,transform, and transfer routines.

There are some great risks of succumbing to politicalpressures.

Business units that demand a quick hit and a

stovepipe implementation of data marts may only

serve to undermine the best laid plans for anintegrated and durable data warehousing program.


19/48

19

Risks of building the data marts first

If the IT department agrees to a bottom-up EDW, a

strictly application specific approach, they may endup with multiple data marts that can not beintegrated into a larger EDW/ODS view and whichcan not support analysis across different marts.

The bottom line is plan and build a reusable dataand technical foundation (technology standards,data modeling principles, and integrated

databases).The Gartner Group estimates that resources

required to manage a disjointed data martenvironment are three times greater than an

integrated data warehouse architecture.


20/48

20

SAPs Vision of Data Marts

If you insist on building data marts, youcan

also use SAPs newly acquired RapidMarts tool from Business Objects.

Built with Data Integrator, SAP RapidMarts are ready-made data marts for

SAP. It has pre-built data flows, businesslogic, and schema that understand theSAP meta-data.

SAP Rapids Marts also include content

that is immediately consumable bybusiness users and can be deployedindependent from an EDWimplementation.

It supports data profiling and cleansing

and can be the first step toward aSource: SAP, Feb 2008

SAP has now inherited atool for Data Marts thatis independent from theSAP NetWeaver Platform


21/48

2121

What Well Cover







Wrap-up

Real time SAP Enterprise Data


22/48

22

Real-time SAP Enterprise Datawarehousing gets better

NW 2004s has more features for updates thatdoes not follow the typical asynchronomous(batch) updates. This include:

1. We can use XML to fill the PSA directly2. Daemon-based update from delta queue

(BW API)3. Daemon-based update of the ODS andminimal loggingNote: XML documents

creates many tags thatwill slow down large

dataloads due to the sizeof each XML record

(relatively large)

However, it works great

for smaller streams ofdata.

Limitations of Real time SAP Enterprise


23/48

23

Limitations of Real-time SAP EnterpriseData warehousing

There are some limitations depending on the version ofSAP BI/BW you use. For versions 3.5 and higher, there

are few limitations and they include:

You can only use real-time to load ODSs or PSA

A normal delta update and a real-time update

cannot happen at the same time for the sameDataSource and/or ODS

For data targets that subsequently store the real-time-supported ODS objects, real time data transfercannot be used

InfoPackages that use real-time updates cannot beassociated with InfoPackage Groups or ProcessChains

Consider Using SAP Exchange Infrastructure (SAP-XI) to generatethe XML documents from non-SAP Systems. This can help build acorporate data hub center that can reduce the number of custom

interfaces in the organizationTip


24/48

2424

What Well Cover







Wrap-up

Common EDW Mistakes Not Using


25/48

25

Common EDW Mistakes Not UsingStandard SAP Solutions

In the 1950s, you could buy a standard Sears house for $2,065 andpay $935 more to have it implemented on your own land

The customers whoselected to buy thestandard house were eitherextremely happy or

totally disappointed.

When Sears examined why,they found a strongcorrelation between level of

modifications to the homeand unhappinessYou buy SAP NetWeaver for its pre-built content and connections to other

SAP applications.

The more you add to the standard

solutions, the harder it will become to

Leveraging SAP Standard Content in The


26/48

26

Leveraging SAP Standard Content in TheEDW

As a guiding principle,map requirements tostandard content beforecustomizing

However, youll probablyalso have external datasources that requirecustom ODSs andInfoCubes

Customizing lower levelobjects will cause higherlevel standard objects tonot work, unless you arewilling to customize thesealso.

BW Content available (BI7.0)

BW Content available (BI7.0)

Cockpits ??? Workbooks 2,211

Queries 4,325

Roles 934

MultiProviders 402

InfoCube 783

DSO objects 687

InfoObjects 14,368

36%

33%

31%

Mostly standard storage objects

Some customization

Highly customized storage objects

An example from a largemanufacturing company

How to Leverage Standard BI Content in


27/48

27

Billing

Number of billing documents

Number biling line items

Billed item quantity

Net weight

Subtotal 1

Subtotal 2

Subtotal 3

Subtotal 4

Subtotal 5

Subtotal 6

Subtotal ANet value

Cost

Tax amount

Volume

Customer

Sold-to

Ship-to

Bill-to

Payer

Customer classCustomer group

~ Customer country~ Customer region~ Customer postal code~ Customer industry code 1End user

Material

Material number

Material entered

Material group

Item category

Product hierarchy

EAN/UPC

Time

Calendar year

Calendar month

Calendar week

Calendar day

Unit

Currency Key

Unit of Measure

Base unit of measure

Sales unit of measure

Volume unit of measure

Weight unit of measure

Billing information

Billing document

Billing item

Billing type

Billing category

Billing date

Creation date

Cancel indicator

Output medium~ Batch billing indicatorDebit/credit reason codeBiling category

Reference document

Payment termsCancelled billing documentDivison for the order header

Pricing procedure

Organization

Company code

Division

Distribution channel

Sales organization

Sales group

Logistics

Plant

Shipping/receiving point

Document details

Sales order document typeSales deal

Sales docuement

Accounting

Cost center

Profit centerControlling area

Account assignment group

Personnel

Sales rep number

LEGEND

Delivered in standard extractorsDelivered in LO extractor

Not in delivered Content -but in R-3

Standardcontent

How to Leverage Standard BI Content inthe EDW

Storage

Requirements

StorageObjectsMap functional requirements to

the standard content before youmake enhancements

+

1. Create a model based on pre-delivered SAP BW content

2. Map your data requirements to the delivered content, and identify gaps

3. Identify where the data gaps are going to be sourced from

Common EDW Mistakes No Tailored


28/48

28

Common EDW Mistakes No TailoredApproach

Each organization has different corporate cultures andconsiderations.

The Top-down approach is preferred in centralized organizations, and

CONTINUE

TOP-DOWN APPROACH

Build a global datawarehouse for thecompany, and proceed

sourcing data from oldlegacy systems drivenfrom a top-downapproach.

C

HANGE

BOTTOM-UP APPROACH

Focus on a bottom-up approachwhere the BW project willprioritize supporting and

delivering local BW solutions,thereby setting the actualestablishment of the globalData Warehouse as secondary,BUT not forgotten.

Common EDW Mistakes loose data


29/48

29

Common EDW Mistakes loose datastandards

Some Many organizations place little value on enforcing data standards.

This include InfoObject, DSO and InfoCube naming standards. It also include naming conventions for queries and InfoAreas.

As a result, these organizations often have a mess where it is hard to understand what is available without researching every field and data store.

It may also lead to problems integrating data with different data types and data lengths due to lack of enforcement

Develop your data standard and have anarchitect enforce them throughout the

lifetime of the EDW.Vchar2(15)

Char(18)JimsQuery

AA Z0986 QueryX0C_K01 0SD_C03

0FIAR_O05 SUDHIRC99

Common EDW Mistakes Lack of


30/48

30

Common EDW Mistakes Lack ofenvironment management

Some organization have a hard-time to say No to the business community.

As a result, their architecture often looks like mix-and-match of systems that wasacquired to put out urgent needs.

In these organizations, multiple portals are common and overlapping reportingsystems is the rule, not the exception.

EDWs are like marriages between ITand Business. You have to work at itconstantly, give it attention, and be

faithful to the solution.

Common EDW Mistakes lack of


31/48

31

Common EDW Mistakes lack oftransport controls

Most companies have strong change management of their R/3 systems. However, it is common that the same organizations have very loose approval processes for their BI systems.

BI is becoming a mission critical system for most organizations and the same processes placed on the R/3 system should be applied to a production BI system.

Dont allow quick-fixes and untested service packs and notes to be applied to the production box without adequate testing. BWQ is not for window dressing!!

If you want a stable BI

system, you have to enforcetestin and controls

Common EDW Mistakes Poor


32/48

32

Common EDW Mistakes PoorPerformance

When you build an enterprise data warehouse, you shouldplan for at least 10-15% of your project time for

performance testing and tuning.

Click-stream analysis have shown the 50% of your casualaudience will hit the refresh button or navigate away fromyour web site if the reports take more than 7 seconds.

If your query takes more then 20 seconds to run, you havemajor problems.

Get substantial amount of memory for caching and makesure your have a fast network and hardware resources.

#1 complaint of EDW is lack of

performance. Consider BIA as partof your infrastructure


33/48

3333

What Well Cover




The many mistakes of EDWs Successes and failures of six large-scale SAP

BI-EDWs


Wrap-up

SAP EDW in 6 large Companies -


34/48

34

SAP EDW in 6 large Companies -Overview

In this EDW case study we are going to look at 6 diverse organizations andsee their lessons learned in their own words

Org

Number of Executive U


35/48

35

SAP as the EDW in an Insurance Company

Go-live Year: 2003 (BW v. 3.0b)

Mistakes Made: Under estimated thetime it would take to get the staff up tospeed and trained in BW. Had noSAP web skills in-house and went with thewrong portal choice (non-SAP)

Successes: Built foundation data storesfirst (AP, AR, GL, etc. before westarted the individual departmentneeds. This created a real EDWfoundation instead of data marts. Now weare building more multiproviders andfewer new data stores. Because we builtthe EDW first, we can now deliversolutions faster.

Technology Challenges: Needed 3 app

Next Steps:

Performance tuning(BIA) and cockpits

Org


Number of Casual users


36/48

36

SAP as the EDW in Oil & Gas Company

Go-live Year: 2001 (BW v. 2.1c)

Mistakes Made: Stated with wrong area(MM). Should have done FI first andthen HR. MM, APO and Motor Vehicle FuelTax reporting was too complex andambitious for the first implementationwhen we were learning.

Successes: Met budgets, deliverables andtimelines. User satisfaction was very highwhen we went from only BExworkbooks to the web templates.Upgrade to BI 7.0 was well received by

developers and users.

Technology Challenges: Did not knowhow to performance tune theworkbooks when we upgraded. Theywent from kilobytes to Megabytes.

Needed on-line user training (CBT)

Next Steps: Addingthe subsidiaries andcorporate entities in

Asia and Europe (650

Org


Number of Casual users SAP as the EDW in another Oil & Gas


37/48

37

SAP as the EDW in another Oil & GasCompany


Mistakes Made: No formal commitment to the

EDW, that evolved over time (3 years). Did nothave the top C-level commitment until 2003 andhad to do a lot of rework to accommodate thenew global vision.

Successes: We are 8 years into the EDW and ithas been adapted as the core platform for

global HR, finance and sales reporting. Wehave most divisions on the system and haveretired six legacy reporting environments.

Technology Challenges: Needed more HWthan originally planned. Performance was a real

problem until 2006 when we started usingthe Broadcaster and cached some reports inmemory.

Org



Next Steps: AddingEuropean training and

rollout (2 more R/3

SAP as the EDW in a Manufacturing


38/48

38

SAP as the EDW in a ManufacturingCompany


Mistakes Made: Started too early with too

ambitious scope. BW was not ready for EDWin 1999. Not until version 3.0b (2002) didwe get a real ODS and could realize our earlierideas of the EDW.

Successes: We kept the scope small andmanageable, and had good consultants.

The turnover rate on the project team has beenlow and the system was allowed to maturewithout business disruptions. We haveconsolidated three reporting groups intoone and saved hundred of thousands ofdollars in licenses each year.

Technology Challenges: Data integrationwas the hardest. We had to spend most ofour project time on masterdata mapping &consolidation.

Or

Number of Executive

Number of Casual user

Next Steps: Add morefunctionality(purchasing) and

rollout to purchasing

SAP th EDW i Hi h T h C


39/48

39

SAP as the EDW in a High-Tech Company

Go-live Year: 2003 (BW v. 3.1c)

Mistakes Made: User interface was notprioritized high enough. Executives and casualusers hated BEx workbooks. We had to relauchthe EDW in 2006 with a new web interface.

Successes: After the relaunch we have hadsuccess with user adaptation and have afunctional steering committee and CFOsponsorship. Closing the financial bookshave gone from 5 days to 3.

Technology Challenges: Was unsure on howto interface our existing portal with SAP BI

content (SSO). Security setup was hard

and advise was too divergent. Process chainsran very slow until we tuned the ABAP.

Or

Number of Executive

Number of Casual user

Next Steps: Add 2more acquired

companies to SAPR/3 and BI.

SAP as the EDW in a Government


40/48

40

SAP as the EDW in a GovernmentOrganization

Go-live Year: 2005 (BW v. 3.5)

Mistakes Made: Source data was in toomany diverse old system with no realstandards. We under estimated the timeit would take in integrate ninedifferent mainframes, some that was 20+years old. Should not used a big-bang go-live.

Successes: Civilian and uniformedpersonnel worked well together and trainingwas well received. The data collection andreporting that used to take 14 days eachmonth to produce, now takes 30 minutes.

Technology Challenges: During the BI7.0 upgrade, the unicode conversiontook long (did not complete over theweekend). The BSP web templates had tobe rebuilt completely.

Org



Next Steps: Addanother maintenance

organization andwork on web

Wh t W ll C


41/48

4141

What Well Cover





BI-EDWs


Wrap-up

Th C t I f ti F t (CIF)


42/48

42

The Corporate Information Factory (CIF)

In 2001, Bill Inmon (the father of DW)and Claudia Imhoff proposed a reporting

architecture known as the CIF.At the heart CIFs reportingstrategy is the EDW. It isthe source of:

1.

Decision Support Systemapplications (APO, CRM, OLAP,Reporting etc).

2. Data Mining and APD

3. Departmental Data Marts

4. Access Media Accelerators (BIA)

Bill Inmon is a SAP BItechnology advisor. He

has advised SAP on howto develop NetWeaver BI

Using the CIF Reducing number of


43/48

43

Using the CIF Reducing number ofPlatforms

NetWeaver helps by:

1.Reducing number ofhardware servers

2.Consolidates the platform

needs for budgeting,planning, forecasting andscheduling

3.Simplifies the platforms forweb access, security,reporting and analysis.

A major CIF decision is how to integrate the solutions inas few platforms as possible.

CIF provides a corporate frameworkfor the EDW; NetWeaver provides the

capabilities to do so with one platformSolution

SAPs Conceptual Enterprise Data


44/48

44

S s Co cep ua e p se a aWarehouse Architecture

SAP recognizes that we do not build EDWs, we

are doing Enterprise Data warehousing. This isan on-going activity that merges informationsystems, people and processes.

EDW is anon-going

activity withcontinuousinvestment

needs.

What Well Co er


45/48

4545

What Well Cover





BI-EDWs


Wrap-up

Resources


46/48

46

COMERIT (Presentations, articles and accellerators)www.comerit.net

Enterprise Wide Data Warehousing with SAP BW

https://www.sdn.sap.com/irj/sdn/go/portal/prtroot/docs/library/uuid/5586d290-0201-0010-b19e-a8b8b91207b8

Enterprise DataWarehousing SAP Help

http://help.sap.com/saphelp_nw70/helpdata/en/29/d9144236bcda2ce10000000a1550b0/frameset.htm

ResourcesResource
http://www.sap.com/index.epx


47/48

Your Turn!


48/48

Your Turn!

How to contact me:Dr. Bjarne Berg

[email protected]

Documents

NW2008 Tips Tricks EDW v10