Build an Advanced Oracle E-Business Suite Architecture (S281594 - Technology Track - OOW 2006) Paul Bramy (Oracle) & Alain

<Insert Picture Here>

Build an Advanced Oracle E-Business Suite Architecture(S281594 - Technology Track - OOW 2006)

Paul Bramy (Oracle) & Alain Roy (IBM)Oracle/IBM Joint Solutions Center Leaders

This presentation has been built on a real customer case.

It represents a concrete follow up of the JSC presentation delivered last year in OOW ( S1854 - How to implement Oracle E-Business Suite on IBM

eServers )

It aims to present how the Oracle solutions & technology stacks can be leveraged & merged to build an advanced and flexible Oracle E-Business Suite Architecture.

It demonstrates how Thousand Concurrent Users and Massive Reporting with high quality of services has been achieved by optimizing the resources through the virtualization delivered by IBM System p & AIX5L.

This project is also a human story with a strong teaming from Oracle & IBM experts.

Presentation Introduction


Agenda

• JSC introduction.

• Project introduction.

• Technical foundations.

• Project presentation.

• Conclusions.

• What is the JSC?

The Oracle/IBM Joint Solutions Center, created in 1998, is a presales team,

permanently staffed by Oracle & IBM, to contribute to the successful delivery of

Oracle/IBM Joint Solutions.

• Why the JSC is unique?

Located inside the PSSC, the IBM’s European Benchmarking and Support

center (PSSC) in Montpellier, France. JSC has access to large amount of

hardware from all the IBM brands (p, x, I, z)

It is staffed by Subject Matter Experts from both IBM & Oracle so we can have

access to the best of our two corporations simultaneously for the benefit of our

joint customers.

Oracle/IBM Joint Solutions CenterJSC Specificities

The living proof of Oracle and IBM’s desire to work closely together

Eight years of experience

What can you do?

Pre Sales Assistance through the whole project/sales cycle

Help me to Convince: Briefings & Conferences (40/year)

Help me to Build: Architecture, Design, Sizing Assistance (300/year)

Help me to Demonstrate: PoC, Benchmarks Assistance (30/year)

Help me to Deliver: hands-on workshops (5/year), Publications (10/year)

Which Products & Solutions are you covering?

Oracle Technologies and Solutions such as Oracle Database & Application

Server (9i, 10g), e-Business Suite 11i on IBM servers System x, System p,

System I & System z.

Expanding since 2006 to PSE, Siebel & JDE. Coming soon Portal Retek.

The living proof of Oracle and IBM’s desire to work closely together

Eight years of experience

Oracle/IBM Joint Solutions CenterJSC Missions

Oracle/IBM Joint Solutions CenterJSC engagement

Q&

A -

Siz

ing

Des

ign

Wor

ksho

psPer

form

ance

Ana

lysi

s

[email protected]

Arc

hite

ctur

e

Impl

emen

tatio

n

Sup

port

Evo

lutio

ns


Agenda





• Conclusions.

Project IntroductionCustomer’s business reasons for change

• Banca Comerciala Romania (BCR) is the biggest Commercial Bank in Romania, having 35% of the Romanian Banking Sector assets andalmost 30% market share. It is the 6th biggest bank in Eastern Europe.

• BCR is part of BCR Financial Group together with BCR Insurance, BCR Asset Management, BCR Financial Investment etc.

• Bank’s business strategy to:

• Remain the no.1 bank on the market by increasing the market share

• Consolidate the capital

• Consolidate the profile of universal bank

• IT Prerequisits to support this strategy:

• Centralized, Integrated

• Technological Consistency

• Client oriented

• Fragmented view of customer , products and accounts information

• Business continuity

• Long cycle “time to market” for new products

• Insular front & back office applications

CHALLENGES

• Centralized Core banking system,meaning:

• Unique customer nomenclature

• Unified products

• Unique account

• 24x7 business availability

• Easy setup of new products

• Capability to setup complex products

• Smooth addition of new channels

WISHES OBJECTIVES

Project IntroductionIT Customer Challenges & Views

• Increased Market Share via new products & delivery channels (Mobile, Internet)

• Increased Customer Satisfaction & Loyalty via High Quality of Services

• Integration of all core solutions

ROLES

• Analysis team leader

• Design and build team

leader

• Analysts

• Technical consultants

RESPONSIBILITIES

• Analyze requirements

• Design, build and unit test

kernel and modules

• Data migration

• System parameterization

Customer Team

Project IntroductionProject teams introduction

ROLES

• Project Director

• System and solution architect

• Business consultant

• Design and build team leader


RESPONSIBILITIES

• Design the solution and

technical architecture for the entire application

• Design, build and unit test

kernel and modules

• Install, configure and support Oracle products

Oracle Team

ROLES

• Hardware team leader


RESPONSIBILITIES

• Install, configure and

support hardware

equipment

IBM Team

ROLES

• Business requirements

definition team leader

• Business consultants

• Conformity officer

• Training team leader

• Testing team leader

RESPONSIBILITIES

• Gather and formalize business requirements

• Organize training events

• Perform system integration

and acceptance tests

• Produce periodic progress

reports

System Integrator Team

JSC Second Level of Support


Agenda





• Conclusions.

• IBM System p & Power 5

• IBM Storage

• AIX5L

• GPFS

REQUIREMENTS

• 1 to 64 cores

• N°1 on Unix Market

• Performance Leader

• A Clear Roadmap

• Pay as you grow withCapacity Upgrade on Demand

• Virtualization &

Flexibility (CPU, Memory, IO, Network)

• Clustered FileSystemfor Shared Appl_Top

and RAC

PRODUCTS VALUES

Technical foundationsIBM Stacks

• Robust & RecognizedTechnologies

• High Performance

• Scalable Infrastructure

• Resource Optimization

• A Clustered Solution &

FileSystem

Without Virtualization

Jay Kruemcke IBM 2003

• DLPAR with micro-partitioning : Dynamically adjust CPU and memory allocation

• Virtual I/O; Virtual LAN

Test LPAR

Agent

CRM LPAR

Agent

Finance LPAR

Agent

Unbalanced & lost resource

5CPU

6CPU

Agent Agent Agent

Adjusted & Balanced resource

Test LPAR CRM LPAR Finance LPAR

0.5CPU

3CPU

8.5CPU

1CPU

Technical foundationsIBM Stacks: Advanced Virtualization

Key Words :EC for Entitlement Capacity, Capped/Uncapped & Partition Weight

With Virtualization

IBM filesystem designed for cluster use, certified with Oracle RACGPFS is capable of holding the database, redo logs,

control files and archived logs

required by Oracle RAC + Binaries

Data is striped across shared local disks or NSD serversMetadata is maintained by all servers in the cluster

File Locking is distributed across the servers in the cluster

Performing and scalableStriping I/O performance

Direct I/O with close to RAW device performance

Supports terabyte file sizes and multiple terabyte file systems

Highly Available – Fault tolerantSupports RAID subsystems and/or performs own file system replication.

Supports on-line dynamic reconfiguration (add, delete, replace disks and nodes;

rebalance the file system)

HACMP

GPFS

RSCT

GPFS

RSCT

Technical foundationsIBM Stacks: General Parallel File-system (GPFS)

• EBS 11.5.10 (PLSQL & FORMS, WORKFLOW)

• ORACLE 10g DATABASE

• REAL APPLICATION CLUSTER

• DATAGUARD

• XML INTEGRATION

REQUIREMENTS

• UNIFIED DEVELOPMENT FRAMEWORK

• SINGLE DATABASE REPOSITORY

• RELIABILITY & AVAILABILITY

• SECURITY & PERFORMANCES

• QUALITY OF SERVICES

PRODUCTS VALUES

Technical foundationsOracle Stacks

• CORE BANKING & FINANCIALS FOUNDATION

• 11i WORFLOW & MESSAGING

• PARTITIONING

• SCALABILTY, HIGH AVAILABILITY & WORKLOAD ISOLATION

• DISASTER RECOVERY

Technical foundationsOracle Stacks: Oracle E-Business Suite

• 11.5.10.2 with Oracle RAC 10.1.0.4

• Main 11i features used in the project

• Apps Concurrent Management

• Apps Users Management

• Apps Security rules

• 3-tier centralized architecture

• Full use of Oracle Technology stack

• Modular approach ( Financial Modules )

• Exchange of messages in XML format

• Advanced Queue technology

Technical foundationsOracle Stack Highlight : The Specific Banking Kernel

•A Development framework has been built by Oracle Consulting on the top of Oracle E business Suite 11.5.10 foundations

•Kernel Definitions• Ensures a tight integration between

the specific banking softwares, the overall IT system and functional 11i modules.

• Secures the overall security and organizational aspects• Reduces any new definition and/or additional coding.

•Main Kernel features• Kernel Validation Services

Check orders versus Bank’s rules and constraints. • Calculation Services

mostly calculation services for interests, fees, taxes, balances…• Prenotation Services

Create and update prenotations against accounts.• Information Services

Provide information about accounts, balances, prenotations...

Technical foundationsOracle Stack Highlight : The Specific Banking Kernel

Database area

Modules area

Kernel area

Applicationframework

Concurrent Manager Interface

(CM API)Core Kernel Interface

(PL/SQL)

Kernel Service Packages

Transaction Management

General Ledger

Interfaces

Forms Interfaces

Oracle Workflow

Server

Domain Service

Packages

Applications

•Module definitions (packages)

•Data management (tables)

•Transactions Performance

•Security access

•Reliability / Availability

Forms server

Daily Banking activity Official reports ATM integratioB.I activity

GPFS or AMSRAC

SingleSingle

Scalable Secured

Virtualized

Technical foundationsOracle stacks: RAC 10g within EBS environments

Oracle 10G R2 RAC node 1

Oracle 10G R2 Oracle 10G R2

RAC node 1RAC node 1

Oracle EBS11.5.10.2

Prod AS node 1

Oracle EBS11.5.10.2 Oracle EBS11.5.10.2

Prod AS node 1Prod AS node 1




Oracle EBS11.5.10.2

Prod AS node 2


Prod AS node 2Prod AS node 2

LPAR# 1.1

LPAR# 1.1

LPAR# 1.2

LPAR# 1.2

LPAR# 2.2

LPAR# 2.2

LPAR# 2.1

LPAR# 2.1

1,7 CPU8 GB

1,7 CPU8 GB

2,5 CPU8 GB

2,5 CPU8 GB

System p5 server 1System p5 server 1

1,1 CPU8 GB

1,1 CPU8 GB


DevDevDev TestTestTestLPAR# 1.3

LPAR# 1.3

1,4 CPU8 GB

1,4 CPU8 GB

LPAR# 2.3

LPAR# 2.3



Oracle 10G R2 Oracle 10G R2 Oracle 10G R2

RAC node 1RAC node 1RAC node 1

Oracle EBS11.5.10.2

Prod AS node 1

Oracle EBS11.5.10.2 Oracle EBS11.5.10.2 Oracle EBS11.5.10.2

Prod AS node 1Prod AS node 1Prod AS node 1

LPAR# 1.1

LPARLPARLPAR

# 1.1# 1.1# 1.1

LPAR# 1.2

LPARLPARLPAR

# 1.2# 1.2# 1.2

2,5 CPU8 GB

2,5 CPU2,5 CPU2,5 CPU

8 GB8 GB8 GB

1,1 CPU8 GB

1,1 CPU1,1 CPU1,1 CPU

8 GB8 GB8 GB

DevDevDevDevLPAR# 1.3

LPARLPARLPAR

# 1.3# 1.3# 1.3




Oracle EBS11.5.10.2

Prod AS node 2


Prod AS node 2Prod AS node 2LPAR# 2.2

LPAR# 2.2

LPAR# 2.1

LPAR# 2.1

2,7 CPU8 GB

2,7 CPU8 GB

TestTestTest

2,4 CPU8 GB

2,4 CPU8 GB

LPAR# 2.3

LPAR# 2.3


Agenda


• 11i Oracle/IBM Customer examples.




• Conclusions.

Project presentation Project challenges through its Numbers

SOLUTION

Branches: 310

Named users: 9783

Conc.users/day: 6530

Accounts: 25M

Orders/day: 500k

Reports/day : 400k

Database size:2.2 TB

Largest table: 1B rows

16.000 DB session peak

Massive OLTP Massive Online Reporting

Large Database

(Day) Opened hours (8AM – 6PM)

Project presentation Project phases: Most important steps

period

Main events

June 2006 Oct 2006

Database size1.2 TB

Database size2.1 TB

250 branchesConnected

New HeadQuarter

activity

310 branchesConnected

Go Live

PlannedATM

integrationFirst Branch

Pilot

250K ordersPer day

400K ordersPer day

D/R reflexion

New Banking

Applications

Nov 2005 Dec 2005 May 2006

Roll up waves

End

Every weekAddiitional branches

Single Datacenter

Two DP rooms *

Clustered IBM System

p p595 running AIX

IBM Storage models

DS8100

Project presentationDatacenter: Physical layout

Remote Branches

And HeadQuarter

IBM IBM

Global networks

IBM IBM

Production Room D/R Room

Routers Routersfirewalls

WAN

Project presentationBusiness area coverage…

• Three type of performance requirements for Three type of business activities

• The Daily activity: representing the opened period of the bank and the OLTP activity at the application level. This period covers each day of the week starting from 8:30 am to 6:30 pm.

• The Daily Over-Night: representing the batch called « End of day » and some additional tasks ( reports + backups). This period covers each night of the week starting at 6:30 pm to 6:30 am the next day.

• The End of month: representing a specific period of the month (last day of the month) starting from 6:30 pm to 6:30 am the next day.

07:02:28

07:40:29

08:18:30

08:56:31

09:34:32

10:12:33

10:50:33

11:28:34

12:06:35

12:44:36

13:22:36

14:00:37

14:38:38

15:16:39

15:54:39

16:32:40

17:10:41

17:48:42

18:26:42

19:04:43

19:42:44

20:20:45

20:58:46

21:36:47

22:14:47

22:52:48

23:30:49

00:08:50

00:46:51

01:24:52

02:02:52

02:40:53

03:18:54

03:56:55

04:34:55

05:12:56

05:50:57

06:28:57

Pro

ject p

resen

tatio

n24 h

ou

rs S

ys

tem

beh

avio

r… (G

lob

al C

PU

activ

ity )

OLT

P D

aily

activ

ity

Daily

batc

h a

ctiv

ity

Project presentationProduction Products Overview

Partition A Partition B

Partition 1 Partition 2

Partition C Partition D

Partition 3 Partition 4

P595 – 32 Ways (24+8) runningAIX5.3

P595 – 32 Ways (24+8)

IBM DS8100

GPFS

Oracle 10gRAC

DB01 DB02 DB03 DB04

(Forms+

Report)

(Forms) (Forms+

Report)

(Forms)

Datafiles

APP01 APP02(Forms+

Report)

(Forms) (Forms+

Report)

(Forms)

Shared APPL_TOP

APP03 APP04

Project presentationProduction Server Overview

DB03

(Forms)

DB04

(Forms+Report)

P595 – 32 Ways (24+8)

APP04

(Forms +Report)

Partition D

CE = 8VP=18

92GB RAMWeight=250

SMT=ON

APP03

(Forms)

Partition 4

CE = 6VP=12

55GB RAMWeight=128

SMT=ON

Partition 3

CE = 6VP=12

55GB RAMWeight=128

SMT=ON

Partition C

CE = 8VP=18

92GB RAMWeight=250

SMT=ON

Shared Pool of 32 Physical ProcessorsMin, Max and Desired

processing units of

capacity

(Capacity Entitlement)

Total CE = 28

4 CPU not alocated

to a specific partition

Min, Max and

Desired number of Virtual

processors : the whole

number of concurrent operations that

the partition can use.

When a partition is running an uncapped mode an uncapped weight should be indicated.

SMT

Simultaneous MultiThreading

Two threads are opened on a Power 5

processor

Project presentationDaily activity – CPU and memory utilization (APPS tier)

0

100

200

300

400

500

600

700

800

07

:30

07

:56

08

:22

08

:48

09

:14

09

:40

10

:06

10

:32

10

:58

11

:24

11

:50

12

:16

12

:42

13

:08

13

:34

14

:00

14

:26

14

:52

15

:18

15

:44

16

:10

16

:36

17

:02

17

:28

17

:54

18

:20

Production Cluster – Server number 2 – Application Partitions APP03 and APP04

CPU utilization

APP04(Forms + report)

APP03

(Forms)

(C.E)Theorical

Partition Capacity

Activity peak

> 100% CE

Memory consumption(free memory)

0

5

10

15

20

25

30

35

40

45

07:3

0

07:5

4

08:1

8

08:4

2

09:0

6

09:3

0

09:5

4

10:1

8

10:4

2

11:0

6

11:3

0

11:5

4

12:1

8

12:4

2

13:0

6

13:3

0

13:5

4

14:1

8

14:4

2

15:0

6

15:3

0

15:5

4

16:1

8

16:4

2

17:0

6

17:3

0

17:5

4

18:1

8

1650 conc/active usersper application partitions

Average 12MB / user

CM + Report processes = 5GB

0

200

400

600

800

1000

1200

07:3

0

08:0

2

08:3

4

09:0

6

09:3

8

10:1

0

10:4

2

11:1

4

11:4

6

12:1

8

12:5

0

13:2

2

13:5

4

14:2

6

14:5

8

15:3

0

16:0

2

16:3

4

17:0

6

17:3

8

18:1

0

Project presentationDaily activity – CPU and memory utilization (DB tier)

Production Cluster – Server number 2 – Database Partitions DB03 and DB04

CPU utilization

DB04(Forms + report)

DB03

(Forms)

(C.E)Theorical

Partition Capacity

Activity peak

> 100% CE

Memory consumption(free memory)

Average 10 MB / user0

5

10

15

20

25

30

35

07

:30

07

:58

08

:26

08

:54

09

:22

09

:50

10

:18

10

:46

11

:14

11

:42

12

:10

12

:38

13

:06

13

:34

14

:02

14

:30

14

:58

15

:26

15

:54

16

:22

16

:50

17

:18

17

:46

18

:14

Se

rve

r 2( A

pp

03+

AP

P04+

DB

03+

DB

04 )

Glo

ba

l CP

U

us

ag

e:

87

.5 %

Pro

ject p

resen

tatio

nD

aily

activ

ity –

Serv

er 2

-G

lob

al C

PU

utiliz

atio

n

0

200

400

600

800

1000

1200

07:30

08:04

08:38

09:12

09:46

10:20

10:54

11:28

12:02

12:36

13:10

13:44

14:18

14:52

15:26

16:00

16:34

17:08

17:42

18:16

0

500

1000

1500

2000

2500

3000

07:30

08:04

08:38

09:12

09:46

10:20

10:54

11:28

12:02

12:36

13:10

13:44

14:18

14:52

15:26

16:00

16:34

17:08

17:42

18:16

DB

04

DB

03

AP

P04

AP

P03

Project presentationCPU utilization summary : 24 hours period

Partitions CPU ( %user +%sys) Free memory

App01

App02

App03

App04

DB01 75 to 80%

DB02 15% (steady)

DB03 60 + peaks 100%

DB04 90 to 100%

The system is rather flat and low.

No important activity.

More free memory than for the daily activity.

Partitions CPU ( %user +%sys) C.E reached vs affected Free memory

App01 80% 5/6 20 GB

App02 80 to 100% 5 / 6 + peaks > 6 15 GB

App03 80 to 90% 4.5 to 5 / 6 20 GB

App04 95 to 100% 5.5 / 6 15 GB

DB01 75 to 80% 6 to 7 / 8 10 GB

DB02 90% 7 / 8 + peaks > 8 10 GB

DB03 75 to 80% 7/8 15 GB

DB04 90 to 100% 6 to 8 / 8 7 GB

CPU and memory consumption of each partition during the daily activity

CPU and memory consumption of each partition during the End of Day activity

Project description Evolutions and Future integrations

• Integrate Headquarter Workload: +5% active users increase but with very specific & strategic type of business requests ( BI, reports, etc..) -> Done…

• Integrate Card system: a critical part of the global system. Current estimations is +200k orders daily. Proof of concept (POC) proposed to evaluate as precisely as possible the impact of the Card System on the current architecture.

• Add contingency ressources: Add Ressources or Nodes to reduce impact of any system failure.

• A Disaster Recovery Site: Study & Design to be conducted for a D/R solution leveraging Dataguard. Potential Stretched RAC.

Project description Card System integration… ( Study in progress)

DB03 DB04DB02DB01 IBM WebsphereMiddle tier

Kernel

IBM System i

Existing Core banking architecture

DB03 DB04DB02DB01

Kernel

DB05IBM

WebsphereMiddle tier

RAC

DB03 DB04DB02DB01 DB05DB06

IBM

W

eb

sp

he

reM

idd

le tie

r

IBM

W

eb

sp

he

r eM

i dd

le t

ier

RACKernel Kernel

Existing Core banking architecture

Extended Core banking architecture

ATMSystems


Agenda


• 11i Oracle/IBM Customer examples.




• Conclusions.

Oracle is the Applications Market Leader

IBM is the Infrastructure Market Leader

IBM & Oracle in clear and deep Partnership

Oracle & IBM Extended Virtual joint team

Oracle & IBM Long Term Roadmaps

11

Why this solution? — Top 10 Reasons

22

33

44

55

11

22

33

44

55

Oracle E-business suite flexibility

Oracle 10gRAC capabilities

IBM Power 5 features

IBM Advanced virtualization

Oracle and IBM skilled teams

Technology reasons Strategic reasons


“Sibcor este unul dintre cele maicompetitive sisteme bancare create

special pentru prima banca de pepiata romaneasca impreuna cu

Oracle; un proiect de mare succes

de care suntem mandri.”

BCR Quote Radu Varduca, IT Manager

Sibcor is one of the most custom developed competitive Core Banking system by Oracle specialy taylored for the no 1 bank in Romania;

we are very proud of our system.

Anca Cont

Catalin BogdanDaniela ZupiniDanut DimaFabienne LepetitFrederic Michiara

Gabriel GheorghiuIonut EnusNadia BendjedouNicoleta MacoveiMihai Teodor Popescu

Octavian Lascu…/…

and all our colleagues involved during this project…

Many thanks to Put it all together…. Again…

Documents

Build an Advanced Oracle E-Business Suite Architecture (S281594 - Technology Track - OOW 2006) Paul Bramy (Oracle) & Alain