What Every Client Should Do on Their Oracle SOA Projects

Raastech, Inc. 2201 Cooperative Way, Suite 600 Herndon, VA 20171 +1-703-884-2223 [email protected]

What Every Client Should Do On Their Oracle SOA Projects

Session 467 Wednesday April 15,2015

9:15 - 10:15 am Room Banyan C

© Raastech, Inc. 2015 | All rights reserved. Slide 2 of 54 @Raastech

Agenda

1. Introduction

2. Architecture

3. Infrastructure

4. Development

5. Things to Avoid

6. Recap & Summary

7. Q&A



About Me

Ahmed Aboulnaga @Ahmed_Aboulnaga

18+ years Oracle experience

Emphasis on Oracle Fusion Middleware

Author of “Oracle SOA Suite 11g Administrator’s Handbook”

Oracle ACE / OCE / OCA

Author / Presenter / Blogger

President of WMOUG / EGOUG

.


About Raastech

Small systems integrator founded in 2009

Headquartered in the Washington DC area

Specializes in Oracle Fusion Middleware

Oracle Platinum Partner & Reseller

Oracle SOA Specialized

100% of consultants are Oracle certified

100% of consultants present at major Oracle conferences

100% of consultants have published books, whitepapers, or articles

Oracle SOA Specialized – 1 in 1,500 worldwide

Oracle Platinum Partner – 1 in 3,000 worldwide


Disclaimer

This presentation is intended for NEW implementations of SOA Suite

Instructions mostly specific to SOA Suite 11g


Why This Presentation

Over 7+ years implementing Oracle SOA Projects

My last implementation is a lot better than my first implementation

I noticed recurring themes among all projects

Generally have 10 key recommendations



Install a Highly Available Infrastructure

Minimum 2 node cluster

Utilize Oracle HTTP Server (WebTier)

Set up for Dev, Test, QA, and Prod

Special consideration needed for:

Adapters

TLOGs

Persistent File Stores

JMS destinations

SOA Suite

RAC Database

Load Balancer

OHS OHS

SOA Suite


Establish a Deployment Process Early On

build,SOA Release 9.13

MDS,Customer.xsd,ns.raastech.com/ApplicationObjectLibrary/Common,29870

SOA,HelloWorldBPEL,default,1.0,true,Code/trunk/src/HelloWorld,26316

OSB,HelloWorldOSB.jar,Code/trunk/src/HelloWorldOSB,22738

SOAUndeploy,HelloWorldMediator,default,1.0,true,Code/trunk/src/HelloWorld,0

bam,/Raastech/CustomerDO,Code/trunk/reports/CustomerDO.xml,dataobject,true,22767

CodeList.txt:


Establish a Deployment Process Early On


Implement Service Level Authentication

Web service URLs are typically identical across all environments

http://soaDV/UpdateBankInfo?wsdl

http://soaQA/UpdateBankInfo?wsdl

http://soaPD/UpdateBankInfo?wsdl

Inadvertent or intentional invocation of

non-authorized services

SOA Suite (Dev)

SOA Suite (QA)

SOA Suite (Prod)

SoapUI

http://soaDV/UpdateBankInfo?wsdl



Solution involves one or both of the following:

Firewall rules

Service authentication

Service authentication can be done a multitude

of ways, with basic authentication the simplest

SOA Suite (Dev)

SOA Suite (QA)

SOA Suite (Prod)

SoapUI

http://soaPD/UpdateBankInfo?wsdl



<soap:Header xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">

<wsse:Security soap:mustUnderstand="1" xmlns:wsse="http://

docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">

<wsse:UsernameToken wsu:Id="UsernameToken-tk6qMWkQ5hl3Md2INvp16Q22"

xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">

<wsse:Username>oratest</wsse:Username>

<wsse:Password Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-

profile-1.0#PasswordText">password123</wsse:Password>

</wsse:UsernameToken>

</wsse:Security>

</soap:Header>

Simply require a <UsernameToken> in every SOAP header

Set different passwords for Dev, QA, and Prod



Establish a gateway (e.g., OAG, OSB)

Simply secure SOA/OSB services with OWSM

SOA Suite

OAG

SOA Suite

OSB

SOA Suite

* *

*

• Oracle API Gateway

• Ideal solution

• Extra licensing/administration cost

• Separate installation of OSB

• Works absolutely fine

• No additional cost

• Secure SOA/OSB services

• Works

• Loses gateway functionality


Securing OSB Services with OWSM

1. Click on the Proxy Service

2. Click on the “Policies” tab

3. Select “From OWSM Policy Store” and click “Add”

4. Select the policy “oracle/wss_username_token_service_policy”

5. Update, activate, and submit the changes


Securing SOA services with OWSM

1. Navigate to the composite, scroll down, and click on the reference

2. Click on the “Policies” tab

3. Attach the policy “oracle/wss_username_token_service_policy”



Setup Purging

A good automatic purging strategy is essential for managing the growth

of your dehydration store.

Oracle’s out-of-the-box scripts do the job, and improved variations can

be found online.

Adhering to a good purge strategy will prevent the tables from growing

too large and compromising your database.

The lack of purging is perhaps the largest culprit behind poor

performance.


Example of Multithreaded/Parallel Purging

DECLARE

MAX_CREATION_DATE TIMESTAMP;

MIN_CREATION_DATE TIMESTAMP;

RETENTION_PERIOD TIMESTAMP;

BEGIN

MIN_CREATION_DATE := TO_TIMESTAMP('2010-01-01','YYYY-MM-DD');

MAX_CREATION_DATE := TO_TIMESTAMP('2011-08-06','YYYY-MM-DD');

RETENTION_PERIOD := TO_TIMESTAMP('2011-08-06','YYYY-MM-DD');

SOA.DELETE_INSTANCES_IN_PARALLEL(

MIN_CREATION_DATE => MIN_CREATION_DATE,

MAX_CREATION_DATE => MAX_CREATION_DATE,

BATCH_SIZE => 10000,

MAX_RUNTIME => 60,

RETENTION_PERIOD => RETENTION_PERIOD,

DOP => 3,

MAX_COUNT => 1000000,

PURGE_PARTITIONED_COMPONENT => FALSE);

END;


Performance Tune the Infrastructure

Audit Level

Auditing in SOA Suite 11g has severe performance implications.

Log all audit information to the database asynchronously, instead of

synchronously.

Since auditing is deferred, there is a very rare chance that some audit

records may be lost.

Do not set compositeInstanceStateEnabled and Immediate to true

as this has severe performance implications.



Audit Level

Results in 25% to 40% performance improvements off the bat

1. Right click on soa-infra SOA Administration Common Properties More SOA Infra Advanced Configuration Properties AuditConfig

2. Apply the following Audit Config settings: compositeInstanceStateEnabled false must be set to false level Production Immediate false must be set to false Deferred true maxThreads 10 flushDelay 5000 batchSize 100



Log Level

Reduces amount of information logged and improves performance

slightly.

1. Right-click on soa-infra Logs Log Configuration 2. Select the dropdown View With Persistent Log Level State 3. Apply the following Log Configuration settings: Log Levels - oracle.bpm ERROR:1 (SEVERE) Log Levels - oracle.integration ERROR:1 (SEVERE) Log Levels - oracle.sdp ERROR:1 (SEVERE) Log Levels - oracle.sdpinternal ERROR:1 (SEVERE) Log Levels - oracle.soa ERROR:1 (SEVERE) Log Levels - oracle.sysman ERROR:1 (SEVERE) Log Levels - oracle.wsm ERROR:1 (SEVERE)



EM Data Display

The Enterprise Manager Console performance is poor, and this helps

improve it by reducing the amount of data retrieved at once on the

page.

1. Right-click on soa-infra SOA Administration Common Properties

2. Check the following checkboxes and set the duration to 12 hours:



Global JCA Retry Count

Impacts the JCA retries.

Limit the global inbound JCA retry count to 5 instead of unlimited.

The GlobalInboundJcaRetryCount takes effect if Jca.retry.count is not

set at the adapter service level.

1. Click on soa_infra Administration System Mbean Browser 2. Select oracle.as.soainfra.config, click on Server AdapterConfig Adapter 3. Set the GlobalInboundJcaRetryCount attribute to 5



BPEL Service Engine

These properties can impact the number of threads allocated to BPEL

processes and may need to be adjusted.

1. Right-click on soa-infra SOA Administration BPEL Properties

2. Set the following BPEL Service Engine Properties:

Parameter Name Current Value

Recommended Setting

Description

Audit Trail 50000 20000 Messages with values greater than this threshold are saved in the AUDIT_DETAILS table instead of AUDIT_TRAIL.

Dispatcher System Threads 10 20 Cleaning and housekeeping work for the engine. Dispatcher Engine Threads 30 100 Responsible for picking and processing instances. Set higher if you have a lot of processes

with many dehydration points. Dispatcher Invoke Threads 200 80 Instantiates new instances in the engine. Tune higher value if there are multiple

branches/flows for faster processing. Basically throttles the load setting of your BPEL Engine. AuditStorePolicy AsyncsyncLocal

Transaction

Will change the audit data storage strategy to save audit data asynchronously, using a separate local transaction.

SyncMaxWaitTime 300 (?) [MUST BE AGREED UPON WITH DEV TEAM] Determines the timeout for request-response operations.



Mediator Service Engine

Most critical are the Parallel Worker Threads and Parallel Locker

Thread Sleep.

1. Right-click on soa-infra SOA Administration Mediator Properties 2. Set the following Mediator Service Engine Properties:

Parameter Name Current Value

Recommended Setting

Description

Metrics Level Enabled Disabled Disable this unless DMS metrics collection is explicitly required.

Parallel Worker Threads 5 20 A value of 20 is used for large environment. Parallel Maximum Rows Retrieved 200 200 Set appropriately; small (200) versus large environments (600).

Parallel Locker Thread Sleep 2 1 Bad parameter. Always set to 1.



JTA Timeout Settings

A recommended value of 120-300 or higher as needed.

If an end to end flow is taking longer than the JTA timeout setting, it will

roll-back the transaction.

If retries are involved, the retry counter is reset and may thus result in

an infinite number of retries. To change this setting, the below

procedure can be used.

1. Click on soa_domain JTA 2. Set the Timeout Seconds setting to 300


Performance Tune the Infrastructure Database Parameter Default Value Tuned Value Comments

AUDIT_TRAIL DB NONE Determines whether auditing is turned on. For performance reasons, it is best set to NONE. There is slight overhead associated with all other audit settings. DISK_ASYNCH_IO TRUE TRUE Controls whether I/O to data files, control files, and log files is asynchronous. Highly recommended to keep the default value of TRU unless ASYNC I/O implementation is

not stable. UNDO_MANAGEMENT AUTO AUTO Specifies which undo space management mode the system should use. It is recommended to have this parameter set to AUTO. PLSQL_CODE_TYPE NATIVE NATIVE PL/SQL compilation mode of NATIVE is typically faster than INTERPRETED. MEMORY_TARGET 0 0 Specifies the system-wide usable memory. Set to 0 when MEMORY_MAX_TARGET is specified. AQ_TM_PROCESSES 2 1 This property controls processing of messages with delay and expiration properties specified. It is strongly recommended not to set AQ_TM_PROCESSES to 0. If it is set

to 1, a one-time manager process is created to monitor the messages. FILESYSTEMIO_OPTIONS SETALL SETALL Oracle recommends setting this to SETALL to enable both asynchronous and direct I/O on file system files where possible.

NLS_SORT BINARY BINARY Specifies the collating sequence for ORDER BY queries and setting to BINARY requires less system overhead. OPEN_CURSORS 2000 1000 (?) OPEN_CURSORS specifies the maximum number of open cursors a session can have at once. It is important to have the value of OPEN_CURSORS set high enough to

prevent your application from running out of open cursors. SESSION_CACHED_CURSORS 2000 1000 The SESSION_CACHED_CURSORS parameter is used to reduce the amount of parsing with statements and cursors. If your cursors are being paged-out of the library

cache, increasing SESSION_CACHED_CURSORS up to the value of OPEN_CURSORS improves the performance. PLSQL_CODE_TYPE NATIVE NATIVE PL/SQL compilation mode of NATIVE is typically faster than INTERPRETED. TRACE_ENABLED TRUE FALSE Setting this to FALSE may result in minimal performance improvement, but you may lose valuable diagnostic information in the event of a database error. UNDO_RETENTION 900 0 By setting UNDO_RETENTION to 0, you enable auto tuned retention. That is, as long as your tablespace can autoextend, you are less likely to get an ORA-01555. PROCESSES 6000 3000 (?) Sets the maximum number of processes that can be started by an instance. This is one of the more important primary parameters to set and tuned to a higher value, as

many other parameter values are deduced from this. SESSIONS 9024 3000 (?) The default value of SESSIONS is derived from PROCESSES. If you alter the value of PROCESSES, you may want to adjust the values of these derived parameters. SGA_MAX_SIZE 40 15G (?) SGA includes Shared Pool, Database buffer cache, and Redo Log buffer. Since Oracle SOA Suite heavily depends upon the underlying database for continuous huge

updates and inserts, the SGA_MAX_SIZE should be around 40 percent of your physical RAM. SGA_TARGET 40 15G (?) This parameter reflects the total size of memory footprint an SGA can consume. SGA_MAX_SIZE and SGA_TARGET should be tuned to the same value. PGA_AGGREGATE_TARGET 13G 5G (?) PGA_AGGREGATE_TARGET specifies the total amount of session PGA memory that Oracle will attempt to allocate across all sessions. Tune it to around 30 percent of

your SGA. CURSOR_SHARING SIMILAR SIMILAR Oracle uses a shared SQL area when applications send similar SQL statements to the database. The sharing of SQL by setting CURSOR_SHARING to SIMILAR reduces

memory use on the database server, thereby increasing system throughput. INITRANS (for table) 1 2 The INITRANS setting controls Initial Transaction Slots (ITLs). A transaction slot is required for any session that needs to modify a block in an object. The contention for

ITL can be reduced by increasing the INITRANS storage parameter of the table and indexes. INITRANS (for indexes) 20 40 SHARED_POOL_SIZE 128M 400M SHARED_POOL_SIZE has to be tuned to provide more room for new cursors and query blocks. It has to be tuned up, if cursors are being aged out due to lack of free

space. DB_BLOCK_CHECKSUM TYPICAL OFF Checksums allow Oracle to detect corruption caused by underlying disks, storage systems, or I/O systems. If your application is I/O intensive and you are short on CPU

capacity, you should disable it. DB_CACHE_SIZE 0 100M Setting SGA_TARGET means this value is auto-tuned. Recommend leaving it auto-tuned.


Configure Log Rotation

Numerous Oracle WebLogic Server logs.

Rotate by size or time.

Managing Logs in Oracle WebLogic Server

http://otechmag.com/magazine/2015/spring/ahmed-aboulnaga.html

Log Location Managed Server Logs Server > [managed_server] > Logging > General Managed Server HTTP Logs Server > [managed_server] > Logging > HTTP Data Source Logs Server > [managed_server] > Logging > Data Source Domain Level Logs soa_domain > Configuration > Logging Managed Server System Out Unfortunately, the WebLogic Server log rotation affects only the .log

files but not the standard .out files, which is generated when you use Node Manager to start up and shut down the managed servers. You must implement a custom solution for the system .out files and many resources online are available to help you with this.






Establish Design Patterns… and stick to them!

For example:

Request-Response (synchronous 2-way flow)

Fire-and-Forget (asynchronous 1-way flow)

Asynchronous with Callback

Bulk Data Load (small data sets)

Bulk Data Load (large data sets)


Asynchronous Design Pattern

OSB Mediator

BPEL

BPEL

OSB

OSB

OAG *

BAM

First touch point should be a gateway, provides:

• Abstraction/virtualization • SLA monitoring • First line of defense • Could be in DMZ

1



Mediator

BPEL

BPEL

OSB

OSB

2 OAG *

BAM

JMS queue provides: • Immediate handoff from caller • Message persistence (which also supports recovery)

OSB



Mediator

BPEL

BPEL

OSB

OSB

OAG

1-way Async *

BAM

Mediator used for: • Routing • Calls BPEL as 1-way async

(eliminates need for another queue)

3

OSB



Mediator

BPEL

BPEL

OSB

OSB

OAG

Synchronous

*

BAM

• Entire sub-process is synchronous • Errors bubbled up to BPEL • BPEL decides how to handle

exceptions • OSB can be used for throttling or

caching

Throttle

Cache

4 OSB



Mediator

BPEL

BPEL

OSB

OSB

OAG *

BAM

Create an error management framework:

• Single async service • Maintains errors in a table • Auto-notify by email as required

5

OSB



Mediator

BPEL

BPEL

OSB

OSB

OAG *

BAM

Create a message resubmission framework:

• Payloads persisted • Manually republished to

original queue for resubmission

6

OSB



Mediator

BPEL

BPEL

OSB

OSB

OAG *

BAM

Use BAM: • To store sensor data • Easily lookup state of transactions

outside of main SOA engines • Keep BAM server on separate

physical servers

7

OSB


Use MDS for all WSDLs

Consider ORAMDS instead of HTTP for composite-to-

external and/or composite-to-composite invocations

Removes the composite loading dependency

(It’s still translated to an optimized call behind the scenes)

<reference name="AsyncStatus" ui:wsdlLocation="oramds:/apps/WSDLs/AsyncStatus.wsdl">

<interface.wsdl interface="http://xmlns.oracle.com/AsyncStatus#wsdl.interface(execute_ptt)"/>

<binding.ws location="oramds:/apps/WSDLs/AsyncStatus.wsdl"

port="http://xmlns.oracle.com/AsynStatus#wsdl.endpoint(AsyncStatus_ep/execute_pt)"

soapVersion="1.1“/>

</reference>


Design an Error Handling Framework

Develop a framework that will serve as your primary error handling

service.

If your BPEL, Mediator, or OSB service throws an error, simply make a

web service call to your framework, which will ultimately persist it in a

table.


Design a Message Recovery Framework

When errored out, synchronous transactions simply bubble up the fault

back to the original caller. Not so with asynchronous transactions.

Two types of errors:

Business Exceptions

System Exceptions

BPEL recovery framework is an

option provided with the product.

Develop your own custom solution.



Avoid: JMS Topics

The first is that there is no simple way to create a JMS topic that is

targeted to all managed servers and accessible via a single JNDI.

Cannot have durable subscribers on distributed topics.

Single message consumption from distributed topic in WebLogic Server 11g http://blog.raastech.com/2012/04/single-message-consumption-from.html Understanding the "singleton" property with WebLogic JMS topics http://blog.raastech.com/2012/07/understanding-singleton-property-with.html 5 Reasons Why Oracle WebLogic Server Sucks http://blog.raastech.com/2012/07/5-reasons-why-oracle-weblogic-server.html

http://blog.raastech.com/2012/04/single-message-consumption-from.html







http://blog.raastech.com/2012/07/understanding-singleton-property-with.html







http://blog.raastech.com/2012/07/5-reasons-why-oracle-weblogic-server.html












Avoid: AIA Foundation Pack 11g

Called Core Services in 12c.

Does not “reduce complexity” and “speed up development.”

Lack of tangible benefits achieved by adopting AIA Foundation Pack:

Licensing cost

Ongoing maintenance cost

Training investment needed

Questionable long term future by Oracle


Avoid: Oracle BAM 11g

Oracle BAM sounds good on paper.

Weak reporting:

No user sorting

No field expansion

Cumbersome navigation

Weak search capabilities

Unable to handle large amounts of data (think millions of records per data object).

Only runs on IE (fixed in 12c).

Avoid BAM RMI Adapter which leads to BPEL threads not being released.


Avoid: Mediator Parallel Routing Rules

See this presentation for details why:

http://www.raastech.com/raastech/library/Raastech_2013_Collaborate_SOASuite11gPerformance.pdf

This is because of the poor design that Oracle has chosen for avoiding

thread starvation

Edit all “*.mplan” files for your asynchronous Mediator operations and

change the executionType as follows:

OLD: executionType="direct"

NEW: executionType="queued"

http://www.raastech.com/raastech/library/ Raastech_2013_Collaborate_SOASuite11gPerformance.pdf


100 messages 98 messages

Wait 1 second

No messages here

Wait 1 second

No messages here

Wait 1 second

No messages here

99 messages

Avoid: Mediator Parallel Routing Rules



Recap

Architecture Infrastructure

• Install Highly Available Infrastructure

• Establish a Deployment Process Early On

• Implement Service Level Authentication

• Setup Purging

• Performance Tune the Infrastructure (11g)

• Configure Log Rotation

Development Things to Avoid

• Establish Design Patterns

• Use MDS for all WSDLs

• Design an Error Handling Framework

• Design a Message Recovery Framework

• JMS Topics

• AIA Foundation Pack 11g

• Oracle BAM 11g

• Mediator Parallel Routing Rules

Applies to Oracle SOA Suite 11g/12c


Recap

Architecture Infrastructure

• Install Highly Available Infrastructure

• Establish a Deployment Process Early On

• Implement Service Level Authentication

• Setup Purging

• Performance Tune the Infrastructure

• Configure Log Rotation

Development Things to Avoid

• Establish Design Patterns

• Use MDS for all WSDLs

• Design an Error Handling Framework

• Design a Message Recovery Framework

• JMS Topics

• AIA Foundation Pack 11g

• Oracle BAM 11g

• Mediator Parallel Routing Rules

Applies to any SOA project


Shameless Plug

Oracle SOA Suite 12c Administrator’s Handbook


Contact Information

Ahmed Aboulnaga

Technical Director

@Ahmed_Aboulnaga

[email protected]


Q&A

Technology

What Every Client Should Do on Their Oracle SOA Projects