SOA Monitoring & Administration: Tips & Tricks

Jeremy TreagueLead Technical Developerjeremy.treague@schreiberfoods.com

Spring 2014 FDUG Meeting

Overview

• Where we are at today with SOA• Recent obstacles & workarounds• Programmatically manage SOA composites• SOA fault handling & monitoring approaches• SOA proactive maintenance tips• Where we are heading with SOA

Where We Are At Today

• We have been using SOA Suite 11.1.1.6 for 1 yearSingle-server Dev & Training EnvironmentsClustered Test & Production Environments

• Hub & Spoke Integration Pattern Manufacturing & Home Office (ERP/WMS) system

integrations 10 domestic plant locations < 200,000 composite instances/day

• Components Used Database Adapter (Oracle & MS SQL) Mediator & BPEL Event Delivery Network (EDN)

Recent Challenges & Workarounds

• Problem: Un-deploying SOA composites took too long

Workaround: Purge off all composite instances, then un-deploy the composite

Recent Challenges & Workarounds

Problem: SOA Managed Servers Intermittently Crashjava.lang.OutOfMemoryError: PermGen space

Preceded by:[STUCK] ExecuteThread: '22' for queue: 'weblogic.kernel.Default (self-tuning)' has been busy for "672" seconds working on the request "oracle.soa.management.internal.ejb.impl.FacadeFinderBean_4vacyo_FacadeFinderBeanImpl", which is more than the configured time 

Solution(s): • Increased MaxPermSize to 1 GB (see Doc ID 1341818.1)• Increased transaction timeout settings • Applied latest recommended patches (WLS & SOA Suite)Good Book: Oracle SOA Suite 11g Performance Tuning Cookbook (by Packt Publishing)

Recent Challenges & Workarounds

Problem:• When polling SQL Server database, our clustered SOA server would

sometimes process unread records twiceResulted in duplicate transactions recorded in our ERP!

Workarounds:• Changed JDBC driver from Microsoft Type 4 to Oracle Type 4 XA

Required installing extended XA stored procedures on MS SQL Server

SOA Suite’s Facade API

• Programmatically manage SOA composites List composite / instance information

Composite StateFaulted InstancesInstance countsDefault vs. non-default revision

Stop/start composites remotely Purge instances Documentation & Examples:

http://docs.oracle.com/cd/E21764_01/integration.1111/e10226/soaadmin_apimanage.htm

http://kmatcha.wordpress.com/2012/08/09/java-client-to-purge-soa-11g-composite-instances-3/

SOA Suite’s Facade API

• JDeveloper libraries to include in CLASSPATH: $MW_HOME\oracle_common\soa\modules\

oracle.soa.mgmt_11.1.1\soa-infra-mgmt.jar $MW_HOME\oracle_common\modules\

oracle.fabriccommon_11.1.1\fabric-common.jar $MW_HOME\jdeveloper\internal\modules\external-fabric-

runtime\soa\modules\oracle.soa.fabric_11.1.1\fabric-runtime.jar $MW_HOME\oracle_common\modules\oracle.jrf_11.1.1\jrf.jar $MW_HOME\wlserver_10.3\server\lib\wlclient.jar

Demo

SOA Error Handling & Monitoring

• Initially we had good handling for “business” faults (expected errors) “Happy Path” Approach Better than nothing

• …but we weren’t handling for “technical” faults (unexpected errors)

• A good error handling & monitoring strategy accounts for both

• Runtime fault handling framework & limitations

ORA-00942: table or view does not existORA-01688: unable to extend tablespaceORA-04061: existing state of package has been invalidated

Business Fault Monitoring Approach

• We persist and communicate business faults using the EDN Send e-mail summary to business and IS support staff Generate human tasks / open IS work orders (future)

• Works great under normal circumstances…but what happens when a PL/SQL exception is raised?

Technical Fault Monitoring Approach

• We use a stand-alone Java process to check for recently faulted composite instances Connects via JDBC to SOA database Queries SOA_INFRA.COMPOSITE_INSTANCE table for

faulted instances (based on state codes found HERE) Checks every 5 minutes and sends e-mail notification “Poor Man’s” monitoring solution – has its limitations

Weblogic Server Monitoring Approach

• We monitor the SOA managed servers using OEM Grid Control 12c and WLS Management Pack

Proactive Maintenance Tips

• Oracle Fusion Middleware (FMW) SOA 11g Infrastructure Database: Installation, Maintenance, Administration and Purging Guide (Doc ID 1384379.1)

• SOA 11g: Bundle Patch Reference (Doc ID 1485949.1)• Oracle WebLogic Server PSUs (Patch Set Updates) (

Doc ID 1306505.1)• SOA Proactive Support Blog: (https

://blogs.oracle.com/soaproactive/)

Where We Are Going (Next 6+ Months)

• Leverage SOA middleware more strategically (outside of manufacturing systems integrations) Integration Technology Decision MatrixGlobal systems integration

• SOA Management Pack for Oracle Enterprise Manager Grid Control 12c

• Bus-style integrations (client call-outs)• RESTful Services for Web/Mobile UI’s• Human Tasks / Workflows (BPM?)

Questions & Comments