SONIC-7: Tuning and Scalability for Sonic Enterprise Messaging Analyzing, testing and tuning ESB/JMS performance David Hentchel Principal Solution Engineer

SONIC-7: Tuning and Scalability for Sonic Enterprise Messaging

Analyzing, testing and tuning ESB/JMS performance

David HentchelPrincipal Solution Engineer

© 2007 Progress Software Corporation2 SONIC-7: Tuning and Scalability for Sonic Enterprise Messaging

Agenda

Methodology: Review the recommended approach to project

and proceduresAnalysis: Understand how to characterize performance

requirements and platform capacityTesting: Learn how to simulate performance scenarios using

the Sonic Test HarnessTuning: Know the top ten tuning techniques for the Sonic

Enterprise Messaging backbone



D I S C L A I M E R

Setting Performance Expectations

System performance is highly dependent on machine, application and product version. Performance levels described here may or may not be achievable in a specific deployment.

Tuning techniques often interact with specific features, operating environment characteristics and load factors. You should test every option carefully and make your own decision regarding the relative advantages of each approach.

D I S C L A I M E R


Agenda

Methodology:

Review the recommended approach to project and procedures

Analysis: Understand how to characterize performance requirements and

platform capacityTesting: Learn how to simulate performance scenarios using the Sonic

Test HarnessTuning: Know the top ten tuning techniques for the Sonic Enterprise

Messaging backbone



Performance Concepts and Methodology

Terms and definitions Performance engineering concepts Managing a performance analysis project

• Skills needed for the project

• Performance Tools

• Project timeline


Performance Engineering Terms

“System Under Test”

“Test Harness”

“Load” = “Sessions” * “Delivery Rate”

“Latency” = ReceiveTime – SendTime“Test Components”

“ExternalComponents”

“Platform” “System Metric”

R

V

“Variable” =client param,app param,system param

V

V

Expert Tip: Limit scope to those test components thatare critical to performance and under your control


Concepts: Partitioning Resource Usage

% C

PU

Partitionable resources can be broken down as the sum of the contributions of each test component on the system

• Total resource usage is limited by system capacity• Becomes the bottleneck as it nears 100%• Goal is linear scalability as additional resource is added• Vertical versus Horizontal scalability

• Total latency is the sum across all resource latencies, i.e.:Latency = CPU_time + Disk_time + Socket_time + wait_sleep

X

Free

Z

Y

( Overhead X Message rate ) = Load∑(Writes/sec)(msg/sec)(writes/msg)svcs

Bottom-Up Rule: Test and tune each component before youtest and tune the aggregate.


Concepts: Computer Platform Resources

Favorite tools: task mgr, perfmon, top, ping –s, traceroute

Disk I/O(read/write)

Network I/O(send/receive)

CPU time

Memory(in use, swap)

# Threads

• Use level of detail appropriate to the question being asked• Machine resource (such as %CPU) artifacts:

• side effects from uncontrolled applications• timing of refresh interval• correlation with test intervals

• Scaling across different platforms and resource types


The Performance Engineering Project

The Project is Goal Driven

Analyze

Test

Tune

For each iteration1. Test performance vs goal2. Identify likeliest area for gain

Startup tasks1. Define project completion goals2. Staff benchmarking skills3. Acquire test harness

Expert Tip: Schedule daily meetings to share results andreprioritize test plan.


Performance Project Skills

Requirements Expert• SLA/QoS levels – minimal & optimal• Predicted load – current & future• Distribution topology

Integration Expert• Allowable design options• Cost to deploy• Cost to maintain

Testing Expert• Load generation tool• Bottleneck diagnosis• Tuning and optimization

SOLUTION

I.E.T.E.

R.E.

Cos

t/Ben

efit

Load/Distribution

Design Options


Tools for a Messaging Benchmark

Configurator – creates conditions to bring system under test into known state

Harness – the platforms and components whose performance response is being measured

Analyzer – tools and procedures to make meaningful conclusions based on result data.

Platform

Component

System Under Test

TestHarness

TestConfigurator

TestAnalyzer


Performance Project Timeline

1 2 3 4 5 6 7 8

DevelopmentProject

Week

Process Dev

Service Dev Sizing

System Test

Deployment Plan

Launch

PerformanceProjectPerf Prototype

Design Application Tuning


Agenda

Methodology: Review the recommended approach to project and

procedures

Analysis:

Understand how to characterize performance requirements and platform capacity

Testing: Learn how to simulate performance scenarios using the Sonic

Test HarnessTuning: Know the top ten tuning techniques for the Sonic Enterprise

Messaging backbone



Performance Analysis

Performance scenarios: requirements and goals• Some generic performance scenarios

System characterization: platforms and architecture

Test cases: specification for benchmark

Utilization(% total)Capacity

(units/sec)

Efficiency(units/msg)

X Load(msg/sec)

=


Performance Scenario Specification

First, triage performance-sensitive processes• substantial messaging load and latency requirements

• impact of resource-intensive services

Document only process messaging & services • leave out Application specific logic – this is a prototype

Set specific messaging performance goals• Message rate and size

• Minimum and average latency required

Try to quantify actual value and risk• Why this use case matters

Rule of Thumb: Focus on broker loads over 10 msg/sec or 1 MByte/sec,and service loads over 10,000 per hour.


Generic Scenario: Decoupled process

Asynchronous, loosely coupled, distributed services.Assumptions: Services allow concurrent, parallel distribution Messaging is lightweight, pub sub End-to-end process completes in real time May be part of a Batch To Real Time patternFactors to analyze: Speed and scalability of invoked services Distributed topology Quality of Service Aggregate Latency over time across batched invocations

Expert Tip: The easiest way to manage decoupled SOA processes isthe ESB Itinerary.


Generic Scenario: Real time data feed

High speed distribution of real time eventsAssumptions: Read-only data pushed dynamically to users Messages are small Service mediation is simple and fast Latency is very important, but QoS needs are modestFactors to analyze: Quality of Service, esp. worst case for outage and

message loss Message rate and fanout (pub sub) Scalability of consumers


Generic Scenario: Simple request reply

Typical web service call that waits for response

Assumptions: client is blocked, pending response small request message, response is larger latency is critical

Factors to analyze: Latency of each component service Load balancing of key services Recovery strategy if loop is interrupted Client network, protocol and security specs


Example: Performance scenario specification

Overall project scope:• Project goals and process• Deployment environment• System architecture

For each Scenario:• Description of business process• Operational constraints (QoS, recovery, availability)• Performance goals, including business benefits

validate CreditCard Event

make a paymentEvent

Midas BillingSystem

Make Payment with Credit Card

make_payment (Progress Adapter)

CC Validate

Payementech(Mocked Object)

Make_payement API

Client Apps

is Billing System Up? Queue

yes

no

Credit Card SyntaxValidation

1

2

3

4

5

7-n

6

8

9

10

XML validation

7


Characterizing platforms and architecture

Scope current and future hardware options available to the project

Identify geographical locations, firewalls and predefined service hosting restrictions

Define predefined Endpoints and Services Define data models and identify required

conversions and transformations.


DMZField DMZ

Platform configuration specification

CPUnumbertypespeed

Networkbandwidthlatencyspeed

Disktypespeed

Firewallcryptoslatency

Memorysizespeed

Rule of Thumb:LAN 15 – 150 MBytes / secondDisk: 2 – 10 MBytes / secondXSLT: 200 – 300 KBytes / second


Network

Memory

Disk

CPU

Platform Profile: Real-time messaging

% capacity

System resource limitations

90%

70%

20%

5%

Rule of Thumb: Real-time, 1 KB messages, broker performance is about 1,000 to 10,000 msg/sec for each 1 gHz cpu power.


Network

Memory

Disk

CPU

Platform Profile: Queued requests

% capacity

System resource limitations

50%

20%

40%

85%

Rule of Thumb: Queued msgs, 1 KB messages, broker performance is about 100 to 1,000 msg/sec for each 1 gHz cpu power.


Architecture Spec: Service distribution

Identify services performance characteristics Identify high-bandwidth message channels Decide which components can be modified Annotate with message load estimates

Back Office PartnerField Front Office

CRM

Fin

an

ce

SF

ACRM

SC

M

ESBESB

Tra

ckin

g

Ser

vice

ESB

Po

S

ERP

Adapter

SCM

Adapter

Partner ESB

Integration Broker


Architecture Spec: Data Integration

Approximate the complexity of data schemas Identify performance critical transformation events Estimate message size Identify acceptable potential services

Data Schemas 1…n Data Schemas 2…m

?

Expert Tip: Transform tools vary in efficiency:XSLT – slowest (but most standard)Semantic modeler – generally faster (e.g. Sonic DXSI)Custom text service – fastest, but not as flexible


DEMO: Test lab setup

Test hardware• guidelines for lab computers• setting up the lab network

Test architecture• location of test components• installation of brokers• configuration of service containers

Test design assets• sample service definitions & wsdls• sample test documents


Specifying Test Cases

Factors to include:• Load, sizes, complexity of messaging• Range of scalability to try (e.g. min/max msg size)• Basic ESB Process model• Basic distribution architecture

Details to avoid:• Application code (unless readily available)• Detailed transformation maps

Define relevant variables:• Fixed factors• Test Variables• Dependent measures


Typical test variables

JMS Client variables:• Connection / session usage

• Quality of Service

• Interaction mode:

• Message size and shape

ESB container variables• Service provisioning and parameters

• Endpoint / Connection parameters

• Process implementation and design

• Routing branch or key field for lookup

Expert Tip: External JMS client variables are easily managedwith the Test Harness.


Example: Test Case Specification

For each identified Test Case there is a section specifying the following: Overview of test

• How this use case relates to the scenario• Key decision points being examined

Functional definition• How to simulate the performance impact• Description of ESB processes and services• Samples messages• Design alternatives that will be compared

Test definition• Variables manipulated • Variables measured

Completion criteria• Throughput and latency goals• Issues and options that may merit investigation


Agenda


proceduresAnalysis: Understand how to characterize performance requirements and

platform capacity

Testing:

Learn how to simulate performance scenarios using the Sonic Test Harness

Tuning: Know the top ten tuning techniques for the Sonic Enterprise

Messaging backbone



Testing Performance

Staging test services in the test bed Staging brokers and containers Configuring the Sonic Test Harness Running performance tests and gathering

data Evaluating results for each test case


Staging Test ServicesDeploying existing services

Appropriate to use actual implementation of a service IF• robust implementation exists

• minimal effort to set up in test environment

• no side effects with other test components

Production ready services merit special treatment:• perform unit load tests to get baseline

• document possible tuning / scaling options


Staging Test ServicesPrototyping proposed new services

Prototype should include:• Correct routing logic for Use Case process

• Approximately correct resource usage

• Generic data

Prototype does not need:• Detailed business logic

• Exception handling code

• Invocation of non-critical library calls

It’s a prototype. Just keep it simple


Staging Test ServicesSimulating non-essential services

Use ‘stub’ service as placeholder for service step that are not performance-sensitive

Can return generic data Ensures ESB process for target use case will run

correctly Useful stub services:

• Transform service• GetFile service• PassThrough service• Enrichment service• Prototype service (version 7.5.2 or later)


Demo: Provisioning test services

StatusInfo

WSI: Address Svc

StatusRequest

StatusQuery

XForm: Build query

DBSvr: Query

TestHarness

WebPortal

Adapter: M/F Callout

AddrInfo

QueryResult

EnrichedResult

WSI: Address Svc

StatusQuery

StatusQuery

PassThru:

DBSvr: Query

PassThru: Sleep

StatusQuery

QueryResult

QueryResult

Business Use Case Performance Test Case


Provisioning test brokers

Test broker must be similar to production• Correct Sonic release and JVM• Expected deployment heap size and settings

CAA configuration• Optimize network for replication channels• Locate on separate host to avoid bottlenecks• If failover testing is part of plan:

– define fault tolerant (JNDI) connections

DRA configuration• Set up subset of clusters and WAN simulations• Measure local broker configs first, then expand

Expert Tip: For each client connection scenario define a namedJNDI Connection Factory and document its characteristics.


Provisioning ESB containers

Use ESB Container to manage service groups• name accordingly to service group role

• plan to reshuffle services during tuning phase

• provision jar files out of sonicfs

Use MF Container to control distribution• name according to domain/host location

• configure Java heap appropriately– for IBM jdk make -Xms = -Xmx– for caching services (e.g. Transform, XML Server), add

extra memory for locally-cached data

Rule of Thumb: Default heap size is fine for most ESB containers;if memory is limited, reduce size, but no smaller than:8MB + ∑(service jar size) + (#Listeners) * (200KB)


Demo: Setting up containers for test

Workbench view of containers• Coding and debugging the prototype

Runtime view of containers• managing the distributed environment

• reinitializing back to a known state


Simulating endpoint producers and consumers

Endpoint protocols and performance• Test Driver options for various protocols

Simulating process/thread configuration Implementing endpoint interaction modes Configuring client Quality of Service (QoS) Generating message content Demo of client/endpoint simulation

System Under Test

TestHarness


Endpoint protocols and performance

JMS• fastest client protocol• strongest range of QoS and Failover

HTTP• moderate performance and QoS• rigid connection model (requires client or router reconnect logic)

Web Services• slowest performance• QoS and recovery depend on WS-* extensions

File-based• flat file pickup / dropoff, FTP• limited to disk speeds (i.e. 1 to 5 MB / sec)• appropriate for batch processing scenarios

JCA• appropriate for EJB server scenarios• limited to EJB transaction speeds (i.e. 100 to 1000 msg / sec)

Rule of Thumb: HTTP acceptors typically achieve ½ to ¼ the performance of comparable JMS/TCP acceptors.


Client session configuration

Broker performance depends on scalability of connections and sessions• JMS best-practice is one thread per session

• JMS sessions can efficiently share a connection

• Use session pool for clients and app servers

For test simulation:• determine allowable range of client threads

• test connection/session numbers up to max # threads

• distribute client processes / drivers across multiple machines, if needed, to avoid client-side bottleneck

Rule of Thumb: Create one connection for every 10 to 20 sessions


Configuring client Quality of Service (QoS)

HTTP and Web Services clients• best possible QoS is at least once• even with WS-Reliability

JMS Client:• CAA w NonPersistentReplicated → exactly once• Many shared sub’s versus one durable sub• NonPersistent Request/Reply → at least once• Discardable send to avoid queue backup• Flow to disk to prevent blocked senders

ESB service:• Exactly once uses JMS transaction• At least once uses client ack• Best effort uses dups_ok ack

Broker:• Sync (default) versus Async disk i/o

Expert Tip: Best compromise is at least once QoS based on CAA, Persistent, Async disk and DUPS_OK.


Generating message content

Simulate message size / distribution for accurate results Message content may trigger ESB routing rules Some services depend on message content

• key values must match existing data / rules• duplicate key value could cause error• services that cache content require accurate key distribution

Simulating content in the client / driver• file-based message pool• message template generation• Java / object message generator• Message properties

StatusRequest

priority=2org=1

Cust Info

Sonic Test Harnesssupports all these

gen random int

gen sample xml

Addr svc lookup

transform rule


Demo: Simulating clients with Test Harness

JNDI connection configuration Producer / Consumer parameters Message generation

System Under Test

TestHarness

Connection

Broker

JNDI

MsgPool

Reply

Request

MessageGeneration


Running performance test iterations

Logistics of test orchestration• managing multiple Test Harness clients

• configuring test intervals

• test warm-up and cool-down

Data collection correlation Ensuring repeatability of results Demo of Test Harness iterations


Logistics of test orchestration

Managing multiple Test Harness clients• Simplest option is multiple command windows

– use telnet sessions for remote hosts– initiate test and warmup– hit <enter> key in each window

• Advanced environments can use distributed driver:– Grinder, SLAMD, JMeter, LoadRunner, …

Configuring test intervals• long enough to detect trend effects• short enough to allow fast iteration across tests

Test warm-up and cool-down• helps eliminate first-time test artifacts• ensures steady-state performance numbers

Rule of Thumb: Start with 10-20 intervals of 30 secs; if steady state is soonreached, reduce to 10 intervals of 10 seconds, plus warmup.


Data collection correlation

Test Harness output:• Throughput (msg/sec)

• Latency (msecs per round trip)

• Message size (bytes)

System measures:• CPU usage (%usr, %sys)

• Disk usage (writes/sec)

Broker metrics:• Messaging rate (bytes/second)

• Peak queue size (bytes)


Ensuring repeatability of results

Experimental method requirement• critical in measuring impact of change

• validate by rerunning identical test

Most common artifacts impacting repeatability• messages left in queue

• duplicate files dropped in file system

• growing database size / duplicate keys

• disconnected Durable subscribers

• cached Service artifacts (ESB default)

Expert Tip: Run initial tests twice in succession; if results differ by morethan 3% or so, investigate why.


Demo of Test Harness iterations

Baseline test Change test harness properties Rerun test Show spreadsheet across tests


Evaluating performanceMeasurement against goal

Short of goal• Perform bottleneck analysis / attempt tuning

• Review option of scaling up resources

• Review design change options

• Give up and re-think goal

Meet or exceed goal• Continue scaling up and tuning ‘til it breaks

• Give up and declare success


Evaluating performanceBottleneck analysis

Review of resource consumption• Determine cpu-bound, disk-bound, net-bound

• Identify components using the resource

• Possibility of offloading to other hosts

Examine trends in scalability tests• Possibility of improving throughput by adding more

client sessions, ESB listeners, clustered brokers, etc.

• Option of rebalancing (# threads, java heap, priority

Go through Top Ten tuning tips and others… Last resort: recode or redesign to save cycles


Evaluating performanceCompare across test runs

Carefully planned test runs yield fertile comparisons:• estimate cost/benefit of a feature or option

• estimate incremental overhead of a tunable parameter

• narrow the field of concerns and alternatives

Advice in collating and analyzing test runs• collect test summary results in spreadsheet

• save raw data and logs in a separate place

• save test config, so you can replicate later

• schedule ad hoc review after each test sequence

Expert Tip: Verify key conclusions by replicating key test runsthat led to that conclusion


Update Scenario

Demo: Example test result matrix

Query Scenario

ORX 1 KB 112 331

ORX 10 KB 146 460

ORX 100 KB 152 1197

XSVR 1 KB 121 68

XSVR 10 KB 632 139

XSVR 100 KB 2113 688

Msg S

ize

DB Svc

Thruput *

Latency **

Test 1

Test 2

Test 3

Test 4

Test 5

Test 6

* kbytes / second** milliseconds


Agenda


proceduresAnalysis: Understand how to characterize performance requirements and

platform capacityTesting: Learn how to simulate performance scenarios using the Sonic

Test Harness

Tuning:

Know the top ten tuning techniques for the Sonic Enterprise Messaging backbone



Performance Tuning with Sonic ESB®

Diagnostics• Review of ESB architecture

• Factors influencing message throughput

• Factors influencing message latency

• Factors influencing scalability

Top Ten Tuning Tips Other tuning issues

• Broker parameters

• Java tuning

• ESB tuning

• Specialized ESB services


Review ESB Architecture

SOA view

SERVICES

COMMUNICATION BACKBONE

NETWORK

SERVICECONTAINER

ESBCONTROL

System view

MessageLog DBMS Cache /

Swap

Th

read

s

VM

Th

read

s

VM

Th

read

s

VM

Th

read

s

VM

I/O Controller

Mem

ory

Network Interface

Router/Switch Router/Switch


ESB System Usage: CPU

Sources of CPU cost:

• Application code

• XML Parsing

• CPU cost of i/o

• Network sockets

• Log/Data disks• Web Service protocol

• Web Service security• Security

• Authorization• Message or channel encryption


ESB System Usage: Disk

Sources of Disk overhead:

• Database services

• Large File / Batch services

• Message recovery log

• Might not be used if other guarantee mechanisms work

• Message data store

• Disconnected consumer

• Queue save threshold overflow

• Flow to disk overflow

• Message storage overhead depends on disk sync behavior

• Explicit file synchronization ensures data retained on crash

• Tuning options at disk, filesystem, JVM and Broker levels

Expert Tip: To estimate best-case message log performance, usethe DiskPerf utility bundled with Test Harness.


ESB System Usage: Network

Sources of Network overhead:

• Client application messages and replies

• Service to service steps within the ESB (except intra-container)

• ESB Web Service callouts

• CAA broker replication messages

• Metadata (JMX, cluster, DRA) messages (normally < 1%)

Computing network bandwidth:

• Network card: 100 mbit ~= 12 MB/sec, 1 gbit ~= 120 MB/sec

• Network switches are individually rated

Computing network load:

• message rate X message size

• include response messages and intermediate steps

• add ack packets (very small) for each message send

Expert Tip: To estimate best-case network performance, usethe DiskPerf utility bundled with Test Harness.


Tip #1: Increase sender and listener threadsto make service & client scale

Increase # Listeners for key entry Endpoints Add more Service/Process instances Warning: Intra-Container ignores endpoint

• split scalable service into separate container• turn intra-container messaging off• note: sub-Process is always intra-container

Container1

ServiceX

…

Container2

ServiceX

… …

Expert Tip: Stop increasing Listeners when CPU usagenears maximum acceptable.


Tip #2: Implement optimal QoSto balance speed versus precision

ESB

QoS

MQ

SettingMessage Loss Events

Duplicate

Msg Events

N/A DiscardableBuffer overflow,

Any crashNever

Best EffortNonPersistent,

DupsOK ack

Broker crash,

Client crashNever

At Least OncePersistent,

DupsOK ackNever Never

Exactly Once Transacted Never Never

Expert Tip: With CAA configured, Best Effort service is equivalentto At Least Once, with substantially lower overhead.

(Based on CAA brokers and fault-tolerant connections)


Tip #3: Re-use JMS objectsto reduce setup costs

Objects with client and broker footprint:• Connection• Session• Sender/Receiver/Publisher/Subscriber• Temporary destination

Tuning strategies:• Reuse JMS objects in client code• Share each Connection across sessions• Share Sessions across Producers and Consumers

– but not across JVM Threads• For low-load topics/queues

– Use Anonymous Producer – Use wildcard or multi-topic subscription

Rule of Thumb: Up to 500 queues per Broker and10,000 topics per broker.


Tip #4: Use intra-container service callsto avoid broker hops

BrokerDispatch Outbox

Marshall Msg

Send Msg

…

…Dispatch Outbox Unmarshall Msg

Marshall Msg Call onMessage

Send Msg …

Receive Msg

Unmarshall Msg

Call onMessage

Instantiate Proc

…

Inter-Container Messaging

Intra-Container Messaging

v7.5: better!faster!


Tip #5: Use NonPersistentReplicated modeto reduce disk overhead

Normal broker mechanisms require disk sync• contributes to latency across the board

• interferes with batching of packets

• limited bandwidth

Disabling disk sync eliminates this overhead• Send mode NonPersistentReplicated

• Optional broker params to disable entirely

• WARNING: Log-based recovery will lose recent messages

• BUT: CAA failover will not

Rule of Thumb: Network overhead for Replication channel isabout ½ the Persistent Msg load of the broker.


Tip #6: Use XCBR instead of CBRto eliminate Javascript overhead

CBR rules implemented via Javascript• dynamic change with complex rules

• very high overhead for runtime engine

XCBR rules extract data fields for comparison• only simple comparisons supported

• no script engine overhead

• use message property data key for best effect

Rule of Thumb: Invocation of the Rhino javascript engine requiresabout 100 milliseconds cpu time on a typical platform.


Tip #7: Use message batchingto accelerate message streams

Message transfer overhead is generally fixed Hidden ack messages amenable to tuning:

• AsyncNonPersistent mode decouples ack latency• Transaction Commit allows 1 ack per N messages• DupsOK ack mode allows ‘lazy’ ack from consumer• Pre-Fetch Count allows batched transmit to consumer

ESB Design option: send one multi-part message instead of N individual messages

XML transforms and other services handle multi-record data efficiently

Producer Broker Consumer

Msg

Msg

…Ack

Msg

Msg

Ack

…


Tip #8: Minimize XML/SOAP operationsto avoid parsing overhead

Bypass SOAP and Web Services processing• Use HTTP Direct Basic instead of SOAP or WS• Risk of invalid XML if source is unreliable

Combine multiple XML parsing steps into one• Save target XPath results as Message props• Also relevant for BPEL correlation ID’s

InputMessage

XMLTransform

CustomJAXB

InputMessage

propY=9propX=1

InputMessage

JAXB ObjMsg part

XCBR

JAXBService


Tip #9: Use high-speed encryptionto reduce security overhead

Default SSL encryption uses old RCA stack• At least 2X slower than more modern options

Change to any JSSE compliant stack:• modify client DSSL_PROVIDER_CLASS to:

progress.message.net.ssl.jsse.jsseSSLImpl

• change broker SSL provider from RSA to JSSE

Use more efficient cipher suites:• RSA_With_Null_MD5 is the smallest and fastest

Reduce broker memory overhead by deleting any unused ciphers


Tip #10: Use stream API'sto improve large message performance

SonicMQ Recoverable File Channels• Uses JMS layer to manage large file xfer• Queue-based initiation of transfer• High-speed JMS pipeline for blocks of data• Recovery continues at point interrupted

Sonic ESB open-source Large Message Service• Provides dynamic provisioning• Interacts with ESB processes

SonicStream API (version 7.5 or later)• Topic-based, pipeline into Java Stream api• No recovery


Broker Tuning Parameters

Core Resources:• JVM heap size• JVM thread, stack limits• DRA, HTTP and Transaction threads• TCP settings

Message storage:• Queue size and save threshold• Pub/sub buffers• Message log and store

Message management• Encryption• Flow control and flow-to-disk• Dead message queue management

Connections• Mapping to NIC’s• Timeouts• Load balancing

Rule of Thumb: For non-trivial queues, multiply defaultsettings by 10 to 100.


Java Tuning Options

‘Fastest’ JVM depends a little on the application and a lot on the platform

VM heap needs to be large enough to process load, but small enough to avoid system swapping

Garbage Collection:• default settings good for optimal throughput• use advanced (jdk4 or later) GC options to

optimize worst-case latency

Rule of Thumb: On windows platforms, the Sun 1.5.0 JVM is10% to 50% slower than the default IBM 1.4.2 JVM.


ESB Tuning Options

Load balancing and scalability of services:• number of distributed service instances

• number of service listener threads

Container Java VM heap size Intra-Container messaging Endpoint and connection parameters

• same principles as JMS client

Expert Tip: Start with small Java heap and only increase -Xms size if it improves performance.


Discussion of Service tuning

Transformations XML Server BPEL Server Database Service DXSI Service


Other fun things you can tune

Database: indexing, query optimization SOA patterns: federated query, temporal

aggregation, split/join, caching XML: DOM, SAX, XStream Disk: Device balancing, RAID, mount params Network: Nagle algorithm, timeouts

Expert Tip: If you save service resources in sonicfs, the ESBcontainer will dynamically load and cache them.


Questions?


Thank you foryour time


Documents

SONIC-7: Tuning and Scalability for Sonic Enterprise Messaging Analyzing, testing and tuning ESB/JMS performance David Hentchel Principal Solution Engineer