SonicMQ for LDIWG

SonicMQ for LDIWG

Kris Kostro, Francesco CalderiniAB/CO

LayoutSonicMQ in CMW JMSCharacteristics of SonicMQConnectivity

Does it fulfill the LDIWG requirements?How could LDIWG be implemented with SonicMQ

SonicMQ in CMW

1999 requirement capture for CMW, product studies, recognized need for MOMMay 2000 Preference for JMS, product evaluationsSept. 2000 presentation by by Mitchell Horovitz, the Technical Product Manager of SonicMQFebruary 2001 SonicMQ has been selected as the messaging product for the CMW project. Purchased 4 licences.August 2001 First real use for timing events delivery

Java Message ServiceIndustry-standard messaging specification

Developed by Sun and other leading vendorsRequired component in J2EE 1.3

Common set of APIs and semanticsPublish and subscribePoint-to-point

Message delivery Quality of Service (QoS)GuaranteedOnce-and-only-onceAt-most-once

Message content (properties), message types, filtering, etc. well definedDeployment architecture not addressed

What is JMS?

SonicMQ JMS implementation

Hierarchical namespace (topics)XML support on top of text message (for DOM2, JAXP, SAX)Multiple levels of Quality of ServiceConnectivity beyond JMS

SonicMQ

JMS implementation (in Java)Market leader for JMSHub-and-spoke architectureScalable (clusters, load balancing, …)High availability (parallel servers, queues, topics, persistent topics, etc)Security (SSL, certificates, HTTP tunneling, firewalls etc.)

Integrating SonicMQ with C/C++ Clients

ConnectivityHTTPC, C++COMFTPSNMPBridges to proprietary systems (MMQ)Bridges to other JMS systems

How can SonicMQ fulfill the LDIWG requirements?

Enterprise Messaging

…and Sonic delivers!

Security

Requires…

Connectivity

Availability

Reliability

Scalability

LDIWG requires

Availability (2, 3)SonicMQ is typically used in areas which much higher availability requirementsIntrinsically high availability architecture. Brokers can be set up as redundant, can be clustered or partitioned into independent domains

Connection Management

Distribution of client connections across cluster

Connection-time load balancingOne or more brokers from list used as initial connection pointsClients redirected to other brokers via weighted round-robin algorithm

High availability of server connectionsConnect time failover

Clients connect to 1st available broker in list

Subsequent failoverReconnect on notification of connection loss

Availability

Removing Bottlenecks and Points of Failure

LDIWG requires

Reliability (4, 5)Publisher down -> watchdog mechanism (outside of SonicMQ)Control frequency -> No, should be assured by the domain, authentication possibleGuaranteed delivery possible

Guaranteed Delivery

Messages delivered even in the most challenging situations

Persistent messages are stored and flushed to disk before being acknowledgedPatent-pending storage mechanism offers reliability and high performance

Dead Message QueueIncludes large message support and client-side persistenceSupports local or distributed (2-phase) transactions

Reliable groups of actions

Reliability

Handles Failure at Any Point in Lifecycle

LDIWG requires (cont.)

Synchronisation (6)Timestamp is part of JMS (ms), in LHC all data will be time-stamped at the source


Latency (7)Latency depends on configuration and required throughput

Performance (8)Guarantee of delivery (different levels)Throughput depends on configuration and QoSExtensive performance figures available

Built to ScaleMulti-threaded Broker

Architected for high capacity*Connections > 2000Destinations > 80,000Messages > 8,000/s (persistent)Latency < 10ms

Actual figures depend on environment

Single ClusterRequired when single broker capacity is exceededMultiple brokers in cluster act as single virtual broker

Communities of ClustersLinked via Dynamic Routing Architecture™ (DRA)

Scalability

*Tested on 4-way 550MHz Windows NT Server (1K message size)

Expanding the Cluster

No limits on cluster size

Current deployments up to 64 brokers

Clients are load-balanced across the clusterMessages routed through inter-broker connections where necessary

Head Office

Scalability

Achieve Linear Scalability

Broker ClusterBroker Cluster

BusinessApplicationBusiness

Application


Application

BrokerBroker

BrokerBroker

Business ApplicationBusiness

Application


Application

BrokerBroker


Adaptability (9)Fully dynamic configuration

Protocol features(10) JMS is an industry standard(11) Supports several platforms (see list)(12) On change semantics must be assured by the producer


Protocol features (cont.)(13) Grouping -> performance issue(14) Last published value ->posibility of persistence with timeout. Request for last value can be implemented outside of SonicMQ (15) JNDI can be used as naming and directory service (16) Hierarchical namespace with wildcards

LDIWG constrains

Constrains(17) Can do much better than 10 messages/s but it is really scalability issue(18) Can do much better for latency, again scalability issue(19) No known blocking problems(20) No flow control inside SonicMQ (domain responsibility)(21) User-based identification and topic-level authorization via ACL(23) Administration utilities available

Comprehensive Security

Encryption Built-in payload encryption

Secure messaging without performance cost of SSL

Channel encryptionSSL with up to 168-bit keys

Authentication Built-in username/password authenticationSupports digital certificates for user authentication

AuthorizationSpecify rights of authenticated users Exploit destination hierarchies and groups of users to simplify administration

Security

Flexible Deployment Options

SonicMQ Explorer

How could LDIWG be implemented with

SonicMQ

How could LDIWG be implemented with SonicMQ

Hierarchical namespace to deal with “private” domain namingXML as common, self-describing data formatBridge for each domain (SonicMQ Bridges technology?)Common administration

Example topics hierarchyCommon root CERNPartitioned into administration domains (PS, LHC, ST, Alarms, CMS ..)Every domain defines it’s own hierarchyAll accelerator domains follow the class/device hierarchy

CERNCERN

SPSSPS CMSCMS

MagnetMagnet BPMBPM Class NClass N

Device instance NDevice instance NMagnet1Magnet1 Magnet2Magnet2

STSTPSPS

Root

Domain

Class

Device

PropertyStatusStatus CurrentCurrent

Can subscribe to status of all magnets: CERN.SPS.Magnet.*.Status



XML as common, self-describing data format

Support within SonicMQUsed for LHC logging following String 2 experienceWill probably be used in other domains (post-mortem, alarms)Self-describing data, no need to negotiate between domains, Web support



Bridge for each domain

CMW – has been done in the past - easyPVSS – has been done (in one direction)DIM – easy to do as there are similar concepts (namespace) and there is JAVA APISmartsockets

SonicMQServer

SonicMQServer

SonicMQClient

SonicMQClient

SonicMQClient

SonicMQClient

CMW AgentCMW Agent

CMW

Server

CMW

Server

CMWServerCMW

Server

SonicMQ Domain Gateway

TopicTopic ListenerListener

ControlsDB

SonicMQ Bridges

Bi-directional message forwarding

Between messaging systems

Across other Internet services

Transparent to client applications

Mappings held in XML configuration file

Administration through SonicMQ

Common exception handling and logging

Connectivity to Internet services andmessaging systems

SonicMQ Bridges

SonicMQBridge

SonicMQBridge

SonicMQ MQSeriesMapping

SonicMQ MQSeriesMapping

MQSeriesSonicMQMapping

MQSeriesSonicMQMapping

SonicMQServer

SonicMQServer

SonicMQClient

SonicMQClient

SonicMQClient

SonicMQClient

MQSeriesBroker

MQSeriesBroker

MqseriesClient

MqseriesClient

MQSeriesClient

MQSeriesClient

TopicTopic

QueueQueue

XML Mapping

Files

SonicMQ Bridges

SonicMQBridge

SonicMQBridge

CMWSonicMQMapping

CMWSonicMQMapping

SonicMQServer

SonicMQServer

SonicMQClient

SonicMQClient

SonicMQClient

SonicMQClient

CMW AgentCMW Agent

CMW

Server

CMW

Server

CMWServerCMW

Server

TopicTopic

ListenerListener

XML Mapping

Files

Reasons to use SonicMQSolid industrial product by market leaderImplements standard, hence certain product independenceScalable: Start with one broker and expand later if neededInexpensiveGood connectivity (some non-standard)Fulfills LDIWG requirements and moreEasy to implement LDIWG

Documents

SonicMQ for LDIWG