SonicMQ for LDIWG
Kris Kostro, Francesco CalderiniAB/CO
LayoutSonicMQ in CMW JMSCharacteristics of SonicMQConnectivity
Does it fulfill the LDIWG requirements?How could LDIWG be implemented with SonicMQ
SonicMQ in CMW
1999 requirement capture for CMW, product studies, recognized need for MOMMay 2000 Preference for JMS, product evaluationsSept. 2000 presentation by by Mitchell Horovitz, the Technical Product Manager of SonicMQFebruary 2001 SonicMQ has been selected as the messaging product for the CMW project. Purchased 4 licences.August 2001 First real use for timing events delivery
Java Message ServiceIndustry-standard messaging specification
Developed by Sun and other leading vendorsRequired component in J2EE 1.3
Common set of APIs and semanticsPublish and subscribePoint-to-point
Message delivery Quality of Service (QoS)GuaranteedOnce-and-only-onceAt-most-once
Message content (properties), message types, filtering, etc. well definedDeployment architecture not addressed
What is JMS?
SonicMQ JMS implementation
Hierarchical namespace (topics)XML support on top of text message (for DOM2, JAXP, SAX)Multiple levels of Quality of ServiceConnectivity beyond JMS
SonicMQ
JMS implementation (in Java)Market leader for JMSHub-and-spoke architectureScalable (clusters, load balancing, …)High availability (parallel servers, queues, topics, persistent topics, etc)Security (SSL, certificates, HTTP tunneling, firewalls etc.)
Integrating SonicMQ with C/C++ Clients
ConnectivityHTTPC, C++COMFTPSNMPBridges to proprietary systems (MMQ)Bridges to other JMS systems
How can SonicMQ fulfill the LDIWG requirements?
Enterprise Messaging
…and Sonic delivers!
Security
Requires…
Connectivity
Availability
Reliability
Scalability
LDIWG requires
Availability (2, 3)SonicMQ is typically used in areas which much higher availability requirementsIntrinsically high availability architecture. Brokers can be set up as redundant, can be clustered or partitioned into independent domains
Connection Management
Distribution of client connections across cluster
Connection-time load balancingOne or more brokers from list used as initial connection pointsClients redirected to other brokers via weighted round-robin algorithm
High availability of server connectionsConnect time failover
Clients connect to 1st available broker in list
Subsequent failoverReconnect on notification of connection loss
Availability
Removing Bottlenecks and Points of Failure
LDIWG requires
Reliability (4, 5)Publisher down -> watchdog mechanism (outside of SonicMQ)Control frequency -> No, should be assured by the domain, authentication possibleGuaranteed delivery possible
Guaranteed Delivery
Messages delivered even in the most challenging situations
Persistent messages are stored and flushed to disk before being acknowledgedPatent-pending storage mechanism offers reliability and high performance
Dead Message QueueIncludes large message support and client-side persistenceSupports local or distributed (2-phase) transactions
Reliable groups of actions
Reliability
Handles Failure at Any Point in Lifecycle
LDIWG requires (cont.)
Synchronisation (6)Timestamp is part of JMS (ms), in LHC all data will be time-stamped at the source
LDIWG requires (cont.)
Latency (7)Latency depends on configuration and required throughput
Performance (8)Guarantee of delivery (different levels)Throughput depends on configuration and QoSExtensive performance figures available
Built to ScaleMulti-threaded Broker
Architected for high capacity*Connections > 2000Destinations > 80,000Messages > 8,000/s (persistent)Latency < 10ms
Actual figures depend on environment
Single ClusterRequired when single broker capacity is exceededMultiple brokers in cluster act as single virtual broker
Communities of ClustersLinked via Dynamic Routing Architecture™ (DRA)
Scalability
*Tested on 4-way 550MHz Windows NT Server (1K message size)
Expanding the Cluster
No limits on cluster size
Current deployments up to 64 brokers
Clients are load-balanced across the clusterMessages routed through inter-broker connections where necessary
Head Office
Scalability
Achieve Linear Scalability
Broker ClusterBroker Cluster
BusinessApplicationBusiness
Application
BusinessApplicationBusiness
Application
BrokerBroker
BrokerBroker
Business ApplicationBusiness
Application
BusinessApplicationBusiness
Application
BrokerBroker
LDIWG requires (cont.)
Adaptability (9)Fully dynamic configuration
Protocol features(10) JMS is an industry standard(11) Supports several platforms (see list)(12) On change semantics must be assured by the producer
LDIWG requires (cont.)
Protocol features (cont.)(13) Grouping -> performance issue(14) Last published value ->posibility of persistence with timeout. Request for last value can be implemented outside of SonicMQ (15) JNDI can be used as naming and directory service (16) Hierarchical namespace with wildcards
LDIWG constrains
Constrains(17) Can do much better than 10 messages/s but it is really scalability issue(18) Can do much better for latency, again scalability issue(19) No known blocking problems(20) No flow control inside SonicMQ (domain responsibility)(21) User-based identification and topic-level authorization via ACL(23) Administration utilities available
Comprehensive Security
Encryption Built-in payload encryption
Secure messaging without performance cost of SSL
Channel encryptionSSL with up to 168-bit keys
Authentication Built-in username/password authenticationSupports digital certificates for user authentication
AuthorizationSpecify rights of authenticated users Exploit destination hierarchies and groups of users to simplify administration
Security
Flexible Deployment Options
SonicMQ Explorer
How could LDIWG be implemented with
SonicMQ
How could LDIWG be implemented with SonicMQ
Hierarchical namespace to deal with “private” domain namingXML as common, self-describing data formatBridge for each domain (SonicMQ Bridges technology?)Common administration
Example topics hierarchyCommon root CERNPartitioned into administration domains (PS, LHC, ST, Alarms, CMS ..)Every domain defines it’s own hierarchyAll accelerator domains follow the class/device hierarchy
CERNCERN
SPSSPS CMSCMS
MagnetMagnet BPMBPM Class NClass N
Device instance NDevice instance NMagnet1Magnet1 Magnet2Magnet2
STSTPSPS
Root
Domain
Class
Device
PropertyStatusStatus CurrentCurrent
Can subscribe to status of all magnets: CERN.SPS.Magnet.*.Status
How could LDIWG be implemented with SonicMQ
Hierarchical namespace to deal with “private” domain namingXML as common, self-describing data formatBridge for each domain (SonicMQ Bridges technology?)Common administration
XML as common, self-describing data format
Support within SonicMQUsed for LHC logging following String 2 experienceWill probably be used in other domains (post-mortem, alarms)Self-describing data, no need to negotiate between domains, Web support
How could LDIWG be implemented with SonicMQ
Hierarchical namespace to deal with “private” domain namingXML as common, self-describing data formatBridge for each domain (SonicMQ Bridges technology?)Common administration
Bridge for each domain
CMW – has been done in the past - easyPVSS – has been done (in one direction)DIM – easy to do as there are similar concepts (namespace) and there is JAVA APISmartsockets
SonicMQServer
SonicMQServer
SonicMQClient
SonicMQClient
SonicMQClient
SonicMQClient
CMW AgentCMW Agent
CMW
Server
CMW
Server
CMWServerCMW
Server
SonicMQ Domain Gateway
TopicTopic ListenerListener
ControlsDB
SonicMQ Bridges
Bi-directional message forwarding
Between messaging systems
Across other Internet services
Transparent to client applications
Mappings held in XML configuration file
Administration through SonicMQ
Common exception handling and logging
Connectivity to Internet services andmessaging systems
SonicMQ Bridges
SonicMQBridge
SonicMQBridge
SonicMQ MQSeriesMapping
SonicMQ MQSeriesMapping
MQSeriesSonicMQMapping
MQSeriesSonicMQMapping
SonicMQServer
SonicMQServer
SonicMQClient
SonicMQClient
SonicMQClient
SonicMQClient
MQSeriesBroker
MQSeriesBroker
MqseriesClient
MqseriesClient
MQSeriesClient
MQSeriesClient
TopicTopic
QueueQueue
XML Mapping
Files
SonicMQ Bridges
SonicMQBridge
SonicMQBridge
CMWSonicMQMapping
CMWSonicMQMapping
SonicMQServer
SonicMQServer
SonicMQClient
SonicMQClient
SonicMQClient
SonicMQClient
CMW AgentCMW Agent
CMW
Server
CMW
Server
CMWServerCMW
Server
TopicTopic
ListenerListener
XML Mapping
Files
Reasons to use SonicMQSolid industrial product by market leaderImplements standard, hence certain product independenceScalable: Start with one broker and expand later if neededInexpensiveGood connectivity (some non-standard)Fulfills LDIWG requirements and moreEasy to implement LDIWG