Building With Open MQ

Linda SchneiderTechnical LeadSun Microsystems, Inc.

Using OpenMQ

What will be covered ?

An introduction to OpenMQ. A Customer example. Basic customer requirements. Building part of the infrastructure.

Warnings:No in depth coverage Assumes basic JMS knowledge

What is OpenMQ ?

Allows heterogenous applications to reliability and asynchronously pass data between each other.Open Source Java Message Service (JMS) implementation (+ additions)Default Messaging Provider for GlassfishUseful on its own for standalone JMS applicationsEnterprise level quality (>8 years in development)Open Source since JavaOne 2006Available as a supported product: Sun Java System Message Queue (SJSMQ)

Overview

Tell Me More

Developer and User discussion forumsStable builds with product releasesEarly access, promoted builds availableNew features, and fixes

Dual license support (GPL v2 and CDDL)Open source version of Java MQ is available from http://mq.dev.java.net

Using OpenMQ ?

An example

To mimic problems faced in designing applications, an example:Represents a complex system with loosely connected applicationsUtilizes various types of messagingIs easy to understandIs at least minimally interesting

Our Example: Santa Claus, Inc

Why, you ask ?

Even if you don't believe in Santa Claus, you must still understand that delivering all those presents would be a daunting taskAnd while its not Christmas which comes but once a year, but requires year round planning and preparation.Just because Santa Claus lives at the North Pole, doesn't imply he can't use technology

Overall System Requirements

Santa Claus, Inc. software applications need do the following:Handle gift selection and deliveryManage resources e.g.giftsreindeerElves

Track general status informationhow many days before Christmasetc.

What are we doing:

Focusing on handling christmas gift processingSteps to design it include:Determining the high level operationComing up with the name and type of destinationsDetermining models used for the messagingDetermining load characteristicsLooking at code for some components

Defining the High Level Operation of the System

What do we need to do ?

What do we need to do ? (cont.)

What are the destinations ?

The Child Queue

The Naughty/Nice Queues

The Wrap Queue

The Stuff to Pack Queue

Select-a-gift Queues

Topic LogChild

A quick overview to design patterns

Some basic design patterns:Pattern Description

Request/Reply

Step OperationsBroadcast One message goes to many consumers

ConduitBatch Messages are processed in a chunk

Time Critical/Sensitive

Message is sent to another application who sends back a responseMessages go through several iterations, the message is persisted at key points where processing it again would be expensive

Multiple consumers send messages to a single destination

Messages must be processed within a short period of time (e.g. under an hour) and can not be lost

More things to think about:

Use persistent messages if it can not afford to be lostUse non-persistent messages for:

non-critical step messages (when it can be repeated)Request/ReplyAnytime a message can be lost on a server crash

Use durables for Topics when it may need to be retrieved laterUse normal or XA transactions when multiple things must process together:

XA if it includes other resources like databases

Processing Queue ChildConduit: many producers to one queuePersistent: would be time consuming to lose message

Processing Naughty and NiceStep Pattern: one step of itNaughty Queue: Non-Persistent

its OK if a child who is bad misses their coal

Nice Queue: Persistent. They must get their present.

Processing NiceStep Pattern: more steps of itMultiple resources so XA

Processing Nice (Select a gift)Request/Reply PatternNon-persistentAction repeated on failure

The Wrap Queue Step operationPersistent: end of an expensive set of steps that they don't want to repeat

The Log Child Topic

Broadcast PatternPersistent because santa wants his database accurate

In this example:

The batch pattern was not usedSanta does use It for processing HR updates for the elves

The time sensitive/critical data pattern was not used:Santa does use it during present delivery on christmas eve

to track where he is

Because he has no time sensitive/critical data, reliability is important however data availability isn't

Performance Requirements

22 billion kids364 days for preparation (since christmas is taken)

31,526,000 seconds

70 children/second must be processedAssume 60% are “nice”Assume 40% downtime to cover outages and normal processing (so goal is approx 100 kids/second)

Performance Requirements (cont)

Naughty Kids use1 Persistent queue (child)1 Non-persistent queue (naughty)1 Persistent Topic (log child)

Nice Kids Use:Persistent Queue (child)Persistent Queue (nice)2 Non-Persistent Queues (Inventory request and reply

queues)Non-persistent queue (Wrap)1 Persistent Topic (log child)

The cold hard requirements

Messages:Child: 100 msgs/second (persistent)Naughty: 40 msgs/second (non-persistent)Log Child: 100msgs/second (persistent)Nice: 60 msgs/second (persistent in XA transaction)Inventory request/reply: 60 msgs/second *2 (non-

persistent)Wrap: 60 msgs/second (persistent)

TOTALS:Persistent: 380 msgs/secondNon-persistent: 160 msgs/second

Some Sample Code

Sending the “Child” messagepublic void doPost(HttpServletRequest request,

HttpServletResponse response) throws ServletException, IOException {

// retrieve initial context (ic) QueueConnectionFactory qcf = (QueueConnectionFactory) ic.lookup("MyConnectionFactory"); Queue destQueue = (Queue)ic.lookup("Child"); QueueConnection connection = qcf.createQueueConnection(); try { QueueSession session = connection.createQueueSession( False, Session.AUTO_ACKNOWLEDGE); QueueSender sender = session.createSender(destQueue); MapMessage msg = session.createMapMessage(); // Set each item msg.setString(“firstname”, request.getParameter(“firstname”)); // … retrieve other properties … ; sender.send(msg); } finally { connection.close(); }}

Processing the “nice” queuepublic void onMessage(Message inMessage) { TextMessage msg = null; try { //Message is of type text and has a unique child id msg = (TextMessage) inMessage; String id = msg.getText(); String[] list = db.getList(id); // makes SQL call String item = null; if (list == null) { // no list, send request String item = getListItem(); //next slide } else { item = list[0]; } //update inventory db.updateInventory(item, id);//makes SQL call // put on packing list pack(item, id); } catch (Exception e) { // things went wrong roll back mdc.setRollbackOnly(); }}

Processing the “nice” queue (step 2)public String getListItem(String childid) throws Exception { QueueConnectionFactory factory = jndiContext.lookup(“MyQueueFactory”); QueueConnection qc = factory.createQueueConnection(); qc.start(); QueueSession session = qc.createSession(true, Session.AUTO_ACKNOWLEDGED); Queue q = session.createQueue(“RandomPresent”); Queue reply = session.createTemporaryQueue(); // get sender and receiver QueueSender sender = session.createSender(q); QueueReceiver receiver = session.createReceiver(q); //send message and wait TextMessage m = session.createTextMessage(childid); m.setJMSReplyTo(reply); //send the message sender.send(m); TextMessage back = (TextMessage) receiver.receive(60*1000); // wait a minute if (back == null) { didn't get anything throw new Exception(“Nothing”); return back.getText();}

Processing the “nice” queue (step 3)

public String pack(String item, String child_id) throws JMSException{ QueueConnectionFactory factory = jndiContext.lookup(“MyQueueFactory”); QueueConnection qc = factory.createQueueConnection(); QueueSession session = qc.createSession(true, Session.AUTO_ACKNOWLEDGED);); Queue q = session.createQueue(“Pack”);

// get sender QueueSender sender = session.createSender(q);

//send message MapMessage m = session.createMapMessage(childid); m.setString(“child_id”, child_id); m.setString(“present”, item); sender.send(m);}

You'll need to fill in the rest

More Information

OpenMQ -- More Information

Visit the product webpagehttp://sun.com/software/products/message_queue

Join the Open Message Queue projecthttps://mq.dev.java.net

Browse the product documentationhttp://docs.sun.com/app/docs/coll/1307.3

Take the free technical traininghttp://www.sun.com/training/catalog/courses/WMT-SMQ-1491.xml

Related Information

Java Composite Application Platform Suitehttp://sun.com/software/javaenterprisesystem/javacaps/

Java System Identity Managerhttp://sun.com/software/products/identity

Project GlassFish https://glassfish.dev.java.net/

The Aquarium, A community forumhttp://blogs.sun.com/theaquarium/

Using OpenMQ

Linda Schneiderlinda.schneider@sun.com

Thank You!

Building With Open MQ

Technology

Building With Open Mq V2

Redhat Open Day - Integracion JBoss Fuse A-MQ

WebSphere MQ 101 - SHARE · WebSphere MQ 101 2 Agenda Introduction Fundamentals ... WebSphere MQ provides the reliable communications and administration ... – While MQ can,

MQ Manager v14 - HelpSystems...Introduction to MQ Manager Overview Halcyon MQ Manager - 6 - Introduction to MQ Manager Overview IBM WebSphere® MQ (formerly MQ Series®) Software that

Enterprise Messaging Infrastructure and use with SIB, MQ ...€¦ · MQ Link Messaging Engine WebSphere MQ 1. Create Foreign Bus 2. Create an MQ Link 3. Create a MQ Link Receiver

MQTT MQ Telemetry Transport MQTT · PDF fileMQTT –MQ Telemetry Transport Peter R. Egli INDIGOO.COM ... (Mobile to Mobile), WSN (Wireless Sensor Networks) ... Building control application

MQ PCF Programming - MQ Technical · PDF fileCapitalware's MQ Technical Conference v2.0.1.5 MQ PCF Programming Mark Taylor, IBM Hursley marke_taylor@uk.ibm.com

June 2014 - Building Rabbit MQ based chat on Android

Amazon MQ - Developer Guide · PDF fileAmazon MQ Developer Guide What Are the Main Beneﬁts of Amazon MQ? What Is Amazon MQ? Amazon MQ is a managed message broker

CS 1 A BCD H Eold.comune.casale-monferrato.al.it/PUT/PEC_exIBL/2.12-Tav... · 2019. 12. 5. · cs 1 mq 369 cs 2 mq 597 vi.pr. mq 2.335 vr.pr.1 mq 2.235 vr.pr.2 mq 200 cs 3 mq 131

IBM MQ: Using Publish/Subscribe in an MQ Network

IBM IMPACT 2014 - AMC-1882 Building a Scalable & Continuously Available IBM MQ Infrastructure

WebSphere MQ Queue Manager Clusters - Capitalware's MQ

A-MQ 7 Sneak Preview - The world's open source leader · A-MQ 7 Sneak Preview David Ingham Senior Engineering Manager June 30, 2016. ... •Application provisioning and monitoring

Connecting MQ to the World: New Bridges · Connecting MQ to the World: New Bridges ... • And open-source connectors for Kafka Introduction. ... • MQ is a natural fit to connect

Flexible MQ Topologies in IBM Integration Bus V10...Capitalware's MQ Technical Conference v2.0.1.5 MQ Environments & Topologies Existing MQ infrastructure, new Integration Node Requirement

IBM MQ update including MQ and security - WebSphere · PDF file@MoragHughson #wuguk MQ Appliance @MoragHughson #wuguk Introducing IBM MQ Appliance • The scalability and security

MQ Manager v15 - HelpSystems · Introduction to MQ Manager Overview Halcyon MQ Manager v15.0 - 6 - Introduction to MQ Manager Overview IBM WebSphere® MQ (formerly MQ Series®) Software

websphere mq admin training | ibm mq online training | websphere mq online training

WebSphere MQ Internals, Performance and Tuningusuaris.tinet.cat/japlan/docs/mq/mq internals and performance-mark... · 1 WebSphere MQ Internals, Performance and Tuning Mark Taylor