User Guide - HUAWEI CLOUD...Contents 1 Getting Started.....1

Distributed Message Service

User Guide

Date 2018-10-29

Contents

1 Getting Started............................................................................................................................... 11.1 Creating a Queue............................................................................................................................................................ 11.2 Creating a Consumer Group........................................................................................................................................... 31.3 Creating a Message.........................................................................................................................................................41.4 Retrieving Messages.......................................................................................................................................................6

2 Managing Queues and Messages............................................................................................. 102.1 Viewing Details of a Queue..........................................................................................................................................102.2 Deleting Queues............................................................................................................................................................112.3 Deleting a Consumer Group......................................................................................................................................... 112.4 Resetting the Message Retrieval Start Position............................................................................................................122.5 Managing Queue Policies.............................................................................................................................................142.6 Querying Messages.......................................................................................................................................................162.7 Viewing the Retrieval Status........................................................................................................................................ 18

3 Managing RabbitMQ Instances................................................................................................203.1 Preparing the Environment...........................................................................................................................................203.2 Creating an Instance..................................................................................................................................................... 213.3 Viewing an Instance......................................................................................................................................................243.4 Restarting an Instance...................................................................................................................................................253.5 Deleting an Instance..................................................................................................................................................... 263.6 Modifying the Information About an Instance............................................................................................................. 283.7 Accessing a RabbitMQ Instance...................................................................................................................................293.8 Connecting to the Management Address of a RabbitMQ Instance.............................................................................. 313.9 Accessing a RabbitMQ Instance in SSL Mode............................................................................................................ 313.10 Resetting the Password of an Instance....................................................................................................................... 333.11 Enabling Public Access to an Instance....................................................................................................................... 34

4 Managing Kafka Instances........................................................................................................ 374.1 Preparing the Environment...........................................................................................................................................374.2 Creating an Instance..................................................................................................................................................... 384.3 Viewing an Instance......................................................................................................................................................414.4 Restarting an Instance...................................................................................................................................................424.5 Deleting an Instance..................................................................................................................................................... 434.6 Modifying the Information About an Instance............................................................................................................. 45

Distributed Message ServiceUser Guide Contents

2018-10-29 ii

4.7 Creating a Topic in a Kafka Platinum Instance............................................................................................................ 464.8 Deleting a Topic in a Kafka Platinum Instance............................................................................................................ 474.9 Accessing a Kafka Platinum Instance.......................................................................................................................... 484.10 Connecting to the Kafka Manager..............................................................................................................................494.11 Accessing a Kafka Platinum Instance in SASL_SSL Mode.......................................................................................504.12 Resetting the Password of an Instance....................................................................................................................... 52

5 Monitoring.................................................................................................................................... 545.1 DMS Metrics................................................................................................................................................................ 545.2 Setting DMS Alarm Rules............................................................................................................................................585.3 Viewing Metrics on the Cloud Eye Console.................................................................................................................59

6 Auditing........................................................................................................................................ 616.1 Operations That Can Be Recorded by CTS..................................................................................................................616.2 Viewing Traces on the CTS Console............................................................................................................................62

7 FAQs...............................................................................................................................................647.1 Does DMS Support FIFO Delivery?............................................................................................................................ 647.2 Does DMS Support "At Least Once" Message Delivery?............................................................................................657.3 How Long Can a Message be Retained in a Queue?....................................................................................................657.4 What Is the Maximum Size of a Message?.................................................................................................................. 657.5 Does DMS Guarantee Zero Message Repetition?........................................................................................................657.6 How Does DMS Ensure Message Security in a Queue?.............................................................................................. 657.7 How Many Queues Can Be Created Under a Project?.................................................................................................667.8 Why Can't I Retrieve the Chosen Number of Messages from the Queue That Contains Only a Few Messages?.......667.9 Why Do Consumers Need to Acknowledge Messages?.............................................................................................. 677.10 Will Duplicate Messages Be Retrieved After "Delay Message Delivery" Is Enabled?............................................. 677.11 Will Duplicate Messages Be Retrieved After Redelivery?.........................................................................................687.12 What Are the Differences Between Kafka Queues and Kafka Platinum Instances?..................................................687.13 Do RabbitMQ or Kafka Platinum Instances Support Cross-VPC Access..................................................................687.14 Do RabbitMQ or Kafka Platinum Instances Support Cross-Subnet Access?............................................................ 697.15 How to Select and Configure a Security Group?....................................................................................................... 697.16 Can I Access DMS Over a Public Network?..............................................................................................................717.17 Where Does a Newly Created Consumer Group Start Message Retrieval?...............................................................717.18 How Is the Performance of Global FIFO Queues?.....................................................................................................727.19 Why Can't I View the Subnet and Security Group Information When Creating a DMS Instance?........................... 72

8 Change History............................................................................................................................ 73

Distributed Message ServiceUser Guide Contents

2018-10-29 iii

1 Getting Started

This chapter describes operations on standard, ActiveMQ, and Kafka queues on the DMSconsole. For details about operations on RabbitMQ queues, see 3 Managing RabbitMQInstances. For details about operations on Kafka platinum queues, see 4 Managing KafkaInstances.

1.1 Creating a Queue

1.2 Creating a Consumer Group

1.3 Creating a Message

1.4 Retrieving Messages

1.1 Creating a Queue

ScenarioThe first task in using DMS is to create one or more queues.

PrerequisitesNone.

Procedure

Step 1 Log in to the management console.

Step 2 Click in the upper left corner of the management console and select a region.

Step 3 Click Service List, and choose Application > Distributed Message Service to launch theDMS console.

Step 4 In the navigation pane, choose Queue Manager.

Step 5 Click Create Queue.

The remaining number of queues that can be created is displayed above the Delete button. Bydefault, 30 queues can be created under a project. To create more than 30 queues, contactcustomer service to increase your quota.

Distributed Message ServiceUser Guide 1 Getting Started

2018-10-29 1

Step 6 Specify queue parameters.

Table 1-1 Parameter description

Parameter Description

Region Region in which the new queue will be created.

Queue Name Unique name of a queue.When creating a queue, a default queue name is generated, which youcan change if required.A queue name is 1 to 64 characters long. Only letters, digits,underscores (_), and hyphens (-) are allowed.The Queue Name parameter cannot be modified after creation of thequeue.

Queue Type l StandardPartition-level FIFO mode and global FIFO mode are supported.

l ActiveMQQueues that use the Advanced Message Queuing Protocol(AMQP). AMQP, an advanced message queue protocol at theapplication layer of the unified messaging service, is an openstandard application layer protocol for message-orientedmiddleware.

l KafkaThe open source Kafka client can be used to create and retrievemessages. High throughput and high reliability are supported.

The default queue type is Standard.

Queue Mode When the queue type is standard, the queue mode can be:l Partition-level FIFO: Messages may be retrieved out of order,

but the concurrent performance is higher.l Global FIFO: Messages are retrieved in the order they were sent.When the queue type is Kafka, the queue mode can be:l High throughput

All message replicas are flushed to disk asynchronously. Selectthe high throughput mode when message delivery performance isimportant.

l High reliabilityAll message replicas are flushed to disk synchronously. Select thehigh reliability mode when message reliability is important.


2018-10-29 2


Dead Letter Queue This parameter is displayed only if the Queue Type is set toStandard.An indicator of whether the dead letter queue is enabled. By default,the dead letter queue is disabled.Dead letter messages are messages that cannot be correctly retrieved.If a message fails to be retrieved for a preset number of times, themessage will be sent to the dead letter queue and retained in the deadletter queue for at least 72 hours. You can then retrieve the messagefrom the dead letter queue.Messages in the dead letter queue are specific to individual consumergroups, and consumer groups cannot retrieve each other's dead lettermessages.Dead letter messages from global FIFO queues are sent to the deadletter queue in the FIFO order.

Message RetentionPeriod (h)

This parameter is displayed only if the Queue Type is set to Kafka.The number of hours for which messages will be preserved in aKafka queue. Messages older than that period will be deleted anddeleted messages are not retrievable to consumer groups.Value range: 1 to 72Default value: 72

MaximumRetrievals

This parameter is displayed only if the Dead Letter Queue isenabled.The maximum number of times a message can be retrieved before itis sent to the dead letter queue.Value range: 1 to 100Default value: 3

Description(optional)

Queue description.The description is 0 to 160 characters long and cannot contain anglebrackets (< and >).

Step 7 Click OK to create the queue.

After the queue is created, you can click the name of a queue to view queue details.

----End

1.2 Creating a Consumer Group

ScenarioA consumer group must be created in order to retrieve messages. A maximum of threeconsumer groups can be created in each queue.

Messages in a queue are retrievable to all consumer groups created in that queue.


2018-10-29 3

Prerequisites

A queue has been created.

Procedure





Step 5 Open the Create Consumer Group dialog box using either of the methods:l Method 1

In the queue list, choose More > Create Consumer Group in the same row as the queuefor which you want to create a consumer group.

l Method 2

a. Click the name of a queue for which you want to create a consumer group.Details of the chosen queue are displayed.

b. On the queue details page, click the Consumer Groups tab.c. Click Create Consumer Group.

Step 6 Specify Consumer Group Name.

A default consumer group name is generated, which you can change if required.

A consumer group name is 1 to 32 characters long. Only letters, digits, underscores (_), andhyphens (-) are allowed. Consumer group names must be unique within their queue.

Step 7 Click OK.

----End

1.3 Creating a Message

Scenario

Messages can be sent to a chosen queue. The size of each message, including the messagebody and attributes, cannot exceed 512 KB. The body includes the message content and a fewbytes of additional JSON information.

If the Kafka SDK is used to produce messages, the maximum size of a single message is 10MB. If the DMS console is used to produce messages, the maximum size of a single messageis 512 KB.

Prerequisites



2018-10-29 4

Procedure





Step 5 Open the Create Message dialog box using either of the methods:

l Method 1

In the queue list, click Create Message in the same row as the queue to which messageswill be sent.

l Method 2

a. Click the name of the queue to which messages will be sent.

b. Click Create Message in the upper right corner of the queue details page.

Step 6 Specify the Message Body and optional Message Labels. If you want to add attributes,specify Name and Value, and then click Add.



Message Body Body of the message to be sent.When using the DMS console to send messages, characters that arereserved in the JSON format will be escaped in the message bodycontent. For example, she said "it's not you, it's me" would beescaped to she said \"it's not you, it's me\".

Delay MessageDelivery

An indicator of whether message delivery delay is enabled.Delivery of messages added to this queue will be delayed for thechosen amount of time.Only standard queues support message delivery delay.

Delivery Delay Amount of time to delay delivery of all messages added to thisqueue.Values: 1 minute, 5 minutes, 30 minutes, 1 hour, or 1 dayDefault value: 5 minutes

Message Labels(optional)

You can use labels to filter the messages you want to retrieve fromthe chosen queue.Each message has a maximum of three labels.A message label is 1 to 64 characters long. Only letters, digits,hyphens (-), and underscores (_) are allowed.Only messages in standard queues have labels.


2018-10-29 5


Message Attributes(optional)

Attribute of a message, which is composed of the name and value.Message attributes are in the key-value format. Attribute names aremandatory and must be unique in the same message.If you enter attribute names or values on the DMS console, spacesare allowed in attribute names and values, but not before and afterthem. For example, " ab c " is converted to "ab c".NOTE

If you use APIs to access DMS, spaces are allowed in, before, and afterattribute names and values.

Message attributes are supplementary to the message body and canbe sent and retrieved along with it. You can specify multiplemessage attributes, as long as the size of the message remainswithin 512 KB.Only messages in standard queues have attributes.

Successfully added attributes will be displayed in the list. To delete an attribute, click Deletein the same row as the attribute.

Step 7 Click OK.

In the queue list, the value of Messages for the chosen queue will increment by one.

In the consumer group list of the chosen queue, the value of Messages Available for thoseconsumer groups will also increment by one if consumer groups exist for the queue.

----End

1.4 Retrieving Messages

ScenarioMessages can be retrieved from a queue.

PrerequisitesA queue has been created. The queue has at least one consumer group and at least onemessage.

NOTE

In standard and ActiveMQ queues, messages are retained for at least 72 hours and will be deleted 72hours later. However, in Kafka queues, messages are retained for 1 to 72 hours, depending on yourchoice.

Procedure




2018-10-29 6



Step 5 Open the Retrieve Message dialog box using either of the methods:

l Method 1

In the queue list, choose More > Retrieve Message in the same row as the queue fromwhich messages will be retrieved.

l Method 2

a. Click the name of the queue from which messages will be retrieved.

b. Click the Consumer Groups tab.

c. In the consumer group list, click Retrieve Message in the same row as theconsumer group who will retrieve messages from the chosen queue.

Step 6 Specify Consumer Group Name, Max. Message Count, Queue Polling Interval, andMessage Type. Optionally specify Message Labels. Then, click Start to retrieve themessages.



Consumer GroupName

Name of a consumer group.If you open the Retrieve Message dialog box by choosing More >Retrieve Message in the queue list, the Consumer Group Nameparameter is by default set to the name of the first consumer groupin the chosen queue.If you open the Retrieve Message dialog box by clicking RetrieveMessage in the consumer group list, the Consumer Group Nameparameter is by default set to the name of the chosen consumergroup.

Queue PollingInterval

Interval at which DMS polls the queue for messages. The value canbe 3s, 5s, 10s, or 30s.Default value: 3s

Message Type Type of messages to be retrieved.Values:l Normal

Only normal messages are retrieved from the chosen queue.l Dead letter

Only dead letter messages generated for the chosen consumergroup are retrieved from the chosen queue.

Default value: NormalIf the Dead Letter Queue parameter is set to off at the queuecreation time, only the Normal option is available in the MessageType drop-down list.


2018-10-29 7


Max. MessageCount

Maximum number of messages to be retrieved in a single pollingcycle. The value can be 5 or 10.Default value: 5

Message Labels Messages can be filtered by labels. Only messages with the chosenlabels are retrieved from the chosen queue.Each message has a maximum of three labels.When "Only retrieve messages containing all entered labels." isselected, only messages that match all labels are retrieved. If it is notselected, messages that match any one label are retrieved.Only messages in standard queues have labels.

After you click the Start button, the console polls the queue at regular intervals for 5 minutes,whether there are messages in the queue. After 5 minutes, the polling stops automatically. Tostop polling before then, click Stop and the polling will stop after the current cycle of pollingcompletes.

If you close the Retrieve Message dialog box or change the Consumer Group Name, Max.Message Count, or Queue Polling Interval while messages are being retrieved, the messageretrieval will stop after the current cycle of polling completes.

Retrieved messages are displayed in the list. To view details such as the message body,message size, ID, attribute count, and attributes, click View Details in the same row as thechosen message. Message retrieval will not be interrupted when you are viewing details.

NOTE

l Messages can be retrieved only once by each consumer group. Information about messages retrieved bythe current consumer group is lost after the Retrieve Message dialog box is closed or another consumergroup is selected to retrieve messages. However, the messages can still be retrieved by other consumergroups.

l Messages from one queue are stored in different partitions to enable simultaneous retrieval of multiplemessages by one consumer group. Each retrieval request can target messages in only one partition,preventing cross-partition management and access from adversely impacting performance.

l If the message queue contains only a few messages and messages are distributed over partitions, thenumber of messages in a single partition may be less than the number of messages you specified to beretrieved in a single polling cycle. This means that each polling cycle will return fewer messages thanexpected. However, all messages in the queue will be retrieved after multiple cycles.

l Once a consumer group specifies a message label, the consumer group must use the label for allsubsequent retrievals. If a consumer group changes a label during the next retrieval, the next retrieval willfail and the messages that fail to be retrieved cannot be retrieved again unless the consumer group resetsthe retrieval start position.

Table 1-4 Parameters in the Messages Retrieved list


Message Body(Condensed)

Content of the message body.Content of messages in Kafka queues is encoded with Base64.


2018-10-29 8


ID Handler of the message.

Body Size Size of the message body.

Operation Clicking View Details in the Operation column displays the completemessage body, complete message ID, and complete message attributesof the message retrieved.The Operation column is not displayed if the message is retrieved froman ActiveMQ or Kafka queue.

----End


2018-10-29 9

2 Managing Queues and Messages

This chapter describes operations on standard, ActiveMQ, and Kafka queues on the DMSconsole. For details about operations on RabbitMQ queues, see 3 Managing RabbitMQInstances. For details about operations on Kafka platinum queues, see 4 Managing KafkaInstances.

2.1 Viewing Details of a Queue

2.2 Deleting Queues

2.3 Deleting a Consumer Group

2.4 Resetting the Message Retrieval Start Position

2.5 Managing Queue Policies

2.6 Querying Messages

2.7 Viewing the Retrieval Status

2.1 Viewing Details of a Queue

ScenarioView details of a chosen queue, such as basic information, the number of retrievablemessages, the number of messages available in the dead letter queue, and consumer groups ofthe queue.

PrerequisitesA queue has been created.

Procedure




Distributed Message ServiceUser Guide 2 Managing Queues and Messages

2018-10-29 10


Step 5 Click the name of a queue for which you want to view details of a chosen queue.

Details of the chosen queue are displayed.

----End

2.2 Deleting Queues

Scenario

Deleting a queue will delete all messages in the queue. Deleted messages cannot berecovered. Exercise caution when deleting queues.

Prerequisites


Procedure





Step 5 Select the boxes next to the queues that will be deleted.

Step 6 Click Delete.

Step 7 Click OK to confirm the deletion.

----End

2.3 Deleting a Consumer Group

Scenario

Deleting a consumer group does not impact normal messages in a queue. Normal messagesstill exist in the queue and can still be retrieved by other consumer groups.

Dead letter messages are specific to consumer groups. If a consumer group is deleted, thedead letter messages of the consumer group are also deleted, regardless of whether they havebeen retrieved. Deleted dead letter messages can no longer be retrieved.

Prerequisites

A consumer group has been created.


2018-10-29 11

Procedure





Step 5 Click the name of a queue for which you want to delete a consumer group.


Step 6 Click the Consumer Groups tab to display the consumer group list. Choose More > DeleteConsumer Group in the same row as the consumer group you want to delete.

Step 7 Click OK to confirm the deletion.

----End

2.4 Resetting the Message Retrieval Start Position

Scenario

Reset the message retrieval start position if you want to skip certain messages that are neverretrieved, or retrieve certain messages again, or retrieve only the messages later than a certainpoint in time regardless of whether these messages have been retrieved.

You can start message retrieval:

l From the beginning of the chosen queueAll messages in the queue are retrieved again, regardless of whether they have beenretrieved.

l From the end of the chosen queueAll messages in the queue are skipped. Only the messages right after the new retrievalstart position will be retrieved.

l From a specified point in timeOnly the messages in the queue that are later than a specified point time are retrieved.

NOTE

After the retrieval start position is reset, messages ahead of the new retrieval start position will beflagged as retrieved messages, regardless of whether they have been retrieved.

Prerequisites

A queue has been created. The queue has at least one consumer group and the consumergroup has retrieved messages.

Procedure



2018-10-29 12




Step 5 Click the name of a queue for which you want to reset the message retrieval start position.


Step 6 On the queue details page, click the Consumer Groups tab.

Step 7 In the consumer group list, choose More > Reset Retrieval Start Position in the same row asthe consumer group for which you want to reset the message retrieval start position.

Step 8 Specify Start Message Retrieval and Force Reset parameters. Click OK.



StartMessageRetrieval

The position at which message retrieval will start during the messageretention period (3 days by default).l From beginning of queue

The consumer group starts retrieving messages from the beginning ofthe queue.

l From a specified point in timeThe consumer group starts retrieving messages from a specified point intime.

l From end of queueThe consumer group starts retrieving messages from the end of thequeue. This means that the consumer group retrieves the messagesnewly sent into the queue.

The default value is From beginning of queue.

Start Point Time at which the consumer group starts retrieving messages from thequeue.The message retrieval start time must fall within the message retentionperiod (3 days by default).This parameter is mandatory only if Start Message Retrieval is set toFrom a Specified Point in Time.

Force Reset An indicator of whether the message retrieval start position is forciblyreset.By default, this parameter is disabled.l Disable

If messages are currently being retrieved, the reset will fail.l Enable

The retrieval start position will be reset even if messages are currentlybeing retrieved. This may cause the message retrieval in progress to fail


2018-10-29 13

----End

2.5 Managing Queue Policies

ScenarioConfigure queue policies if multiple users or services need to access the same message queue.

Procedure





Step 5 Click the name of a queue for which you want to add a queue policy.


Step 6 On the queue details page, click the Policy Management tab.

Step 7 On the Policy Management tab page, click Create Queue Policy.

Step 8 Configure queue policy parameters.

Table 2-2 Queue policy parameters


Queue Name Name of the queue to which the new policy will be applied.

Permission Values:l Allowl DenyThe default permission is Allow.

Policy Type Values:l User-basedl Service-basedThe default policy type is User-based.


2018-10-29 14


User This parameter is configurable if Policy Type is User-based.Values:l All users

If Permission is set to Allow, all users can perform the specifiedactions on the chosen queue.If Permission is set to Deny, all users cannot perform the specifiedactions on the chosen queue.

l Specified usersIf Permission is set to Allow, only the specified users can performthe specified actions on the chosen queue.If Permission is set to Deny, the specified users cannot perform thespecified actions on the chosen queue.The user list can be any comma-separated combination of thefollowing three:– domain IDs– domain names– Uniform Resource Names (URNs)URN is a unique resource ID in the format ofurn:csp:service:region_id:domain_id:resourcetype:resource orurn:csp:service:region_id:domain_id:resource.Fields:– csp: cloud service provider– service: abbreviation of a service name, for example, SMN– region_id: region ID, for example, cn-north-1– domain_id: domain ID– resourcetype: resource type– resource: resource name, for example, Topic_testURN example:urn:csp:smn:cn-north-1:e23bf08ebb924730b452426c60849564:user:Topic_test

The default authorized users is All users.


2018-10-29 15


Service This parameter is configurable if Policy Type is Service-based.l FunctionGraph

If Permission is set to Allow, only the FunctionGraph service canperform the specified actions on the chosen queue.If Permission is set to Deny, the FunctionGraph service cannotperform the specified actions on the chosen queue.

l SMNIf Permission is set to Allow, only the Simple Message Notification(SMN) service can perform the specified actions on the chosenqueue.If Permission is set to Deny, the SMN service cannot perform thespecified actions on the chosen queue.

Action Actions on the chosen queue.Values:l GetQueuel CreateGroupl GetGroupsl DeleteGroupl ProduceMessagesl ConsumeMessages

----End

2.6 Querying Messages

Scenario

Query messages by message ID or queue for troubleshooting.

If the body size of a message is greater than 3 KB, you can download the message after thequery.

When querying messages by queue, you can query messages only within the persistentstorage time range.

Table 2-3 Query mode comparison

QueryMode

QueryCriteria

QueryType

Description

Query bymessageID

MessagesID

Exactquery

Based on a message ID, you can exactly locate thecorresponding message to obtain its attributes.


2018-10-29 16

QueryMode

QueryCriteria

QueryType

Description

Query byqueue

Queue+timerange

Rangequery

Based on a queue and time range, you can obtainall the messages meeting the query criteria inbatches. In this case, multiple messages arequeried, causing difficulties in matching.

Prerequisites

A queue has been created. The queue has at least one message.

Procedure





Step 5 Click the name of a queue for which you want to query the messages.


Step 6 On the queue details page, click the Message Query tab.

Step 7 Enter a message ID in the search box and press Enter. The queried message body andattributes are displayed.

NOTE

When querying messages by message ID, you can only query messages in the chosen queue. Otherwise,the query fails and an error message is displayed, indicating that the specified message ID is invalid.



ID Message ID.

Queue ID Queue ID.

Message Body Message body.

Message Attributes List of attributes.

Message Tags Message label.

Message Size Message size.

Created Date and time when a message was created.


2018-10-29 17


Operation This parameter is displayed only when the body size of amessage is greater than 3 KB.After the query, you can download the message, which will beautomatically saved as a .txt file.

Step 8 On the Message Query page, click the Query by Queue tab.

Step 9 Specify the queue and time range for query, and press Enter. Queried message details aredisplayed.

You can set the time range to 72 hours at most.

Step 10 In the Operation column, click View Details. The complete message bodies, IDs, andattributes are displayed.



ID Message ID.

Message Body Message body.

Message Attributes List of attributes.

Message Tags Message label.

Message Size Message size.

Created Date and time when a message was created.

Operation If the body size of a message is greater than 3 KB, you candownload the message after the query.The downloaded message will be automatically saved as a .txtfile.

----End

2.7 Viewing the Retrieval Status

Scenario

View the retrieval status, including the number of consumer instances, number of messagesaccumulated in the queue, retrieval model, retrieval rate, and consumer instance details.

Prerequisites

A queue and a consumer group have been created.


2018-10-29 18

Procedure





Step 5 Click the name of a queue for which you want to view the retrieval status.

Step 6 Click the Consumer Groups tab.

Step 7 Choose More > View Retrieval Status in the same row as the chosen consumer group.

Table 2-6 Retrieval status parameters


Consumer Group Name Name of a consumer group.

Consumer Instances Total number of consumer instances of a consumergroup.

Messages Available Number of consumable messages of a consumergroup.

Total Retrieval TPS Total retrieval rate of a consumer group.

Retrieval Model Currently, only the multicast model is supported.

Table 2-7 Client parameters


Consumer Instance ID ID of a consumer instance.

IP Address IP address of a client.

Retrieval TPS Retrieval rate of a consumer instance.

Language Development language of a client.

Version Version of the development language of a client.

----End


2018-10-29 19

3 Managing RabbitMQ Instances

3.1 Preparing the Environment

3.2 Creating an Instance

3.3 Viewing an Instance

3.4 Restarting an Instance

3.5 Deleting an Instance

3.6 Modifying the Information About an Instance

3.7 Accessing a RabbitMQ Instance

3.8 Connecting to the Management Address of a RabbitMQ Instance

3.9 Accessing a RabbitMQ Instance in SSL Mode

3.10 Resetting the Password of an Instance

3.11 Enabling Public Access to an Instance

3.1 Preparing the EnvironmentBefore creating RabbitMQ instances, you need to create a VPC and configure security groupsand subnets for it. A VPC provides an isolated, user-configurable, and user-manageablevirtual network environment for RabbitMQ instances. Using VPCs enhances public cloudresource security and simplifies network deployment.

Once you have created a VPC, you can use it for all RabbitMQ instances you subsequentlycreate.

Procedure



NOTE

Select the same region as the application service of the tenant.

Distributed Message ServiceUser Guide 3 Managing RabbitMQ Instances

2018-10-29 20

Step 3 Click Service List, and choose Network > Virtual Private Cloud to launch the VPCconsole.

Step 4 On the Dashboard page, click Create VPC.

Step 5 Create a VPC as guided by the VPC console, retaining the default values of the VPCparameters unless otherwise required.

For more information on how to create a VPC, see the Virtual Private Cloud User Guide.

After a VPC is created, a subnet is also created in the subnet. If the VPC needs more subnets,go to Step 6.

Step 6 In the navigation pane, choose Virtual Private Cloud. On the Virtual Private Cloud page,click the name of the VPC in which you intend to create a subnet. On the Subnets tab page,click Create Subnet.

Create a subnet as guided by the VPC console, retaining the default values of the subnetparameters unless otherwise required.

For more information on how to create a subnet, see the Virtual Private Cloud User Guide.

Step 7 In the navigation pane, choose Security Group. On the Security Group page, click CreateSecurity Group.

Create a security group for the VPC, retaining the default values of the security groupparameters unless otherwise required.

For more information on how to create a security group, see the Virtual Private Cloud UserGuide.

----End


Scenario

Currently, DMS provides the RabbitMQ instance service. RabbitMQ instances are physicallyisolated for each tenant. You can customize the computing capabilities and storage space of aRabbitMQ instance based on your service requirements.

RabbitMQ is an open-source service based on the Advanced Message Queuing Protocol(AMQP). It is used to store and forward messages in a distributed system. A RabbitMQ serveris compiled in Erlang (supporting high concurrency, distribution, and robust fault tolerance),and a RabbitMQ client can be compiled in various programming languages, including Python,Ruby, .NET, Java, JMS, C, PHP, ActionScript, XMPP, STOMP, and AJAX.

AMQP is an open standard application-layer protocol for message-oriented middleware. Itskey features are message orientation, queuing, routing, reliability, and security.

Prerequisites

A VPC configured with security groups and subnets is available.

For more information on how to create a VPC, security group, and subnet, see the VirtualPrivate Cloud User Guide.


2018-10-29 21

Procedure




Step 4 In the navigation pane, choose RabbitMQ Queue.

Step 5 Click Buy Instance in the upper right corner of the page.

By default, you can create a maximum of 100 instances for each project. To create more than100 instances, contact customer service to increase your quota.

Step 6 Specify the instance parameters listed in Table 3-1.



Billing Mode Billing mode.

Current Region Region in which DMS is applied. To select a different region, use theregion selector at the upper left of the main menu bar.

AZ Availability zone.

Instance Name Name of an instance, which must meet the following requirements:l Cannot be empty.l Must start with a letter.l Must be 4 to 64 characters long.l Can contain only letters, digits, underscores (_), and hyphens (-).


Instance description, which can contain a maximum 1024 characters.

MessageEngine

Currently, only RabbitMQ is supported.

Version Version of the message engine.

Instance Type Type of a RabbitMQ instance.l Single-node: Only one RabbitMQ node is deployed.l Cluster: Multiple RabbitMQ nodes are deployed to implement

highly reliable message storage.

Specifications l Single-node type: 2 vCPUs 4 GB, 4 vCPUs 8 GB, 8 vCPUs 16 GB,and 16 vCPUs 32 GB.

l Cluster type: 4 vCPUs 8 GB, 8 vCPUs 16 GB, and 16 vCPUs 32 GB.


2018-10-29 22


Storage Space Total storage space of a RabbitMQ instance. For details on how to selecta disk type, see Disk Types and Disk Performance.l Value range of a single-node instance: 100–90000 GBl Value range of a cluster instance: 100 GB x Number of nodes to

90000 GB, 200 GB x Number of nodes to 90000 GB, and 300 GB xNumber of nodes to 90000 GB

VPC VPC in which a RabbitMQ instance is located.l A VPC enables you to provision logically isolated, configurable, and

manageable virtual networks for your RabbitMQ instances.l If you are unsure of security group rules in the selected VPC, click

View VPC to show more details about the VPC.

Subnet Name and IP address range of the subnet in which a RabbitMQ instanceis located.

Security Group Security group that controls access to a RabbitMQ instance.A security group is a set of rules that control access to ECSs. It providesaccess policies for mutually trusted ECSs with the same securityprotection requirements in the same VPC.

Public Access An indicator of whether public access is enabled.NOTE

l In comparison with intra-VPC access, packet loss and jitter may occur and theaccess delay increases during public access. Therefore, you are advised toenable public access only during the service development and testing phase.

l After you enable public access to a RabbitMQ instance, messages received byand sent from the instance are all transmitted in plaintext. You are advised toenable this function only during the service development and testing phase.

Elastic IPAddress

Elastic IP address (EIP) used for accessing a RabbitMQ instance forwhich Public Access is enabled.Select an existing EIP from the drop-down list. If no EIP is available,click Create Elastic IP to buy an EIP on the displayed page on the VPCconsole.NOTE

l This parameter is displayed only when Public Access is enabled.

l If you manually unbind or delete an EIP from the VPC console, the publicaccess function of the corresponding DMS RabbitMQ instance isautomatically disabled.

SSL An indicator of whether SSL is enabled when a client is accessing aninstance.If SSL is enabled, data will be encrypted before transmission forenhanced security.

Username Username used for accessing a RabbitMQ instance.


2018-10-29 23

https://support.huaweicloud.com/en-us/productdesc-evs/en-us_topic_0014580744.html


Password Password used for accessing a RabbitMQ instance. You need to enterand then confirm the password.The password of a RabbitMQ instance must meet the followingcomplexity requirements:l Cannot be empty.l Must be 8 to 32 characters long.l Must contain at least two of the following character types:

– Lowercase letters– Uppercase letters– Digits– Special characters (`~!@#$%^&*()-_=+\|[{}]:'",<.>/?)

Time Window Time for O&M operations.During the maintenance time window, services can still be used, butoccasionally there may be temporary service interruptions. Services willquickly recover after any interruption. Scheduled maintenance occursinfrequently (typically once every few months).You can select 22:00-02:00, 02:00-06:00, 06:00-10:00, 10:00-14:00,14:00-18:00, or 18:00-22:00 as the time window, during which serviceO&M personnel perform maintenance operations on an instance.

Step 7 After specifying the parameters described in the preceding table, click Apply Now.

Step 8 Confirm the instance information, and click Submit.

Step 9 After the new RabbitMQ instance has been created, return to the RabbitMQ Queue page toview and manage your RabbitMQ instances.

1. It takes 10 to 20 minutes to create a RabbitMQ instance.2. After a RabbitMQ instance has been successfully created, by default its status should be

Running.3. If the new RabbitMQ instance fails to be created, delete the unsuccessful instance

creation task by following the procedure in 3.5 Deleting an Instance and then create theRabbitMQ instance again. If the RabbitMQ instance creation fails a second time, contactcustomer service.

----End


ScenarioView detailed information about a RabbitMQ instance on the DMS console.

PrerequisitesA RabbitMQ instance has been created.


2018-10-29 24

Procedure



NOTE




Step 5 Search for a RabbitMQ instance by specifying the status and name. Table 3-2 describes thevarious possible statuses of a RabbitMQ instance.

Table 3-2 RabbitMQ instance status description

Status Description

Creating The instance is being created.

Running The instance is running properly.Only RabbitMQ instances whose status is Running can provide the DMSRabbitMQ service.

Faulty The instance is not running properly.

Starting The instance is being started.

Restarting The instance is being restarted.

Stopping The instance is being stopped.

Stopped The instance has been stopped.RabbitMQ instances whose status is Stopped no longer provide anyexternal service.

Frozen The instance has been frozen due to insufficient account balance. You canunfreeze the instance by topping up your account in My Order.

Step 6 Click the name of the chosen RabbitMQ instance and view detailed information about theinstance on the displayed page.

----End


Scenario

Restart one or more RabbitMQ instances at a time on the DMS console.


2018-10-29 25

When a RabbitMQ instance is being restarted, message retrieval and creation requests of theclient will be rejected.

Prerequisites

The status of the RabbitMQ instance you want to restart is either Running or Faulty.

Procedure



NOTE




Step 5 Select one or more RabbitMQ instances in the instance list.

Step 6 Click Restart on the top of the instance list.

Step 7 Click OK.

It takes 1 to 30 minutes to restart a RabbitMQ instance. After it is successfully restarted, theRabbitMQ instance status should be Running.

NOTE

Restarting a RabbitMQ instance only restarts the instance process and does not restart the VM where theinstance is located.

To restart a RabbitMQ instance, you can also choose Operation > Restart in the same row as thechosen RabbitMQ instance on the RabbitMQ Queue page.

----End


Scenario

Delete one or more RabbitMQ instances at a time. With one click on the DMS console, youcan delete multiple RabbitMQ instances that failed to be created.

Deleting a RabbitMQ instance will delete the data in the instance without any backup.Exercise caution when performing this operation.


2018-10-29 26

Prerequisitesl The RabbitMQ instance you want to delete already exists.l The status of the RabbitMQ instance you want to delete is either Running, Faulty, or

Stopped.l RabbitMQ instances put into commercial use in yearly/monthly subscription mode

cannot be deleted. To disable such a RabbitMQ instance, choose Operation > More >Unsubscribe in the same row as the instance.

ProcedureDeleting a RabbitMQ instance



NOTE




Step 5 Select one or more RabbitMQ instances in the instance list.

RabbitMQ instances whose status is Creating, Starting, Stopping, or Restarting cannot bedeleted.

Step 6 Click Delete on the top of the instance list.

Step 7 Click OK.

It takes 1 to 30 minutes to delete a RabbitMQ instance.

NOTE

To delete a RabbitMQ instance, you can also choose Operation > Delete in the same row as the chosenRabbitMQ instance on the RabbitMQ Queue page.

----End

Deleting a RabbitMQ instance that failed to be created



NOTE




If there are RabbitMQ instances that failed to be created, Instance Creation Failures andquantity information will be displayed.


2018-10-29 27

Step 5 Click the icon or quantity next to Instance Creation Failures.

The Instance Creation Failures page is displayed.

Step 6 Delete RabbitMQ instances that failed to be created in either of the following ways:l To delete all RabbitMQ instances that failed to be created at once, click Clear Failed

Instance.l To delete a single RabbitMQ instance that failed to be created, click Delete in the same

row as the chosen RabbitMQ instance.

----End

3.6 Modifying the Information About an InstanceAfter creating a RabbitMQ instance, you can adjust some parameters of the instance based onyour service requirements.

Prerequisites

A RabbitMQ instance has been created.

Procedure



NOTE




Step 5 Click the name of a RabbitMQ instance for which you want to modify its information.

Step 6 Click behind a parameter to modify it.

You can modify the following parameters:

l Instance Namel Time Windowl Descriptionl Security Group

Step 7 Click to save the modification.

NOTE

Click to undo modification.

----End


2018-10-29 28

3.7 Accessing a RabbitMQ InstanceDMS RabbitMQ instances are compatible with the open-source RabbitMQ protocol. Fordetails on how to access a DMS RabbitMQ instance in different languages, see the RabbitMQtutorials at http://www.rabbitmq.com/getstarted.html.

This section uses the demo provided by DMS to describe how to access a RabbitMQ instancein a VPC.

Prerequisitesl A RabbitMQ instance has been created by following the instructions in 3.2 Creating an

Instance, and the username and password used to create the instance have beenrecorded.

l The Connection Address of the instance has been recorded from the instance details.

l An ECS has been created, and its VPC, subnet, and security group configurations are thesame as those of the RabbitMQ instance.

Accessing the Instance in CLI Mode

Step 1 Log in to the ECS.

NOTE

Ensure that the VPC, subnet, and security group configurations of the ECS are the same as those of theRabbitMQ instance.

Step 2 Install Java Development Kit (JDK) or Java Runtime Environment (JRE), and add thefollowing lines to .bash_profile in the home directory to configure the environment variablesJAVA_HOME and PATH:export JAVA_HOME=/opt/java/jdk1.8.0_151 export PATH=$JAVA_HOME/bin:$PATH

Run the source .bash_profile command for the modification to take effect.

NOTE

Use Oracle JDK instead of ECS's default JDK (for example, OpenJDK), because ECS's default JDKmay not be suitable for the sample project. Obtain Oracle JDK 1.8.111 or later from the Oracle officialwebsite.

Step 3 Run the following command to download RabbitMQ-Tutorial.zip (code package of thesample project):$ wget https://dms-demo.obs.myhwclouds.com/RabbitMQ-Tutorial.zip

Step 4 Run the following command to decompress RabbitMQ-Tutorial.zip:$ unzip RabbitMQ-Tutorial.zip

Step 5 Run the following command to navigate to the RabbitMQ-Tutorial directory, which containsthe precompiled JAR file:$ cd RabbitMQ-Tutorial

Step 6 Create messages using the sample project.$ java –cp .:rabbitmq-tutorial.jar Send host port user password


2018-10-29 29

http://www.rabbitmq.com/getstarted.html

http://www.oracle.com/technetwork/java/javase/downloads/index.html


Press Ctrl+C to exit.

Step 7 Retrieve messages using the sample project.$ java –cp .:rabbitmq-tutorial.jar Recv host port user password

To stop retrieving messages, press Ctrl+C to exit.

----End

Java Sample CodeAccessing a RabbitMQ instance and creating messages

ConnectionFactory factory = new ConnectionFactory(); factory.setHost(host); factory.setPort(port);

factory.setUsername(user); factory.setPassword(password); Connection connection = factory.newConnection(); Channel channel = connection.createChannel();

channel.queueDeclare(QUEUE_NAME, false, false, false, null);

String message = "Hello World!"; channel.basicPublish("", QUEUE_NAME, null, message.getBytes("UTF-8")); System.out.println(" [x] Sent '" + message + "'");

channel.close(); connection.close();

Accessing a RabbitMQ instance and retrieving messages

ConnectionFactory factory = new ConnectionFactory(); factory.setHost(host); factory.setPort(port); factory.setUsername(user); factory.setPassword(password); Connection connection = factory.newConnection(); Channel channel = connection.createChannel();

channel.queueDeclare(QUEUE_NAME, false, false, false, null); System.out.println(" [*] Waiting for messages. To exit press CTRL+C");

Consumer consumer = new DefaultConsumer(channel) { @Override public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException { String message = new String(body, "UTF-8"); System.out.println(" [x] Received '" + message + "'");


2018-10-29 30

} }; channel.basicConsume(QUEUE_NAME, true, consumer);

3.8 Connecting to the Management Address of aRabbitMQ Instance

Access the open-source RabbitMQ cluster management tool by entering the managementaddress of a RabbitMQ instance in the address bar of a browser.

Procedure

Step 1 Create a Windows ECS with the same VPC, subnet, and security group configurations as theRabbitMQ instance available. For details, see Purchasing and Logging In to a WindowsECS.

Step 2 Click the name of the RabbitMQ instance. On the displayed instance details page, copy theManagement Address of the instance.

Step 3 Paste the management address to the address bar of a browser. You can then manageRabbitMQ clusters.

----End

3.9 Accessing a RabbitMQ Instance in SSL ModeWhen an instance is created, if SSL is enabled, data will be encrypted before transmission forenhanced security.

This section describes intra-VPC access to a RabbitMQ instance with SSL enabled.

Prerequisitesl A RabbitMQ instance has been created by following the instructions in 3.2 Creating an

Instance, and the username and password used to create the instance have beenrecorded.

l The Connection Address of the instance has been recorded from the instance details.l An ECS has been created, and its VPC, subnet, and security group configurations are the

same as those of the RabbitMQ instance.


Step 1 Log in to the ECS. If public network access is enabled, log in to the server for runningcommands.

NOTE

Ensure that the VPC, subnet, and security group configurations of the ECS are the same as those of theRabbitMQ instance.

Step 2 Install JDK or JRE, and add the following lines to .bash_profile in the home directory toconfigure the environment variables JAVA_HOME and PATH:export JAVA_HOME=/opt/java/jdk1.8.0_151 export PATH=$JAVA_HOME/bin:$PATH


2018-10-29 31

https://support.huaweicloud.com/en-us/qs-ecs/en-us_topic_0021831611.html



NOTE


Step 3 Run the following command to download RabbitMQ-Tutorial-SSL.zip (code package of thesample project):$ wget https://dms-demo.obs.myhwclouds.com/RabbitMQ-Tutorial-SSL.zip

Step 4 Run the following command to decompress RabbitMQ-Tutorial-SSL.zip:$ unzip RabbitMQ-Tutorial-SSL.zip

Step 5 Run the following command to navigate to the RabbitMQ-Tutorial-SSL directory, whichcontains the precompiled JAR file:$ cd RabbitMQ-Tutorial-SSL

Step 6 Create messages using the sample project.$ java –cp .:rabbitmq-tutorial-sll.jar Send host port user password

Press Ctrl+C to exit.

Step 7 Retrieve messages using the sample project.$ java –cp .:rabbitmq-tutorial-sll.jar Recv host port user password


----End

Java Sample CodeAccessing a RabbitMQ instance and creating messages

ConnectionFactory factory = new ConnectionFactory(); factory.setHost(host); factory.setPort(port);

factory.setUsername(user); factory.setPassword(password); factory.useSslProtocol(); Connection connection = factory.newConnection(); Channel channel = connection.createChannel();

channel.queueDeclare(QUEUE_NAME, false, false, false, null);

String message = "Hello World!"; channel.basicPublish("", QUEUE_NAME, null, message.getBytes("UTF-8")); System.out.println(" [x] Sent '" + message + "'");


2018-10-29 32



channel.close(); connection.close();

Accessing a RabbitMQ instance and retrieving messages

ConnectionFactory factory = new ConnectionFactory(); factory.setHost(host); factory.setPort(port); factory.setUsername(user); factory.setPassword(password); factory.useSslProtocol(); Connection connection = factory.newConnection(); Channel channel = connection.createChannel();

channel.queueDeclare(QUEUE_NAME, false, false, false, null); System.out.println(" [*] Waiting for messages. To exit press CTRL+C");

Consumer consumer = new DefaultConsumer(channel) { @Override public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException { String message = new String(body, "UTF-8"); System.out.println(" [x] Received '" + message + "'"); } }; channel.basicConsume(QUEUE_NAME, true, consumer);


ScenarioIf you forget the password of a RabbitMQ instance, reset the password so that you cannormally access the instance.

NOTE

l You can reset the password of a RabbitMQ instance only if its status is Running.

l Resetting a password requires you to obtain a verification code by using a registered mobile phonenumber. Ensure that you have successfully registered a mobile phone number.

l Only the tenant administrator account has the permission to reset a password.

PrerequisitesA RabbitMQ instance has been created.

Procedure



NOTE




2018-10-29 33


Step 5 Click More > Reset Password in the Operation column of a RabbitMQ instance for whichyou want to reset its password.

Step 6 On the display Reset Password page, specify New Password and Confirm Password.

NOTE

l Cannot be empty.l Must be 8 to 32 characters long.l Must contain at least two of the following character types:


Step 7 Click Send Verification Code.

NOTE

l You must register a mobile phone number before resetting a password.l A verification code is valid within 10 minutes. If the verification code expires, get a new one.l A verification code will expire after three incorrect attempts. In this case, get a new one.

Step 8 Enter the verification code and click OK.l If the password is successfully reset, a success message will be displayed.l If the password fails to be reset, a failure message will be displayed. If you still fail to

reset the password after multiple attempts, contact customer service.

NOTE

The system will display a success message only after the password is successfully reset on all nodes.

----End

3.11 Enabling Public Access to an Instance

ScenarioEnable public access to a RabbitMQ instance so that you can access the instance over a publicnetwork.

NOTE

l You can enable public access to a RabbitMQ instance only if its status is Running.l In comparison with intra-VPC access, packet loss and jitter may occur and the access delay increases

during public access. Therefore, you are advised to enable public access to RabbitMQ instances onlyduring the service development and testing phase.

l After you enable public access to a RabbitMQ instance, messages received by and sent from theinstance are all transmitted in plaintext. You are advised to enable this function only during theservice development and testing phase.

PrerequisitesA RabbitMQ instance has been created, and public access has not been enabled for theinstance.


2018-10-29 34

Procedure



NOTE




Step 5 Click the name of a RabbitMQ instance for which you want to enable public access.

Step 6 Click on the right of Public Access.

Step 7 Select an EIP from the Elastic IP Address drop-down list and click .

It takes 10s to 30s to enable public access.

The system is automatically redirected to the Background Task Management tab page, onwhich the progress of the current task is displayed. If the status of the task turns toSucceeded, public access is successfully enabled.

NOTE

l If an EIP exists in the Elastic IP Address drop-down list box, click View Elastic IP to view thedetails of the EIP on the displayed page on the VPC console.

l If no EIP exists in the Elastic IP Address drop-down list box, click Create Elastic IP to buy an EIPon the displayed page on the VPC console.

l The selected EIP fails to be bound if it has been used (for example, it is bound to another RabbitMQinstance or used by another service).

----End

Disabling Public Access


Step 2 Click in the upper left corner of the management console and select a region and aproject.

NOTE




Step 5 Click the name of a RabbitMQ instance for which you want to disable public access.

The instance details page is displayed.

Step 6 Click on the right of Public Access.


2018-10-29 35

Step 7 Click .

It takes 10s to 30s to disable public access.

The system is automatically redirected to the Background Task Management tab page, onwhich the progress of the current task is displayed. If the status of the task turns toSucceeded, public access is successfully disabled.

----End


2018-10-29 36

4 Managing Kafka Instances

4.1 Preparing the Environment





4.6 Modifying the Information About an Instance

4.7 Creating a Topic in a Kafka Platinum Instance

4.8 Deleting a Topic in a Kafka Platinum Instance

4.9 Accessing a Kafka Platinum Instance

4.10 Connecting to the Kafka Manager

4.11 Accessing a Kafka Platinum Instance in SASL_SSL Mode


4.1 Preparing the EnvironmentBefore creating Kafka platinum instances, you need to create a VPC and configure securitygroups and subnets for it.

A VPC provides an isolated, user-configurable, and user-manageable virtual networkenvironment for Kafka platinum instances. Using VPCs enhances public cloud resourcesecurity and simplifies network deployment.

Once you have created a VPC, you can use it for all Kafka platinum instances yousubsequently create.

Procedure


Distributed Message ServiceUser Guide 4 Managing Kafka Instances

2018-10-29 37


NOTE


Step 3 Click Service List, and choose Network > Virtual Private Cloud to launch the VPCconsole.

Step 4 On the Dashboard page, click Create VPC.

Step 5 Create a VPC as guided by the VPC console, retaining the default values of the VPCparameters unless otherwise required.

For more information on how to create a VPC, see the Virtual Private Cloud User Guide.

After a VPC is created, a subnet is also created in the subnet. If the VPC needs more subnets,go to Step 6. Otherwise, go to Step 7.

Step 6 In the navigation pane, choose Virtual Private Cloud. On the Virtual Private Cloud page,click the name of the VPC in which you intend to create a subnet. On the Subnets tab page,click Create Subnet.

Create a subnet as guided by the VPC console, retaining the default values of the subnetparameters unless otherwise required.

For more information on how to create a subnet, see the Virtual Private Cloud User Guide.

Step 7 In the navigation pane, choose Security Group. On the Security Group page, click CreateSecurity Group.

Create a security group for the VPC, retaining the default values of the security groupparameters unless otherwise required.

For more information on how to create a security group, see the Virtual Private Cloud UserGuide.

----End


Scenario

DMS provides the Kafka platinum instance service. Kafka platinum instances are physicallyisolated and exclusively occupied by each tenant. You can customize the computingcapabilities and storage space of a Kafka instance based on your service requirements.

Prerequisites

A VPC configured with security groups and subnets is available.

For more information on how to create a VPC, security group, and subnet, see the VirtualPrivate Cloud User Guide.


2018-10-29 38

Procedure




Step 4 In the navigation pane, choose Kafka Platinum.

Step 5 Click Buy Instance in the upper right corner of the page.

By default, you can create a maximum of 5 instances for each project. To create more than 5instances, contact customer service to increase your quota.

Step 6 Specify the instance parameters listed in Table 4-1.



Billing Mode Billing mode.

Current Region Region in which DMS is applied. To select a different region, use theregion selector at the upper left of the main menu bar.

AZ Availability zone.

Instance Name Name of an instance, which must meet the following requirements:l Cannot be empty.l Must start with a letter.l Must be 4 to 64 characters long.l Can contain only letters, digits, underscores (_), and hyphens (-).


Instance description, which can contain a maximum 1024 characters.

Version Kafka version.

Bandwidth Attainable bandwidth during intra-VPC access.Unit: byte/s.Options: 300 MB, 600 MB, and 1200 MB.

MaximumNumber ofPartitions

Maximum number of partitions that can be created for a Kafka platinuminstance. If the total number of partitions of all topics exceeds thisthreshold, topic creation will fail.Options:l When Bandwidth is 300 MB: 900l When Bandwidth is 600 MB: 1800l When Bandwidth is 1200 MB: 1800


2018-10-29 39


Storage Space Total disk space for storing Kafka data. If the available disk space is lessthan 5%, you can retrieve but cannot create messages. Messages do notoccupy any disk space after the retention period ends.Disks are formatted when an instance is created. As a result, the actualavailable disk space is 93% to 95% of the total disk space.l When Bandwidth is set to 300 MB, the value range of Storage

Space is 1200–90000.l When Bandwidth is set to 600 MB, the value range of Storage

Space is 2400–90000.l When Bandwidth is set to 1200 MB, the value range of Storage

Space is 4800–90000.NOTE

l High I/O + 300 MB/s bandwidth: If the average message size is 1 KB, thetransactions per second (TPS) can reach 300,000 in high throughput scenariosand 150,000 in synchronous replication scenarios.

l Ultra-high I/O + 300 MB/s bandwidth: If the average message size is 1 KB,the TPS can reach 300,000 in high throughput scenarios and 200,000 insynchronous replication scenarios.

l Ultra-high I/O + 600 MB/s bandwidth: If the average message size is 1 KB,the TPS can reach 600,000 in high throughput scenarios and 300,000 insynchronous replication scenarios.

l Ultra-high I/O + 1200 MB/s bandwidth: If the average message size is 1 KB,the TPS can reach 1,200,000 in high throughput scenarios and 400,000 insynchronous replication scenarios.

VPC VPC in which a RabbitMQ instance is located.l A VPC enables you to provision logically isolated, configurable, and

manageable virtual networks for your RabbitMQ instances.l If you are unsure of security group rules in the selected VPC, click

View VPC to show more details about the VPC.

Subnet Name and IP address range of the subnet in which a RabbitMQ instanceis located.

Security Group Security group that controls access to a RabbitMQ instance.A security group is a set of rules that control access to ECSs. It providesaccess policies for mutually trusted ECSs with the same securityprotection requirements in the same VPC.

SASL_SSL An indicator of whether SSL authentication is enabled when a client isaccessing an instance.If SSL is enabled, data will be encrypted before transmission forenhanced security.

Username Username for accessing a Kafka platinum instance.This parameter is available only when SSL is enabled.


2018-10-29 40


Password This parameter is available only when SSL is enabled.Password for accessing a Kafka platinum instance. You must specify apassword and then confirm the password.The password of a Kafka platinum instance must meet the followingrequirements:l Cannot be empty.l Must be 8 to 32 characters long.l Must contain at least two of the following character types:


Time Window Time for O&M operations.You can select 22:00-02:00, 02:00-06:00, 06:00-10:00, 10:00-14:00,14:00-18:00, or 18:00-22:00 as the time window, during which serviceO&M personnel perform maintenance operations on an instance.

Step 7 After specifying the parameters described in the preceding table, click Apply Now.

Step 8 Confirm the instance information, and click Submit.

Step 9 After the new Kafka platinum instance has been created, return to the Kafka Platinum pageto view and manage your Kafka platinum instances.

1. It takes 10 to 20 minutes to create a Kafka platinum instance.

2. After a Kafka platinum instance has been successfully created, by default its statusshould be Running.

3. If the new Kafka platinum instance fails to be created, delete the unsuccessful instancecreation task by following the procedure in 3.5 Deleting an Instance and then create theKafka platinum instance again. If the Kafka platinum instance creation fails a secondtime, contact customer service.

----End


Scenario

View detailed information about a Kafka platinum instance on the DMS console.

Prerequisites

A Kafka platinum instance has been created.


2018-10-29 41

Procedure



NOTE




Step 5 Search for a Kafka platinum instance by specifying the status and name. Table 4-2 describesthe various possible statuses of a Kafka platinum instance.

Table 4-2 Kafka platinum instance status description

Status Description

Creating The instance is being created.

Running The instance is running properly.Only Kafka platinum instances whose status is Running can provide theDMS Kafka platinum service.

Faulty The instance is not running properly.

Starting The instance is being started.

Restarting The instance is being restarted.

Stopping The instance is being stopped.

Stopped The instance has been stopped.Kafka platinum instances whose status is Stopped no longer provide anyexternal service.

Frozen The instance has been frozen due to insufficient account balance. You canunfreeze the instance by topping up your account in My Order.

Step 6 Click the name of the chosen Kafka platinum instance and view detailed information aboutthe instance on the displayed page.

----End


Scenario

Restart one or more Kafka platinum instances at a time on the DMS console.


2018-10-29 42

When a Kafka platinum instance is being restarted, message retrieval and creation requests ofthe client will be rejected.

Prerequisites

The status of the Kafka platinum instance you want to restart is either Running or Faulty.

Procedure



NOTE




Step 5 Select one or more Kafka platinum instances in the instance list.

Step 6 Click Restart on the top of the instance list.

Step 7 Click OK.

It takes 1 to 30 minutes to restart a Kafka platinum instance. After it is successfully restarted,the Kafka platinum instance status should be Running.

NOTE

Restarting a Kafka platinum instance only restarts the instance process and does not restart the VMwhere the instance is located.

To restart a Kafka platinum instance, you can also choose Operation > Restart in the same row as thechosen Kafka platinum instance on the Kafka Platinum page.

----End


Scenario

Delete one or more Kafka platinum instances at a time. With one click on the DMS console,you can delete multiple Kafka platinum instances that failed to be created.

Deleting a Kafka platinum instance will delete the data in the instance without any backup.Exercise caution when performing this operation.


2018-10-29 43

Prerequisitesl The Kafka platinum instance you want to delete already exists.l The status of the Kafka platinum instance you want to delete is either Running, Faulty,

or Stopped.l Kafka platinum instances put into commercial use in yearly/monthly subscription mode

cannot be deleted. To disable such a Kafka platinum instance, choose Operation > More> Unsubscribe in the same row as the instance.

ProcedureDeleting a Kafka platinum instance



NOTE




Step 5 Select one or more Kafka platinum instances in the instance list.

Kafka platinum instances whose status is Creating, Starting, Stopping, or Restarting cannotbe deleted.

Step 6 Click Delete on the top of the instance list.

Step 7 Click OK.

It takes 1 to 30 minutes to delete a Kafka platinum instance.

NOTE

To delete a Kafka platinum instance, you can also choose Operation > Delete in the same row as thechosen Kafka platinum instance on the Kafka Platinum page.

----End

Deleting a Kafka platinum instance that failed to be created



NOTE




If there are Kafka platinum instances that failed to be created, Instance Creation Failuresand quantity information will be displayed.


2018-10-29 44

Step 5 Click the icon or quantity next to Instance Creation Failures.

The Instance Creation Failures page is displayed.

Step 6 Delete Kafka platinum instances that failed to be created in either of the following ways:l To delete all Kafka platinum instances that failed to be created at once, click Clear

Failed Instance.l To delete a single Kafka platinum instance that failed to be created, click Delete in the

same row as the chosen Kafka platinum instance.

----End

4.6 Modifying the Information About an InstanceAfter creating a Kafka platinum instance, you can adjust some parameters of the instancebased on your service requirements.

Prerequisites


Procedure



NOTE




Step 5 Click the name of a Kafka platinum instance for which you want to modify its information.

Step 6 Click behind a parameter to modify it.

You can modify the following parameters:

l Instance Namel Time Windowl Descriptionl Security Group

Step 7 Click to save the modification.

NOTE

Click to undo modification.

----End


2018-10-29 45

4.7 Creating a Topic in a Kafka Platinum InstanceAfter creating a Kafka platinum instance, you can create a topic in the instance.

Prerequisites


The status of the Kafka platinum instance is Running.

Procedure



NOTE




Step 5 Click the name of a Kafka platinum instance for which you want to create a topic.


Step 6 Click the Topic Management tab, and click Create Topic.

The Create Topic page is displayed.

Step 7 Specify the topic parameters listed in the following table.

Table 4-3 Topic parameters


Topic Name When you create a topic, a default topic name is generated, whichyou can change as required. A topic name contains a maximum of 64characters. Only letters, digits, underscores (_), and hyphens (-) areallowed.The topic name cannot be changed after a topic is created.

Partitions A larger number of partitions for a topic indicates more messagesretrieved concurrently.If this parameter is set to 1, messages will be retrieved in a FIFOorder.Value range: 1 to 20Default value: 3


2018-10-29 46


Replicas A higher number of replicas indicates higher reliability. Data isautomatically backed up on each replica. After one Kafka brokerbecomes faulty, data is still available on other brokers.If this parameter is set to 1, only one set of data is available.Value range: 1 to 3Default value: 3

Message RetentionPeriod (h)

The number of hours for which messages will be preserved in a topic.Messages older than that period will be deleted and deleted messagesare not retrievable to consumer groups.Value range: 1 to 168Default value: 72

SynchronousFlushing

An indicator of whether a message is immediately flushed to diskonce it is created.On: A message is immediately flushed to disk once it is created,featuring higher reliability.Off: A message is stored in the memory instead of being immediatelyflushed to disk once it is created.

Step 8 Click OK.

----End

4.8 Deleting a Topic in a Kafka Platinum Instance

PrerequisitesA Kafka platinum instance has been created, and a topic has been created in this instance.

The status of the Kafka platinum instance is Running.

Procedure



NOTE




Step 5 Click the name of a Kafka platinum instance for which you want to delete a topic.



2018-10-29 47

Step 6 Click the Topic Management tab.

Step 7 Select one or more topics in the topic list.

Step 8 Click Delete Topic on the top of the topic list.

NOTE

To delete a topic, you can also choose Operation > Delete in the same row as the chosen topic on theTopic Management page.

Step 9 Click OK.

----End

4.9 Accessing a Kafka Platinum InstanceDMS supports Kafka platinum instances, each of which is physically isolated and exclusivelyoccupied by one tenant. After creating a Kafka platinum instance, you can use an open-sourceKafka client to create and retrieve messages in the instance.

This section describes how to use an open-source Java client to access a Kafka platinuminstance in the same VPC as the client. For details on how to use Kafka clients in otherlanguages, visit https://cwiki.apache.org/confluence/display/KAFKA/Clients.

Prerequisitesl A Kafka platinum instance has been created, and its Connection Address has been

recorded from the instance details.l An ECS has been created, and its VPC, subnet, and security group configurations are the

same as those of the Kafka platinum instance.



NOTE

Ensure that the VPC, subnet, and security group configurations of the ECS are the same as those of theKafka platinum instance.



NOTE


Step 3 Download an open-source Kafka client.

If the version of the created Kafka platinum instance is 0.10.2_plus, download the client athttps://archive.apache.org/dist/kafka/0.10.2.0/kafka_2.11-0.10.2.0.tgz.


2018-10-29 48

https://cwiki.apache.org/confluence/display/KAFKA/Clients



https://archive.apache.org/dist/kafka/0.10.2.0/kafka_2.11-0.10.2.0.tgz

If the version of the created Kafka platinum instance is 1.1.0, download the client at http://mirrors.tuna.tsinghua.edu.cn/apache/kafka/1.1.0/kafka_2.11-1.1.0.tgz.

Step 4 Decompress the Kafka client package. The following example uses a Kafka platinum instanceof version 0.10.2.0.

tar -zxf [kafka_tar]

Step 5 Access the [base_dir]/kafka_2.11-0.10.2.0/bin directory.

Step 6 Run the following command to create messages:

./kafka-console-producer.sh --broker-list [connection-address] --topic [topic-name]

After running the preceding command, you can send a message to the Kafka instance bywriting it and pressing Enter. Each line of content is sent as a message.

For example, connection-address in the preceding command is set to192.168.0.192:9092,192.168.0.183:9092,192.168.0.155:9092, which indicates the IPaddresses and ports of three brokers.

To stop creating messages, press Ctrl+C to exit.

Step 7 Run the following command to retrieve messages:

./kafka-console-consumer.sh --bootstrap-server [connection-address] --topic [topic-name] --from-beginning



----End

4.10 Connecting to the Kafka ManagerKafka Manager is an open-source Kafka cluster management tool. Perform the followingprocedure to access the Kafka Manager.

Procedure

Step 1 Create a Windows ECS with the same VPC, subnet, and security group configurations as theKafka platinum instance available. For details, see Purchasing and Logging In to aWindows ECS.

Step 2 Click the name of the Kafka platinum instance. On the displayed instance details page, copythe address of the Kafka Manager.


2018-10-29 49

http://mirrors.tuna.tsinghua.edu.cn/apache/kafka/1.1.0/kafka_2.11-1.1.0.tgz




Step 3 Paste the address to the address bar of a browser. You can then manage Kafka clusters.

----End

4.11 Accessing a Kafka Platinum Instance in SASL_SSLMode

When an instance is created, if SASL_SSL is enabled, data will be encrypted beforetransmission for enhanced security.

This section describes how to use a Java client to access a Kafka platinum instance withSASL_SSL enabled in the same VPC.

Prerequisitesl A Kafka platinum instance has been created, and its Connection Address has been

recorded from the instance details.l An ECS has been created, and its VPC, subnet, and security group configurations are the

same as those of the Kafka platinum instance.


Step 1 Download the client.truststore.jks certificate.

To obtain the certificate, download the required package at http://static.huaweicloud.com/upload/files/dms/dmskafkasasl.zip and decompress the following from the package:l client.truststore.jks: client certificatel dms.kafka.sasl.client-1.0.0.jar: SASL packagel dms_kafka_client_jaas.conf: client configuration file


NOTE

Ensure that the VPC, subnet, and security group configurations of the ECS are the same as those of theKafka platinum instance.



NOTE


Step 4 Download an open-source Kafka client.

If the version of the created Kafka platinum instance is 0.10.2_plus, download the client athttps://archive.apache.org/dist/kafka/0.10.2.0/kafka_2.11-0.10.2.0.tgz.

If the version of the created Kafka platinum instance is 1.1.0, download the client at http://mirrors.tuna.tsinghua.edu.cn/apache/kafka/1.1.0/kafka_2.11-1.1.0.tgz.


2018-10-29 50

http://static.huaweicloud.com/upload/files/dms/dmskafkasasl.zip

http://static.huaweicloud.com/upload/files/dms/dmskafkasasl.zip



https://archive.apache.org/dist/kafka/0.10.2.0/kafka_2.11-0.10.2.0.tgz



Step 5 Decompress the Kafka client package. The following example uses a Kafka platinum instanceof version 0.10.2.0.

tar -zxf [kafka_tar]

Step 6 Add the following lines to the consumer.properties/producer.properties file:sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \username="**********" \password="**********"; sasl.mechanism=PLAIN

security.protocol=SASL_SSLssl.truststore.location=/opt/kafka_2.11-1.1.0/config/client.truststore.jksssl.truststore.password=*********

In the preceding configurations, username and password indicate the username andpassword specified when SASL_SSL is enabled, and ssl.trustore.location indicates the pathfor storing the certificate file.

Step 7 Access the [base_dir]/kafka_2.11-0.10.2.0/bin directory.

Step 8 Run the following commands to create messages:

./kafka-console-producer.sh --broker-list [connection-address] --topic [topic-name] --producer.config ../config/producer.properties

After running the preceding command, you can send a message to the Kafka instance bywriting it and pressing Enter. Each line of content is sent as a message.


To stop creating messages, press Ctrl+C to exit.

Step 9 Run the following commands to retrieve message:

./kafka-console-consumer.sh --bootstrap-server [connection-address] --topic [topic-name] --from-beginning --consumer.config ../config/consumer.properties



----End


2018-10-29 51


Scenario

If you forget the password of a Kafka platinum instance, reset the password so that you cannormally access the instance.

NOTE

l You can reset the password of a Kafka platinum instance only if SSL is enabled for this instance.

l You can reset the password of a Kafka platinum instance only if its status is Running.

l Resetting a password requires you to obtain a verification code by using a registered mobile phonenumber. Ensure that you have successfully registered a mobile phone number.

l Only the tenant administrator account has the permission to reset a password.

Prerequisites


Procedure



NOTE




Step 5 Click More > Reset Password in the Operation column of a Kafka platinum instance forwhich you want to reset its password.

Step 6 On the display Reset Password page, specify New Password and Confirm Password.

NOTE

l Cannot be empty.

l Must be 8 to 32 characters long.

l Must contain at least two of the following character types:

– Lowercase letters

– Uppercase letters

– Digits

– Special characters (`~!@#$%^&*()-_=+\|[{}]:'",<.>/?)

Step 7 Click Send Verification Code.


2018-10-29 52

NOTE

l You must register a mobile phone number before resetting a password.

l A verification code is valid within 10 minutes. If the verification code expires, get a new one.

l A verification code will expire after three incorrect attempts. In this case, get a new one.

Step 8 Enter the verification code and click OK.l If the password is successfully reset, a success message will be displayed.l If the password fails to be reset, a failure message will be displayed. If you still fail to

reset the password after multiple attempts, contact customer service.

NOTE

The system will display a success message only after the password is successfully reset on all nodes.

----End


2018-10-29 53

5 Monitoring

5.1 DMS Metrics

5.2 Setting DMS Alarm Rules

5.3 Viewing Metrics on the Cloud Eye Console

5.1 DMS Metrics

Standard Queue

Table 5-1 Monitoring metrics of standard queues

Dimension

Monitoring Metric Description

Queue Queued Messages The current number of messages in a queue.The following messages are not counted into themetric:l Messages older than 72 hours

Messages older than 72 hours will be deletedfrom a queue and deleted messages are notcounted into the metric.

l Dead letter messagesDead letter messages are specific to consumergroups and stored in dead letter queues.Therefore, generation and expiry of dead lettermessages do not affect the metric.

Message Size (Bytes) The total size of messages sent to a queue duringthe previous one minute.This metric gives you an overview of messageload flowing into a queue.

Request Count The total number of requests sent to a queueduring the previous one minute.

Distributed Message ServiceUser Guide 5 Monitoring

2018-10-29 54

Dimension


Consumergroup

Accumulated Messages The current number of messages that are notretrieved by a consumer group.The following messages are not counted into themetric:l Messages older than 72 hours

Messages older than 72 hours will be deletedfrom a queue. Deleted messages can no longerbe retrieved by a consumer group andtherefore they are not counted into the metric.

l Dead letter messagesDead letter messages are flagged as retrievedmessages and therefore not counted into themetric.

Retrieved Messages The accumulated number of messages retrievedby a consumer group since the consumer group iscreated.Dead letter messages are counted into the metricbecause dead letter messages are flagged asretrieved messages.After the retrieval start position is reset, messagesahead of the new retrieval start position will beflagged as retrieved messages, regardless ofwhether they have been retrieved.

Skipped Messages The accumulated number of messages that aredeleted and never retrieved. The count of skippedmessages is accumulated since a consumer groupis created.Messages older than 72 hours will be deletedfrom a queue. Deleted messages can no longer beretrieved by a consumer group.After the retrieval start position is reset, messagesahead of the retrieval start position will beflagged as retrieved messages and the number ofskipped messages will be reset to zero.

Available Dead LetterMessages

The current number of dead letter messages thatare not retrieved by a consumer group.Dead letter messages older than 72 hours will bedeleted from a queue. Once deleted, dead lettermessages can no longer be retrieved by aconsumer group and therefore they are notcounted into the metric.


2018-10-29 55

Dimension


Historical Dead LetterMessages

The accumulated number of dead letter messagesgenerated for a consumer group since theconsumer group is created.The count of historical dead letter messagesincludes the retrieved dead letter messages,remaining dead letter messages, and expired deadletter messages.

ActiveMQ Queue

Table 5-2 Monitoring metrics of ActiveMQ queues

Dimension


Queue Queued Messages The current number of messages in a queue.The following messages are not counted into themetric:l Messages older than 72 hours

Messages older than 72 hours will be deletedfrom a queue and deleted messages are notcounted into the metric.

l Dead letter messagesDead letter messages are specific to consumergroups and stored in dead letter queues.Therefore, generation and expiry of dead lettermessages do not affect the metric.




2018-10-29 56

Dimension


Consumergroup

Accumulated Messages The current number of messages that are notretrieved by a consumer group.The following messages are not counted into themetric:l Messages older than 72 hours

Messages older than 72 hours will be deletedfrom a queue. Deleted messages can no longerbe retrieved by a consumer group andtherefore they are not counted into the metric.

l Dead letter messagesDead letter messages are flagged as retrievedmessages and therefore not counted into themetric.

Retrieved Messages The accumulated number of messages retrievedby a consumer group since the consumer group iscreated.Dead letter messages are counted into the metricbecause dead letter messages are flagged asretrieved messages.After the retrieval start position is reset, messagesahead of the new retrieval start position will beflagged as retrieved messages, regardless ofwhether they have been retrieved.

Skipped Messages The accumulated number of messages that aredeleted and never retrieved. The count of skippedmessages is accumulated since a consumer groupis created.Messages older than 72 hours will be deletedfrom a queue. Deleted messages can no longer beretrieved by a consumer group.After the retrieval start position is reset, messagesahead of the retrieval start position will beflagged as retrieved messages and the number ofskipped messages will be reset to zero.


2018-10-29 57

Kafka Queue

Table 5-3 Monitoring metrics of Kafka queues

Dimension


Queue Queued Messages The current number of messages in a queue,excluding those deleted after the MessageRetention Period (value range: 1 hour to 72hours; default value: 72 hours).



Consumergroup

Accumulated Messages The current number of messages that are notretrieved by a consumer group, excluding thosedeleted after the Message Retention Period(value range: 1 hour to 72 hours; default value: 72hours).

5.2 Setting DMS Alarm Rules

Scenarios

Setting DMS alarm rules allows you to customize the monitored objects and notificationpolicies and determine the running status of your DMS at any time.

This section describes how to set DMS alarm rules, including alarm rule names, monitoredobjects, monitoring indicators, alarm thresholds, monitoring periods, and notifications.

Prerequisites


Procedure






2018-10-29 58

Step 5 In the same row as the chosen queue, choose More > View Metrics.

Step 6 In the navigation pane, choose Alarm Management > Alarm Rules.

Step 7 On the Alarm Rules page, click Create Alarm Rule to add an alarm rule, or modify anexisting alarm rule.

For more information about DMS alarm rules, see the Cloud Eye User Guide.

----End

5.3 Viewing Metrics on the Cloud Eye Console

Scenario

Cloud Eye monitors the following metrics in real time:

l Queue metrics– Queued Messages– Message Size– Request Count

l Consumer group metrics– Accumulated Messages– Consumed Messages– Skipped Messages– Available Dead Letter Messages– Historical Dead Letter Messages

Prerequisites

A queue has been created. The queue has at least one consumer group and at least onemessage.

Procedure





Step 5 Display the metrics you wish to view.l For queue metrics:

a. In the same row as the chosen queue, choose More > View Metrics.b. Click View Graph. The Cloud Eye console then displays metrics of the queue.

l For consumer group metrics:


2018-10-29 59

a. Click the name of a queue for which you want to view details about the queue.b. Click the Consumer Groups tab to display the consumer group list.c. In the same row as the chosen consumer group, choose More > View Metrics.d. Click View Graph. The Cloud Eye console then displays metrics of the consumer

group.

NOTE

Metric values are reported to Cloud Eye every minute.

----End


2018-10-29 60

6 Auditing

6.1 Operations That Can Be Recorded by CTS

6.2 Viewing Traces on the CTS Console

6.1 Operations That Can Be Recorded by CTSWith CTS, you can record operations associated with DMS for later query, audit, andbacktrack operations.

Table 6-1 DMS operations that can be recorded by CTS

Operation ResourceType

Trace Name

Creating a queue queue createQueue

Deleting a queue queue deleteQueue

Creating a consumer group group createGroup

Deleting a consumer group group deleteGroup

Successfully creating a task forcreating a Kafka platinum instance

kafka createDMSInstanceTaskSuccess

Successfully creating a Kafkaplatinum instance

kafka createDMSInstanceSuccess

Failing to create a Kafka platinuminstance

kafka createDMSInstanceFailure

Successfully creating a task fordeleting a Kafka platinum instance

kafka deleteDMSInstanceTaskSuccess

Successfully deleting a Kafkaplatinum instance

kafka deleteDMSInstanceSuccess

Distributed Message ServiceUser Guide 6 Auditing

2018-10-29 61

Operation ResourceType

Trace Name

Successfully creating a task fordeleting Kafka platinum instancesin batches

kafka batchDeleteDMSInstanceTaskSuc-cess

Successfully creating a topic for aKafka platinum instance

kafka Kafka_platinum_create_topicSuccess

Successfully deleting a topic for aKafka platinum instance

kafka Kafka_platinum_delete_topicSuccess

Successfully creating an order forcreating a RabbitMQ instance

rabbitmq createDMSInstanceOrderSuccess

Successfully creating a task forcreating a RabbitMQ instance

rabbitmq createDMSInstanceTaskSuccess

Successfully creating a RabbitMQinstance

rabbitmq createDMSInstanceSuccess

Failing to create a RabbitMQinstance

rabbitmq createDMSInstanceFailure

Successfully creating a task fordeleting a RabbitMQ instance

rabbitmq deleteDMSInstanceTaskSuccess

Successfully deleting a RabbitMQinstance

rabbitmq deleteDMSInstanceSuccess

Successfully creating a task fordeleting RabbitMQ instances inbatches

rabbitmq batchDeleteDMSInstanceTaskSuc-cess

6.2 Viewing Traces on the CTS Console

Scenario

View traces of the last seven days on the CTS console.

Procedure



Step 3 Click Service List, and choose Management & Deployment > Cloud Trace Service.

Step 4 In the navigation pane, choose Trace List.

Step 5 Specify the filters used for querying traces. The following filters are available:l Trace Source: Select DMS.


2018-10-29 62

l Resource Type: Select group or queue.l Search By:

Select an option from the drop-down list.When you select Trace name, you also need to select a specific trace name.When you select Resource ID, you also need to select a specific resource ID.When you select Resource name, you also need to select a specific resource name.

l Operator: Select a specific operator (a user other than tenant).l Trace Status: Available options include All trace status, normal, warning, and

incident. You can only select one of them.l Start time and end time: You can specify the time period to query traces.

Step 6 Click on the left of a trace to expand its details.

Figure 6-1 Expanding trace details

Step 7 Click View Trace in the Operation column. In the displayed View Trace dialog box, thetrace structure details are displayed.

Figure 6-2 View Trace

----End


2018-10-29 63

7 FAQs

7.1 Does DMS Support FIFO Delivery?

7.2 Does DMS Support "At Least Once" Message Delivery?

7.3 How Long Can a Message be Retained in a Queue?

7.4 What Is the Maximum Size of a Message?

7.5 Does DMS Guarantee Zero Message Repetition?

7.6 How Does DMS Ensure Message Security in a Queue?

7.7 How Many Queues Can Be Created Under a Project?

7.8 Why Can't I Retrieve the Chosen Number of Messages from the Queue That ContainsOnly a Few Messages?

7.9 Why Do Consumers Need to Acknowledge Messages?

7.10 Will Duplicate Messages Be Retrieved After "Delay Message Delivery" Is Enabled?

7.11 Will Duplicate Messages Be Retrieved After Redelivery?

7.12 What Are the Differences Between Kafka Queues and Kafka Platinum Instances?

7.13 Do RabbitMQ or Kafka Platinum Instances Support Cross-VPC Access

7.14 Do RabbitMQ or Kafka Platinum Instances Support Cross-Subnet Access?

7.15 How to Select and Configure a Security Group?

7.16 Can I Access DMS Over a Public Network?

7.17 Where Does a Newly Created Consumer Group Start Message Retrieval?

7.18 How Is the Performance of Global FIFO Queues?

7.19 Why Can't I View the Subnet and Security Group Information When Creating a DMSInstance?

7.1 Does DMS Support FIFO Delivery?When creating a queue using DMS, you can select Standard and then Global FIFO. In aglobal FIFO queue, messages are retrieved in the order they were sent. Global FIFO queues

Distributed Message ServiceUser Guide 7 FAQs

2018-10-29 64

are applicable to scenarios where the order of messages is important. In a partition-level FIFOqueue, messages might be retrieved in an order different from which they were sent. Partition-level FIFO queues are applicable to scenarios where high concurrency is important.

7.2 Does DMS Support "At Least Once" MessageDelivery?

Yes. DMS stores message replicas in multiple servers to achieve redundancy and highavailability. In rare cases, a server storing message replicas may be unavailable when theclient requests or deletes messages. If this happens, the message replicas will not be deletedfrom that server and may be sent when the connection is restored.

This process is called "at-least-once" delivery. To avoid any adverse impact from processingthe same message multiple times, ensure that your application processes messagesidempotently.

7.3 How Long Can a Message be Retained in a Queue?In standard and ActiveMQ queues, messages are retained for at least 72 hours and will bedeleted 72 hours later. However, in Kafka queues, messages are retained for 1 to 72 hours,depending on your choice.

7.4 What Is the Maximum Size of a Message?The size of each message, including the message body, labels, and attributes, cannot exceed512 KB.

Messages are sent one by one on the DMS console. The Message Body input box displays theremaining number of bytes allowed in a message as you are creating the message. If themessage size exceeds 512 KB, you cannot click OK to create the message.

Multiple messages can be sent by a single API, but the aggregated message size cannotexceed 512 KB. If the aggregated message size exceeds 512 KB, messages fail to be createdand the API caller receives an error message "The message size is {message size}, larger thanthe size limit {max allowed size}."

If the Kafka SDK is used to produce messages, the maximum size of a single message is 10MB. If the DMS console is used to produce messages, the maximum size of a single messageis 512 KB.

7.5 Does DMS Guarantee Zero Message Repetition?DMS does not guarantee zero message repetition. Messages may be sent more than once ifnetwork exceptions occur.

7.6 How Does DMS Ensure Message Security in a Queue?DMS uses a reliable authentication mechanism to prevent unauthorized access to messages.


2018-10-29 65

l Before you manage messages through the DMS console, you must complete youridentity authentication.

l If you use APIs to send and retrieve messages, you must obtain a token, and add theobtained token to the API request header. In addition, the API request must be signedusing the access key ID/secret access key (AK/SK pair), so that this API request can besent through the API gateway. For details on how to obtain a token and AK/SK pair, seethe DMS API Reference.

7.7 How Many Queues Can Be Created Under a Project?You can create multiple projects. By default, 30 queues can be created under a project. Tocreate more than 30 queues, contact customer service to increase your quota.

7.8 Why Can't I Retrieve the Chosen Number of Messagesfrom the Queue That Contains Only a Few Messages?

l Messages in a global FIFO queue are stored in a single partition. Therefore, eachretrieval request can return the chosen number of messages unless the available numberof messages in the queue is less than the chosen number of messages.

l Messages in other types of queues are distributed to multiple partitions. This achieveshigh concurrency, allowing multiple requests from a consumer group to be processed atthe same time. To avoid performance loss caused by cross-partition access andmanagement, a single retrieval request from a consumer group returns messages in onlyone partition. Therefore, when a queue holds a small number of available messages, asingle retrieval request may not return the chosen number of messages even if thenumber of available messages in the queue is greater than the chosen number ofmessages.For example, as shown in Figure 7-1, 12 messages exist in the queue and are distributedinto three partitions.


2018-10-29 66

Figure 7-1 A single retrieval request retrieves messages from only one partition

If the chosen number of messages is 10, only four messages can be retrieved fromPartition 0. After retrieval from Partition 0 is complete, messages are retrieved insequence from Partition 1 and Partition 2.

7.9 Why Do Consumers Need to Acknowledge Messages?The reliability in sending and retrieving messages is the result of joint efforts from DMS,message producers, and message consumers. DMS considers a message retrieved only after aconsumer acknowledges the message (that is, acknowledges the message retrieval).

While a message is being retrieved, it remains in the queue. It cannot be retrieved againwithin 30s since the start of retrieval. If the message is not acknowledged within this period,the message can be retrieved again.

NOTE

Only messages retrieved using APIs need to be acknowledged. Messages retrieved using the DMSconsole are automatically acknowledged.

7.10 Will Duplicate Messages Be Retrieved After "DelayMessage Delivery" Is Enabled?

Yes. After Delay Message Delivery is enabled, delivery of messages will be delayed for aspecified time period. In the event of a network error, messages may be retrieved in duplicate.


2018-10-29 67

Therefore, set your application as an idempotent application so it is not affected when itprocesses a message multiple times.

7.11 Will Duplicate Messages Be Retrieved AfterRedelivery?

Yes. Messages in standard queues can be flagged to be redelivered during acknowledgment byusing an SDK. Redelivered messages can be retrieved a second time.

In the event of a network error, redelivered messages may be retrieved in duplicate.Therefore, set your application as an idempotent application so it is not affected when itprocesses a message multiple times.

7.12 What Are the Differences Between Kafka Queues andKafka Platinum Instances?

Kafka queues and Kafka platinum instances differ in the following aspects:

l Creation:When you apply for a Kafka platinum instance, an exclusive instance is created and runson an ECS in a VPC. You can create multiple topics in a Kafka platinum instance andcustomize partitions. For a Kafka queue, only one topic can be created, and threepartitions are configured by default.

l Resource allocation:The Kafka platinum instance is a new service provided by DMS. After creating a Kafkaplatinum instance, you can exclusively use physically isolated resources in this instance.

l Client connection:Kafka queues require tenant identity authentication for logical isolation. Therefore, tokenauthentication is added to the DMS SDK by using the open-source Kafka SDK. Kafkaplatinum instances support client-based access in a VPC. With the secure access controlprovided by the VPC, you can access Kafka platinum instances by using the open-sourceSDK.

7.13 Do RabbitMQ or Kafka Platinum Instances SupportCross-VPC Access

Kafka platinum instances do not support cross-VPC access. VPCs are isolated from eachother, and ECSs cannot access Kafka platinum instances that belong to a different VPC fromthese ECSs.

RabbitMQ instances support cross-VPC access. By establishing a peering connection betweentwo VPCs, ECSs in one VPC can access RabbitMQ instances in the other VPC.

For more information on how to create and use VPC peering connections, see VPC PeeringConnection.


2018-10-29 68

http://support.huaweicloud.com/en-us/usermanual-vpc/en-us_topic_0046655035.html

http://support.huaweicloud.com/en-us/usermanual-vpc/en-us_topic_0046655035.html

7.14 Do RabbitMQ or Kafka Platinum Instances SupportCross-Subnet Access?

No. To access a RabbitMQ or Kafka platinum instance, you must deploy your client on anECS in the same VPC and subnet as the RabbitMQ or Kafka platinum instance.

7.15 How to Select and Configure a Security Group?Kafka platinum instances only support intra-VPC access and RabbitMQ instances supportpublic network access. The following describes how to configure a security group.

Intra-VPC Access

To access a RabbitMQ or Kafka platinum instance, you must deploy your client on an ECS inthe same VPC and subnet as the RabbitMQ or Kafka platinum instance.

In addition, before you can access the instance through your client, you must configurecorrect rules for the security groups of both the ECS and RabbitMQ or Kafka platinuminstance.

1. You are advised to configure the same security group for the ECS and RabbitMQ orKafka platinum instance. The security group contains a rule allowing any network accessfrom within the group by default.

2. If different security groups are configured, refer to the following configurations:

NOTE

l The following example assumes that security groups sg-ECS and sg-RabbitMQ areconfigured for the ECS and RabbitMQ instance respectively.

l You can specify a security group or IP address as the remote end in the following rules.

To ensure that your client can access the RabbitMQ instance, add the following rule tothe security group configured for the ECS:

To ensure that your client can access the RabbitMQ instance, add the following rule tothe security group configured for the RabbitMQ instance:


2018-10-29 69

Public AccessA client can access a RabbitMQ instance only after rules are correctly configured for thesecurity group of the instance.

For example, for security group sg-RabbitMQ, you need to configure either of the followingrules in the inbound direction:

1. Protocol: TCP; port number: 5672; source IP address: 0.0.0.0/02. Protocol: Any; source IP address: 0.0.0.0/0

Figure 7-2 and Figure 7-3 show the rules.

Figure 7-2 Rule 1 for the security group


2018-10-29 70

Figure 7-3 Rule 2 for the security group

7.16 Can I Access DMS Over a Public Network?Whether you can access DMS over a public network depends on the DMS queue type andaccess mode.l RESTful API

Public access is supported. You can access standard, Kafka, and ActiveMQ queues overa public network by using a RESTful API.

l TCP APIPublic access is not supported. To access DMS by using a TCP API, you must deployyour message client on an ECS.

l Open-source Kafka APIPublic access is supported. Alternatively, you can access DMS from an ECS. Note thatthe DMS endpoints differ by access mode.

l ActiveMQ APIPublic access is not supported. To access DMS by using an ActiveMQ API, you mustfirst apply for an ECS on the HUAWEI CLOUD management console.

l Kafka platinum instancePublic access is not supported. To access a Kafka platinum instance over a publicnetwork, you must apply for an ECS on the HUAWEI CLOUD management console.

l RabbitMQ instancePublic access is supported. For details, see section 3.11 Enabling Public Access to anInstance.

7.17 Where Does a Newly Created Consumer Group StartMessage Retrieval?

l Kafka API– When a newly created consumer group starts message retrieval for the first time, it

ignores all messages that have already been sent, and only retrieves messages sentafter its creation. To set the message retrieval start position to the earliest message,change the default value of auto.offset.reset from latest to earliest in theConsumer configuration on your Kafka client.


2018-10-29 71

– When the consumer group starts message retrieval for a second time, it retrievesmessages starting from the last retrieval position.

– To start message retrieval from a specific position, reset the retrieval start positionon the DMS console as required. The reset takes effect only for the queue of theconfigured consumer group, and does not affect other consumer groups.

l RESTful API/TCP API/ActiveMQ API– When a consumer group starts message retrieval for the first time, it retrieves

messages starting from the earliest in the queue.– When the consumer group starts message retrieval for a second time, it retrieves

messages starting from the last retrieval position.– To start message retrieval from a specific position, reset the retrieval start position

on the DMS console as required. The reset takes effect only for the queue of theconfigured consumer group, and does not affect other consumer groups.

7.18 How Is the Performance of Global FIFO Queues?Compared with partition-level FIFO queues, global FIFO queues have restrictions inapplication, deteriorating their performance. For example, to ensure that messages areglobally retrieved in the order they were sent, you cannot configure multiple partitions toimprove concurrent message retrieval.

In global FIFO queue mode, the message retrieval order is ensured by DMS queues, andplaces requirements on your service.

l Messages must be created in order and then sent to message queues in synchronousmode, which is less efficient than asynchronous message transmission mode.

l Messages must be retrieved and acknowledged in order, and messages in a queue cannotbe concurrently consumed using multiple clients or threads.

You are advised to use the global FIFO queue mode in scenarios with high requirements forordered message processing, such as processing transactions or financial transactionmessages.

7.19 Why Can't I View the Subnet and Security GroupInformation When Creating a DMS Instance?

This may be because you do not have the permissions of the server administrator and VPCadministrator. For details on how to add user permissions, see Changing User Permissions.


2018-10-29 72

https://support.huaweicloud.com/en-us/usermanual-iam/en-us_topic_0080335069.html

8 Change History

Released On What's New

2018-10-29 This issue is the fifteenth official release, which incorporates thefollowing changes:l Revised description in the previous issue.

2018-08-15 This issue is the fourteenth official release, which incorporates thefollowing changes:l Added the function of enabling SSL for RabbitMQ instances.l Added section 3.8 Connecting to the Management Address of a

RabbitMQ Instance.l Added section 3.9 Accessing a RabbitMQ Instance in SSL Mode.l Added the function of enabling SASL_SSL for Kafka platinum

instances.l Added section 4.10 Connecting to the Kafka Manager.l Added section 4.11 Accessing a Kafka Platinum Instance in

SASL_SSL Mode.l Added section 4.12 Resetting the Password of an Instance.l Added support for enabling synchronous replication when topics are

created for Kafka platinum instances.

2018-07-10 This issue is the thirteenth official release, which incorporates thefollowing changes:l Added section 3.10 Resetting the Password of an Instance.l Added public access to RabbitMQ instances.l Added cross-VPC access to RabbitMQ instances.l Added section 7.19 Why Can't I View the Subnet and Security

Group Information When Creating a DMS Instance?.

Distributed Message ServiceUser Guide 8 Change History

2018-10-29 73


2018-06-15 This issue is the twelfth official release, which incorporates the followingchanges:l Added the Kafka platinum instance function.l Added section 4.7-Accessing a RabbitMQ Instance in chapter 3

Managing RabbitMQ Instances.l Added chapter 7 FAQs.

2018-05-14 This issue is the eleventh official release, which incorporates thefollowing changes:l Added the message query function.l Added the function of viewing the retrieval status.l Added the RabbitMQ instance function.

2018-04-20 This issue is the tenth official release, which incorporates the followingchanges:l Added chapters 5 Monitoring and 6 Auditing.

2018-03-15 This issue is the ninth official release, which incorporates the followingchanges:l Added the AMQP queue type.l Added the Delay Message Delivery function to messages in normal

and FIFO queues.l Revised description in the previous issue.

2018-01-15 This issue is the eighth official release, which incorporates the followingchanges:l Told users that the retention period of messages in a Kafka queue is

user configurable.l Revised description in the previous issue.

2017-12-18 This issue is the seventh official release, which incorporates thefollowing changes:l Revised description in the previous issue.

2017-10-16 This issue is the sixth official release, which incorporates the followingchanges:l Added description of message labels.l Added description of Kafka queues.l Added section 2.5 Managing Queue Policies.


2018-10-29 74


2017-08-17 This issue is the fifth official release, which incorporates the followingchanges:l In section 1.1 Creating a Queue, added the Dead Letter Queue

parameter.l In section 1.4 Retrieving Messages, add the Message Type

parameter.l Added section 2.4 Resetting the Message Retrieval Start Position.

2017-05-31 This issue is the fourth official release, which incorporates the followingchanges:l Added five Cloud Eye monitoring metrics.l Added the Queue Type parameter.

2017-04-21 This issue is the third official release, which incorporates the followingchanges:l Revised description in the previous issue.

2017-02-13 This issue is the second official release, which incorporates the followingchanges:l Revised description in the previous issue.

2016-08-25 This issue is the first official release.


2018-10-29 75

Documents

User Guide - HUAWEI CLOUD...Contents 1 Getting Started.....1