Federated Design Guidelines for EMC ViPR SRM 3 - Meetupfiles.meetup.com/...Federated-Design-Guidelines-for-ViPR-SRM-3.7.pdf · White paper . FEDERATED DESIGN GUIDELINES FOR EMC VIPR

White paper

FEDERATED DESIGN GUIDELINES FOR EMC VIPR SRM 3.7

Abstract

This white paper explains how to achieve a federated design with EMC ViPR SRM 3.7. October 2015

FEDERATED DESIGN GUIDELINES FOR EMC ViPR SRM 3.7

Copyright © 2015 EMC Corporation. All Rights Reserved. EMC believes the information in this publication is accurate of its publication date. The information is subject to change without notice. The information in this publication is provided “as is.” EMC Corporation makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. For the most up to date listing of EMC product names, see EMC Corporation Trademarks on EMC.com. VMware, VMware vCenter, and VMware View are registered trademarks or trademarks of VMware, Inc. in the United States and/or other jurisdictions. All other trademarks used herein are the property of their respective owners. Part Number H14600

2


Table of Contents

Executive summary.................................................................................................. 4 About ViPR SRM ................................................................................................................. 4 Audience ............................................................................................................................ 4 Solution scope ................................................................................................................... 4

Terminology ............................................................................................................ 5

Multi-Sites Design Patterns ...................................................................................... 6 Scenario #1 – Centralized Site ............................................................................................ 6 Scenario #2 – Isolated Sites ............................................................................................... 7 Scenario #3 – Centralized + Isolated Sites .......................................................................... 7 Scenario #4 – Low Latency ................................................................................................. 8

Considerations ........................................................................................................ 9

Calculating the network bandwidth .......................................................................... 9

Data Gateway ........................................................................................................ 10 Introduction ..................................................................................................................... 10 Assumptions .................................................................................................................... 11 Requirements ................................................................................................................... 11 Installing the Data Gateway .............................................................................................. 11

From the Centralized-Management Interface ................................................................ 11 From the command line .................................................................................................... 14 Reconfiguring the Data Gateway ....................................................................................... 15 Deep dive ......................................................................................................................... 16 Multi-Site Patterns with the Data Gateway ........................................................................ 19 Option #1 - Mirroring ........................................................................................................ 20

Necessary changes on the Regional Backends ............................................................. 20 Option #2 - Rebalancing ................................................................................................... 21 Option 3 – Collectors ........................................................................................................ 22

Appendix A – Performing Data Enrichment ............................................................. 24 From the Centralized-Management Interface .................................................................... 24

Appendix B – Building the Federated Solution ........................................................ 27 Configuring the Federated Frontend .................................................................................. 27 Configuring the Backend instances .................................................................................. 28

Granting Permissions ................................................................................................... 28 Network and Firewall access ......................................................................................... 29

Appendix C – Expanding the default configuration ................................................. 30

3


Executive summary This article provides general guidelines on how to achieve a Federated Design for EMC ViPR SRM 3.7.

About ViPR SRM

ViPR SRM leverages state-of-the-art visibility and forecasting from various devices and technologies, processing and storing millions of indicators. When designing a ViPR SRM solution, it should be taken into consideration how to avoid common availability pitfalls and, in the case of a failure, how to gracefully reestablish the services to maintain consistency and performance.

Audience

This white paper is intended for Solution Architects, Customer Support, and EMC partners interested in knowing the current recommended architecture to achieve a Federated Design.

Solution scope

The proposed solutions described in this white paper are tailored towards time series data with accompanying procedures and instrumental building blocks. This is to say that other data types present in SRM are not covered here, such as events, flows (Netflows), topology and alerting.

4


Terminology

Term Definition Load Balancer Arbiter The Arbiter is the main component responsible for

deciding where to send metrics related to a particular device. The Arbiter will generate a key-value list (balancing list) containing entries for devices and Backends.

Load Balancer Connector The Load Balancer Connector is capable of sending known device metrics to their corresponding Backend based on the balancing list generated by the Arbiter.

Property Tagging Filter The Property Tagging Filter has been designed to add new properties to raw values based on the properties they already have.

Variable Handling Filter The Variable Handling Filter is designed to allow variables to be handled in a specific way depending on the criteria on which they match. As an example, it is possible to block or allow Metrics that fit under a filter condition.

vApp ViPR SRM is offered as a binary install as well as a VMware Virtual Appliance (vApp).

5


Multi-Sites Design Patterns

Scenario #1 – Centralized Site

The first design pattern presented here is where the collectors are deployed under various regions and the data stream flows into a centralized datacenter where the Frontend and Backend instances will be deployed. This pattern could be used for example for customers that have smaller satellite offices and that are not looking for a full deployment of SRM under each one of them but would still want to do have the visibility of their IT resources.

Having this perspective, it would be possible to deploy the vApps with the Collector only option over the satellite offices and a full install of the vApp on the centralized location.

Figure 1 Centralized Site

6


Scenario #2 – Isolated Sites

Figure 2 Isolated Sites

Isolated sites, by themselves, are not what one would think of as a Federated solution. Understanding that this argument holds itself true, the isolated sites are natural candidates for a Federation design. For one, once customers have done the discovery of their devices (on a per location basis), we can leverage the power of data enrichment and add the metadata for the devices discovered (e.g.: location, business units, departments, owner, etc). This newly added information will serve as the base for customized reports on the centralize site.

Scenario #3 – Centralized + Isolated Sites

Augmenting the previous example, this is a mixed scenario with users under each datacenter being able to access their regional information through on premise Frontends, as well as a central designated location (Figure 3).

The benefit of this approach is that of keeping Report generation performance at an optimal level, as the databases are close to the Frontend. To achieve this, a copy of all of the regional databases to the central datacenter is needed. This will involve the deployment of more Backend virtual machines with the proper compute, network and storage requirements. In effect, the central federated site will have the sum of all of the regional databases or a gross amount close to it.

7


Figure 3 Centralized and Isolated Sites

Scenario #4 – Low Latency

Figure 4 Low latency

For this solution, the number one concern is latency. This implementation will trade-off the Report generation, including property updates that come from the Property-Store refreshment task, making it slower as the Frontend needs to traverse a longer path to retrieve the data points for the reports to be fully generated. Depending on the particular case, the Property-Store could take several hours to complete.

8


Considerations Architecting for Federation involves certain consideration and trade-offs. There are several questions that one should ask prior to sketching and presenting a Federation design. Overall, there are three main topics that need to be addressed: Data locality, Latency and Bandwidth.

Consideration Concept Trade-offs Data Locality

Bring databases closer to the Federated Frontend.

Number of required databases and bandwidth between the collectors and the Backends.

Latency

This is a key consideration as it has a direct impact on report generation and the import of properties. Schedules Reports are a possible and viable solution to enhance the customer experience.

Report generation will be impacted significantly if the Latency between Frontend and Backend goes beyond the recommended value of 5ms. The report generation time may be impacted, especially for the reports with several thousands of rows and more formulas.

Bandwidth Collected data needs to be transferred from the Collection layer to the Backends.

The amount of needed bandwidth to move metrics, under a small pooling period, could be prohibitive under certain scenarios due to the impact on traversing the WAN. More details below.

Calculating the network bandwidth Typically, one metric (point in time) is close to 600 bytes in average. To illustrate, in a case where 4000000 (4 million) metrics are being collected we estimated the total traffic between all APG Collectors to all APG Backend to be 64Mbits/s. Here is how this number is computed:

600 bytes x 4000000 = 2400000000 bytes

2400000000 bytes / 1000000 = 2400 Mbytes

2400 Mbytes * 8 = 19200 Mbits

19200 Mbits / 300 (5 mins polling) = 64Mbits/s (for each 5 minutes polling)

9


Data Gateway

Introduction

Figure 5 Federated Design Overview

The Data Gateway can essentially duplicate that information that it receives from another collecting section and send that to one or more parties. With a Data Gateway in place it is possible to replicate the same information on different databases, thereby making it possible to have local, regional and global databases.

At the proposed architecture depicted on Figure 5, the Data Gateway will be placed in front of each Backend, listening on the TCP port for that particular Backend. Its operation consists of duplicating the flow of data and forwarding it to a remote Backend. This configuration eases the management of this architecture, as there is no need for any change on the Collection Layer.

10


Throughout this document we will review other design patterns, including the one above.

Assumptions

The following discussion takes into illustrate the use of a Federated Design when deploying SRM via a virtual appliance solution (vApp). The same concepts still apply when a binary installation is chosen.

vApp was chosen due to its value proposition, that is, speed of deployment, consistency and easy of scalability. This is not so say that binary cannot be used. At times, when evaluating what option to choose from (vApp or binary), one must evaluate the environment that SRM will be deployed and verify which option fits better.

Requirements

There are a few parameters that need to be taken into consideration when deploying the Data Gateway and moving towards a Federated Design.

It is important to understand that there is not a single solution for a Federation but different approaches. Careful evaluation of the customer environment and knowledge of the trade-offs should be taken into consideration.

Component Requirements

Primary Backend (1 database)

2 GB of extra RAM/120 GB extra for Storage

Scale-Out Backend (up to 4 databases)

8 GB of extra RAM/ up to 480 GB extra for Storage

Table 1 Backend requirements

Installing the Data Gateway

There are two options when installing the new Data Gateway component, from the command line interface or through the Centralized Management. We will demonstrate both approaches in this white paper.

NOTE: Currently the Data Gateway is not being shipped with the default ViPR SRM releases. In order to have access to it, please contact the ASD Product Delivery Engineering team at https://asdcse.sites.emc.com/default.aspx

From the Centralized-Management Interface

1. Navigate to the Centralized-Management, click on the server you wish to install the Data Gateway-collector and then click on the Block category.

11

https://asdcse.sites.emc.com/default.aspx


2. Choose the block for the Data Gateway (Figure 6).

Figure 6 Installing from the Centralized-Management Interface

3. Click on the launch button and proceed with the installation (Figure 7).

Figure 7 Installation Steps

12


4. Next, answer the questions related to the Global and Regional backend. The following table summarizes these questions.

Question Comments

Data Gateway Socket Collector Listening Port.

The network port where the collectors will send data to be routed to the Regional and Global Backends. Default value: 2060.

Hostname or IP address to send data to (Global).

The hostname or IP address of the collector host where Load Balancer Connector on the Global site.

Network port to send data to (Global).

The network port of the collector host where Load Balancer Connector is installed on the Global site. Default value: 2020.

Hostname or IP address to send data to (Regional).

The hostname or IP address of the collector host where Load Balancer Connector on the Regional site. By default, 127.0.0.1

Network port to send data to (Regional).

The network port of the collector host where Load Balancer Connector is installed (Regional). Default value: 2020

Table 2 Data Gateway configuration

The questions above will vary depending on the Federated solution adopted. For more information, see the section on Multi-Site Patterns with Data Gateway for more information.

5. Finally, in order to be able to filter the raw data flow, register the Data Enrichment module on the Centralized-Management.

Figure 8 Registering for Data Enrichment

13


From the command line

1. When installing from the command line, it is necessary to log in as root on all of the VMs where the Backend are present and install the necessary building blocks for the Data Gateway. To begin with, let’s install the collector-manager. The other components that make the Data Gateway will be installed later.

[root@SRM bin]# ./manage-modules.sh install data-gateway Required dependencies, in processing order: [1] java '8.0u25' v8.0u25 [2] module-manager '1.7u1' v1.7u1 [3] license-manager 'Default' v5.4 [4] I collector-manager 'Data-Gateway' (none) => v5.6 [5] I failover-filter 'Data-Gateway' (none) => v5.2 [6] I variable-handling-filter 'Data-Gateway' (none) => v1.13 [7] jdbc-drivers 'Default' v2.3 [8] I property-tagging-filter 'Data-Gateway' (none) => v2.8 [9] I data-gateway 'Data-Gateway' (none) => v3.6.3u99 > 4 not modified, 5 to install > 11.5 MB space required / 31 GB available ? Enter the step to modify, 'yes' to accept them, or 'no' to cancel the operation [yes] > […] Starting installation of data-gateway v3.6.3u99 from data-gateway-3.6.3u99... * Gathering information... * 'data-gateway v3.6.3u99' will be registered with instance name 'Data-Gateway'. * It will be installed in '/opt/APG/Block/data-gateway/Data-Gateway'. * Unpacking files... * Installing files... 100% * 9 files have been installed. * Finalizing installation... ? Data Gateway socket collector listening port [2060] > ? Hostname or IP address to send data to (Global) > global_fqdn ? Network port to send data to (Global) [2020] > ? Hostname or IP address to send data to (Regional) [127.0.0.1] > ? Network port to send data to (Regional) [2020] > ? Do you want to start the installed services now? (yes/no) [y] > * Starting 'collector-manager Data-Gateway'... [ OK ] Installation complete.

Table 3 Data Gateway install

14


Reconfiguring the Data Gateway

It is always possible to reconfigure the Data Gateway SolutionPack Block by navigating to the SolutionPacks->Other Components section on Centralized Management.

Figure 9 Reconfiguring the Data Gateway Solution Pack Block

15


Deep dive

The above diagram depicts the internals of the Data Gateway. The Socket Collector is the raw data stream entry point. Here the Data Gateway will listen on a defined TCP port. If not changed by the user, it will listen on port 2060.

When using the Data Gateway as a simple replication tool, a copy of the incoming data stream will be placed into each one of the outbound socket connectors. Out of the box, this is the expected behavior.

Metrics can also be filtered on the data stream directed to the Global Backend (right side of the Figure 10). For that to happen, a Property-Tagging-Filter facility is provided

Figure 10 Data Gateway details

Figure 11 Filtering metrics

16


so that users can use the Centralized-Management interface to block/allow metrics (Figure 11).

On the next step, in front of the Property-Tagging-Filter (PTF), there is a Variable Handling Filter (defined under VHF-Global.xml) whose sole purpose is to filter metrics based on a property called isglobal. This property could be defined earlier on the PTF for all of the metrics that are not supposed to be sent to the Global Backend.

http://www.watch4net.com/VariableHandlingFilter variable-handling-filter.xsd "> <handling-configuration id="BlockMetrics"> <matcher class="com.watch4net.apg.v2.collector.plugins.variablehandlingfilter.matchers.APGFilterMatcher"> <parameter name="filter">!isglobal=='true'</parameter> </matcher> <handler class="com.watch4net.apg.v2.collector.plugins.variablehandlingfilter.handlers.BlockAllHandler"/> </handling-configuration> </configuration>

Figure 12 Blocking filters inside of the VHF-Global

For example, on the Global Datacenter, after discussing with your customer about his expectations for what kind of reports he is looking for under the Global Datacenter, a subset of the metrics are needed. This is key to proper architecture of the solution as it directly implicates in less bandwidth and less databases on the Global datacenter. As a working use case, let’s assume that the set of custom reports will be tailored to Capacity Planning which will mostly be reviewed by upper management. In this case, we do not need to have all of the metrics being collected from all of our devices. The approach here should be, start from the mockup of the custom reports and then make your way back to understand what metrics are needed. Doing the opposite is an error-prone and time-consuming approach. Under this example we will only include information related to capacity metrics (e.g.: FreeSpace, UsedCapacity, FreeCapacity, etc.) so one could drop all of the metrics that are related to performance which will not bring any value to these reports. In this case, metrics such as ReadRequests, WriteRequests or those with the unit property set to “MB/s”, “IOPS”,”Pkts/s” or the like should be blocked.

Note: At any time, when removing metrics from the data flow, it is also important to modify the default reports on the Frontend that would otherwise rely on these metrics. In general this is one of the singular points of every Federated Design. That is, to decide what should be presented on the Frontend of the Global instance. If not properly planned and understood, the lack of metrics could result in half populated reports, with blank columns and missing data onto specific rows. On the other hand, the fact that a filter is in place will represent shrink the amount of bandwidth required to ship the data from the Regional datacenter to the Global one.

17


The PTF facility (Figure 11) allows for customization of what metrics should be filtered, meaning that the user can add or remove columns to be used as a filter with the exception of the property isglobal, which is required for the VHF filter. Other than that, users can add new parameters for their filters as needed. On Figure 11 we are using two keys for matching our rule: the metric name (defined as the name property) and the metric unit (defined as the property unit), but this is not a general rule, just a suggestion. Table 4 provides more examples that could be used as filters.

The task of mining for properties which could be used to filter out metrics can be daunting as the available data could be of a significant size. One option available is to use the Management of Database Metrics to explore the properties currently attached to a given metric. Another handy and extremely powerful tool is the mnrcli that, as stated on Gitbhub where the tool is hosted, provides an interactive console to navigate into EMC ViPR SRM and EMC Service Assurance Suite data.

Filter Condition

Individual metrics name=='ReadRequests’ | name=='WriteRequests'

Units unit=='Pkts/s' | unit=='MB/s' | unit=='IOPS' or

unit=’%/s’ | unit==’IOPS’

Performance Metrics for Arrays

devtype==’Array’ & name=’%Requests%’

Table 4 Example of filters

The Failover filter is a component that will enhance the data availability, working as a buffer zone for the situations where the connection to the remote Backend is interrupted. Under these circumstances, the Failover filter will store the data stream locally on files until the connection is reestablished with the remote Backend.

By default, for each of the two Failover filters installed on the Data Gateway, it is possible to retain close to 60 GB with a real-time value of approximately 128 Bytes (Figure 13) summing up to 120GB (Table 1 Backend requirements). When installing the Data Gateway on the Collectors, make sure that these values are considered.

<?xml version="1.0" encoding="UTF-8"?> <config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.watch4net.com/APG/Filter/Failover config.xsd" xmlns="http://www.watch4net.com/APG/Filter/Failover">

<availability-check period="60000" /> <local-queue size="10000" buffer-compensation="5000" /> <file-transfer buffer-size="32768" />

<temporary-storage path="tmp-regional" file-size="100000" max-capacity="461373440” /> </config>

Figure 13 Regional Failover-Filter

18


The last components are the Socket Connectors that bridge the data flow next endpoint, sending the flow of data to another component defined on its configuration by an IP and port.

Note: If your project requires more filters and modifications to what’s currently available on the Data Gateway SolutionPack Block, you can use the provided Data Gateway as a base to deploy your customized solution and then make the necessary modifications and add-ons on top of that. On ViPR SRM 3.7, there is a mechanism that will detect if a file was modified, for instance the default collecting.xml, and if it can still be used with the current iteration of the SolutionPack. See Appendix C for more details.

Multi-Site Patterns with the Data Gateway

We already discussed some of the reference design for the multi-site Federation. Here we will present three solutions that could be used to implement the discussed patterns, each one with its advantages and trade-offs.

Each of the three approaches have their own pros and cons. For instance, on the one hand, option 1 is easier to implement, it requires less components to be installed on the global site and on the regional site. On the other hand, this approach will require the same amount of databases to be installed on the global site. For instance, if the sum of the databases from the regional websites accounts for 20 Additional Backends, the same number of virtual machines needs to be configured on the global site. This implies careful planning in terms of resources (CPU, Storage and Network).

Options Data Gateway Location

Advantages Trade-offs

1 In front of the Backends (Mirroring)

Fewer components need to be installed.

Require the same number of databases on the Primary site and on the secondary site (this is true when mirroring the databases).

2 In front of the Backends + Remote LBC

Fewer components need to be installed. No need for the extra databases to be deployed on the remote site upfront.

No guarantee that the metrics will reside on the same databases on both datacenters.

3 Before the Load-Balancer Connector

Ratio of 1:1 with the LBCs. Can ship data to a remote site and use another LBC there (see diagram below). This saves the architecture of having to put extra databases upfront on the remote site.

More components need to be installed than Option #1.

19


Option #1 - Mirroring

The core value of Option 1 is for a customer who is looking for a mirroring solution. In practical terms, that is to say that a given metric A, found on the database apg1 at the Regional datacenter will also be found on the Global datacenter under the database apg1.

On the Global Datacenter there will be more instances of the Scale-Out Backend that should be mapped to a 1:1 ratio with the Regional Backends. The sole difference will be for the mapping between the Regional Primary Backend, where the ‘apg’ database is located and a scale-out backend instance on the Global Datacenter. In this case, it is possible to fit up to 4 Primary Backend databases into one Scale-out Backend (Figure 6).

Necessary changes on the Regional Backends

As part of the newly installed Data Gateway, a Socket-Collector was installed, which by default, listens on the TCP port 2060. This port needs to be modified so that it now listens on the Backend port.

Remember, as we are now in front of the Backend, intercepting all of the traffic directed to it, this socket collector will listen on what was initially the Backend data port. This port is defined under: /opt/APG/Backends/APG-Backend/${Backend Instance}/conf/socketinterface.xml

Figure 14 Option #1 - Mirroring

20


By default, these are the port numbers for the Scale-Out Backend vApp.

Scale-Out Backend Instance

Original Port New Port

apg1 2100 3100

apg2 2200 3200

apg3 2300 3300

apg4 2400 3400

Table 5 Scale-Out Backend default ports

Option #2 - Rebalancing

The second option (Rebalancing) is a variation of the first option, whereas before there was a 1:1 relation between the databases on the Regional site and on the Global site which implies having to plan for all of the necessary databases upfront, we now have the possibility to use a Load Balancer Connector on the Global site to spread the metrics to another set, albeit smaller, set of databases.

21


This is likely to be the case when metrics are being filtered before being shipped to the Global site. The number of databases can be increased if needed as the LBC will be instructed by a local Arbiter.

As detailed on Option #1 the changes to the Backend for the Regional site still apply in this case.

Option 3 – Collectors

This method is the only one for which there is no need to modify any of the Backend configuration on the Regional site. The split of the raw data will happen at the collection layer resulting in one Data Gateway instance per Load Balancer Connector instance. The Data Gateway will be placed between the LBC and the Collector Manager instances. As in option #2 the raw data will flow from the Regional sites into the Global site landing on a LBC.

One possible way to configure this scenario is to swap the port numbers that the LBC and the Data Gateway listen on by default. This means that the LBC will listen on port 2060 and the

22


Data Gateway will listen on port 2020. By doing this, we avoid having to reconfigure all of the SolutionPacks to point to the Data Gateway, reducing the amount of changes to only two.

The following table is summarizes the configuration changes on the Data Gateway.

Parameter Value

Data Gateway Socket Collector Listening Port. 2020

Hostname or IP address to send data to (Global).

Global FQDN/IP

Network port to send data to (Global). 2020

Hostname or IP address to send data to (Regional).

127.0.0.1

Network port to send data to (Regional). 2060

23


Appendix A – Performing Data Enrichment One common scenario for a Federated Design relates to the Global datacenter where key business decisions are made, such as those related to capacity planning and growth. Within this scenario, it becomes interesting to know what datacenter or customer the data set being analyzed belongs to. This situation can be addressed with the Property-Tagging-Filter (PTF).

The Load Balancer Connector (LBC), present on all of the Collector VMs, is the perfect fit for this task. The LBC has more than one placeholder for data enrichment, and as the metrics have to flow through the LBC, they will be evenly tagged before reaching any of the Backends.

From the Centralized-Management Interface

1. Navigate to the Centralized-Management then click on Data Enrichment found on the left side menu.

Figure 15 Installing from the Centralized-Management Interface

2. Click Register a new module. Select a Collector VM (under Servers), Collecting (under Categories) and Collector-Manager::Load-Balancer::PTF-Location (under Modules) then click on the Register button to proceed. Do this process for all of the collectors.

IMPORTANT: Repeat the previous step for all of the Collector servers in the data center ViPR SRM deployment. Having all of the Load Balancer Connectors registered is a key component of the tagging strategy. You can register new Load Balancer Connectors after the fact (i.e.: with the addition of new collectors).

24


Figure 16 Registering the new module.

3. Expand the location field and enter the specific information that reflects the datacenter where the collector resides. The keyword in order to tag all of the devices, is @MATCHALL under the “Device type” and “Device name” keys. Next, input the name of the data center location under the “Device Location” property. The following illustrates a tagging configuration for the NYC data center and the same construct should be followed by the other locations.

Figure 17 Tagging all of the devices with the data center location

4. After all of the modifications are in place, click Save.

5. On the Save Data Enrichment page, select all of the servers that were previously registered (Step 2) and click Update. This will synchronize the tagging information done on step 3.

25


Figure 18 Synchronizing the new Data Enrichment

The newly created property, location, will be available on the Frontend and can be used on Reports.

26


Appendix B – Building the Federated Solution

Configuring the Federated Frontend

The most common solutions to install the Federated Frontend are either using a vApp install, or a binary install. Although both methods will result in the same result, we will demonstrate here the vApp option only. For more information on the binary installation please refer to the EMC ViPR SRM Installation and Configuration Guide.

For the vApp solution we can deploy the All-in-One (AIO) VM which will contain the entire set of tools needed for the Federated Frontend, which includes a local MySQL database, the emc-watch4net health collector (to gauge the Frontend own health).

After the vApp is deployed we are now ready to manage the access to the other databases.

1. Logon to the Federated Frontend host.

2. Verify to server registration:

/opt/APG/bin/manage-resources.sh list

Output example: ./bin/manage-resources.sh list [ "dba/FLOW-TF-DATA", "dba/FLOW-RPE2-LIVE", "dba/FLOW-COMPLIANCE-POLICY", "dba/FLOW-SOM-ARCH", "dba/FLOW-UCS-LIVE", "dba/FLOW-OUTAGE-DB", "dba/FLOW-TF-APP", "dba/FLOW-TF-HOSTS", "dba/FLOW-VIPR-EVENTS", "dba/FLOW-VMWARE-EVENTS", "mgmt/APG-DB", "dba/FLOW-COMPLIANCE-BREACH", "dba/FLOW-TF-DSTADDR", "dba/FLOW-EVENTS-GENERIC", "dba/FLOW-EVENTS-GENERICARCH", "dba/FLOW-WHATIF-SCENARIOS", "dba/FLOW-TF-ALLPORTS", "dba/FLOW-COMPLIANCE-CONFIGCHANGE", "dba/APG-DB", "dba/FLOW-TF-CONV", "dba/FLOW-VNX-LIVE", "dba/FLOW-PROSPHERE-LIVE", "dba/FLOW-VMWARE-TASKS", "dba/FLOW-COMPLIANCE-RULE", "dba/FLOW-SOM-LIVE", "dba/FLOW-TF-PROTO", "dba/FLOW-TF-NOSNMP", "dba/FLOW-PROSPHERE-ARCH", "dba/FLOW-RPE2-ARCH", "dba/FLOW-TF-SRCADDR" ]

27


3. In case there is a requirement to remove access to a database from the Federated Frontend, this can be done through the manage-resources utility. To illustrate the process, this is how one would remove a FLOW database entry:

/opt/APG/bin/manage-resources.sh delete dba/FLOW-PROSPHERE-LIVE

4. To add a new time series database entry, use the following command:

/opt/APG/bin/manage-resources.sh create dba/APG-DB<DATACENTER_IDENTIFIER>? "{ 'type': 'jdbc', 'datasource': { 'maxActive': '10', 'maxIdle': '10', 'validationQuery': 'SELECT 1', 'testOnBorrow': 'false', 'testWhileIdle': 'true', 'validationQueryTimeout': '5', 'timeBetweenEvictionRunsMillis': '10000', 'minEvictableIdleTimeMillis': '60000', 'maxWait': '10000', 'removeAbandoned': 'true', 'removeAbandonedTimeout': '60', 'logAbandoned': 'true', 'driverClassName': 'com.mysql.jdbc.Driver', 'username': 'read_apg', 'password': ' {B1CD6397A44F1D312AEA544BFA424DA62FDE2F8623F8BBEBDC36CDB87F2C395BF916D6AB44C2627217BF48DE318E0BAE}', 'url': 'jdbc:mysql://${IP OF REMOTE DATABASE}:53306/apg<index>?autoReconnect=true' }, 'settings': {'cachegrp': 'DB'} } "

Notes:

• Ensure that each APG-DB entry is unique. A good naming convention is to append the name of the data center and its database index (e.g.: APG-DB-NYC1, APG-DB-NYC2, and so on). Also, make sure that the jdbc url contains the correct value for the apg database (e.g.: apg1, apg2, apg3, apg4).

• The mgmt/* entries will provide access to the Management of Database Metrics, assure that those are configured.

For more information please refer to the APG-Administration-Tool found under the /opt/APG/Doc/ directory.

Configuring the Backend instances

Granting Permissions

Next, connect into each one of the Backends (Primary and Additional Backends) to grant privileges to the Federated Tomcat. This needs to be done on all of the databases, for all of the Datacenters.

28


1. Logon to the Backend host

2. cd to /opt/APG/bin and run ./mysql-client.sh

Username = root

Database = mysql

Password = watch4net

3. Grant privileges access to each of the databases for the Frontend host FQDN, Frontend

host IP-Address.

mysql> grant all privileges on apg.* to 'apg'@’${IP_ADDRESS}’ identified by 'watch4net'; mysql> grant all privileges on apg.* to 'apg'@’${FQDN}’ identified by 'watch4net'; mysql> flush privileges;

4. Verify the privileges by running “show grants for ‘apg’@’%’;

5. Exit MySQL

You can validate the connection from the Federated Frontend to a remote database with:

/opt/APG/Databases/MySQL/Default/bin/mysql -h <DB IP> -P 53306 -u apg -p ${APG_DATABASE}

Where, on a default deployment of the vApp, ${APG_DATABASE} could be one of the following values:

VM Purpose (vApp based) ${APG_DATABASE}

Primary Backend apg

Additional Backend apg1, apg2, apg3, apg4

Network and Firewall access

The Federated Frontend must be able to access and receive data from MySQL (TCP port 53306). On the vApp deployments there is no need for further configuration. Adjustments might be required on a binary deployment. For more information please refer to the EMC ViPR SRM Installation and Configuration Guide.

29


Appendix C – Expanding the default configuration Here is an example of expanding the original Data Gateway with an inline-calculation filter. You can see that when an update to the SolutionPack Block is issued the collection.xml file does not changes.

/opt/APG/bin/manage-modules.sh install inline-calculation-filter Data-GatewayRequired dependencies, in processing order: [1] java '8.0u51' v8.0u51 [2] module-manager '1.8' v1.8 [3] license-manager 'Default' v5.5 [4] I collector-manager 'Default' (none) => v5.7u1 [5] I inline-calculation-filter 'Data-Gateway' (none) => v6.2 > 3 not modified, 2 to install > 12.1 MB space required / 104.2 GB available ? Enter the step to modify, 'yes' to accept them, or 'no' to cancel the operation [yes] > Starting installation of Collector-Manager v5.7u1 from collector-manager-5.7u1-linux-x64... * Gathering information... * 'Collector-Manager v5.7u1' will be registered with instance name 'Default'. * It will be installed in '/opt/APG/Collecting/Collector-Manager/Default'. * Unpacking files... * Installing files... 100% * 66 files have been installed. * Finalizing installation... * Installing service 'collector-manager Default'... [ installed ] Installation complete. Starting installation of Inline-Calculation-Filter v6.2 from inline-calculation-filter-6.2-linux-x64... * Gathering information... * 'Inline-Calculation-Filter v6.2' will be registered with instance name 'Data-Gateway'. * It will be installed in '/opt/APG/Collecting/Inline-Calculation-Filter/Data-Gateway'. * Unpacking files... * Installing files... 100% * 16 files have been installed. * Finalizing installation... Installation complete.

30

Documents

Federated Design Guidelines for EMC ViPR SRM 3 - Meetupfiles.meetup.com/...Federated-Design-Guidelines-for-ViPR-SRM-3.7.pdf · White paper . FEDERATED DESIGN GUIDELINES FOR EMC VIPR