Download pdf - BCO Best Practice Webinars - Designing the BCO Data Integration Architecture

Copyright 12/4/2012 BMC Software, Inc 1

BMC Capacity Optimization - Best Practice Webinar Designing the BCO data integration architecture

Int'l Toll: 913-312-0828 US/CAN Toll free: 866-816-1982

Argentina: 0800 444 6440

Australia: 1 800 612 415

Austria: 0800 295 780

Bahamas: 1 800 389 0491

Belgium: 0 800 75 636

Brazil: 0800 891 0266

Bulgaria: 00 800 115 1141

Chile: 123 0020 6707

China, Northern Region: 10 800 714 1509

China, Southern Region: 10 800 140 1376

Colombia: 01 800 518 1171

Czech Republic: 800 700 715

Denmark: 80 883 277

Dominican Republic: 1 888 752 0002

France: 0 800 914 176

Germany: 0 800 183 0299

Greece: 00 800 161 2205 6440

Hong Kong: 800 968 066

Hungary: 06 800 112 82

India: 000 800 1007 613

Indonesia: 001 803 017 6440

Ireland: 1 800 947 415

Israel: 1 80 925 6440

Italy: 800 789 377

Japan: 00348 0040 1009

Latvia: 8000 3523

Lithuania: 8 800 3 09 64

Luxembourg: 800 2 3214

Malaysia: 1 800 814 723

Mexico: 001 800 514 6440

Monaco: 800 39 593

Netherlands: 0 800 022 1465

New Zealand: 0 800 451 520

Norway: 800 138 41

Panama: 00 800 226 6440

Peru: 0800 54 129

Philippines: 1 800 111 010 55

Poland: 00 800 112 41 42

Portugal: 800 827 538

Russia: 810 800 2915 1012

Singapore: 800 101 2320

Slovenia: 0 800 80439

South Africa: 0 800 982 304

South Korea: 003 0813 2344

Spain: 900 937 665

Sweden: 02 079 3266

Switzerland: 0 800 894 821

Taiwan: 00 801 127 186

Thailand: 001 800 156 205 2068

Trinidad and Tobago: 1 800 205 6440

United Kingdom: 0 808 101 7156

Uruguay: 0004 019 0348

Venezuela: 0 800 100 8540

INTERNATIONAL TOLL FREE

December 4th, 2012

BMC Capacity Optimization - Best Practice Webinar Designing the BCO data integration architecture Speaker: Giuseppe Nardiello Sr Mgr BCO Product Management Panelists: Sudheer Apte Marco Colombo Charlie Geisler Giorgio Gasparini Mike Paska Mike West Evan Wetstone


BMC Capacity Optimization - Best Practice Webinar Legal Notice

The information contained in this presentation is the confidential information of BMC, Inc. and is being provided to you with the

express understanding that without the prior written consent of BMC, customers and partners may not discuss or otherwise disclose

this information to any third party or otherwise make use of this information for any purpose other than for which BMC intended.

All of the recommendations and information described herein are at the sole discretion of BMC and are subject to change and/or cancellation, and in no way should this content be viewed as

guarantees or warrantees on BMCs part.


BMC Capacity Optimization - Best Practice Webinar Topic: Designing the BCO data integration architecture

We will discuss topics such as: How do I select the right data source for managing a specific

environment? Do I need to deploy new data collection capabilities or can I rely on already deployed management tools? How do I design the data integration architecture from a logical perspective?

We will NOT discuss topics such as: How to integrate BCO to extract data from external reporting

tools or applications How do I size the BCO infrastructure based on the number of

systems to manage? How many connectors do I need? How do I make it scalable? How do I design the BCO infrastructure from a physical perspective?

How do I configure a connector to a specific data source? How do I manage historical recovery? How to ensure BCO resilience?


BMC Capacity Optimization - Best Practice Webinar Agenda

A (short) introduction to BMC Capacity Optimization (BCO) architecture

data integration

Designing data integration in BCO general criteria

some real-world use cases

Conclusions

Some of your use cases will be selected to be discussed please have them at hand !

F I R S T P A R T

S E C O N D P A R T


technical and business-oriented dashboards & scheduled, on-demand and by-exception reports

BMC Capacity Optimization High-level architecture

Web Interface (OOTB and customizable views)

Capacity Database (CDB)

Data Integration Engine

Financial Metrics Facility Metrics Performance Metrics Business KPIs Configuration Metrics Events

Storage Data Center Facilities Physical/Mainframe/Virtual/Clouds Databases & Middleware Applications Networks Business Drivers

Reporting Engine Analytical and Modeling Engines

Capacity Manager Storage Admin Service Manager IT Financial Manager IT Operation Manager Cloud Admin

BMC Capacity Optimization


AIX Topas / CEC

BMC Atrium CMDB

BMC Atrium Discovery and Dependency Mapping (ADDM)

BMC Patrol

BMC Performance Assurance

BMC ProactiveNet Performance Mgmt

BMC Cloud Lifecycle Management

BMC Capacity Management for Mainframe (CMM)

Citrix Xen server

EMC Storage Scope Control Center

nMon for AIX

Microsoft Performance Monitor (perfmon)

Microsoft SCOM 2007

NetApp Data Fabric Manager (DFM)

Oracle Enterprise Manager (OEM)

Solaris Prstat

Solaris Extended Accounting

UNIX OS configuration (AIX, Solaris, )

UNIX CLI tools (DiskFree, IOstat, )

UNIX System Activity Report (SAR)

VMware vCenter

BMC Capacity Optimization Connectors

CA Unicenter TNG

HP SiteScope

HP Business Availability Center (BAC)

HP Load Runner

HP OVO EPC Agents (Coda Agents)

HP OpenView Performance Agents

HP OpenView Performance Insight

HP OpenView Performance Manager

NetIQ AppManager

Oracle AWR / STATSPACK

SolarWind Orion NPM

TeamQuest Enterprise Database

Zenoss

(*) OOTB connectors are supported by BMC - please verify the availability for the specific platform, version and metrics with BMC representatives

(**) Market Zone Direct (MZD) connectors are supported connectors that require a separate license

(***) Template connectors are provided as-is (at no additional cost) and are nor supported by BMC - please read BMC support policies for template connectors

CSV file

NCSA and custom text log files

SQL queries to RDBMS (e.g. Oracle, IBM DB2 and MS SQL Server)

SNMP interfaces (standard/proprietary MIBs)

HTTP and JMX interfaces

Template Connectors (***)

OOTB Connectors (*)

Custom connectors can be developed from an

Eclipse-based Integrated Development

Environment (IDE) provided as part of BCO SDK

Specific Connectors General-purpose Connectors

Custom connectors (SDK)

MarketZone Direct Connectors (**)

IBM Tivoli Monitoring

HP Reporter

HP uCMDB

MS SCOM DW

Entuity

EMC SMI-S

Hitachi Device Manager

more to come...


Movri Integration for BMC Capacity Optimization

Moviri Integration for BMC Capacity Optimization v3.0

HP Reporter

IBM Tivoli Monitoring provides supported connectors to:

HP Reporter HP Reporter 3.8, 3.9

HP uCMDB HP uCMDB 9 and 10

IBM Tivoli Monitoring ITM 6.2.1, 6.2.2, 6.2.3

MS SCOM Data Warehouse MS SCOM DW 2007, 2012

Entuity Entuity EYE 2012 and 12.5

Entuity

MS SCOM DW

BMC Capacity

Optimization uCMDB

NEW

ENHANCED

ENHANCED

ENHANCED


Sentry Software Integration for BMC Capacity Optimization

provides supported connectors to

EMC SMI-S provider EMC Symmetrix DMX Series (DMX-4, DMX-3, DMX-2) EMC Symmetrix V-Max Series EMC CX4 Series, CX3 Series, CX Series EMC VNX series

Hitachi Device Manager Hitachi Adaptable Modular Storage 2000 family Hitachi Universal Storage Platform (USP, USP-V/USP-VM) Hitachi Virtual Storage Platform (VSP)

Sentry Software Integration for BMC Capacity Optimization v2.0

NEW

NEW

BMC Capacity

Optimization

EMC

Hitachi


BMC Capacity Optimization Visibility, Planning and Optimization

Technical and business oriented views

Scheduled, on-demand and by-exception reports Operation Manager

IT Financial Manager

Service Manager

Storage Admin

Capacity Manager

Database Admin BMC

Capacity Optimization


BMC Capacity Optimization OOTB Views

DataCenter Overview

vSphere datastore allocation AIX PowerVM frame

vSphere infrastructure


BMC Capacity Optimization for Mainframes OOTB Views

z/OS Image Summary View 4 Hour Rolling Average

4 Hour Rolling Average MAX value KPIs

Mid Short term mainframe capacity summary

z/OS Image MIPs consumed




data integration



Conclusions


F I R S T P A R T

S E C O N D P A R T


BMC Capacity Optimization General criteria

When designing the BCO data integration archicturecture, you are required to evaluate several factors, such as: - data availability (e.g. integration of already adopted management tool vs

new data collection mechanism to be deployed) - data quality (e.g. data collection mechanisms guaranteed to return CPU

utilization values


BMC Capacity Optimization General Criteria

BCO has the ability to virtually integrate any existing data source thanks to BCO flexible and scalable data integration layer

Therefore, when discovering the existing management environment and identifying potential data sources to be integrated, you might be tempted to consider any availble data source

However, only data sources that are would provide useful data to support Capacity Management use cases should be considered

In general, make sure that you understand how existing or new data sources are supporting your Capacity Management use cases of interest

Also take into account that your environment can change over time in terms of available data sources

#1 It is not about data integration it is about Capacity Mgmt

Best Practices: Dont integrate data sources that you are not sure how to leverage

Be driven by your Capacity Management use cases

Adopt a stepwise approach perfect knowledge is not always available


BMC Capacity Optimization General Criteria - Real-World Use Case

Case 1: Can we integrate WebSphere Application Server (WAS)? First of all, integrating WAS data in BCO may mean different use cases:

perform a seasonal analysis on WAS instances (e.g. heap / thread pool size) execute extrapolation models on applications running on WAS containers

based on technical proxy (e.g. number of servlet calls) of Business KPIs Depending on which use case is of interest, the available integration options

can be very different (e.g. a custom connector to IBM Tivoli Monitoring for WAS or a custom connector to JMX provider or more simply parse an application log file, respectively)

While these use case are valid, do not forget that Capacity Management is done OS level (e.g. to right size CPU or memory resources) not to identify AS level configuration

Case 2: Can we integrate BMC EUEM? Measured response times provided by BMC End-User Experience

Management (EUEM) or similar tools can be collected to provide trending analysis or to calibrate what-if scenarios but the latter constitutes quite an advanced use case




BCO can provide support for easily geneating seasonal and trending analyses and reports on any metric collected from the integrated data sources

However, not all metrics that might be potentially available from a data source need to be collected unless useful from a Capacity Management perspective

BCO supported connectors usually collect only a subset of all potentially available metrics, that correspond to typically useful deliverables (e.g. views)

BCO supported connectors usually also provide a way of filtering specific Datasets of interest (e.g. filtering out NIC-related metrics)

While BCO allows additional custom metrics to be added, existing BCO Data Sets usually provide a quite comprehensive set of metrics can be expected to be collected for each managed resource type

#2 Horses for courses

Best Practices: Dont integrate metrics that you are not sure how to leverage


Perform a match & gap analysis before adding custom metrics




Case 1: Can we integrate NetBackup metrics? Again it is important to identify the intended use cases in order to evaluate

whether it might be useful to integrate specific metrics, for example: job execution times: to execute trend analyses volumes of backup data: to be correlate to number of clients

While some other metrics may not be integrated as being more performance oriented or more useful for an administration purpose, as for example: disk performance, drive performance, ... real-time library drive status

Case 2: Should I create a new custom metrics for these NetBackup metrics (and in case for each different backup technology of interest)? BCO Datasets in #13 WKLGEN (Business Driver Generic Metrics) contains

many (56) predefined metrics that can used to map the previously listed metrics when used as Business KPIs: ELAPSED_TIME or BYSET_ELAPSED_TIME TOTAL_EVENTS or BYSET_EVENTS



Data sources providing the same metric can still differ in terms of: Support only for specific platforms (e.g. not for HP IVM) Support for different time granularities (e.g. 5m or 1h, possibly depending

on how original data collection has been configured) Support also for by-metrics (e.g. storage utilization by tablespace, or

process-level CPU utilization, by custom workloads or transaction classes) Availability of historical data (based on configured retention policies, that

usually cannot be changed easily)

Since BCO provides decoupling between data collection and data model for managed systems, it is possible to have different data sources providing different set of metrics (provided corresponding connectors are shared)

#3 Not all data sources are created equal

Best Practices: Evaluate your specific needs in terms of metrics


Consider using different data sources for different platforms



Case 1: Can we leverage BPPM data for AIX workload characterization? BCO 9.0 can leverage BPPM (9.0) data collection for performing capacity

management activities on AIX platforms (both standalone and partitioned) However BPPM does not provide support workload characterization (that is

resource utilization breakdown by custom group of processes running on an LPAR) that is supported by BPA (a BPA agents needs to run on each LPAR)

Case 2: Do I need to use BPA for collecting vSphere data or can I use BCO? BCO 9.0 collects an extensive set of vSphere metrics, but there are cases

where an higher level of detail is required, for example for accounting or chargeback purposes

BPA the ability to collect (via agent-less or agent-based data collection depending on the specific platform) data from inside the VMs (at the virtual node level) related to transaction classes

Example: banking applications accessed via Citrix terminal servers with SAP (Linux) servers virtualized with VMware - each Citrix server runs multiple applications that are accounted individually and also Linux guests run multiple services (e.g. WAS JVMs) that need to be accounted individually




BCO may collect metrics related to a managed system (aka BCO entity) from diverse data sources - and this is what usually is used to combine configuration data (e.g. from a CMDB) with system metrics (e.g. from BPPM)

However, notice that BCO does not support multiple datasources for the same metric for the same systems (e.g. to compare their data quality) BCO has been designed to supports Capacity Management use cases, including ability to generate automated forecasting and automated by-exception reporting, to provide resource utilization visibility also from a service/business perspective and to provide IT cost visibility (showback/chargeback) that require

Therefore, it is highly discouraged to configure BCO to collect the same set of metrics from different datasources unless this is done in a test environment, for a limited time period and only to evaluate the best data sources to integrate in the production environment

#4 Data sources should not be multiplied beyond necessity

Best Practices: Evaluate in advance the quality of data sources (in case, in your test env)

Dont collect twice the same metric in your production env




Case 1: Can we collect the same VM metrics both from the hypervisor level (e.g. vSphere or Hyper-V) and from inside the VM? In general, metrics measured from inside the VM and at the hypervisor level

may not provide the same values - as matter of fact they do not represent exactly the same metrics

In order to make this distinction clear, metrics collected at the VM level are referred to a different entity (Virtual Node) that is distinct from the VM entity associated to vSphere or Hyper-V hosts and clusters

Case 2: Can we collect the same metrics from two data sources (in our test environment) whitout creating custom shadow metrics? The simplest approach consists in not sharing the connectors extracting data

from the different data sources so that managed entities are populated as completely separated entities

Notice that also with this appoach, the complexity of the BCO environment (and the size of the database) increases for example, entities of the same type can be counted twice and be visible under different names



When multiple equivalent BCO data integration options are available, always balance the advantage of the most complex and complete possible integration with the benefits of a simpler while still robust integration approach

BCO provides several mechanisms that can ensure an automatic and reliable data integration: last counter mechanism: retrieve only new data or data that were missed

in a previous run (also used to retrieve historical data) done mechanism: parse only new files not already parsed in previous runs

Better control of connector execution, support for migration from test to production environments and for database cleanup is being introduced in the upcoming Service Pack

#5 Keep it simple

Best Practices: Perform a cost / benefit analysis


Consider also how your environment may change over time



#5 Keep it simple

Case 1: How can we integrate network metrics from a Network Management solution that does not provide any interface that can be leveraged? This is a real case that was solved by having reports scheduled on a regular base from

the Network Management solution, so as to allow BCO to automatically parse all reports files via a custom connector (few lines of code)

In some cases it is required to keep the integration simple as there are no other way of integrating closed solutions

Case 2: How can we integrate storage management metrics from a Storage Management solution that had a complex API (WBEM) for extracting data? This is a real case that was solved by having reports scheduled on a regular base from

the Storage Management solution, so as to allow BCO to automatically parse all reports files via a custom connector (few lines of code)

Even when some different options are available, do not understimate how effective a simple integration approach can be at least as a first integration step that allows to quickly demonstrate BCO value

As a matter of fact, as high as 80% of data integrations can be easily done via general purpose connectors or via simple parsing


BMC Capacity Optimization General criteria - summary





#5 Keep it simple


b u t m o s t a n d f o r e m o s t


BMC Capacity Optimization From criterias to a decision tree

Identify the use cases of interest

Is there an OOTB or a MZD connector?

Identify the required metrics

Identify the potential datasources

Identify potential integration mechanisms and interfaces

Is there a template/community connector?

Is a general-purpose connector viable?

create a custom connector

PHASE 1: use cases

PHASE 2: metrics and data sources

PHASE 3: connectors

/




data integration



Conclusions


F I R S T P A R T

S E C O N D P A R T


Question

Answer

BMC Capacity Optimization Your cases


BMC Capacity Optimization - Best Practice Webinar Upcoming Webinars

January 8th

Managing BPA to BCO data integration

February 5th

Deploying and configuring BCO-CLM integration

March 5th

BCO for Storage Capacity Management

Thanks for attending

and ... completing the survey!


Learn more at www.bmc.com