30
IBM Tivoli Monitoring for Transaction Performance: Enterprise Transaction Performance Version 1.1.0 Warehouse Enablement Pack Implementation Guide SC09-7791-00

IBM Tivoli Monitoring for Transaction Performance ...publib.boulder.ibm.com/tividd/td/ITMFTP/SC09-7791...2 Overview The following sections provide an overview of Tivoli Enterprise

  • Upload
    others

  • View
    1

  • Download
    0

Embed Size (px)

Citation preview

Page 1: IBM Tivoli Monitoring for Transaction Performance ...publib.boulder.ibm.com/tividd/td/ITMFTP/SC09-7791...2 Overview The following sections provide an overview of Tivoli Enterprise

IBM Tivoli Monitoring for Transaction Performance: Enterprise Transaction Performance

Version 1.1.0

Warehouse Enablement Pack Implementation Guide

SC09-7791-00

Page 2: IBM Tivoli Monitoring for Transaction Performance ...publib.boulder.ibm.com/tividd/td/ITMFTP/SC09-7791...2 Overview The following sections provide an overview of Tivoli Enterprise

IBM Tivoli Monitoring for Transaction Performance: Enterprise Transaction Performance Warehouse Pack Implementation Guide

2

Edition notice First Edition

Copyright Notice

© Copyright IBM Corporation 2002. All rights reserved. May only be used pursuant to a Tivoli Systems Software License Agreement, an IBM Software License Agreement, or Addendum for Tivoli Products to IBM Customer or License Agreement. No part of this publication may be reproduced, transmitted, transcribed, stored in a retrieval system, or translated into any computer language, in any form or by any means, electronic, mechanical, magnetic, optical, chemical, manual, or otherwise, without prior written permission of IBM Corporation. IBM Corporation grants you limited permission to make hardcopy or other reproductions of any machine-readable documentation for your own use, provided that each such reproduction shall carry the IBM Corporation copyright notice. No other rights under copyright are granted without prior written permission of IBM Corporation. The document is not intended for production and is furnished “as is” without warranty of any kind. All warranties on this document are hereby disclaimed, including the warranties of merchantability and fitness for a particular purpose.

U.S. Government Users Restricted Rights—Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corporation.

Trademarks

IBM, the IBM logo, Tivoli, the Tivoli logo, AIX, Cross-Site, NetView, OS/2, Planet Tivoli, RS/6000, Tivoli Enterprise, Tivoli Enterprise Console, Tivoli Ready, and TME are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. Lotus is a registered trademark of Lotus Development Corporation and/or IBM Corporation in the United States or other countries, or both. Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both. UNIX is a registered trademark of The Open Group in the United States and other countries. C-bus is a trademark of Corollary, Inc. in the United States, other countries, or both.

PC Direct is a trademark of Ziff Communications Company in the United States, other countries, or both and is used by IBM Corporation under license. ActionMedia, LANDesk, MMX, Pentium, and ProShare are trademarks of Intel Corporation in the United States, other countries, or both. For a complete list of Intel trademarks, see http://www.intel.com/sites/corporate/tradmarx.htm. SET and the SET Logo are trademarks owned by SET Secure Electronic Transaction LLC. For further information, see http://www.setco.org/aboutmark.html.

Java and all Java-based trademarks and logos are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States and other countries.

Other company, product, and service names may be trademarks or service marks of others. Notices

References in this publication to Tivoli Systems or IBM products, programs, or services do not imply that they will be available in all countries in which Tivoli Systems or IBM operates. Any reference to these products, programs, or services is not intended to imply that only Tivoli Systems or IBM products, programs, or services can be used. Subject to valid intellectual property or other legally protectable right of Tivoli Systems or IBM, any functionally equivalent product, program, or service can be used instead of the referenced product, program, or service. The evaluation and verification of operation in conjunction with other products, except those expressly designated by Tivoli Systems or IBM, are the responsibility of the user. Tivoli Systems or IBM may have patents or pending patent applications covering subject matter in this document. The furnishing of this document does not give you any license to these patents. You can send license inquiries, in writing, to the IBM Director of Licensing, IBM Corporation, North Castle Drive, Armonk, New York 10504-1785, U.S.A.

Page 3: IBM Tivoli Monitoring for Transaction Performance ...publib.boulder.ibm.com/tividd/td/ITMFTP/SC09-7791...2 Overview The following sections provide an overview of Tivoli Enterprise

IBM Tivoli Monitoring for Transaction Performance: Enterprise Transaction Performance Warehouse Pack Implementation Guide

3

Contents

1 About this document .......................................................................................................5 1.1 Related Documentation ..........................................................................................................5

2 Overview ...........................................................................................................................7 2.1 Overview of Tivoli Enterprise Data Warehouse ....................................................................7 2.2 Overview of IBM Tivoli Monitoring for Transaction Performance - Enterprise Transaction Performance .....................................................................................................................................8

3 Installing and Configuring...............................................................................................9 3.1 Prerequisites ...........................................................................................................................9 3.2 Supported hardware and software.........................................................................................9 3.3 Limitations...............................................................................................................................9

Late data load ................................................................................................................................................................... 9 Buckets set...................................................................................................................................................................... 10

3.4 Database sizing considerations...........................................................................................10 3.5 Data sources and targets......................................................................................................11 3.6 Pre-installation steps ............................................................................................................11 3.7 Installation procedure...........................................................................................................11 3.8 Post-installation steps ..........................................................................................................12

4 Maintaining .....................................................................................................................13 4.1.1 Backing up and restoring ................................................................................................................................ 13 4.1.2 Pruning ........................................................................................................................................................... 13 4.1.3 Reinstalling the ETP warehouse enablement pack......................................................................................... 13

5 ETL processes................................................................................................................14 5.1 APF_c05_Initialize_Process .................................................................................................14 5.2 APF_c10_CDW_Process.......................................................................................................14

6 Generic schema implementation ..................................................................................15 6.1 Component configuration.....................................................................................................16

6.1.1 Component type (table CompTyp) .................................................................................................................. 16 6.1.2 Component (table Comp)................................................................................................................................ 16 6.1.3 Component relationship type (table RelnTyp)................................................................................................. 17 6.1.4 Component relationship rule (table RelnRul) .................................................................................................. 17 6.1.5 Component relationship (table CompReln) ..................................................................................................... 18 6.1.6 Attribute type (table AttrTyp) ........................................................................................................................... 18 6.1.7 Attribute rule (table AttrRul) ............................................................................................................................ 19 6.1.8 Attribute domain (table AttrDom)..................................................................................................................... 21 6.1.9 Component attribute (table CompAttr) ............................................................................................................ 21

6.2 Component measurement ....................................................................................................23 6.2.1 Measurement group type (table MGrpTyp) ..................................................................................................... 23 6.2.2 Measurement group (table MGrp)................................................................................................................... 23 6.2.3 Measurement group member (table MGrpMbr) .............................................................................................. 23 6.2.4 Measurement unit category (table MUnitCat) ................................................................................................. 24 6.2.5 Measurement unit (table MUnit)...................................................................................................................... 24

Page 4: IBM Tivoli Monitoring for Transaction Performance ...publib.boulder.ibm.com/tividd/td/ITMFTP/SC09-7791...2 Overview The following sections provide an overview of Tivoli Enterprise

IBM Tivoli Monitoring for Transaction Performance: Enterprise Transaction Performance Warehouse Pack Implementation Guide

4

6.2.6 Time summary (table TmSum) ....................................................................................................................... 24 6.2.7 Measurement source (table MSrc).................................................................................................................. 24 6.2.8 Measurement type (table MsmtTyp) ............................................................................................................... 24 6.2.9 Component measurement rule (table MsmtRul) ............................................................................................. 25 6.2.10 Measurement (table Msmt) ............................................................................................................................. 26

6.3 Helper tables..........................................................................................................................27 6.4 Exception tables....................................................................................................................27 6.5 Incremental extraction ..........................................................................................................27

6.5.1 Extract Control (table Extract_Control) ........................................................................................................... 28 6.5.2 Extract Log (table Extract_Log) ...................................................................................................................... 29

7 Data mart schema information......................................................................................30

Page 5: IBM Tivoli Monitoring for Transaction Performance ...publib.boulder.ibm.com/tividd/td/ITMFTP/SC09-7791...2 Overview The following sections provide an overview of Tivoli Enterprise

IBM Tivoli Monitoring for Transaction Performance: Enterprise Transaction Performance Warehouse Pack Implementation Guide

5

1 About this document This document describes the warehouse enablement pack for IBM Tivoli Monitoring for Transaction Performance: Enterprise Transaction Performance. It covers the following topics:

• Installing and configuring the warehouse pack

• The data flow and data structures used by the warehouse pack

With this warehouse pack, you can enable a set of extract, transform and load (ETL) utilities to extract and move data from an Enterprise Transaction Performance database to a central data warehouse database. Database administrators and programmers require the type of information provided in this document to install the ETL tools and understand what data in the central data warehouse during what is referred to as the central data warehouse ETL.

Note: The IBM Tivoli Monitoring for Transaction Performance 5.1 contains the component Enterprise Transaction Performance, which previously was known as Tivoli Application Performance Management (TAPM). So, every reference to TAPM has to be intended as the Enterprise Transaction Performance (ETP) component of IBM Tivoli Monitoring for Transaction Performance 5.1.

1.1 Related Documentation You can access many Tivoli® publications online using the Tivoli Information Center, which is available on the Tivoli Customer Support Web site:

http://www.tivoli.com/support/documents/

The following Tivoli Enterprise™ Data Warehouse documents are available on the Tivoli Enterprise Data Warehouse Documentation CD:

• Tivoli Enterprise Data Warehouse Release Notes, GI11-0857

Provides late-breaking information about Tivoli Enterprise Data Warehouse and lists hardware requirements and software prerequisites.

• Installing and Configuring Tivoli Enterprise Data Warehouse, GC32-0744

Describes how Tivoli Enterprise Data Warehouse fits into your enterprise, explains how to plan for its deployment, and gives installation and configuration instructions. It provides an introduction to the built-in program for creating and running reports, and contains maintenance procedures and troubleshooting information.

• Enabling an Application for Tivoli Enterprise Data Warehouse, GC32-0745

Provides information about connecting an application to Tivoli Enterprise Data Warehouse. This book is for application programmers who use Tivoli Enterprise Data Warehouse to store and report on their application’s data, data warehousing experts who import Tivoli Enterprise Data Warehouse data into business intelligence applications, and customers who use their local data in the warehouse.

The DB2 library contains important information about the database and data warehousing technology provided by IBM DB2, DB2 Data Warehouse Center, and DB2 Warehouse Manager. Refer to the DB2 library for help in installing, configuring, administering, and troubleshooting DB2. The DB2 library is available on the Tivoli Customer Support Web site. After you install DB2, its library is also available on your system.

The following DB2 documents are particularly relevant for people working with Tivoli Enterprise Data Warehouse:

• IBM DB2 Universal Database for Windows Quick Beginnings, GC09-2971

Guides you through the planning, installation, migration (if necessary), and setup of a partitioned database system using the IBM DB2 product on Microsoft Windows.

• IBM DB2 Universal Database for UNIX Quick Beginnings, GC09-2970

Page 6: IBM Tivoli Monitoring for Transaction Performance ...publib.boulder.ibm.com/tividd/td/ITMFTP/SC09-7791...2 Overview The following sections provide an overview of Tivoli Enterprise

IBM Tivoli Monitoring for Transaction Performance: Enterprise Transaction Performance Warehouse Pack Implementation Guide

6

Guides you through the planning, installation, migration (if necessary), and setup of a partitioned database system using the IBM DB2 product on UNIX.

• IBM DB2 Universal Database Administration Guide: Implementation, SC09-2944

Covers the details of implementing your database design. Topics include creating and altering a database, database security, database recovery, and administration using the Control Center, a DB2 graphical user interface.

• IBM DB2 Universal Database Data Warehouse Center Administration Guide, SC26-9993

Provides information on how to build and maintain a data warehouse using the Data Warehouse Center.

• IBM DB2 Warehouse Manager Installation Guide, GC26-9998

Provides the information to install the following Warehouse Manager components: Information Catalog Manager, warehouse agents, and warehouse transformers.

• IBM DB2 Universal Database and DB2 Connect Installation and Configuration Supplement, GC09-2957

Provides advanced installation considerations and guides you through the planning, installation, migration (if necessary), and set up a platform-specific DB2 client. Once the DB2 client is installed, you then configure communications for both the client and server, using the DB2 GUI tools or the Command Line Processor. This supplement also contains information on binding, setting up communications on the server, the DB2 GUI tools, DRDA™ AS, distributed installation, the configuration of distributed requests, and accessing heterogeneous data sources.

• IBM DB2 Universal Database Message Reference Volume 1, GC09-2978 and IBM DB2 Universal Database Message Reference Volume 2, GC09-2979

Lists the messages and codes issued by DB2, the Information Catalog Manager, and the Data Warehouse Center, and describes the actions you should take.

Page 7: IBM Tivoli Monitoring for Transaction Performance ...publib.boulder.ibm.com/tividd/td/ITMFTP/SC09-7791...2 Overview The following sections provide an overview of Tivoli Enterprise

IBM Tivoli Monitoring for Transaction Performance: Enterprise Transaction Performance Warehouse Pack Implementation Guide

7

2 Overview The following sections provide an overview of Tivoli Enterprise Data Warehouse and the IBM Tivoli Monitoring for Transaction Performance - Enterprise Transaction Performance warehouse pack.

2.1 Overview of Tivoli Enterprise Data Warehouse Tivoli Enterprise Data Warehouse provides the infrastructure for the following:

• Extract, transform, and load (ETL) processes through the IBM DB2 Data Warehouse Center tool

• Schema generation of the central data warehouse

• Report interfaces

As shown in Figure 1, Tivoli Enterprise Data Warehouse consists of a centralized data store where historical data from many management applications can be stored, aggregated, and correlated.

Figure 1. Tivoli Enterprise Data Warehouse overview

The central data warehouse uses a generic schema. As new components or new applications are added, more data is added to the database; however, no new tables or columns are added in the schema.

A data mart is a subset of a data warehouse that contains data tailored and optimized for the specific reporting needs of a department or team.

The central data warehouse ETL reads the data from the operational data stores of the application that collects it, verifies the data, makes the data conform to the schema, and places the data into the central data warehouse.

The data mart ETL extracts a subset of data from the central data warehouse, transforms it, and loads it into one or more star schemas, which can be included in data marts to answer specific business questions.

A program that provides these ETLs is called a warehouse enablement pack, or warehouse pack.

Page 8: IBM Tivoli Monitoring for Transaction Performance ...publib.boulder.ibm.com/tividd/td/ITMFTP/SC09-7791...2 Overview The following sections provide an overview of Tivoli Enterprise

IBM Tivoli Monitoring for Transaction Performance: Enterprise Transaction Performance Warehouse Pack Implementation Guide

8

2.2 Overview of IBM Tivoli Monitoring for Transaction Performance - Enterprise Transaction Performance

The IBM Tivoli Monitoring for Transaction Performance - Enterprise Transaction Performance warehouse pack extracts Enterprise Transaction Performance data aggregated hourly into the central data warehouse.

Enterprise Transaction Performance summarizes transaction data at individual endpoints to produce a single summary record at the end of the collection interval and logs individual transaction instance data. The summary record is aggregated to hourly records uploaded, using the Tivoli RDBMS Interface Module (RIM), into a relational database. For more information, see the “Setting up the database" and "Using Performance Data" chapters in the IBM Tivoli Monitoring for Transaction Performance Enterprise Transaction Performance User’s Guide Version 5.1, GC23-4803-00.

Note that the ETP ETL for TEDW will use only the aggregated data from the ETP RIM database and not the instance data.

Note that there is no data mart ETL processes provided. At this time, application performance specific reporting is provided by the “Tivoli Decision Support for Application Performance” Guide.

Collect performance data on the Endpoints

Aggregate data at hour level

Upload data from Gateway to the RIM database

Enterprise Transaction Performance task

Load data from the ETP RIM DB to CDW DB

Insert the data in the CDW generic data model

Central Data Warehouse ETL

Upload data from Endpoint to Gateway

Page 9: IBM Tivoli Monitoring for Transaction Performance ...publib.boulder.ibm.com/tividd/td/ITMFTP/SC09-7791...2 Overview The following sections provide an overview of Tivoli Enterprise

IBM Tivoli Monitoring for Transaction Performance: Enterprise Transaction Performance Warehouse Pack Implementation Guide

9

3 Installing and Configuring

3.1 Prerequisites • IBM DB2 Universal Database Enterprise Edition Version 7.2

• IBM DB2 Universal Database Enterprise Edition Version 7.2 FixPack 6

• Tivoli Enterprise Data Warehouse required e-fixes to IBM DB2 UDE v7 FixPak 6 (1.1-TDW-0002)

• Tivoli Enterprise Data Warehouse Version 1.1

• Tivoli Enterprise Data Warehouse 1.1 Fix Pack 1 (1.1-TDW-FP01a)

• Tivoli Enterprise Data Warehouse 1.1 E-fix 2 (1.1-TDW-0005E)

Note: Tivoli Enterprise Data Warehouse 1.1 Fix Pack 2 (1.1-TDW-FP02) supersedes e-fix 1.1-TDW-0005E. When 1.1-TDW-FP02 becomes available, install 1.1-TDW-FP02 instead of 1.1-TDW-0005E. You can obtain the Tivoli Enterprise Data Warehouse e-fixes and fix pack from the Tivoli Enterprise Data Warehouse Web site: http://www.ibm.com/software/sysmgmt/products/support/TivoliEnterpriseDataWarehouse.html. Click the Downloads link in the Self-help section. Refer to the Tivoli Enterprise Data Warehouse required e-fixes to IBM DB2 UDE v7 FixPak 6 documentation (1.1-TDW-0002.README) for instructions on obtaining IBM DB2 Universal Database Enterprise Edition Version 7.2 Fixpak 6.

3.2 Supported hardware and software Enterprise Transaction Performance warehouse pack, Version 1.1.0, supports IBM Tivoli Monitoring for Transaction Performance - Enterprise Transaction Performance, Version 5.1.0 (or Versions n.n.n and m.m.m). It supports all versions of DB2, Informix, Microsoft SQL Server, Oracle, and Sybase database products as documented in the IBM Tivoli Monitoring for Transaction Performance - Enterprise Transaction Performance Release Notes, GC23-4803-00.

3.3 Limitations Late data load

When the Enterprise Transaction Performance (ETP) ETL runs it pulls data from the ETP database. When data is extracted from the ETP database the ETL will always pull an entire day of data instead of pulling a partial day’s data. For example, if the ETP ETL runs at 7 p.m. on Wednesday, then the data for Wednesday will not be copied by the ETL. The data for Wednesday will be copied to the central data warehouse when the ETP ETL runs on Thursday.

In some error cases, there is the potential that the ETL will not copy all of the data from the ETP database. This condition can happen when there is some delay in getting the data to the ETP database. For example, if ETP is collecting data hourly, then the ETP database may have data from an endpoint up to Wednesday at 9 a.m. If the gateway goes down for 3 days, the data from that endpoint will not make it to the ETP database for 3 days. In most production environments, the ETP ETL will run daily. This means that on Thursday the ETP ETL will pull over the measurement data up to 9 a.m. for the endpoint. On Saturday when the gateway comes back up, the data from 9 a.m. on Wednesday will arrive along with the data for Thursday and Friday. The late data for Thursday and Friday will be moved into the warehouse, but the late data for Wednesday will never be copied to the Warehouse.

If all of the following conditions occur, then data will not be copied to the central data warehouse database:

1. The ETP database has partial days data in the ETP database relative to an application name, user name, transaction name, and hostname.

2. For some reason the data associated with the remainder of the day is delayed in arriving to the ETP database.

3. The data has not arrived to the database when the ETP ETL runs on the next day.

Page 10: IBM Tivoli Monitoring for Transaction Performance ...publib.boulder.ibm.com/tividd/td/ITMFTP/SC09-7791...2 Overview The following sections provide an overview of Tivoli Enterprise

IBM Tivoli Monitoring for Transaction Performance: Enterprise Transaction Performance Warehouse Pack Implementation Guide

10

ETP development is aware of this small window where data may not be copied to the warehouse. Additional information about this problem can be obtained by contacting ETP support and referencing Solution ID #1049180.

Buckets set

A problem could occur interpreting ETP bucket limits stored in the central data warehouse if a ETP collection has been set with a number of buckets less than 7 (the default value). The problem occurs because the ETP stores the upper limit boundary for the unset buckets as 0. For example if you start a ETP collection as “wmarstartcoll -a app1 -d 2 -b 2500,10000 <endpoint>” the upper limit boundary values stored in the file (and database) are 2500,1000,0,0,0,0, 2147483647 while they should be 2500,1000,2147483647, 2147483647, 2147483647, 2147483647, 2147483647.

The suggested workaround for this is to not leave unset values for the bucket limits in the wmarstartcoll command (for example: wmarstartcoll -a app1 -d 2 -b 2500,1000,2147483647, 2147483647, 2147483647, 2147483647 <endpoint>).

This problem will not impact the accuracy of the measurement data in the warehouse in anyway.

3.4 Database sizing considerations You will need to consider the volume of data you expect to collect for execution of the Enterprise Transaction Performance ETL and define an appropriate amount of space for the Tivoli central data warehouse database.

The ETP database stores amounts of data based on the combination of hosts, users, applications and transactions recorded at the Tivoli endpoint. For more information, see the “Setting up the database" chapter in the IBM Tivoli Monitoring for Transaction Performance Enterprise Transaction Performance User’s Guide Version 5.1, GC23-4803-00

The amount of data that will be moved from the ETP source database to the Tivoli central data warehouse database can be computed basing on the following formulas. The data warehouse tables that are greatly impacted from the ETP amount of data are: the Component, the Component Attribute, the Measurement and the ETP staging tables stored in the TEDW.

In order to compute the number of rows stored in the Component table of the Tivoli central data warehouse, you can use this formula:

Components = Hosts + Users per host + Applications per user + Transactions per application

The Component attribute table will have a number of rows equals to:

Component attributes = Host * 2 + Transactions per application * Tx attributes

Where the Tx attributes is 1 for the ETP minimum configuration, 15 for the ETP medium configuration (Buckets) and 21 for the ETP maximum configuration (user’s defined metrics).

The number of measurements per day stored in the Measurement table is given from this formula:

Measurements Per Day = Transactions per application * Measurement types * 24

Where the Measurement type is 1 for the ETP minimum configuration, 7 for the ETP medium configuration (Buckets) and 14 for the ETP maximum configuration (user’s defined metrics).

As example you can see later in the “Generic schema implementation” the ETP sample data having 2 hosts, 2 users, 3 applications and 2 transactions:

Components = 2 + (1+2) + (2+1) + (2+1) = 11

Component attributes = 2*2 + 3*21 = 67

Measurements Per Day = 3 * 14 * 24 = 1008

In addition, you should estimate some additional space for the central database because of the staging tables utilized by the ETP ETL.

Page 11: IBM Tivoli Monitoring for Transaction Performance ...publib.boulder.ibm.com/tividd/td/ITMFTP/SC09-7791...2 Overview The following sections provide an overview of Tivoli Enterprise

IBM Tivoli Monitoring for Transaction Performance: Enterprise Transaction Performance Warehouse Pack Implementation Guide

11

3.5 Data sources and targets

The source database for the ETL has to be the database in which ETP (ex TAPM) is storing data. This means that the source database is the database for which the Tivoli RIM object “tapm” is configured. ETP 5.1 supports all the databases supported by the Tivoli Framework version 3.6.5 and 3.7.1 (Oracle, MS_SQL, Sybase, DB2 and Informix). The current ETP ETL supports only DB2 and Oracle databases.

Before you define the warehouse source, you must verify that the ETP database exists and that you can connect to it. For DB2, you must catalog the database with ODBC. For Oracle, you must supply the system data source name (DSN) by which the database is registered in ODBC on the agent site. Use the ODBC applet, accessible from the Windows NT Control Panel, to add the data source as a system data source. The ODBC driver to be used for DB2 is "IBM DB2 ODBC DRIVER" Version 7.01.00.50 from IBM. The ODBC driver to be used for Oracle is "DataWHSE 3.60 32-bit Oracle8" Version 3.60 from MERANT.

The data source name assigned to ETP after the installation is TAPM_RIM. Before running the ETL processes, you must be sure that the Warehouse Sources and the Warehouse Targets specific to the ETP application are correctly defined (username, password, hostname, system data source).

You must also do the following:

1. From the Data Warehouse Center window, expand the Warehouse Sources folder and then the item APF_TAPM_Source.

2. Click the Tables folder. The tables in the warehouse source are displayed on the right-hand side of the window.

3. If the Schema property is empty and the Name of the tables include the schema name, right-click each table and select Properties. The Properties window opens.

4. On the Source Table page, modify the Table schema and Table name properties with the values of the ETP database source. (This depends on how you create the ETP database. Refer to the ETP documentation for further detail). The table name should be the table name without "TAPM" to prevent problems when exporting the metadata to an ETL interchange file (*.tag), which can be imported at another Data Warehouse Center.

Refer to Installing and Configuring Tivoli Enterprise Data Warehouse for instructions on how to install Tivoli Enterprise Data Warehouse.

3.6 Pre-installation steps The ETP ETL is a replacement of TAPM ETL already shipped with Tivoli Service Level Advisor, Version 1.1. If you already have installed the TAPM ETL it is fully compatible with the new IBM Tivoli Monitoring for Transaction Performance 5.1 – Enterprise Transaction Performance, but further releases of ETL will be based on the ETP ETL. Also if in this release functionalities have not been changed, it’s recommended to replace the TAPM ETL with the ETP ETL.

In order to do that, uninstall the TAPM ETL warehouse pack as described in the instructions in chapter Removing Tivoli Enterprise Data Warehouse from your enterprise of Installing and Configuring Tivoli Enterprise Data Warehouse.

3.7 Installation procedure To install the IBM Tivoli Monitoring for Transaction Performance - Enterprise Transaction Performance warehouse pack, perform the following steps:

1. Make sure that IBM Tivoli Monitoring for Transaction Performance - Enterprise Transaction Performance is installed and the APF data source is available.

2. Make sure that all prerequisite product patches are applied.

3. Make sure that Tivoli Enterprise Data Warehouse is installed. For instructions about installing Tivoli Enterprise Data Warehouse, refer to Installing and Configuring Tivoli Enterprise Data Warehouse.

4. Perform any pre-installation steps as described in Pre-installation steps.

Page 12: IBM Tivoli Monitoring for Transaction Performance ...publib.boulder.ibm.com/tividd/td/ITMFTP/SC09-7791...2 Overview The following sections provide an overview of Tivoli Enterprise

IBM Tivoli Monitoring for Transaction Performance: Enterprise Transaction Performance Warehouse Pack Implementation Guide

12

1. Install the warehouse pack as described in the instructions in Installing and Configuring Tivoli Enterprise Data Warehouse.

2. Optionally, install language support for the warehouse pack as described in the instructions in Installing and Configuring Tivoli Enterprise Data Warehouse.

3. Perform the post-installation steps described in Post-installation steps.

3.8 Post-installation steps

Page 13: IBM Tivoli Monitoring for Transaction Performance ...publib.boulder.ibm.com/tividd/td/ITMFTP/SC09-7791...2 Overview The following sections provide an overview of Tivoli Enterprise

IBM Tivoli Monitoring for Transaction Performance: Enterprise Transaction Performance Warehouse Pack Implementation Guide

13

4 Maintaining 4.1.1 Backing up and restoring

4.1.2 Pruning

Remove old data from the Msmt table using with a combination of triggers and warehouse processes. CDW_C05_PurgeMsmt_Process is a process in the CDW_TivoliEnterpriseDataWarehouse_v1.1.0_Subject_Area subject area. You schedule when you want to run the process: weekly, monthly, daily. What data gets pruned is controlled by the Prune_Msmt_Control table. The default value set by Enterprise Transaction Performance is 31 days. You can modify the value by running the following SQL statement, where X is a date duration whose format is yyyymmdd (for example: X = 00000108 for 0000 years, 01 month, 08 days).

UPDATE TWG.Prune_Msmt_Control

SET PMSMTC_AGE_IN_DAYS = X

WHERE TMSUM_CD = 'H'

AND MSRC_CD = 'APF'

4.1.3 Reinstalling the ETP warehouse enablement pack

When the ETP warehouse enablement pack is uninstalled, ETP data in the central data warehouse is not automatically removed. This is because data in the central data warehouse is not tied to a specific application. It is a historical record of activity and conditions in your enterprise. Although it might have been placed in the warehouse by one application, it can be read by other applications, including those you write yourself. Therefore, Tivoli Enterprise Data Warehouse does not provide an automated way to delete data from the central data warehouse, aside from the data pruning mechanisms.

If you do not delete the data, a subsequent reinstallation of the ETP warehouse enablement pack can lead to a potential duplication of data in the central data warehouse. To avoid this situation, do the following after an ETL uninstallation:

1. Set PMSMTC_AGE_IN_DAYS = -1 in the PRUNE_MSMT_CONTROL table as described in “Pruning.” Note that the PMSMTC_AGE_IN_DAYS value is set to -1 because the purge process does not delete the data for the current day.

2. Run the Warehouse PurgeMsmt process.

3. Reinstall the ETP warehouse enablement pack.

4. Run the ETP warehouse enablement pack steps.

Moreover after step 4

Enterprise Transaction Performance warehouse enablement pack will retrieve all the data from the ETP database. If you want a lesser amount of data in the warehouse, you must run the prune job/process after the install to remove the unwanted measurements (but leaves the component data that central data warehouse needs).

Page 14: IBM Tivoli Monitoring for Transaction Performance ...publib.boulder.ibm.com/tividd/td/ITMFTP/SC09-7791...2 Overview The following sections provide an overview of Tivoli Enterprise

IBM Tivoli Monitoring for Transaction Performance: Enterprise Transaction Performance Warehouse Pack Implementation Guide

14

5 ETL processes This warehouse pack has the following processes.

5.1 APF_c05_Initialize_Process This process initializes the incremental extract logic. It should be run only one time after you install the component. You can also run it anytime you want to reload all the data from the Tivoli Application Performance Management database to the central data warehouse.

This process has the following step:

• APF_c05_s010_init

This step resets the values of the Tivoli Enterprise Data Warehouse Extract_Control and Prune_Msmt_Control tables used respectively to enable data to be incrementally extracted by the ETL processes and to remove old data from the Msmt table. After you run this step, the next process will be able to load all the data from the ETP in the central data warehouse.

5.2 APF_c10_CDW_Process This process loads the data from the Enterprise Transaction Performance database to the central data warehouse.

This process has the following steps:

• APF_c10_s010_rim

• APF_c10_s020_comp

• APF_c10_s030_compattr

• APF_c10_s040_msmt

• APF_c10_s050_msmt

Page 15: IBM Tivoli Monitoring for Transaction Performance ...publib.boulder.ibm.com/tividd/td/ITMFTP/SC09-7791...2 Overview The following sections provide an overview of Tivoli Enterprise

IBM Tivoli Monitoring for Transaction Performance: Enterprise Transaction Performance Warehouse Pack Implementation Guide

15

6 Generic schema implementation Before reading this section, read about the generic schema for the Tivoli Enterprise Data Warehouse central data warehouse, which is described in Enabling an Application for Tivoli Enterprise Data Warehouse. That document defines the content of each table and explains the relationships between the tables in this document.

Shaded columns (*) in the following tables can be translated by the application. Installing and Configuring Tivoli Enterprise Data Warehouse contains instructions for installing support for additional languages.

The sample data illustrated in the following tables is taken from the following ETP data:

Host name User name Application name Transaction name

host1 Administrator Iexplorer download

host2.dev.tivoli.com Administrator Netscape download

host2.dev.tivoli.com Goofy MSWord print

Hostname/Origin will be mapped into one of two types: IP_HOST or TAPM_HOST.

IP_HOST is a component common to all applications in the warehouse. Rules specified in Enabling an Application for Tivoli Enterprise Data Warehouse are used to determine if the Hostname or Origin slot values meet these requirements. If not, a TAPM_HOST component is created.

The component types involved in an ETP transaction are: TAPM_TX, TAPM_APPL, TAPM_HOST, TAPM_USER.

To retain the links between these components we need three CompReln entries per TAPM_TX:

TAPM_TX to TAPM_APPL, TAPM_APPL to TAPM_USER, TAPM_USER to TAPM_HOST (or IP_HOST)

A TAPM_TX is identified by:

• its name

• the name of its parent TAPM_APPL

• the name of its grandparent TAPM_USER

the name of its greatgrandparent TAPM_HOST (or IP_HOST)

A TAPM_APPL is identified by:

• its name

• the name of its parent TAPM_USER

• the name of its grandparent TAPM_HOST (or IP_HOST)

A TAPM_USER is identified by:

• its name

• the name of its parent TAPM_HOST (or IP_HOST)

Page 16: IBM Tivoli Monitoring for Transaction Performance ...publib.boulder.ibm.com/tividd/td/ITMFTP/SC09-7791...2 Overview The following sections provide an overview of Tivoli Enterprise

IBM Tivoli Monitoring for Transaction Performance: Enterprise Transaction Performance Warehouse Pack Implementation Guide

16

Note that Comp_Corr_Val is not required for this method. The TAPM_HOST component is changed to IP_HOST component if it is a fully qualified host name.

6.1 Component configuration 6.1.1 Component type (table CompTyp)

CompTyp_Cd CHAR(17)

CompTyp_Parent_CdCHAR(17)

CompTyp_Nm * VARCHAR(120)

CompTyp_Strt_DtTm TIMESTAMP

CompTyp_End_DtTmTIMESTAMP

IP_HOST IP Host 2002-06-30-12.00.00.000000

9999-01-01-12.00.00.000000

TAPM_TX TAPM Transaction 2002-06-30-12.00.00.000000

9999-01-01-12.00.00.000000

TAPM_APPL TAPM Application 2002-06-30-12.00.00.000000

9999-01-01-12.00.00.000000

TAPM_USER TAPM User 2002-06-30-12.00.00.000000

9999-01-01-12.00.00.000000

TAPM_HOST TAPM Host 2002-06-30-12.00.00.000000

9999-01-01-12.00.00.000000

6.1.2 Component (table Comp)

Comp_ID INTEGER

CompTyp_Cd CHAR (17)

Centr_Cd CHAR(6)

Cust_ID INTEGER

Comp_Corr_IDINTEGER

Comp_Nm VARCHAR (254)

Comp_Corr_Val VARCHAR (254)

Comp_Strt_DtTm TIMESTAMP

Comp_End_DtTm TIMESTAMP

Comp_Ds VARCHAR (254)

1 TAPM_HOST CDW 1 0 host1 2002-06-30-12.00.00.000000

9999-01-01-12.00.00.000000

2 IP_HOST CDW 1 0 host2.dev.tivoli.com

2002-06-30-12.00.00.000000

9999-01-01-12.00.00.000000

3 TAPM_USER CDW 1 1 administrator 2002-06-30-12.00.00.000000

9999-01-01-12.00.00.000000

4 TAPM_USER CDW 1 1 administrator 2002-06-30-12.00.00.000000

9999-01-01-12.00.00.000000

5 TAPM_USER CDW 1 2 goofy 2002-06-30-12.00.00.000000

9999-01-01-12.00.00.000000

6 TAPM_APPL CDW 1 4 IExplorer 2002-06-30-12.00.00.000000

9999-01-01-12.00.00.000000

Page 17: IBM Tivoli Monitoring for Transaction Performance ...publib.boulder.ibm.com/tividd/td/ITMFTP/SC09-7791...2 Overview The following sections provide an overview of Tivoli Enterprise

IBM Tivoli Monitoring for Transaction Performance: Enterprise Transaction Performance Warehouse Pack Implementation Guide

17

Comp_ID INTEGER

CompTyp_Cd CHAR (17)

Centr_Cd CHAR(6)

Cust_ID INTEGER

Comp_Corr_IDINTEGER

Comp_Nm VARCHAR (254)

Comp_Corr_Val VARCHAR (254)

Comp_Strt_DtTm TIMESTAMP

Comp_End_DtTm TIMESTAMP

Comp_Ds VARCHAR (254)

7 TAPM_APPL CDW 1 5 MSWord 2002-06-30-12.00.00.000000

9999-01-01-12.00.00.000000

8 TAPM_APPL CDW 1 3 Netscape 2002-06-30-12.00.00.000000

9999-01-01-12.00.00.000000

9 TAPM_TX CDW 1 6 download 2002-06-30-12.00.00.000000

9999-01-01-12.00.00.000000

10 TAPM_TX CDW 1 8 download 2002-06-30-12.00.00.000000

9999-01-01-12.00.00.000000

11 TAPM_TX CDW 1 7 print 2002-06-30-12.00.00.000000

9999-01-01-12.00.00.000000

6.1.3 Component relationship type (table RelnTyp)

RelnTyp_Cd CHAR(6)

RelnTyp_Nm * VARCHAR(120)

PCHILD Parent Child relationship

6.1.4 Component relationship rule (table RelnRul)

CompTyp_Source_Cd CHAR(17)

CompTyp_Target_Cd CHAR(17)

RelnTyp_Cd CHAR(6)

RelnRul_Strt_DtTm TIMESTAMP

RelnRul_End_DtTm TIMESTAMP

IP_HOST TAPM_USER PCHILD 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

TAPM_HOST TAPM_USER PCHILD 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

TAPM_USER TAPM_APPL PCHILD 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

TAPM_APPL TAPM_TX PCHILD 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

Page 18: IBM Tivoli Monitoring for Transaction Performance ...publib.boulder.ibm.com/tividd/td/ITMFTP/SC09-7791...2 Overview The following sections provide an overview of Tivoli Enterprise

IBM Tivoli Monitoring for Transaction Performance: Enterprise Transaction Performance Warehouse Pack Implementation Guide

18

6.1.5 Component relationship (table CompReln)

CompReln_ID INTEGER

Comp_Source_ID INTEGER

Comp_Target_IDINTEGER

RelnTyp_CdCHAR(6)

CompReln_Strt_DtTm TIMESTAMP

CompReln_End_DtTm TIMESTAMP

1 1 3 PCHILD 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

2 1 4 PCHILD 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

3 2 5 PCHILD 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

4 4 6 PCHILD 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

5 5 7 PCHILD 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

6 3 8 PCHILD 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

7 6 9 PCHILD 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

8 8 10 PCHILD 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

9 7 11 PCHILD 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

6.1.6 Attribute type (table AttrTyp)

AttrTyp_Cd CHAR(17)

AttrTyp_Nm * VARCHAR(120)

TX_DETAIL TAPM Transaction detail

METRIC_1 TAPM Application defined metric 1 name

METRIC_2 TAPM Application defined metric 2 name

METRIC_3 TAPM Application defined metric 3 name

METRIC_4 TAPM Application defined metric 4 name

METRIC_5 TAPM Application defined metric 5 name

METRIC_6 TAPM Application defined metric 6 name

BUCKET_1_ULB TAPM Bucket 1 Upper Limit Boundary

Page 19: IBM Tivoli Monitoring for Transaction Performance ...publib.boulder.ibm.com/tividd/td/ITMFTP/SC09-7791...2 Overview The following sections provide an overview of Tivoli Enterprise

IBM Tivoli Monitoring for Transaction Performance: Enterprise Transaction Performance Warehouse Pack Implementation Guide

19

AttrTyp_Cd CHAR(17)

AttrTyp_Nm * VARCHAR(120)

BUCKET_2_ULB TAPM Bucket 2 Upper Limit Boundary

BUCKET_3_ULB TAPM Bucket 3 Upper Limit Boundary

BUCKET_4_ULB TAPM Bucket 4 Upper Limit Boundary

BUCKET_5_ULB TAPM Bucket 5 Upper Limit Boundary

BUCKET_6_ULB TAPM Bucket 6 Upper Limit Boundary

BUCKET_7_ULB TAPM Bucket 7 Upper Limit Boundary

BUCKET_1_LLB TAPM Bucket 1 Lower Limit Boundary

BUCKET_2_LLB TAPM Bucket 2 Lower Limit Boundary

BUCKET_3_LLB TAPM Bucket 3 Lower Limit Boundary

BUCKET_4_LLB TAPM Bucket 4 Lower Limit Boundary

BUCKET_5_LLB TAPM Bucket 5 Lower Limit Boundary

BUCKET_6_LLB TAPM Bucket 6 Lower Limit Boundary

BUCKET_7_LLB TAPM Bucket 7 Lower Limit Boundary

6.1.7 Attribute rule (table AttrRul)

CompTyp_Cd CHAR(17)

AttrTyp_Cd CHAR(17)

AttrRul_Strt_DtTm TIMESTAMP

AttrRul_End_DtTm TIMESTAMP

AttrRul_Dom_Ind CHAR

TAPM_HOST LAST_IP_ADDRESS 9999-01-01 12:00:00.0

N

TAPM_HOST TME_LABEL 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

N

TAPM_TX TX_DETAIL 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

N

TAPM_TX METRIC_1 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

N

TAPM_TX METRIC_2 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

N

TAPM_TX METRIC_3 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

N

TAPM_TX METRIC_4 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

N

Page 20: IBM Tivoli Monitoring for Transaction Performance ...publib.boulder.ibm.com/tividd/td/ITMFTP/SC09-7791...2 Overview The following sections provide an overview of Tivoli Enterprise

IBM Tivoli Monitoring for Transaction Performance: Enterprise Transaction Performance Warehouse Pack Implementation Guide

20

CompTyp_Cd CHAR(17)

AttrTyp_Cd CHAR(17)

AttrRul_Strt_DtTm TIMESTAMP

AttrRul_End_DtTm TIMESTAMP

AttrRul_Dom_Ind CHAR

TAPM_TX METRIC_5 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

N

TAPM_TX METRIC_6 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

N

TAPM_TX BUCKET_1_ULB 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

N

TAPM_TX BUCKET_2_ULB 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

N

TAPM_TX BUCKET_3_ULB 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

N

TAPM_TX BUCKET_4_ULB 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

N

TAPM_TX BUCKET_5_ULB 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

N

TAPM_TX BUCKET_6_ULB 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

N

TAPM_TX BUCKET_7_ULB 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

N

TAPM_TX BUCKET_1_LLB 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

N

TAPM_TX BUCKET_2_LLB 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

N

TAPM_TX BUCKET_3_LLB 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

N

TAPM_TX BUCKET_4_LLB 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

N

TAPM_TX BUCKET_5_LLB 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

N

Page 21: IBM Tivoli Monitoring for Transaction Performance ...publib.boulder.ibm.com/tividd/td/ITMFTP/SC09-7791...2 Overview The following sections provide an overview of Tivoli Enterprise

IBM Tivoli Monitoring for Transaction Performance: Enterprise Transaction Performance Warehouse Pack Implementation Guide

21

CompTyp_Cd CHAR(17)

AttrTyp_Cd CHAR(17)

AttrRul_Strt_DtTm TIMESTAMP

AttrRul_End_DtTm TIMESTAMP

AttrRul_Dom_Ind CHAR

TAPM_TX BUCKET_6_LLB 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

N

TAPM_TX BUCKET_7_LLB 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

N

6.1.8 Attribute domain (table AttrDom)

AttrDom_ID INTEGER

CompTyp_Cd CHAR(17)

AttrTyp_Cd CHAR(17)

AttrDom_Strt_DtTm TIMESTAMP

AttrDom_End_DtTm TIMESTAMP

AttrDom_Val VARCHAR(254)

AttrDom_Ds VARCHAR(254)

6.1.9 Component attribute (table CompAttr)

CompAttr_ID INTEGER

Comp_ID INTEGER

AttrTyp_Cd CHAR(17)

CompAttr_Strt_DtTm TIMESTAMP

CompAttr_End_DtTm TIMESTAMP

CompAttr_Val VARCHAR(254)

1 1 LAST_IP_ADDRESS 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

146.84.181.70

2 2 LAST_IP_ADDRESS 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

146.84.182.71

3 1 TME_LABEL 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

Host1_ep

4 2 TME_LABEL 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

Host2_ep

5 9 TX_DETAIL 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

HTML page download

6 9 BUCKET_1_ULB 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

100

7 9 BUCKET_2_ULB 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

500

8 9 BUCKET_3_ULB 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

1000

Page 22: IBM Tivoli Monitoring for Transaction Performance ...publib.boulder.ibm.com/tividd/td/ITMFTP/SC09-7791...2 Overview The following sections provide an overview of Tivoli Enterprise

IBM Tivoli Monitoring for Transaction Performance: Enterprise Transaction Performance Warehouse Pack Implementation Guide

22

CompAttr_ID INTEGER

Comp_ID INTEGER

AttrTyp_Cd CHAR(17)

CompAttr_Strt_DtTm TIMESTAMP

CompAttr_End_DtTm TIMESTAMP

CompAttr_Val VARCHAR(254)

9 9 BUCKET_4_ULB 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

2000

10 9 BUCKET_5_ULB 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

5000

11 9 BUCKET_6_ULB 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

10000

12 9 BUCKET_7_ULB 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

2147483647

13 9 BUCKET_1_LLB 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

0

14 9 BUCKET_2_LLB 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

100

15 9 BUCKET_3_LLB 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

500

16 9 BUCKET_4_LLB 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

1000

17 9 BUCKET_5_LLB 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

2000

18 9 BUCKET_6_LLB 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

5000

19 9 BUCKET_7_LLB 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

10000

20 9 METRIC_1 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

KBytes_transferred

21 9 METRIC_2 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

Number_of_objects

22 9 METRIC_3 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

HTTP_Server_requests

Page 23: IBM Tivoli Monitoring for Transaction Performance ...publib.boulder.ibm.com/tividd/td/ITMFTP/SC09-7791...2 Overview The following sections provide an overview of Tivoli Enterprise

IBM Tivoli Monitoring for Transaction Performance: Enterprise Transaction Performance Warehouse Pack Implementation Guide

23

CompAttr_ID INTEGER

Comp_ID INTEGER

AttrTyp_Cd CHAR(17)

CompAttr_Strt_DtTm TIMESTAMP

CompAttr_End_DtTm TIMESTAMP

CompAttr_Val VARCHAR(254)

23 9 METRIC_4 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

HTTP_Client_requests

24 9 METRIC_5 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

Failed_requests

25 9 METRIC_6 2002-06-30-12.00.00.000000

9999-01-01 12:00:00.0

Network_bandwith

… … … … … …

6.2 Component measurement 6.2.1 Measurement group type (table MGrpTyp)

MGrpTyp_Cd CHAR(6)

MGrpTyp_Nm * VARCHAR(120)

GROUP Aggregate Types or Group Functions

PMIN TAPM Minimum data collection

PMED TAPM Medium data collection

PMAX TAPM Maximum data collection

ETP can collect data according to 3 types of configurations: Min, Med and Max. If Med collection configuration is set on an application, the Min is included. If Max collection configuration is set on an application, the Med is included.

6.2.2 Measurement group (table MGrp)

MGrp_Cd CHAR(6)

MGrpTyp_Cd CHAR(6)

MGrp_Parent_Cd CHAR(6)

MGrp_Nm * VARCHAR(120)

AVG_E GROUP NULL Average value exists MIN_E GROUP NULL Minimum value exists MAX_E GROUP NULL Maximum value exists

TOT_E GROUP NULL Total value exists

AVG_E PMIN NULL Average value exists

MAX_E PMIN NULL Maximum value exists

MIN_E PMIN NULL Minimum value exists

TOT_E PMED NULL Total value exists

TOT_E PMAX NULL Total value exists

6.2.3 Measurement group member (table MGrpMbr)

MGrp_Cd CHAR(6)

MGrpTyp_Cd CHAR(6)

MsmtTyp_ID INTEGER

AVG_E PMIN 1 MAX_E PMIN 1

Page 24: IBM Tivoli Monitoring for Transaction Performance ...publib.boulder.ibm.com/tividd/td/ITMFTP/SC09-7791...2 Overview The following sections provide an overview of Tivoli Enterprise

IBM Tivoli Monitoring for Transaction Performance: Enterprise Transaction Performance Warehouse Pack Implementation Guide

24

MIN_E PMIN 1 TOT_E PMED 2 TOT_E PMED 3 TOT_E PMED 4 TOT_E PMED 5 TOT_E PMED 6 TOT_E PMED 7 TOT_E PMED 8 TOT_E PMAX 9 TOT_E PMAX 10 TOT_E PMAX 11 TOT_E PMAX 12

TOT_E PMAX 13

TOT_E PMAX 14

6.2.4 Measurement unit category (table MUnitCat)

MunitCat_Cd CHAR(6)

MunitCat_Nm * VARCHAR(120)

TM Time Duration

QTY Quantity

6.2.5 Measurement unit (table MUnit)

MUnit_Cd CHAR(6)

MUnitCat_Cd CHAR(6)

Munit_Nm * VARCHAR(120)

QTY QTY Quantity MSec TM milliseconds

6.2.6 Time summary (table TmSum)

The period over which a measurement may be summarized.

TmSum_Cd CHAR

TmSum_Nm * VARCHAR(120)

H Hourly

6.2.7 Measurement source (table MSrc)

MSrc_Cd CHAR(6)

MSrc_Parent_Cd CHAR(6)

MSrc_Nm * VARCHAR(120)

Tivoli NULL Tivoli Application

APF Tivoli IBM Tivoli Monitoring for Transaction Performance - Enterprise Transaction Performance 5.1

6.2.8 Measurement type (table MsmtTyp)

MsmtTyp_ID INTEGER

MUnit_Cd CHAR(6)

MSrc_Cd CHAR (6)

MsmtTyp_Nm * VARCHAR(120)

MsmtTyp_Ds * VARCHAR(254)

1 MSec APF ResponseTime Transaction Response Time

2 QTY APF Bucket_1 Transactions Succeeded in Bucket 1

Page 25: IBM Tivoli Monitoring for Transaction Performance ...publib.boulder.ibm.com/tividd/td/ITMFTP/SC09-7791...2 Overview The following sections provide an overview of Tivoli Enterprise

IBM Tivoli Monitoring for Transaction Performance: Enterprise Transaction Performance Warehouse Pack Implementation Guide

25

MsmtTyp_ID INTEGER

MUnit_Cd CHAR(6)

MSrc_Cd CHAR (6)

MsmtTyp_Nm * VARCHAR(120)

MsmtTyp_Ds * VARCHAR(254)

3 QTY APF Bucket_2 Transactions Succeeded in Bucket 2

4 QTY APF Bucket_3 Transactions Succeeded in Bucket 3

5 QTY APF Bucket_4 Transactions Succeeded in Bucket 4

6 QTY APF Bucket_5 Transactions Succeeded in Bucket 5

7 QTY APF Bucket_6 Transactions Succeeded in Bucket 6

8 QTY APF Bucket_7 Transactions Succeeded in Bucket 7

9 QTY APF Metric_1 Application defined metric 1

10 QTY APF Metric_2 Application defined metric 2

11 QTY APF Metric_3 Application defined metric 3

12 QTY APF Metric_4 Application defined metric 4

13 QTY APF Metric_5 Application defined metric 5

14 QTY APF Metric_6 Application defined metric 6

Note: MsmtType_ID 1 is mandatory (ETP Minimum configuration). The MsmtType_ID values from 2 to 8 are present only when an ETP Medium collection configuration is set and data is produced. The MsmtType_ID values from 9 to 14 are present only when an ETP Maximum collection configuration is set and data is produced.

6.2.9 Component measurement rule (table MsmtRul)

CompTyp_Cd CHAR(17)

MsmtTyp_ID INTEGER

TAPM_TX 1

TAPM_TX 2

TAPM_TX 3

TAPM_TX 4

TAPM_TX 5

TAPM_TX 6

TAPM_TX 7

TAPM_TX 8

TAPM_TX 9

TAPM_TX 10

TAPM_TX 11

TAPM_TX 12

TAPM_TX 13

TAPM_TX 14

Page 26: IBM Tivoli Monitoring for Transaction Performance ...publib.boulder.ibm.com/tividd/td/ITMFTP/SC09-7791...2 Overview The following sections provide an overview of Tivoli Enterprise

IBM Tivoli Monitoring for Transaction Performance: Enterprise Transaction Performance Warehouse Pack Implementation Guide

26

6.2.10 Measurement (table Msmt)

Msmt_ID BIGINT

Comp_ID INTEGER

MsmtTyp_ID INTEGER

TmSum_Cd CHAR

Msmt_Strt_DtDATE

Msmt_Strt_Tm TIME

Msmt_Min_Val FLOAT

Msmt_Max_Val FLOAT

Msmt_Avg_Val FLOAT

Msmt_Tot_Val FLOAT

Msmt_Smpl_Cnt INTEGER

Msmt_Err_Cnt INTEGER

1 9 1 H 2001-11-08

00:00 8.0 1532.0 700.0 41

2 9 2 H 2001-11-08

00:00 2.0

3 9 3 H 2001-11-08

00:00 0.0

4 9 4 H 2001-11-08

00:00 0.0

5 9 5 H 2001-11-08

00:00 0.0

6 9 6 H 2001-11-08

00:00 0.0

7 9 7 H 2001-11-08

00:00 0.0

8 9 8 H 2001-11-08

00:00 0.0

9 9 9 H 2001-11-08

00:00 10.2

10 9 10 H 2001-11-08

00:00 42.0

11 9 11 H 2001-11-08

00:00 100.0

12 9 12 H 2001-11-08

00:00 79.0

13 9 13 H 2001-11-08

00:00 2.0

14 9 14 H 2001-11-08

00:00 1024.0

15 10 1 H 2001-11-08

00:00 2.0 14.0 3.0 21 1

16 10 2 H 2001-11-08

00:00 2.0

17 10 3 H 2001-11-08

00:00 5.0

Page 27: IBM Tivoli Monitoring for Transaction Performance ...publib.boulder.ibm.com/tividd/td/ITMFTP/SC09-7791...2 Overview The following sections provide an overview of Tivoli Enterprise

IBM Tivoli Monitoring for Transaction Performance: Enterprise Transaction Performance Warehouse Pack Implementation Guide

27

Msmt_ID BIGINT

Comp_ID INTEGER

MsmtTyp_ID INTEGER

TmSum_Cd CHAR

Msmt_Strt_DtDATE

Msmt_Strt_Tm TIME

Msmt_Min_Val FLOAT

Msmt_Max_Val FLOAT

Msmt_Avg_Val FLOAT

Msmt_Tot_Val FLOAT

Msmt_Smpl_Cnt INTEGER

Msmt_Err_Cnt INTEGER

18 10 4 H 2001-11-08

00:00 2.0

19 10 5 H 2001-11-08

00:00 1.0

20 10 6 H 2001-11-08

00:00 0.0

21 10 7 H 2001-11-08

00:00 0.0

22 10 8 H 2001-11-08

00:00 0.0

23 10 9 H 2001-11-08

00:00 10.2

24 10 10 H 2001-11-08

00:00 42.0

25 10 11 H 2001-11-08

00:00 100.0

26 10 12 H 2001-11-08

00:00 79.0

27 10 13 H 2001-11-08

00:00 2.0

28 10 14 H 2001-11-08

00:00 1024.0

29 11 1 H 2001-11-08

01:00 1.0 5.0 3.0 6.0

… … … … … … … … … … … …

6.3 Helper tables

6.4 Exception tables

6.5 Incremental extraction

The incremental extract for ETP is implemented in the following way.

New ETP data in the source database is discovered by looking at the integer ascending sequence IDs of the following tables: HOSTS, TRANSACTIONS, ID_INFO, DATA32, DATA64 and DATABUCKETS. The SQL to select from the data source is something like:

Page 28: IBM Tivoli Monitoring for Transaction Performance ...publib.boulder.ibm.com/tividd/td/ITMFTP/SC09-7791...2 Overview The following sections provide an overview of Tivoli Enterprise

IBM Tivoli Monitoring for Transaction Performance: Enterprise Transaction Performance Warehouse Pack Implementation Guide

28

select x,y,x from source_table s, extract_control ec

where s.marker_column > ec.from_seq and

and s.marker_column <= ec.to_seq

Because the join happens between the source_table (in the ETP database) and the extract_control table (in the central data warehouse database), a temporary version of extract_control is created in the source database, and the marker values copied over to it (from the extract_control in the central data warehouse) before the join happens. This makes it unnecessary to store the extract_control and extract_log tables as permanent tables in the source database.

When the incremental extraction is run, the following actions are done:

--- create a temporary table temp_extract_control to hold the extract window in the ETP database

--- get the from_intseq value from the extract_control in the CDW and copy it to ETP

--- update the temp_extract_control with the max tx_class_id, data_id, host_id and tx_id for the ETP database

--- create the source staging tables at CDW

--- do the extract from ETP to CDW

--- update the extract control with the to_intseq value at CDW

--- write extract information to the extract log. This fires the trigger that updates extract control to close the window.

6.5.1 Extract Control (table Extract_Control)

ExtCtl_Source VARCHAR(120)

ExtCtl_Target VARCHAR(120)

ExtCtl_From_RawSeq CHAR

ExtCtl_To_RawSeq CHAR

ExtCtl_From_IntSeq BIGINT

ExtCtl_To_IntSeq BIGINT

ExtCtl_From_DtTm TIMESTAMP

ExtCtl_To_DtTm TIMESTAMP

ID_INFO_INIT APF.STAGE_ID_INFO_COPY

3 3 9999-01-01 00:00:00.0

9999-01-01 00:00:00.0

DATA32_INIT APF.STAGE_METRICDATA32_COPY

20 20 9999-01-01 00:00:00.0

9999-01-01 00:00:00.0

DATA64_INIT APF.STAGE_METRICDATA64_COPY

20 20 9999-01-01 00:00:00.0

9999-01-01 00:00:00.0

DATABUCKETS_INIT

APF.STAGE_METRICDATABUCKETS_COPY

21 21 9999-01-01 00:00:00.0

9999-01-01 00:00:00.0

HOSTS_INIT APF.STAGE_HOST_ATTR_COPY

5 5 9999-01-01 00:00:00.0

9999-01-01 00:00:00.0

TRANSACTIONS_INIT

APF.STAGE_TX_ATTR_COPY

2 2 9999-01-01 00:00:00.0

9999-01-01 00:00:00.0

Page 29: IBM Tivoli Monitoring for Transaction Performance ...publib.boulder.ibm.com/tividd/td/ITMFTP/SC09-7791...2 Overview The following sections provide an overview of Tivoli Enterprise

IBM Tivoli Monitoring for Transaction Performance: Enterprise Transaction Performance Warehouse Pack Implementation Guide

29

6.5.2 Extract Log (table Extract_Log)

ExtLog_Source VARCHAR(120)

ExtLog_Target VARCHAR(120)

ExtLog_Done_DtTm TIMESTAMP

ExtLog_From_RawSeq CHAR

ExtLog_To_RawSeq CHAR

ExtLog_From_IntSeq BIGINT

ExtLog_To_IntSeq BIGINT

ExtLog_From_DtTm TIMESTAMP

ExtLog_To_DtTm TIMESTAMP

ID_INFO_INIT APF.STAGE_ID_INFO_COPY

2001-12-06 16:00

0 3 1970-01-01 00:00:00.0

9999-01-01 00:00:00.0

DATA32_INIT APF.STAGE_METRICDATA32_COPY

2001-12-06 16:00

0 20 1970-01-01 00:00:00.0

9999-01-01 00:00:00.0

DATA64_INIT APF.STAGE_METRICDATA64_COPY

2001-12-06 16:00

0 20 1970-01-01 00:00:00.0

9999-01-01 00:00:00.0

DATABUCKETS_INIT

APF.STAGE_METRICDATABUCKETS_COPY

2001-12-06 16:00

0 21 1970-01-01 00:00:00.0

9999-01-01 00:00:00.0

HOSTS_INIT APF.STAGE_HOST_ATTR_COPY

2001-12-06 16:00

0 5 1970-01-01 00:00:00.0

9999-01-01 00:00:00.0

TRANSACTIONS_INIT

APF.STAGE_TX_ATTR_COPY

2001-12-06 16:00

0 2 1970-01-01 00:00:00.0

9999-01-01 00:00:00.0

Page 30: IBM Tivoli Monitoring for Transaction Performance ...publib.boulder.ibm.com/tividd/td/ITMFTP/SC09-7791...2 Overview The following sections provide an overview of Tivoli Enterprise

IBM Tivoli Monitoring for Transaction Performance: Enterprise Transaction Performance Warehouse Pack Implementation Guide

30

7 Data mart schema information This warehouse pack provides no data marts or reports. At this time, application performance specific reporting is provided by Tivoli Decision Support for Application Performance.

---------------------- End of Document ----------------------