IBM PureData System for Transactions Technical Deep …praxiumgroup.com/adug-www/prev_pres/ADUG...

IBM expert integrated system

IBM PureData System for Transactions

Technical Deep Dive

Maria N. Schwenger, PureSystems Specialistschwenge@us.ibm.com

Jonathan Rossi, PureSystems Specialistrossij@us.ibm.com

PureSystems

IBM® PureData® System for Transactions - Agenda

Feature Overview

System Architecture and Setup

System Console Overview

Patterns and Deployment

Database Console Overview

Applications

Monitoring

Security

IBM expert integrated systems

Feature Overview

Slides - 10

PureSystems

Expertise builds champions

PureSystems

Systems with integrated expertise that combine the flexibility of a general purpose system, the elasticity of cloud and the simplicity of an appliance

Fundamentally changing the experience and economics of IT

The Time Has Come For a New Breed of Systems

Built-in Expertise: Capturing and automating what experts do – from the infrastructure to the application

Integration by Design: Deeply integrating and tuning hardware and software – in a ready-to-go workload optimized system

Simplified Experience: Making every part of the IT lifecycle easier; Integrated management of the entire system; A broad, open ecosystem of optimized solutions

PureSystems

IBM Family of PureSystems® – OverviewIBM PureData System

Infrastructure

Delivering Infrastructure Services

Application Platform

Delivering Platform Services

Data Platform

Delivering Data Services6

PureSystems

2013 Announcements

PureSystems

Building blocks definition

PureSystems

• Workload optimized performance

• Data load ready in hours

• Integrated management

• Automated maintenance

• Single point of support

Data Platform

Delivering Data Services

PureSystems

Database services that handle large volumes of transactions with high availability, scalability and integrity

Data Warehouse services for complex analytics and reportingon data up to petabyte scale -with minimal administration

Operational Warehouse services for continuous ingest of operational data, complex analytics, and a large volume of concurrent operational queries

Different data workloads have different characteristics

System for Transactions

System for Analytics

System for Operational Analytics

powered by Netezza technology

PureSystems

All Inclusive Comprehensive System

Minimum Set-up Time Pre-installed, Pre-integrated Single pane-of-glass management,

administration and maintenance

IBM PureData Systems for TransactionsIntegrated Design, Built-in Expertise and Simplified Experience

Servers

Storage

Networking

Virtualization in PureSystems Management (PSM) Node

Management

Data Management

Databases

Deployment

PureSystems

IBM PureData System for TransactionsSimplified Experience – Single Pane of Glass

IBM PureData Console

Workload Console

ChassisCompute ITE Compute ITE

Compute ITE Compute ITE

Storage Unit

Network

System Console

PureSystems

IBM PureData System for TransactionsLoading Dock to Loading Data in Just Hours

To Loading Data

From Loading Dock

In Just Hours

Plug-in system and network No assembly required

Power-on Integrate system in your network Use the Workload Console to:

Create the clustered instances Configure administrative access

to the instance Create and Deploy a database

Use the Database Operation Console to: Configure access control Add disk storage to the database

Load Objects and Data Use IBM Data Studio

System Architecture and Setup

Slides - 36

PureSystems

System Architecture and SetupHardware, Software and Setup

PureSystems

Hardware ArchitectureSmall, Medium and Large Configuration

42U41U40U39U38U37U36U35U34U33U32U31U30U29U28U27U26U25U24U23U22U21U20U19U18U17U16U15U14U13U12U11U10U09U08U07U06U05U04U03U02U01U

PSM 2PSM 1

System Console

Top of Rack Switch

Compute ITE

V7000 Expansion

V7000 Controller

V7000 Expansion

V7000 Controller

V7000 Expansion

V7000 Controller

V7000 Expansion

V7000 Controller

Top of Rack Switch

Compute ITE

On-line Upgrade On-line Upgrade

PureSystems

Storage System • Utilizes Storwize v7000

chassis and expansion units

• Holds up to 192 disks inlarge configuration

• 1:3 workload balanced SSDto HDD ratio

• Easy tier

Hardware ArchitectureHardware Details

Flex System Chassis • Up to 2 per rack (Large)

• Holds up to 14 computenodes each

• Back plane with fullyredundant Network (Ethernet)and Storage (SAN) connectivity Pure System Manager

• 2 per rack for redundancy

• Integrate management forall system resources

Compute Nodes• Up to 24 per rack

• Serves as CF and Membernodes for pureScale instances

Balanced HDD & SSD • 48-disk module (12 SSD+ 36 HDD) to optimize forperformance & cost

• RAID 10 for extremestorage reliability

• Up to 9.5 TB SSD and64 TB HDD capacity

External Network Connectivity• Dual 10Gb Ethernet Switches for external and

rack-to-rack communication

PureSystems

Software ArchitectureIncluded with IBM PureData System

Compute ITEs Red Hat Enterprise Linux® 6.2 IBM DB2 Enterprise Server Edition 10.1 FP1 with

DB2 pureScale Feature Storage optimization Feature Workload Management Q Replication for business continuity

TSM client for policy based backup and restore

PureSystems Manager Nodes Optim® Performance Manager IBM Data Studio System Console

PureSystems

Software ArchitectureDriving principles for the design of DB2 for PureData

Extreme Capacity Designed for OLTP

Start small

Grow easily, with your business

Move from small to medium to large config

Application Transparency Avoid the risk and cost of changing applications

because of the database topology

Automatic Work Load Balancing, Automatic Client Reroute, and Client Affinity

Continuous Availability Maintain service across unplanned events

Maintain service across planned events using maintenance

PureSystems

Software ArchitecturePureData System DB2 Features

Combining DB2 with proven IBM clustering technologies such as RSCT, GPFS etc.

SAN based databaseSAN based database

Interrupt free RDMA calls through the network

Clients connect using EthernetClients connect using Ethernet

Clustering Services for hosts (RSCT) and storage (GPFS)

PureSystems

Software ArchitectureCF and DB2 Members

● DB2 member in PureData:

DB2 engine address space – can be logical or physical

All members access the same shared data (True data sharing)

Each member has its own buffer pool, memory, log files, etc.

● Cluster caching facility (CF) in PureData

Inherits golden standards from System z parallel sysplex

Group Buffer Pool (GBP), Global Lock Management (GLM), and Shared Communication Area (SCA)

Efficient communications between CFs and Members via RDMA

Multiple DB2 member in a cluster

Cluster caching facility (CF) for global lock and buffer pool management

PureSystems

Software ArchitectureCluster Elasticity

● IBM PureData – Choose from pre-configured topology

● Up to 3 PureData instances for the small configuration

● Up to 6 for medium and 12 PureData instances for large.

PureSystems

Software ArchitecturePureData Database – Ease of Use

● No cluster manager scripting or configuration required

Setup automatically upon installation

Automatic install of all components across all desired hosts

Automatically configured using best practices

● Application Transparency

No cluster awareness in the application

No affinity routing (Can be configured if desired)

No application / data partitioning

Automatic Work Load Balancing

● Member hardware failure is transparent to the applications

● Automatic connections reroute to the active members

● Member re-integration in cluster is transparent to the applications

● DBAs add capacity without re-tuning and re-testing

System Console Overview

Slides - 12

PureSystems

Launch ConsoleSystem Console

https://<Management_Host_Name>

1. System Console

2. Workload Console

PureSystems

System ConsoleManaging Hardware Resources

Infrastructure Map

Compute Nodes

Management Nodes

Storage Devices

Network Devices

PureSystems

Managing Hardware Resources Hardware Infrastructure Map

View performance data

Check Status

Check LED lights

Check temperature for each part

Monitor jobs associated with the hardware device

See name and serial number of each hardware device

PureSystems

Managing Hardware ResourcesHardware Compute Nodes

1. Power On

2. Power Off

3. Start

4. Suspend

5. Maintenance

6. Monitoring

PureSystems

Managing Hardware ResourcesHardware Storage Devices

Select a Storage Device to monitor Events Jobs Types Firmware Status Capacity Location Temperature Physical Cores Disk Drives Operating System volumes Storage volumes LUNs Storage controller ports Storage node statistics LEDs

PureSystems

Managing Hardware ResourcesHardware Network Devices

View Network Device Events Jobs Switch type Description Location Firmware level Software version Model Customer ports Network ports

PureSystems

Managing Hardware ResourcesHardware Hosts

Configure Hosts Access granted to Description Storage volumes IP addresses

Managing Hosts Jobs Description Status Created On Updated On CPU Memory In cloud group On compute node Operating system volume Storage volume Access grants DB2 pureScale instances

PureSystems

Managing System SettingsConfigure Email, Date/Time, DNS, Events, and Backup/Restore

Configure Email System Console System System Settings Expand Mail Delivery Specify SMTP server etc.

Managing Date / Time System Console System System Settings Expand Date and Time Specify IP address of NTP server etc.

Configure Event Forwarding System Console System System Settings Expand Event Forwarding Specify SNMP destination and download MIB file

Managing DNS System Console System System Settings Expand Domain Name Service (DNS) Specify DNS Server IP addresses

Managing Backup / Restore System Console System System Settings Expand Backup and Restore Complete External Backup Server configuration

Patterns and Deployments

Slides - 10

PureSystems

Patterns of ExpertiseDecades of Experience

Decades of clients experience captured, lab tested and optimized into a deployable form

What is a pattern? The pre-defined architecture of cluster,

database For each cluster

Pre-installation of an OS Pre-integration across components Pre-configured and tuned Pre-configured security Pre-configured monitoring Complete lifecycle management

What is a deployment? Using predefined pattern to deploy Repeatable deployment with full lifecycle

management

PureSystems

Do It Yourself OR Built by ExpertsTaking drudgery out of complexity

Install

Point and clickManual, brittle

Manage

Manual optimizations Pre-optimized by experts

Deploy

Pattern-basedTime consuming

TraditionalSystem

PureDataSystem

Traditional System – build-it yourself Takes several weeks Procure HW/SW/Network and configure Define HA topology Set up storage pools Install multiple operating systems Install database instances Set up backup policies Test, tune and reconfigure

PureData System – built-in expertise Just in hours Choose topology pattern and click OK.

PureSystems

Instance Patterns

Automatically creates, configures, and deploys an Instance cluster topology with built-in redundancy and high performance

Small Configuration Rack – Maximum 3 Instances

3 Instances of 2 CF and 2 Members on 2 ITE each

2 Instances of 2 CF + 2 Members on 2 ITE and 2 CF + 2 Members on 4 ITE

1 Instance of 2 CF and 4 Members on 6 ITE

Instance PatternsSmall Configuration

Legend:

CF and Member collocated

Only CF

Only Member

PureSystems

Small Configuration

Storage PatternsOptimized for Instance / Databases

/db2plog/db2inst1/dbname1 /db2fs/db2inst2/dbname1 /db2mlog/db2inst2/dbname1

/db2plog/db2inst3/dbname1 /db2fs/db2inst3/dbname4 /db2mlog/db2inst3/dbname4

/db2fs/db2inst1/dbname1 /db2mlog/db2inst1/dbname1 /db2plog/db2inst2/dbname1

/db2fs/db2inst3/dbname1 /db2mlog/db2inst3/dbname1 /db2plog/db2inst3/dbname4

V7000 – 1 (Pool – 1)

EXP – 1 (Pool – 2)

1st Instance / 1st DB

2nd Instance / 1st DB

3rd Instance / 1st DB

3rd Instance / 2nd DB

Workload Console Overview

Slides - 8

PureSystems

Launch ConsoleWorkload Console

https://<Management_Host_Name>

1. System Console

2. Workload Console

PureSystems

Creating / Managing Instances, Databases and ToolsWorkload Console

DB2 pureScale Instances

Databases

Database Patterns

Database Workload Standards

PureSystems

Creating InstanceDatabase DB2 pureScale Instances

Create DB2 pureScale Instance Name Description Size – Small/Medium/Large Compatibility mode Database version Database level Port number Maximum number of

databases Click OK

PureSystems

Creating Database PatternDatabase Database Patterns

Create Database Pattern Name Description Pattern Database version Database level Database size Compatibility mode Schema file Advanced options Click OK

PureSystems

Deploying Database Using a PatternDatabase Databases

Deploy Database Name Description Default user Password Instance Name Click OK

PureSystems

Administering a DatabaseEasy Management

Manage Database Schedule database backups Create a database image List all database images

Database Storage Management Add more storage

PureSystems

Manage InstanceGrow Instance

Growing DB2 pureScale Instance Change size to large Click Submit

PureSystems

Simplified maintenance with pre-integrated fixes

• All hardware firmware and OS software patches integrated and tested together at the factory

• Can apply hardware and OS maintenance with zero downtime

• Single line of support

• Integrated stack support

Reduce risk and eliminate manual errors when applying maintenance

PureSystems

High Availability and Disaster Recovery

PureData System for Transactions is highly availableby design Redundant hardware, networking and storage Recovery from hardware and software failures with DB2 pureScale Additional capacity can be added and utilized when needed

PureData systems can be configured with Q-Replication or InfoSphere Change Data Capture (CDC) to provide DR capabilities Not installed on PureData - installed on separate replication server Q Replication is now licensed with system CDC not bundled/installed/licensed with system

CDC/Q-Replication

Applications

Slides - 8

PureSystems

PureData Systems for TransactionsApplication Transparency

Work Load Balancing (WLB) Distributes application requests in a balanced fashion among DB2 members WLB is handled based upon capacity information returned by the pureScale server Lower capacity value than other members indicate that the member is busier than others

Automatic Client Reroute (ACR) Provides a mechanism to reroute application requests from the failing members to the other

available members With ACR enabled, the planned or unplanned outages do not affect applications so long as there is

one surviving DB2 member Seamless ACR happens when failover was at the start of the UOW Non-seamless ACR returns -30108 error and connection is rerouted automatically

Client Affinity Controls how DB2 members are used by the applications Client wants to connect to a particular member only If failover happens, it will reroute connection to other member defined in the list Should failed member become available again, client will reconnect to the available server again

A must read white paper for DB2 pureScale application: http://tinyurl.com/wlbacr or http://www.ibm.com/developerworks/data/library/long/dm-1206purescaleenablement/index.html

Monitoring

Slides - 16

PureSystems

PureData Systems for TransactionsMonitoring

Single Pane of Glass for Monitoring – IBM PureData Console

System Monitoring Management Nodes Compute Nodes Network Storage Hosts

Database Monitoring Single Sign-on and Trusted Context Monitored databases End-to-end Performance Monitoring

of Java and non-Java applications (License included)

PureSystems

Database performance management from integrated console

PureSystems

Database Performance MonitorKey Performance Monitoring Health Summary

Alerts Configuration

Dashboards Overview SQL Statements Buffer Pool and I/O Locking Logging Memory System Utilities Utilities Workload Connection Extended Insight

Reports Custom Reports Predefined Reports

PureSystems

Database Performance MonitorExtended (Top-down) and Core (Bottom-up) Monitoring

Application Program

Top-down

With Extended InsightBottom-up

With Core Capabilities

OPM Server

OPM Repository

PureData DB2 Servers

Security

Slides - 11

PureSystems

System Level SecurityRole Based Security – Separation of Duties

Hardware Administration

Database configuration Management

Security Administration

Cloud group Administration

Workload Administration

Auditing Administration

Role Based Security –

Separation of Duties

PureSystems

IBM PureData System for TransactionsSecurity Model

At System Level

Allows users to view / manage system console and hardware elements

Create users & user groups

At Instance Level Controls access to instance and to databases

within that instance Create instance users & user groups

At Database Level Access to objects within the database Create database users & user groups

PureSystems

System Level SecurityAuthentication Methods

LDAP based User authentication and group membership is managed through a central authority Users connecting to database can still be defined at LDAP even if you choose not to

use LDAP for system-level authentication

Operating System OS level users and groups

Can mix local and LDAP users and groups

PureSystems

Looking for fast time-to-value with the IBM’s workload optimized systems

Require “out of the box” Continuous Availability for businessapplication workloads

Are struggling to deal with increasing volumes of data and skyrocketing data management costs

Require maximum business application transaction performance to maintain a competitive edge

May or may not have DB2 pureScale and/or DB2 installed May have looked at DB2 pureScale in the past but were not ready for it

Currently have PureApplication System installed and looking to expand OLTP environments

Reduce complexityThe entire system

lifecycle is simplified from acquisition to

retirement

Accelerate time to valueData management expertise is built in

each system and ready for immediate use

Improve IT economicsEach system tuned

for the different needs of different analytic and transaction processing

When to use PureData System for Transactions?

PureSystems

Backup Slides

PureSystems

Hardware ArchitectureAvailable Configurations

Continued ….61

PureSystems

Hardware ArchitectureAvailable Configurations (Continued…)

PureSystems

Hardware ArchitectureStorage Design – Small Rack

Small Rack has 12 SSD and 36 HDD 6 MDISK for HDD (Each with 6 drive – RAID 10 *) 2 MDISK for SSD (Each with 6 drive – RAID 10) 2 Storage Pool each with 3 HDD MDISK + 1 SSD MDISK Each storage pool space – 9.3 TB (13% of SSD) Each storage pool is divided into generic volumes (VDISKS)

48 Internal Drives 8 MDisks 2 Pools 128 VDisks

* RAID 10 – Disks are mirrored and striped63

IBM PureData System for Transactions Technical Deep …praxiumgroup.com/adug-www/prev_pres/ADUG...

Documents

Cost/Benefit Case for IBM PureData System for Analyticslocussolutions.com/wp-content/uploads/2016/07/Cost-Benefit-Case... · Cost/Benefit Case for IBM PureData System for Analytics

PureData Systems Takes Managed Service Provider from ... · Case Study: Managed Services PureData Systems Takes Managed Service Provider from Inflexible Siloes to …

Universal PMML Plug-in for IBM PureData System for Analytics · · 2018-01-25PureData System for Analytics IBM PureData System for Analytics, ... analytics with IBM Netezza.

IBM Database Conversion Workbench 3 - …ibmtvdemo.edgesuite.net/software/analytics/learning-centers/data... · IBM Analytics IBM Database Conversion Workbench 3.5 IBM PureData System

ADUG 21-Oct 2013 Grahame Grieve

IM Forum 2013 PureData v2.0 FINAL.ppt

IBM Smarter Business 2012 - PureSystems - PureData

IBM PureData for Analytics (Netezza) To IBM Integrated ... · This guide documents different aspects of migration from PureData for Analytics Systems (aka PDA aka ... The higher performance

IBM Fluid Query 1 - KPI Digitalkpidigital.com/...PureData...Logical_Data_Warehouse_Solution_Brief.pdf · Netezza Platform Software (NPS) releases ... to data across any system from

Presentation adug afghan drug user group washington first part

Sa lgoh I An1ay Adug Netuu'1Van - SIL International · paunlarin ang kasanayan sa pagbasa at pagkaunawa, d.agdagan ang kaalaman, patindihin ang pagmamalaki sa unang wika, at ihanda

IBM PureData System for Analytics - NDM Technologies › datawarehouse › pdf › IBM PureData System D… · IBM PureData™ System for Analytics is a high-performance, scalable,

IBM Predictive Customer Intelligence - Sogeti.nl Customer... · Predictive Customer Intelligence Architecture Overview PureData for Analytics Deep customer analytics Actionable customer

DB2 Tech Talk PureData Systems Presentation.pdf

IBM PureData System: A Giant Leap Forward in Delivering ... · IBM PureData System. IBM PureData System: A Giant Leap Forward in Delivering Data Services. Building your own database

IBM PureData System for Analytics Overview - Meetupfiles.meetup.com/13471152/PDA-Netezza-Overview-Indy-Accel.pdf · IBM PureData System for Analytics Overview Chris Jackson ... (Powered

IBM PureData System for Analytics N3001 Overview

Addressing DDF Issues and Best Practices - ADUGalabamadb2usersgroup.org/2017-spring-presentations/ADUG-stale... · Addressing DDF Issues and Best Practices Adrian Burke DB2 SWAT team

Click to edit Master title style IBM PureData System for Operational Analytics James Cho Chief Architect PureData Transaction System And Operational Analytics

IBM PureData System for Analytics