COMP-13: The “Weakest-Link” in your High Availability system How to make sure your HA is really highly available Hugo Loera Senior Technical Support Engineer

COMP-13: The “Weakest-Link” in your High Availability system

How to make sure your HA is really highly available

Hugo Loera Senior Technical Support Engineer

© 2007 Progress Software Corporation2 COMP-13: The “Weakest-Link” in your High Availability system

Goals

To consider the entire system architecture for High Availability (HA).

Provide a summary of all the fault tolerance features that provide HA.

To think of possible disaster scenarios.


Agenda

Your system chain Keeping your hardware protected Keeping your network connected DB availability Protecting WebSpeed™ and AppServer™ tiers Sonic™ fault tolerance features


Your system chain

The strength of a computer system is based on it’s weakest link, much like a chain.

Goal: to remove all of the single points of failure for an application. User

Software

Hardware

Facilities

Network


Your system chain

System Components Redundant Components

Software

Application Additional application tiers

Database Clusters, Replication

OS Computer cluster node

Hardware

Network Alternate network links

Storage Devices Redundant File ServersRAID, Remote data replication devices

CPU Clusters

Power Source UPS, Power Plant


Your system chain

Application ServersApplication Servers

It is necessary to provide redundancy

Data BaseData Base

WebSpeedWebSpeed

SonicMQ®

HardwareHardware

FacilitiesFacilities

Sonic Adapters

Sonic Adapters

Internet

NetworkNetwork

Alternate siteAlternate site

Name Server Fault Tolerance Name Server Fault Tolerance

Clustered ServersClustered ServersMultiple

links

Multiple

links

CAA

MultipleWebServers

SonicMQ®

Cluster & ReplicationCluster & Replication

Multiple WSBrokers

Multiple WSBrokers

Multiple ABL-Adapters



Agenda

Your system chain Keeping your hardware protected Keeping your network connected DB availability Protecting WebSpeed™ and AppServer™ tiers Sonic fault tolerance features



Clustered ServersClustered Servers

Keeping your hardware protected



Hardware• Storage Devices

– RAID, Remote data replication device

• CPU– Cluster hardware

• Facilities– Air conditioning – Fire and flood protection– Power Source

UPS, Power Plant, Alternative Power Supplies

Alternate site


RAID Configuration


RAID (redundant array of independent disks)• Striping (provides load balancing)

• Mirrored configurations (for increased reliability)

Fiber Channel


Node 1

Application

OS

Local Storage

Cluster Manager Common

Storage

DB

DB

DB

OE Enterpris

e

Cluster name

or

Cluster IP

Address

Node 2

Application

OS

Cluster Manager

Local StorageOE

Enterprise

Heartbeat


HA Cluster


Node 1

Application

OS

Local Storage

Cluster Manager Common

Storage

DB

DB

DB

OE Enterpris

e

Cluster name

or

Cluster IP

Address

Node 2

Application

OS

Cluster Manager

Local StorageOE

Enterprise


XFailover

Cluster


Agenda





links

Multiple

links

Keeping your network connected



Network

• Alternate network paths

• Redundant network links

• Configure a Domain Name Server (DNS)

Site BSite A

Users site



If using alternate site• Use a “virtual” DNS entry for

– Your DB server– Application server– Web servers– Programs File Server

• Hostnames should be virtual names

Consider user VPN access for emergencies


Agenda




Clustered ServersClustered Servers


DB availability

Multiple

links

Multiple

links


DB availability

Cluster configuration• Protects from hardware failure

• Quick recovery time

• Easy to register DB once the cluster has been configured.

OpenEdge® cluster support• Included with the Enterprise RDBMS

• Allows use of the procluster command to:– register the DB with the cluster manager – disable DB registration– start and stop the DB– query the status of the DB


Vendor specific cluster commands

DB availability


DB availability

OE provides a common interface• For different cluster products.

• To integrate OE commands into the cluster environment.

..stop

procluster DB stop

pscluster

Node 1

Cluster Manager

OE Enterpris

e

Caa_stop…

Caa_stop…


DB availability

OpenEdge DB replication• Protects from storage

problems• Site fault tolerance

protection• Can be manually activated

Source(live)

Target

Client Client

TransitionFailover

Live

Client Client

X TransitionFailback

Target Live

ClientClient


DB availability

Combining OpenEdge Replication with Clusters

Node 1

DB Broker

Operating System

Cluster Manager

Common Storage

Local StorageOpenEdge Enterprise

DB & Replication

Node 2

Operating System

Cluster Manager


DB & Replication

Target DB

Replication AgentReplication

Server

OpenEdge Enterprise

& Replication

Source DB

Site A

Site B



Node 1

Operating System

Cluster Manager

Common Storage


DB & Replication

Node 2

Operating System

Cluster Manager


DB & Replication

Target DB

Replication AgentReplication

Server

OpenEdge Enterprise

& Replication

Source DB

Site A

Site B

DB Broker

DB availability

X



Common Storage

Node 2

Operating System

Cluster Manager


DB & Replication

DB

DB Broker

OpenEdge Enterprise

& Replication

Source DB

Site A

Site B

Node 1

Operating System

Cluster Manager


DB & Replication

X X X

DB availability

X


Agenda





links

Multiple

links


Protecting WebSpeed and AppServer tiers


MultipleWebServers

Multiple WSBrokers

Multiple WSBrokers





NameServer Fault Tolerance• Using UDP broadcast address

• Will provide instant access to alternate brokers

• Can use Name Server Load Balancing

Redundant brokers can be registered• AppServer brokers

• WebSpeed brokers



Name Server Replication based fault tolerance

Client

Host2

NS1

AppServerBroker

WebSpeedBroker

Subnet

LAN1 Gb

Backbone

Messenger

WebServerINTERNET

Host1

NS1

AppServerBroker

WebSpeedBroker


Getting the UDP broadcast address on UNIX

$ netstat -iName Mtu Net/Dest Address Ipkts Ierrs Opkts Oerrs Collis Queue lo0 8232 loopback localhost 2318894 0 42318894 0 0 0 le0 1500 psc-bed oasis 3625559 16 79807506 0 0 0

$ ifconfig le0ce0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index2 inet 192.128.5.30 netmask ffff0000 broadcast 192.128.255.255



Getting the UDP broadcast address on WindowsC:\>ipconfig

Windows IP Configuration

Ethernet adapter Local Area Connection:

Connection-specific DNS Suffix . : blabla.speak.com IP Address. . . . . . . . . . . . : 192.128.3.24 Subnet Mask . . . . . . . . . . . : 255.255.0.0 Default Gateway . . . . . . . . . : 192.128.3.1

IP address 192.128. 3. 24 Subnet Mask 255.255. 0. 0

__________________UDP Broadcast=

IP address 192.128. 3. 24 NOT Subnet Mask 0. 0.255.255 OR

__________________UDP Broadcast= 192.128.255.255



Connect(“-H 192.128.255.255 -S 5162 -AppService asbroker1”)

NameServer=NS1

portNumber=5162hostName=192.128.255.255

NAME SERVER REPLICATION

Client

Host2

NS1

AppServerBroker

Subnet

LAN

Host1

NS1

AppServerBroker




Using redundant WebSpeed brokers• Messenger has to use UDP broadcast address• Distributed standalone messengers can be configured• Can use Name Server Load Balancing

Can use multiple web servers• For fault tolerance• Load balancing• Multiple products are currently available for web server

fault tolerance and load balancing



Stand alone WebSpeed Messenger ubroker.properties configuration

## Sample definition of a NameServer#[NameServer.NS1] location=remote hostname=192.128.255.255 environment=NS1 srvrLogFile=$WRKDIR/NS1.ns.log autoStart=0 ...

## WebSpeed Messenger Properties#[WebSpeed.Messengers] certStorePath=$DLC/certs/ controllingNameServer=NS1


Messenger

WebServerINTERNET


Name Server Replication based fault tolerance

Host2

NS1

WebSpeedBroker

Subnet

LAN

Messenger

WebServerINTERNET

Host1

NS1

WebSpeedBroker


ABL application working with SonicMQ

OpenEdge Database

Unified BrokerOpenEdge ABL

*.w, *.p ABL-JMS API

ABL Adapterbroker

Sonic MQ



Agenda






links

Multiple

linksMultipleWebServers

CAA

SonicMQ®


Multiple WSBrokers

Multiple WSBrokers



Sonic fault tolerance features



What is SonicMQ® ?

A Standards-Based Enterprise Messaging Server

User benefits• Uses standards-based infrastructure• Enterprise-class reliability • Configuration monitoring• Can include message transformation and routing

using ESB (Enterprise Service Bus).


Sonic Components

Domain • Domain Manager

– Agent Manager– Manager Broker– Directory Service

Containers• MQ Container provides runtime

support for Sonic components• Can include ESB Containers to hosts

ESB services

Messaging Nodes• Message Broker• Broker Cluster


DomainContainer1

QueueTopic

Container2

ESB Container

Management Broker

AgentManager

DirectoryService

Message BrokerQueueTopic


Continuous Availability Architecture features

SonicMQ's CAA includes:• Resilient client connections

• Replication brokers

• Fault tolerant management services and communications

Use some or all of these features to provide a continuously available messaging infrastructure.



Resilient client connections

JMS ClientLost connection

Recovered connection

Fault Tolerant Broker

Broker preserves client context

Fault tolerant client runtime preserves session



Replicated Brokers

Creates a primary and backup broker Backup broker replicates all acceptors on

primary Primary accepts clients and sends

replications data to backup Both brokers are functionally equivalent



Replicated brokers

Replication connection

ping, ping, ping, …

Broker DB

Broker DB

JMSClient

Broker(Standby)

Broker(Active)



Broker roles: failing over

JMS Client

Broker(new active)

Lost connection

Recovered connection

Broker(formerly active)

Replicated Broker pair


XX


Continuously available management communications

Options include:• A cluster of management brokers

• Replicated management brokers

Recommended



Broker 1


Domain

ESB Container

JMS Client

Broker 2

Interbroker Connection

Replicated brokers Broker Cluster

Node

Host B

Host A


Continuously available management

Container1

Broker1(Backup)

Container2

Agent Manager (Backup)

Container3


Agent Manager (Primary)

Broker1(Primary)

DirectoryService

(Primary)

DirectoryService

(Backup)


Example of a clustered management topology

Shared directory

store

Management broker

Agent Manager

Directory Service

Container1

Management broker

Agent Manager

Directory Service

Container2

Cluster

Primary

Primary

Backup

Backup

Managementclient

Managementclient

Managementclient



Site B

Site A

The big picture


In Summary

Consider all possible components involved in the Application.

Make a plan that supports your HA configuration.

Progress products have all the required technology for HA.


For more Information, go to…

PSDN

• DB–12: Achieving High Availability with Clusters and OpenEdge Replication http://www.psdn.com/library/entry!default.jspa?categoryID=254&externalID=850

• OpenEdge Replication How–To http://www.psdn.com/library/entry!default.jspa?categoryID=254&externalID=841

• Messaging Fundamentals for Sonic ESB http://www.psdn.com/library/entry!default.jspa?categoryID=91&externalID=2143

Documentation:• Open Edge 10.1B Installation and Configuration Guide• OpenEdge Getting Started: Application and Integration Services • Progress SonicMQ Deployment Guide V7.5


Questions?


Thank you foryour time


Documents

COMP-13: The “Weakest-Link” in your High Availability system How to make sure your HA is really highly available Hugo Loera Senior Technical Support Engineer