Oracle Database 12c High Availability...GDS . Catalog . Standby . Global Service . Managers . Global Service . Managers Clients (Users) Clients (Users) App/Middleware App/Middleware

Oracle Database 12c Global Data Services (GDS) Nagesh Battula Sr. Principal Product Manager, Oracle Mark Dilman Consulting Member of Technical Staff, Oracle Al Fischer IT Manager, Database Services, Amway

2 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

Safe Harbor Statement

THE FOLLOWING IS INTENDED TO OUTLINE OUR GENERAL PRODUCT DIRECTION. IT IS INTENDED FOR INFORMATION PURPOSES ONLY, AND MAY NOT BE INCORPORATED INTO ANY CONTRACT. IT IS NOT A COMMITMENT TO DELIVER ANY MATERIAL, CODE, OR FUNCTIONALITY, AND SHOULD NOT BE RELIED UPON IN MAKING PURCHASING DECISIONS. THE DEVELOPMENT, RELEASE, AND TIMING OF ANY FEATURES OR FUNCTIONALITY DESCRIBED FOR ORACLE’S PRODUCTS REMAINS AT THE SOLE DISCRETION OF ORACLE.

Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 3

Optimize Oracle Active Data Guard/Oracle GoldenGate: Oracle Database 12c Global Data Services [CON8704]

Oracle Active Data Guard and Oracle GoldenGate enable distribution of application workloads across replicated databases. However, when applications are spread across multiple databases and data centers, it can be challenging to efficiently use all the databases for best performance and availability. In this session, Oracle Product Development experts introduce a unique new scalability and availability feature of Oracle Database 12c: Global Data Services. It enables region-based workload routing, load balancing, and service failover across replicated databases, within and across data centers. Learn how you can maximize utilization of your replication investments with Global Data Services while enabling a seamless distributed private data cloud for your IT.

Abstract


Program Agenda

Workload Management Challenges of Replicas – 5 min

Introduction to Oracle Global Data Services (GDS) – 10 min

GDS Architecture & Use Cases– 15 min

Amway, “Increase the Nines” – 10 min

GDS Internals – 15 min

Summary – 5 min


Database Workload Management

Sub-optimal resource utilization – No automated load balancing

Application outages when replicas fail – No Service HA

Workload on replicas managed in silos – No centralized workload management

Solution: A holistic automated workload management framework for replicated databases

Challenges in Replicated Environments

?

Oracle GoldenGate

Primary Active Standby Active Standby

Active Data Guard

Data Center #1 APAC Region

Data Center #2 EMEA Region


Oracle Database 12c: Global Data Services (GDS)

Workload routing & balancing, Service failover & management for replicas (1 or more data centers) – Takes into account

DB instance load, Network latency Workload management policies (region affinity,

load balancing goals, DB cardinality, DB role, replication lag tolerance)

Benefits – Better performance & availability for applications – Global resource management with a single

framework

Workload Management for Replicas

Oracle GoldenGate

Primary Active Standby Active Standby

Active Data Guard



Global Data Services

Global Service Failover

Load Balancing Load Balancing

GDS Unified Framework


Automated Workload Management for Replicas

Workload Routing ( Region-based & Lag based) Load Balancing (Connect-time & Run-time )

Global Service Failover Role-based Global Services

Centralized Service Management

GDS Capabilities


GDS Configuration – In a Nutshell

Global Service Manager (GSM): – Regional listener to the incoming database connections

– Performs Connect-time load balancing

– Publishes Run-time load balancing advisory

– Service failover & management

GDS Catalog: stores GDS configuration metadata

GDSCTL: CLI to administer GDS configuration (GUI support of GDS in EMCC DB Plug-in 12.1.0.5)

GDS Region: Group of databases and clients in close network proximity, e.g., East, West

GDS Pool: Databases that offer a common set of global services, e.g., HR, Sales

Global Service: Database Service provided by multiple databases with replicated data

– Local service + {Locality, replication lag, database cardinality, load balancing goals}

– Establish workload management policies via Service attributes



Clients (Users) Clients (Users)

App/Middleware App/Middleware

Replication

GDS Framework

GDSCTL and EMCC

GDS Catalog Primary

GDS Catalog Standby

Global Service Managers


SALES POOL (sales_report_srvc, sales_entry_srvc)

SALES Databases


GDS Enabled Applications – Blue Print



GDSCTL and EMCC

GDS Catalog Primary

GDS Catalog Standby



Clients (Users) Clients (Users)

App/Middleware App/Middleware

Replication

SALES POOL (sales_report_srvc, sales_entry_srvc)

SALES Databases

GDS is a shared infrastructure

One GDS configuration manages – 300 Database Instances – 1000 Global Services – 20 GDS Pools – 10 GDS Regions – 5 GSMs per Region – 100 Mid-tier connection pools

Databases in GDS – Must be licensed for Oracle Active Data Guard

and/or Oracle GoldenGate – Oracle Streams & 3rd Party replication

technologies are supported – Must be Oracle Database EE 12.1 or greater – Can be Oracle Multitenant or non-Multitenant

HR POOL(hr_apac_srvc, hr_emea_srvc)

Replication

HR Databases

GDS Framework


GDS Deployment

Install GSM software on GSM servers – Min of 1 GSM per region – Recommended 3 GSMs/region

Setup GDS Administrator accounts & privileges Configure GDS

– Create GDS Catalog – Add GSMs, Regions, Pools, Databases, Global Services

Setup client connectivity

High Level Steps


Client Connectivity in GDS

(DESCRIPTION=

(FAILOVER=on)

(ADDRESS_LIST=

(LOAD_BALANCE=ON)

(ADDRESS=(global_protocol_address_information))

(ADDRESS=(global_protocol_address_information)))

(ADDRESS_LIST=

(LOAD_BALANCE=ON)

(ADDRESS=(global_protocol_address_information))

(ADDRESS=(global_protocol_address_information)))

(CONNECT_DATA=

(SERVICE_NAME=global_service_name)

(REGION=region_name)))

Clients connect to GSM instead of the database listener

– GDS forwards the connection to the local listener (bypassing the SCAN listeners)

TNS-entries must contain two lists of addresses:

– One list of local GSMs for load balancing and intra-region failover

– Another list of addresses for remote GSMs for inter-region failover

Clients specify global service name and which region they are in.

APAC’s GSMs

EMEA’s GSMs


Global Service Attributes Preferred

– Databases designated to provide the Global Service

Available – Databases that provide Global Service if not enough Preferred databases are

running – If one of the preferred databases fails, then GSM maintains the cardinality of the

Global service by starting the service on an Available database

Preferred_All – All databases in a GDS Pool are preferred for the Global Service

Options for Add Service : {-preferred_all | -preferred dbname_list [-available dbname_list] }

– GDSCTL>add service -service sales_qry_srvc -gdspool sales -preferred sfo -available bos

– GDSCTL>add service -service sales_reporting_srvc -gdspool sales -preferred_all

Service Placement


Global Service Attributes Role based Global Service

– Service should be active only when the database is either a primary or standby – Can be started on a database if its role matches global service’s role attribute

– Options for Add Service : [-role {PRIMARY | PHYSICAL_STANDBY [-failover_primary] | LOGICAL_STANDBY |

SNAPSHOT_STANDBY}]

– GDSCTL>add service -gdspool sales -service sales_reporting_srvc -preferred_all -role physical_standby

Lag Tolerance – Establish application's tolerance for non-current data – Specify the lag limit for the Global Service in seconds

– Options for Add or Modify Service : -lag {lag_value | ANY}

– GDSCTL>add service -service sales_reader_lag180_srvc -gdspool sales -preferred_all -role physical_standby -lag 180

Service Placement


Achieve geographical affinity between clients and Databases Options for Add or Modify Service

[-locality {ANYWHERE | LOCAL_ONLY [-region_failover]}]

Locality ANYWHERE Client connections and work requests are routed to any region for load balancing or failover GDSCTL>add service -service sales_reader_srvc -gdspool sales –preferred_all –locality

ANYWHERE

Locality LOCAL_ONLY Regardless of load, GDS will not route to databases in other regions

GDSCTL>add service -service sales_reader_srvc -gdspool sales –preferred_all –locality LOCAL_ONLY

Locality LOCAL_ONLY –region_failover Client connections and work requests are routed to another region when all databases in a region have failed

GDSCTL>add service -service sales_reader_srvc -gdspool sales –preferred_all –locality LOCAL_ONLY –region_failover

Global Service Attributes Locality Based Routing


CLBGOAL – GDS supports connect-time load balancing for all

clients

– Directs connection requests to the best database instance in GDS pool

– Takes into account

– Load statistics from all GDS pool databases

– Inter-region network latency, locality and CLB goal

– Options for Add Service : [-clbgoal {SHORT | LONG}]

– GDSCTL>add service -service sales_clb_srvc -gdspool sales –preferred_all -clbgoal LONG

RLBGOAL – GDS supports Runtime load balancing feature of

connection pools for Oracle database integrated clients (OCI, JDBC, ODP.NET, WLS)

– Publishes RLB Advisory to clients

– Takes into account – Per-service performance data from pool databases

– Inter-region network latency, locality and RLB goal

– Based on advisory, clients distribute workload requests across persistent connections spanning GDS Pool database instances

– Options for Add Service : [-rlbgoal {SERVICE_TIME | THROUGHPUT}]

– GDSCTL>add service -service sales_rlb_srvc -gdspool sales –preferred_all –rlbgoal SERVICE_TIME

Global Service Attributes Load Balancing


Graphic Section Divider


Oracle Global Data Services

Application handles multi-master conflict resolution

GDS can route all workloads to nearest and best Replica in the client’s region

Global Data Services (GDS) Region Affinity in Oracle GoldenGate Multi-Master

Oracle GoldenGate

Master

Data Center 1 Data Center 2

Order Entry Service

Clients Clients

Master

Global Service Manager (GSM)

Load Balancing LEGEND:

Order Entry Service

Reporting Service

Reporting Service

GDSCTL>add service –service reporting_srvc –gdspool sales –preferred_all –locality LOCAL_ONLY –region_failover GDSCTL>add service –service order_entry_srvc –gdspool sales –preferred_all –locality LOCAL_ONLY –region_failover



Application handles multi-master conflict resolution

GDS provides connect-time and run-time load balancing (within and across data centers) for all work requests

Global Data Services (GDS) Load Balancing in Oracle GoldenGate Multi-Master

Oracle GoldenGate

Master


Order Entry Service

Clients Clients

Master



Order Entry Service

GDSCTL>add service -service order_entry_srvc -gdspool sales –preferred_all -clbgoal LONG GDSCTL>add service -service reporting_srvc -gdspool sales –preferred_all -clbgoal LONG

Reporting Service

Reporting Service



With GDS, route Read Write workload to primary/master

Balance Read Only workload on a set of replicas

Maximize utilization and performance

Global Data Services (GDS) Balance Read Workloads in Active Data Guard & Oracle GoldenGate

Standby

Replication

Primary Standby


Order Entry Service

Reporting Service

Clients Clients

Reporting Service



Active Data Guard Example: GDSCTL>add service -service reporting_srvc -gdspool sales –preferred_all –role PHYSICAL_STANDBY -clbgoal LONG GDSCTL>add service -service order_entry_srvc -gdspool sales –preferred_all –role PRIMARY

Replication



Global Data Services (GDS) Role-based Global Services for Active Data Guard

Active Data Guard

Standby


Clients Clients Order Entry Service runs on Primary

Reporting Service runs on Standby

Primary

Reporting Service

Order Entry Service





Global Data Services (GDS) Role-based Global Services for Active Data Guard

Active Data Guard


Clients Clients

Standby

Reporting Service

Order Entry Service

Order Entry Service runs on Primary

Reporting Service runs on Standby

Upon Data Guard role change, GDS fails over services based on Role

Primary



GDSCTL>add service -service order_entry_srvc -gdspool sales –preferred_all –role PRIMARY GDSCTL>add service -service reporting_srvc -gdspool sales –preferred_all –role PHYSICAL_STANDBY –failover_primary



Specify replication lag limit for a service.

GDS ensures that service runs on Active Data Guard standby(s) with lag less than this limit

Improved data quality

Global Data Services (GDS) Replication Lag Tolerance in Active Data Guard

Replication

(Oracle Active Data Guard )

Standby or Replica

Replication

Primary or Master

Standby or Replica

Order Entry Service

Reporting Service

Clients Clients

Reporting Service



Replication Lag

exceeded threshold




Global Data Services (GDS) Replication Lag Tolerance in Active Data Guard

Replication

(Oracle Active Data Guard)

Replication

Primary or Master

Standby or Replica

Data Center 1

Order Entry Service

Reporting Service

Clients Clients Data Center 2 Specify replication lag

limit for a service. GDS ensures that

service runs on Active Data Guard standby(s) with lag less than this limit

Improved data quality

Replication Lag

exceeded threshold GDSCTL>add service -service reporting_srvc -gdspool sales

–preferred_all –role PHYSICAL_STANDBY –lag 180



Standby or Replica



Inter-database Service failover within and across regions

Higher availability and improved manageability

Global Data Services (GDS) Service Failover for Active Data Guard & Oracle GoldenGate

Replication

Standby or Replica


Order Entry Service

Clients Clients

Reporting Service

Reporting Service


Load Balancing

LEGEND:

Active Data Guard Example: GDSCTL>add service -service reporting_srvc -gdspool sales –preferred_all –role PHYSICAL_STANDBY -failover_primary

Primary or master




Inter-Database Service Failover

Oracle Database 12c - GDS Demo Viewlet


Global Data Services 12c A Customer’s Point of view

Al Fischer

CORPORATE IT

Global Database Services

Oracle Global Data Services Al Fischer Manager IT – Database Services

30 C O R P O R A T E I T

31 C O R P O R A T E I T 31

Regional DBA Team Members Global DBA Team

Australia/New Zealand

North America

Latin America

Russia Europe

South Africa

Japan Korea

India

China

AITS Malaysia SE Asia

Global and Regional Database Teams


MAGIC - Managing Awards Growth Income and Compensation

Bonus Value / Points Value (BV/PV) on sales

Monthly rewards based on sales that month

Multi-level Marketing For downline sales, they get their full benefit, but the upline also accumulates BV/VP

Achievement level based on number sales that month

Multiple months at higher levels lead to greater rewards

Crucial for business that ABO’s can see current BV/PV near or at real-time

Read only operation

Outage of BV/PV has direct consequence on sales

Outage of BV/PV has significant business cost.

How we pay our Amway Business Owners


Business Challenge How many 9’s at what cost?

99.9 is 8.76 hours a year 99.99 is 52 minutes a year 99.999 is 5 minutes a year

BV/PV inquiry time Traverse the line of sponsorship Up to the minute level information

Proactive patching and security PCI compliance

Amway crown jewels

Total Cost Hardware / storage Resources to manage

Uptime

Response time

Security

TCO


Existing Architectural overview

Active Data Guard for: DR Backups Read workloads


GDS advantages Leverage existing infrastructure resource utilization Primary database DR database Data Guard Broker Existing services

No application change except for TNS connectivity

Already separated for read-only and OLTP operations

Minimal set-up (DBA time) Quick deployment Adaptable for maintenance 2-3 hours after delivery of

servers

UpTime Read only transactions can get

another ‘9’ without major infrastructure changes

36 C O R P O R A T E I T 36 C O R P O R A T E I T 36 C O R P O R A T E I T

GDS POC

37 C O R P O R A T E I T 37 C O R P O R A T E I T 37 C O R P O R A T E I T 37 C O R P O R A T E I T

High level Steps Amway tested GDS on Oracle Multitenant Databases Install GSM software on multiple nodes

Download / unzip ./runInstaller -> root.sh

Set up GDS Admin accounts and priv’s alter user gsmuser account unlock :

Configure GDS Create GDS catalog Add GSMs

GDSCTL>start gsm –gsm <gsmname> Add Services

Set up Client connectivity


Test Case Result

Crash Primary / Standby

Services failover as expected

Crash GDS Catalog

All services remain available

Active Data Guard Switchover

Services restart on expected nodes

Lag Tolerance

Service stops / restarts on other node if lag exceeds threshold

Amway Test Cases


Test Case Benefit

Load Balancing

Faster return for query

RAC database

Higher availability

Moving from one container database to another

Higher availability Ease of upgrade

Amway Future Steps


Next Steps for Amway • Demo to Business User • Include in Oracle Database 12c upgrade plans • Procure GSM servers and Catalog

QA and Production • Verify in our QA environment

• We have ADG in both Production and QA • Include GDS as part of our High Availability offerings to

other programs/projects • Global Datawarehouse?

• Enjoy the benefits


Contact Information Al Fischer

Database Services Manager 616-787-1807 - Work [email protected]

Ron Swanson Principal Oracle DBA 616-787-5387 – Work [email protected]

mailto:[email protected]�

mailto:[email protected]�




GDS Internals

Administration

DB Health Monitoring

Connection Management

Fault Management

Change Management

Key Workflows


GDS – Administrative Operations ADD / MODIFY / REMOVE :

– GDSPOOL – REGION – DATABASE – BROKERCONFIG – SERVICE – …

START / STOP SERVICE


GDS – Administration Flow

GDS Catalog DB

Pool Database

1. To modify GDS configuration (e.g. To add a global Service), GDS Administrator uses GDSCTL and connects to catalog DB via GSM

2. GSM routes request to catalog

3. GDSCTL establishes connection to catalog DB

4. GDS Administrator runs GDSCTL commands and modifies the catalog

5. Catalog notifies GSMs of changes

Pool Database

6. Global Master GSM modifies Pool Databases

GSM GSM

Global Master


GDS – Component Architecture CLIENTS DATABASES

GSMs

APP SERVER

CONNECTION POOL

APPLICATION

US-EAST EUROPE

Connection Pool

Application

Registration, Performance Stats

Network RTT


GDS – Managing Database Connections For High Performance and Availability

For all clients – Connect-time Load Balancing (CLB) across databases

Choose the best DB instance to connect For Oracle integrated connection pool based clients

– Run-time load balancing (RLB) Selecting a cached connection for a work request

– Gravitation Moving unused connections to heavily loaded instances

– Proactive handling of instance UP and DOWN events


GDS – Connect-time Load Balancing (CLB) CLIENTS DATABASES

GSMs EUROPE US-EAST

APP SERVER

CONNECTION POOL

APPLICATION

Connection Pool

Application


APP SERVER

CONNECTION POOL

APPLICATION

GDS – Run-time Load Balancing (RLB) CLIENTS DATABASES

GSMs EUROPE US-EAST FAN over ONS

40%

20%

30%

10%

RLB advisory

Connection Pool

Application

Regional Master


GDS – Run-time Load Balancing Graph (I) Standalone Identical Database Servers - Simple External Load

Routing responds gracefully to changing system conditions


GDS – Run-time Load Balancing Graph (II) Standalone Asymmetrical Database Servers - No External Load DB b: 4 CPUs DB c: 3 CPUs DB d: 2 CPUs

GDS does intelligent load balancing even across asymmetrical database servers


APP SERVER

CONNECTION POOL

APPLICATION

GDS – Gravitation of Connections CLIENTS DATABASES

GSMs EUROPE US-EAST FAN

40%

20%

30%

10%

20%

50%

RLB advisory

Connection Pool

Application

BACKUP


APP SERVER

CONNECTION POOL

APPLICATION

GDS – Connection Failover CLIENTS DATABASES


40%

20%

30%

10%

‘Instance Down’

Connection Pool

Application

RLB advisory

0%

30%


APP SERVER

CONNECTION POOL

APPLICATION

GDS – Reconnecting to a DB Instance CLIENTS DATABASES


40%

30%

30%

0%

‘Instance UP’

Connection Pool

Application

RLB advisory

20%

10%


GDS – Global Service Failover

Global Service : SALES

6) Global Master GSM observes that service SALES went down on database Sales-1 and starts it on database Sales-2.

Sales-2

2) Connection Pool has established connections to the service SALES on the database Sales-1

Sales-1

8) Connection Pool creates connections to service SALES on the database Sales-2 and workload is redirected there

Global Service : SALES

App Server

1) Client is connected to the Application Server

7) Regional Master GSM observes that service SALES is up on Sales-2 and notifies the connection pool

GSM

3) DB Sales-1 crashes

Client

DOWN UP

4) Regional Master GSM observes that service SALES went down and notifies the connection pool

5) Connection Pool closes connections to Sales-1

Regional Master

GSM Global Master




Automated Workload Management for Replicas Feature Oracle GDS 3rd Party Load

Balancers

Locality based routing

Connect-time database load balancing

Publish Run-time database load balancing advisory for Oracle integrated connection pool based clients

Replication lag based database workload routing

Inter-database global Service failover

Automatic role based global Services

Centralized management of database Services across replicas

Native integration for Active Data Guard

Cost Effectiveness Incl. with Active Data Guard or Oracle GoldenGate

Additional $$$

GDS is a complete Dynamic Workload Management capability for Oracle Replicated Databases


Active Data Guard – Data Protection, DR – Query Offload

GoldenGate – Active-active – Heterogeneous

Active Replica

Edition-based Redefinition, Online Redefinition, Data Guard, GoldenGate – Minimal downtime maintenance, upgrades, migrations

RAC – Scalability – Server HA

Flashback – Human error

correction

Production Site Enterprise Manager Cloud Control – Coordinated Site Failover Application Continuity – Application HA Global Data Services – Service Failover / Load Balancing

Oracle Maximum Availability Architecture

RMAN, Oracle Secure Backup – Backup to disk, tape or cloud


Global Data Services

Provides workload routing, load balancing, service failover & management for replicated databases

Key benefits – Higher availability

Failover a Service across databases located anywhere

– Improves application response time Workload routing & real-time load balancing

– Better resource utilization Centralized administration of global resources

Summary


Resources OTN GDS Portal:

http://www.oracle.com/goto/gds

OTN HA Portal: http://www.oracle.com/goto/availability

Maximum Availability Architecture (MAA): http://www.oracle.com/goto/maa

MAA Blogs: http://blogs.oracle.com/maa

Exadata on OTN: http://www.oracle.com/technetwork/database/exadata/index.html

Oracle HA Customer Success Stories on OTN: http://www.oracle.com/technetwork/database/features/ha-casestudies-098033.html


Documents

Oracle Database 12c High Availability...GDS . Catalog . Standby . Global Service . Managers . Global Service . Managers Clients (Users) Clients (Users) App/Middleware App/Middleware