30
©Continuent 2013 Tungsten University: Introduction to Continuent Tungsten 2.0 Robert Hodges, CEO Giuseppe Maxia, Director of QA

Tungsten University: Introduction to Continuent Tungsten 2.0

Embed Size (px)

DESCRIPTION

Continuent Tungsten 2.0 is a major step forward that includes key feature improvements to help you manage very large datasets on MySQL. It also sets the stage for operation in cloud environments like Amazon. We will cover the following details to help you plan for using the new release: - Major improvements in Continuent Tungsten 2.0, including replication and connectivity upgrades - Major improvements in cluster management for cross-site and cross-cloud operations, especially in Amazon AWS - How to upgrade from Continuent Tungsten 2.0 from older Continuent Tungsten releases - How to create multi-master, multi-site clusters - How to use parallel replication to eliminate slave lag - How to set connection pooling to boost connectivity throughput - Showcase Continuent Tungsten roadmap for new features including our new Tungsten Management API and the upcoming Continuent Tungsten GUI.

Citation preview

Page 1: Tungsten University: Introduction to Continuent Tungsten 2.0

©Continuent 2013

Tungsten University: Introduction to

Continuent Tungsten 2.0Robert Hodges, CEO

Giuseppe Maxia, Director of QA

Page 2: Tungsten University: Introduction to Continuent Tungsten 2.0

©Continuent 2013

Introducing Continuent

2

• The leading provider of clustering and replication for open source DBMS

• Our Product: Continuent Tungsten

• Clustering - Commercial-grade HA, performance scaling and data management for MySQL

• Replication - Flexible, high-performance data movement

Page 3: Tungsten University: Introduction to Continuent Tungsten 2.0

©Continuent 2013

Continuent Tungsten 2.0 Topics

3

• What’s in Continuent Tungsten 2.0?

• What is the release schedule?

• How do I upgrade from 1.5.x?

• What’s coming next?

Page 4: Tungsten University: Introduction to Continuent Tungsten 2.0

©Continuent 2013

What’s in Continuent Tungsten 2.0?

4

Page 5: Tungsten University: Introduction to Continuent Tungsten 2.0

©Continuent 2013

New Feature Overview

5

• Tungsten 2.0 Replicator integrated into clustering

• TPM updates installations more e"ciently

• Ability to create read-only slaves

• Better connector load balancing and connector management

• Cookbook test for installed cluster

• Documentation build-out

Page 6: Tungsten University: Introduction to Continuent Tungsten 2.0

©Continuent 2013

Overall Replication Improvements

6

• Parallel apply to reduce slave lag

• Transaction history log much more robust

• Support for multiple replication services

• Simpler integration with stand-alone replication for multi-master

Page 7: Tungsten University: Introduction to Continuent Tungsten 2.0

©Continuent 2013

Parallel Apply Concepts

7

cust1cust2cust3

cust1cust2cust3

appappapp

Shards

Channels

Master Slave

Page 8: Tungsten University: Introduction to Continuent Tungsten 2.0

©Continuent 2013

Parallel Apply Intro

8

• Best for apps with many independent schemas

• Enabling parallel apply and selecting channels

tpm install \ --repl-svc-parallelization-type=disk \ --channels=10

• Take replicator o#ine cleanly to make changes!

• Restore is complicated if you change channels!

• Changes position reporting on slave!

• Up to 200x reduction in slave lag

Page 9: Tungsten University: Introduction to Continuent Tungsten 2.0

©Continuent 2013

Replication Services

9

• Replicators support multiple services per replicator (clusters use only one)

• Service named after cluster

• Catalog data in schema tungsten_<svcname>

• THL stored in /opt/continuent/thl/<svcname>

Page 10: Tungsten University: Introduction to Continuent Tungsten 2.0

©Continuent 2013

New Status Monitoring Commands

10

• Extensions to ‘trepctl status’

• trepctl status -name shards - Transactions per shard

• trepctl status -name stages - Stage con$guration

• trepctl status -name stores - Log/queue info

• trepctl status -name tasks - Task performance

• trepctl services - Show replication services

• relativeLatency setting to help detect stalls (also visible with cctrl ls -l)

Page 11: Tungsten University: Introduction to Continuent Tungsten 2.0

©Continuent 2013

Transaction History Log

11

• THL has new metadata

• ##charset property tracks SQL character set

• Service property shows source of transaction

• Shard property is derived from schema and enables parallel apply

• Epoch number printed on each transaction

• Many improvements to log consistency checks

• Easier to $x logs or regen from a particular point

Page 12: Tungsten University: Introduction to Continuent Tungsten 2.0

©Continuent 2013

Multi-Master Operation

12

• Standalone 2.0 replicators link clusters

• New standalone installation options for setting up multi-master clusters

• Replicators track transaction source to cluster

• Certi$ed multi-master topology for MySQL

Page 13: Tungsten University: Introduction to Continuent Tungsten 2.0

©Continuent 2013

PARALLEL REPLICATION DEMO

13

Page 14: Tungsten University: Introduction to Continuent Tungsten 2.0

©Continuent 2013

TPM - Changing Properties

14

# Recover current settings

shell> ./tools/tpm reverse

# Update to settings on all hosts

shell> export CONTINUENT_PROFILES=/opt/continuent/software/confshell> cd /opt/continuent/software/continuent-tungsten-2.0.1-###shell> ./tools/tpm fetch --hosts=<hostname>,autodetect \ --user=tungsten --directory=autodetectshell> ./tools/tpm reverseshell> ./tools/tpm update mycluster --repl-java-mem-size=2048

# Update to settings on a single host

shell> cd /opt/continuent/tungstenshell> ./tools/tpm update --repl-java-mem-size=2048

Page 15: Tungsten University: Introduction to Continuent Tungsten 2.0

©Continuent 2013

Connectivity Upgrades

15

• Load balancing reads to most advanced slave

(router.properties)dataSourceLoadBalancer_RO_RELAXED=com.continuent.tungsten.router.resource.loadbalancer.MostAdvancedSlaveLoadBalancer

• Old password support (handles MySQL 4.1 libs)

• Experimental connection pooling

(router.properties)# c3p0 pool:#connectionProviderClass=com.continuent.tungsten.router.resource.C3P0PooledConnectionProvider

Page 16: Tungsten University: Introduction to Continuent Tungsten 2.0

©Continuent 2013

Management Support - Archive Slaves

16

• Archive slave is for reads but not failover

• Setting an archive slave

datasource pdrepl1 set archive

• Clearing an archive slave

datasource pdrepl1 clear archive

• Compare with standby slave (failover but no reads)

Page 17: Tungsten University: Introduction to Continuent Tungsten 2.0

©Continuent 2013

Cookbook and Testing

17

• Cookbook allows simple setup of common cluster and replication topologies

• New command to test an installed cluster!

cd $CONTINUENT_ROOT/tungsten/./cookbook/tungsten-cookbook \ --test-installed

Page 18: Tungsten University: Introduction to Continuent Tungsten 2.0

©Continuent 2013

Documentation Build-Out

18

• Adding new quick-start guides

• Adding new operations guides

• Adding reference guides for all tools and utilities

• Check out docs in Tungsten 2.0 section

• Suggestions for improvement welcomed!

Page 19: Tungsten University: Introduction to Continuent Tungsten 2.0

©Continuent 2013

MULTI-MASTER CLUSTERING AND TEST

19

Page 20: Tungsten University: Introduction to Continuent Tungsten 2.0

©Continuent 2013

What Is the Release Schedule?

20

Page 21: Tungsten University: Introduction to Continuent Tungsten 2.0

©Continuent 2013

Continuent Tungsten Release Schedule

21

Release When What

2.0.0 Feb 2013 Controlled release to selected customers

2.0.1 June 2013* GA release to everybody

2.0.x Q3 2013 Follow-on roadmap improvements

* We will release when 2.0.1 is fully baked

Page 22: Tungsten University: Introduction to Continuent Tungsten 2.0

©Continuent 2013

How Do I Upgrade from Tungsten Enterprise 1.5.x?

22

Page 23: Tungsten University: Introduction to Continuent Tungsten 2.0

©Continuent 2013

TPM Upgrade Procedure

23

shell> export CONTINUENT_PROFILES=/opt/continuent/software/confshell> cd /opt/continuent/softwareshell> tar -xf continuent-tungsten-2.0.1-149shell> cd continuent-tungsten-2.0.1-149shell> ./tools/tpm fetch --hosts=<hostname>,autodetect --user=tungsten --directory=autodetectshell> ./tools/tpm reverseshell> ./tools/tpm update

Page 24: Tungsten University: Introduction to Continuent Tungsten 2.0

©Continuent 2013

What’s Coming Next in Tungsten?

24

Page 25: Tungsten University: Introduction to Continuent Tungsten 2.0

©Continuent 2013

Roadmap for Continuent Tungsten 2.0.x

25

• Backup/restore usability (master backups)

• Connectivity performance (esp. login)

• Security for untrusted environments

• More multi-master support

• Cluster management API

• Graphical User Interface

Page 26: Tungsten University: Introduction to Continuent Tungsten 2.0

©Continuent 2013

Preview of Administrative API

26

Page 27: Tungsten University: Introduction to Continuent Tungsten 2.0

©Continuent 2013

Preview of GUI Administration

27

Page 28: Tungsten University: Introduction to Continuent Tungsten 2.0

©Continuent 2013

Wrapping Up

28

Page 29: Tungsten University: Introduction to Continuent Tungsten 2.0

©Continuent 2013

Tungsten University Sessions

29

• Introduction to Continuent Tungsten 2.0 (May 21)

• Load a Vertica Data Warehouse with MySQL Data (May 30 and June 4)

Send any feedback to: [email protected]

Page 30: Tungsten University: Introduction to Continuent Tungsten 2.0

©Continuent 2012.

Continuent Web Page:http://www.continuent.com

Tungsten Replicator 2.0:http://code.google.com/p/tungsten-replicator

Our Blogs:http://scale-out-blog.blogspot.comhttp://datacharmer.org/bloghttp://!yingclusters.blogspot.comhttp://www.continuent.com/news/blogs

560 S. Winchester Blvd., Suite 500 San Jose, CA 95128 Tel +1 (866) 998-3642 Fax +1 (408) 668-1009e-mail: [email protected]