Tungsten University: Unleashing the Power of Tungsten Connectors

Preview:

DESCRIPTION

Continuent Tungsten clusters use the Tungsten Connector to ensure your applications transparently connect to the master. This enables failover and seamless switching of masters for maintenance. However, you can do far more. Tungsten Connector allows you to make better use of hardware by load balancing SQL traffic to slaves. There is also a wealth of configuration settings to help Tungsten Connector manage connections efficiently. Learn standard deployment patterns, how to create users, control load balancing, how to distribute reads across local and remote sites, and other configuration tricks. We also provide advice about how to adapt your applications to use load balancing more efficiently. Course Topics - Deployment options and techniques - Configuring users with regard to data services - Application configuration samples - Read/write splitting options and demos - Load balancing strategies - Optimization of connection and request speed - Understanding and fixing connectivity issues

Citation preview

©Continuent 2013

Tungsten University:Unleashing the Power of

Tungsten ConnectorsGilles Rayrat, Director of Operations/France

©Continuent 2013

What is Tungsten Connector?

2

Connector

App

NativeprotocolCOM_QUERY

COM_INIT_DBCOM_DROP_DB

©Continuent 2013

A Key Tungsten Component

3

• Provide access to the master

• Handle database failures

• Can access slaves for reads

©Continuent 2013

Deployments

4

©Continuent 2013

Deployment 1: app servers

5

App Server+ Connector

App Server+ Connector

App Server+ Connector

App Server+ Connector

App Server+ Connector

©Continuent 2013

Deployment 2: database servers

6

Software Load Balancer

...

App Server App Server App Server App Server App Server App Server

©Continuent 2013

Deployment 3: dedicated hosts

7

...

Hardware Load Balancer

App Server App Server App Server App Server App Server App Server

...

©Continuent 2013

Deployment best practice

8

Install with port 3306!

tools/tpm con!gure <data service name>

<options>

--connector-listen-port=3306

©Continuent 2013

Closer view

• Performance

• Failure handling

9

©Continuent 2013

Packet Streaming

• Fast operations, Low overhead

• Full transparency

10

Connector

App

Protocol packetCOM_QUERY

select * from t

Protocol packetOK

ResultSet Rows:1

©Continuent 2013

Closer view

• Performance

• Failure handling

11

©Continuent 2013 12

Connector-Manager link

Master Slave

App Logic

Tungsten Connector

Replicator Replicator Replicator

App Logic

Tungsten Connector

Manager Manager Manager

Monitoring and control

Monitoring and control

Slave

©Continuent 2013

Node Failure

“Pause” incoming connection requests+

+

Transparent retry (autocommit tx only)

13

Planned switch:wait for disconnect

Unplanned Failover:break connections

©Continuent 2013

Demo?

14

asia

worldeurope

u1

u2

u3

u4

©Continuent 2013 15

Installation

• tools/tpm con!gure <data service name> <options>--dataservice-connectors=node1,node2,...

• --connector-listen-port --connector-user --connector-password --connector-smartscale=true

• tools/tpm help con!gure

©Continuent 2013 16

Specifying Users

• --connector-user=user1

• additional users in

tungsten-connector/conf/user.map

©Continuent 2013 17

user map

<user> <password> <data service> [<preferred site>]

tungsten_testing private world europemyuser p4$$ asiaalice - othersvc

No need to restart

©Continuent 2013 18

Connection Protocol

App

version #, ..., salt

Database, user,Salted password

©Continuent 2013 19

A Man in the Middle

App

Connector

?

©Continuent 2013 20

• Workaround:

@script /opt/continuent/share/gen_user_map.sh

• output identical to regular user.map

• Can be any script: SQL requests, encrypting, etc.

• Accepts parameters

Clear text passwords

©Continuent 2013

Application con!guration

21

©Continuent 2013

Application con!guration - PHP

22

<?php

$host = “u1”; 

$port = 9999;

$username = “tungsten_testing”;

$password = “secret_private”;

$dbname = “myapp";

$connection = mysqli_connect($host, $username, $password, $dbname, $port);

......... More application logic here .........

?>

©Continuent 2013

Application con!guration - Java

23

• Hibernate

hibernate.connection.url=jdbc:mysql://u1:9999/mydb

• JBoss

<connection-url>jdbc:mysql://u1:9999/database</connection-url>

• JDBC

jdbc:mysql://u1:9999/dbname

©Continuent 2013

Application con!guration - Ruby on Rails

24

prod:

  adapter: mysql2

  encoding: utf8

  database: my_db_name

  username: tungsten_testing

  password: private

  host: u1

  port: 9999

©Continuent 2013

Read/write splitting

• Automatic

• Application Controlled

25

©Continuent 2013

Automatic R/W splitting

• No application changes

• Only outside transaction boundaries

• 2 variants:

• Smart Scale

• @direct connections

26

©Continuent 2013

Automatic R/W splitting - smartScale

• AKA ‘session consistency’ QoS

• Ability to see session’s writes

27

©Continuent 2013

SmartScale

28

App

Connector

MasterDBMS

TransactionHistory Log

SlaveDBMS

Slavereplay

position

write

Session xSession y...

©Continuent 2013 29

SmartScale

App

Connector

MasterDBMS

TransactionHistory Log

SlaveDBMS

Slavereplay

position

read

Session x

©Continuent 2013 30

SmartScale

App

Connector

MasterDBMS

TransactionHistory Log

SlaveDBMS

Slavereplay

position

read

Session x

©Continuent 2013 31

• Enabling:

tpm configure <data service name> <other options> --connector-smartscale=true --connector-smartscale-sessionid= DATABASE | USER | CONNECTION | PROVIDED_IN_DBNAME

SmartScale

©Continuent 2013 32

• Choose you sessionId:

• PROVIDED_IN_DBNAME (++)=> set dbname to ”mydb?sessionId=12345”

• CONNECTION (+)

• USER (-)

• DATABASE (--)

SmartScale

©Continuent 2013

Demo?

33

©Continuent 2013 34

Limitations:

• single connector instance

• select function_that_writes();

• ephemeral SQL objects

SmartScale

©Continuent 2013 35

Trade-o":

• Can read outdated data from other sessions

• Poor e!ciency in write intensive environments

SmartScale

©Continuent 2013

Automatic R/W splitting - directReads

• Blind R/W splitting

• Fast

• Similar limitations:

• ephemeral SQL objects

• select function_that_writes();

• Reads potentially outdated data

36

©Continuent 2013

DirectReads

37

• Enablingtpm configure <ds> <opts> --connector-rwsplitting=true

• Other users in user.map@direct user2@direct user3

• Can be backed by a c3p0 pool

©Continuent 2013

Read/write splitting

• Automatic

• Application Controlled

38

©Continuent 2013 39

Application-driven R/W splitting

• More e#cient...

• ...but potentially dangerous

• 3 possibilities:

• SQL comments

• QoS in database name

• @hostoption

©Continuent 2013

Selective R/W Splitting

• Must enable directReads

tpm update ...--connector-rwsplitting=true \--property=selective.rwsplitting=true

• Embedded SQL comment:

40

!!! mysql -u... -c

/* TUNGSTEN_URL_PROPERTY=qos=RO_RELAXED */select @@hostname;

©Continuent 2013

Demo?

41

©Continuent 2013

QoS at connection time

• No connector con!guration change

• connect to “mydb@qos=RO_RELAXED”

42

©Continuent 2013

Demo?

43

©Continuent 2013

Host-based routing

• /etc/hosts127.0.0.1 master.localhost127.0.0.2 slave.localhost

• tpm configure <data service name> <other options> --connector-rw-addresses=master.localhost --connector-ro-addresses=slave.localhost

44

©Continuent 2013

Demo?

45

©Continuent 2013 46

SOR (multi site)

• Getting a local slave

<user> <password> <data service> [<preferred site>]

tungsten_testing private world europe

©Continuent 2013

Demo?

47

©Continuent 2013 48

Performance

• Pool for DirectReads

tpm configure <ds> <opts> --connector-rwsplitting=true --property=directReadsUsePool=true

• JDBC driver

https://docs.continuent.com/wiki/display/TEDOC/Improving+Connection+Speed

--property=jdbc.driver.options=

cacheServerCon!guration=true\&characterSetResults=UTF8

©Continuent 2013

Getting connector information

• “tungsten connection status” for last request

• “tungsten show processlist” for real client addresses

49

mysql> tungsten connection status;+---------------------------------------------------------------+| message |+---------------------------------------------------------------+| internal(OPEN) DIRECT TO u1@default(master:ONLINE) STATUS(OK) |+---------------------------------------------------------------+1 row in set (0.01 sec)

mysql> tungsten show processlist;+------------+------+----------+-----------+----------+---------+------+-------+------+| DataSource | Id | User | Host | db | Command | Time | State | Info |+------------+------+----------+-----------+----------+---------+------+-------+------+| u1 | 1342 | tungsten | mac:64477 | tungsten | Sleep | 20 | | NULL |+------------+------+----------+-----------+----------+---------+------+-------+------+1 row in set (0.13 sec)

©Continuent 2013

Troubleshooting

• tungsten-connector/log/connector.log

• Lots of connections in TIME_WAIT state

• Too many open !les

• OutOfMemory error

• New connections hanging

50

©Continuent 2013

Tungsten University Sessions

51

• Manage multi-master or fan-in replication topologies - March 7th & 12th

• Zero-downtime maintenance and schema operations - March 21st & 26th

• Geographically distributed MySQL clusters - April 11th & 16th

• Replicate between MySQL and Oracle - May 2nd & 7th

©Continuent 2013

Continuent Web Page:http://www.continuent.com

Tungsten Replicator 2.0:http://code.google.com/p/tungsten-replicator

Our Blogs:http://scale-out-blog.blogspot.comhttp://datacharmer.org/bloghttp://www.continuent.com/news/blogs

560 S. Winchester Blvd., Suite 500 San Jose, CA 95128 Tel +1 (866) 998-3642 Fax +1 (408) 668-1009e-mail: sales@continuent.com

Recommended