Highly available database clusters with JDBC [email protected]

Highly available database clusters with JDBC

[email protected]

http://c-jdbc.objectweb.org/ - [email protected] ObjectWebCon’05 - 2 - 19/01/2005

Scaling the database tier – Alternative 1 (SMP)

Internet

Web frontend

App. server

Well-known database

vendor here

Database

Well-known hardware +database vendors here

Cons Cost Scalability limit


Scaling the database tier – Alternative 2 (shared disks)

Internet

Web frontend

App. server Database Disks

Another well-knowndatabase vendor

Consstill expensive hardwareavailability


Internet

scalabilityhigh availabilitywithout modifying the client applicationdatabase vendor independenton commodity hardware

Database clustering middleware

JDBC

JDBCJDBC


RAIDb concept

Redundant Array of Inexpensive DatabasesRAIDb controller

gives the view of a single database to the clientbalance the load on the database backends

RAIDb levels offers various tradeoff of performance and fault tolerance


RAIDb levels

RAIDb-0partitioningno duplication and no fault toleranceat least 2 nodes

table 2 & 3 table ...

RAIDb controller

table n-1table 1 table n

SQL requests


RAIDb levels

RAIDb-1mirroringperformance bounded by write broadcastat least 2 nodes

Full DB Full DB

RAIDb controller

Full DBFull DB Full DB

SQL requests


RAIDb levels

RAIDb-2partial replicationat least 2 copies of each table for fault toleranceat least 3 nodes

table x table y

RAIDb controller

table x & yFull DB table z

SQL requests


C-JDBC overview

Middleware implementing RAIDb100% Java implementationopen source (LGPL)

Two componentsgeneric JDBC driver (C-JDBC driver)C-JDBC Controller

Read-one, Write all approachprovides eager (strong) consistency

Supports heterogeneous databases


architectural overview

JVM

Applicationserver

C-JDBC JDBC driver

C-JDBCcontroller

JVM

MySQL JDBC driver

MySQLdatabase


Inside the C-JDBC Controller

XML configuration file

C-JDBC Controller

MySQL

C-JDBC driver

Java client program

(Servlet, EJB, ...)

XML engine

MySQL

Virtual database

Database Backend

Connection Manager

Database Backend

Connection Manager

Request Manager

Request Cache

Scheduler

Load balancer

MySQL JDBC driver

MySQL JDBC driver

Configuration &

administrationAdministration console

RMIRMI

Recovery Log

Authentication Manager

MySQL

Database Backend

Connection Manager

MySQL JDBC driver

Virtual database

Database Backend

Connection Manager

Database Backend

Connection Manager

Request Manager

Request Cache

Scheduler

Load balancer

Recovery Log

Authentication Manager

Oracle

Oracle JDBC driver

Oracle JDBC driver

Oracle

SocketsSockets

JMX


Scaling existing databases

add open source databases for more fault tolerance and better performances

rules for on-the-fly query rewriting to handle heterogeneity

MySQL MySQL

C-JDBC Controller RAIDb-2

MySQL JDBC driver

MySQLOracle

Java client program

C-JDBC driver

JVM

C-JDBC driver

C-JDBC driver

JVM

JVM

Oracle JDBC driver

EJB Container JOnAS, WebLogic,

JBoss, WebSphere, ...

Servlet container Tomcat, Jetty, ...


TPC-W benchmark(Amazon.com)

Nearly linear speedups with the shopping mix

0

200

400

600

800

1000

1200

1400

1600

0 1 2 3 4 5 6

Number of nodes

Th

rou

gh

pu

t in

req

ues

ts p

er m

inu

te

Single Database

Full replication

Partial replication


Horizontal & Vertical scalability

DB 6DB 5

DB native JDBC driver

DB 7

C-JDBC driver

DB 1 DB 2


DB 3


DB 4

C-JDBC controller Full replication




C-JDBC driverJVM

Client program

C-JDBC driver

JVM

Client program

C-JDBC driver

JVM

Client program

C-JDBC driver

C-JDBC driver

adapt and evolve according to user needs

advanced caching functionalities

dynamic adding of database backends

fault tolerant transaction log

integrated backup and checkpointing

grid & edge-side servers support


Administration

JMX Graphical Consolecluster administrationbackup/checkpoint mgtcluster monitoringSQL profilingintegrated SQL console (iSQL)

Administration APIintegration in any existing administration infrastructureJMX notifications for all internal eventsJMX MBeans for monitoring


J2EE end-to-end reliability

ObjectWeb exclusivityOn-demand availability

no SPOF, transparent failover

On-demand scalability scalability, flexibility

Internet


C-JDBC today

Web site~200.000 hits/month>28.000 downloadsEU (18 countries) 36%, US 28%, Japan 12%, China

5%, Canada 4%, Australia 4%, India 3%, Brazil 2%, …

Community27 committers both industrial & [email protected]: >200 subscribers, 200-300

msgs/monthtranslation in japanese, italian, chinese, turkish, french ,

german

RPM on JPackage.orgProfessional support by Emic Networks


What’s next?

Tribe (.objectweb.org)replacement for JGroupsuniform total order broadcast optimized for clusters

LeWYS (.objectweb.org)hardware and software monitoringmonitoring repository

Distributed query executionOptimized support for edge-side servers and

interconnected clusters


Demos

Tuesday 18INRIA booth: 12.00 to 1pmObjectWeb booth: 2pm to 2.30pm

Wednesday 19INRIA booth: 10.30am to 11amObjectWeb booth: 1pm to 1.30pm


Q&A_________

Thanks to all users and contributors ...

http://c-jdbc.objectweb.org

Bonus slides


Current limitations

JDBC onlyDistributed joinsUpdatable ResultSetsXA support through XAPool onlytransparent controller failover not supported

when using horizontal scalability with JGroupsnetwork partition/reconciliation not supported

Documents

Highly available database clusters with JDBC [email protected]