Paris NoSQL User Group - In Memory Data Grids in Action (without transactions chapter)

In Memory Data Grid in Actionwith Oracle Coherencefor Paris NoSQL User Group

Cyrille Le Clerc

Transactions chapter will be presented during another session

Wednesday, May 25, 2011

Speaker

Cyrille Le Clerc

@cyrilleleclerc

blog.xebia.fr

Open Source (Apache CXF, ...)

In Memory Data Grid

Large Scale

“you build it, you run it”

Once upon a time...

- Released Coherence in 2001- Started as a distributed cache

- Released Gigaspaces XAP in 2001- Started as a data grid

On the Financial side

• Very low latency

• Rich queries & transactions

• Scalability

• Data consistency

Needs within financial market :

Let’s define an In Memory Data Grid ...

Let’s define an In Memory Data Grid

eXtreme Scale

This is an In Memory Data Grid

This is Network Attached Memory

Similarities with NoSQL document orientedPartitioned, distributed Hastable, schema-less, value is not opaque, scale-out scalability

Very fastIn memory (persistence coming), business logic inside the data

Consistent and AvailableTransactional, redundant

Written in Java, data are POJOs Not necessary

Clients in Java, Microsoft, etc8

Use cases for this presentation

Train Booking System

trains, stations, seats, booking and passengers

eCommerce Web Site

warehouse stocks

canon-eos: 1ipod : 1headphone : 1iphone: 1...

ipad : 1 iphone: 1

barbie : 1iphone: 1cabbage-doll: 1

{ "name": "Barbie Computer", "stock": 637, "weigth" : 200 }

warehouse & customers shopping carts

In Memory Data Grids Key Principles

Store Everything in a Mainframe !

3 To of RAM80 x 5.2 GHtz coresMuch more than $1,000,000

IBM z11http://ibm.com/

Spread on Inexpensive Servers

Mainframe Cheap Servers !http://1userverrack.net/

http://ibm.com/

Partition Data

MainFrame

Smallservers

Partition gamma

Partition beta

Partition alpha

Partition for scalability

Duplicate Data

sync synchronization

Duplicate data for high availability

Partition alpha

Master

Standby Backup

Data Access Patterns

This is not traditional Java EE coding style !

Can apply very complex business logic inside the data

Stored Procedures Style

Change management challenge !

Pattern : Targeted Operation

Pattern: Targeted Operation

Partition gamma

Search Trains

Partition beta

Search Trains

Partition alpha

Search Trains

{ "train-id": "tgv-3071-20110512", "time" : 2011/05/12 12:15, "departure" : "Paris", "arrival" : "Marseille", "seats" : 3, }

Book Train Tickets

“train-id” is indexed

Pattern : Map Reduce Style Operation

Pattern: Map Reduce

Partition gamma

Search Trains

Partition beta

Search Trains

Partition alpha

Search Trains

{ "departure": "Paris", "arrival": "Marseille", "time" : 2011/05/12 12:00, "seats" : 3, }

Distributed “Search Train Ticket”Wednesday, May 25, 2011

Pattern: Map Reduce

Partition gamma

Search Trains

Partition beta

Search Trains

Partition alpha

Search Trains

{ "Paris -> Marseille : 12:15", "Paris -> Marseille : 13:15"}

Distributed “Search Train Ticket”

{ #NONE# }

{ "Paris -> Lyon -> Marseille : 12:40"}

Pattern: Map Reduce

Partition gamma

Search Trains

Partition beta

Search Trains

Partition alpha

Search Trains

Distributed “Search Train Ticket”

{ "Paris -> Marseille : 12:15", "Paris -> Lyon -> Marseille : 12:40", "Paris -> Marseille : 13:15"}

This is not traditional Java EE coding style

Don’t forget “Map Reduce” = “Distributed Table Scan”

Use Indexes

Change management

CAP Theorem & In Memory Data Grids

CAP Theorem and In Memory Data Grid

Consistency

Availability

PartitionTolerance

Only 2 of these 3 properties can be

achieved at any given moment in time

Brewer’s Conjecture

http://lpd.epfl.ch/sgilbert/pubs/BrewersConjecture-SigAct.pdf

CAP Theorem and In Memory Data Grid

Consistency

Availability

PartitionTolerance

Only 2 of these 3 properties can be

achieved at any given moment in time

Brewer’s Conjecture

http://lpd.epfl.ch/sgilbert/pubs/BrewersConjecture-SigAct.pdf

Data Grids

Cross Data Center Data Consistency

TokyoNew York

London

World wide replicationfor financial market

West Coast

East Coast

Warehouse stocks

propagation delay !

West Coast

East Coast

set stock to 146

West Coast

East Coast

set stock to 146

set weight 175reconciliation API needed !

West Coast

East Coast

set stock to 146

set weight 175Network partitioning

Data Modeling

Dominant Question Driven Design

Constrained Tree Schema

Denormalized

Opposite to Relational which is Domain Driven Design

Because RPC matters

Due to dominant questions and CTS

Data Modeling

TrainStopdate

TrainStationcodename

Traincodetype

Seatnumberprice

Bookingreduction

Passengername

Typical relational data model

Data Modeling

Find the root entity and denormalize

TrainStopdate

Seatnumberprice

Bookingreduction

Passengername

Reference data

Duplicated in each grid node

Root entity

Partitioning ready entities tree

Traincodetype

Data Modeling

Remove unused data

TrainStopdate

Seatnumberprice

Bookingreduction

Passengername

booked

Traincodetype

Partitioned

Replicated

Data Modeling

TrainStopdate

Seatnumberpricebooked

Traincodetype

Data Grid Ready data structure

Partitioned

Replicated

Data Modeling is Hard !

Two root entities for the same MoneyTransfer !

from to

CashWitdrawaldateamount

MoneyTransferiddateamount

Accountnumber

MoneyTransferIniddateamount

MoneyTransferOutiddateamount

Accountnumber

Split MoneyTransfer

Accountnumber

Split MoneyTransfer

Accountnumber

Data Grid Ready data structure

Grid Internals

Data Serialization

Used for data transfer and byte oriented storage

Hot topic like Apache Thrift, Apache Avro, Google Protocol Buffer

Must support evolvable data structure

Data Storage

Store Java Beans in the grid

Store byte arrays in the grid

No need to unmarshall for inprocess operations

Beware of garbage collector !

Pay unmarshalling at each read and write

Slightly more garbage collector friendlyLow-level / byte-oriented APIs to read data

Communication Protocols

UDP Multi Cast (Coherence, Gigaspaces)

TCP/IP (Websphere eXtreme Scale)

48Wednesday, May 25, 2011

Topology

Partitions made of shards : 1 primary + 0..* backups)

Dynamic shards location (changes at runtime and at restart)

Can use dedicated “directory servers” or embed it in the “data nodes”

JVM and Memory

Many editors recommend tiny 1.4 Go JVM !

More than ten JVM per server

Garbage collector hell

Management hell

More and more IMDG support large heaps

Raw Java Mapping with Oracle Coherence

hand-coded serializationJUnit is your friend !

public class Train extends AbstractEvolvable implements PortableObject { enum Type { HIGH_SPEED, NORMAL }

/** Key of the Cache */ String code;

/** Indexed */ String name;

Type type;

List<Seat> seats = new ArrayList<Seat>();

int version;

List<TrainStop> trainStops = new ArrayList<TrainStop>();

@Override public int getImplVersion() { return 1; }

@Override public void readExternal(PofReader pofReader) throws IOException { this.code = pofReader.readString(0); this.name = pofReader.readString(1); this.type = (Type) pofReader.readObject(2); pofReader.readCollection(3, this.seats); pofReader.readCollection(4, this.trainStops); this.version = pofReader.readInt(5); }

@Override public void writeExternal(PofWriter pofWriter) throws IOException { pofWriter.writeString(0, this.code); pofWriter.writeString(1, this.name); pofWriter.writeObject(2, this.type); pofWriter.writeCollection(3, this.seats, Seat.class); pofWriter.writeCollection(4, this.trainStops, TrainStop.class); pofWriter.writeInt(5, this.version); }}

TrainStopdate

Traincodetype

JPA Style Mapping with Websphere eXtreme Scale

sub entities can have cross relations

@Entity(schemaRoot=true)public class Train { @Id String code; @Index @Basic String name; @OneToMany(cascade=CascadeType.ALL) List<Seat> seats = new ArrayList<Seat>(); @Version int version;

TrainStopdate

Traincodetype

Map API with Oracle Coherence

NamedCache trainCache = CacheFactory.getCache("train-cache");

/** Save */ void persist(Train train) { trainCache.put(train.getCode(), train); } /** Find by key */ Train findByCode(String code) { return (Train) trainCache.get(code); }

/** Find by Query Language */ Train findByTrainName(String name) { Filter filter = QueryHelper.createFilter("name = :name" , Collections.singletonMap("name", name)); Set<Map.Entry<String, Train>> trainEntrySet = trainCache.entrySet(filter); if (trainEntrySet.isEmpty()) { return null; } else { return trainEntrySet.iterator().next().getValue(); } }

Map API

JPA Style with Websphere eXtreme Scale

/** Save */void persist(Train train) { entityManager.persist(train);}

/** Find by key */Train findByCode(String code) { return (Train) entityManager.find(Train.class, code);}

/** Query Language */Train findByTrainName(String name) { Query q = entityManager.createQuery("select t from Train t where t.name=:name"); q.setParameter("name", name);

return (Train) q.getSingleResult();}

JPA Style Entity Manager

Creating Indexes

Map reduce (without index) = Distributed Table Scan !

Indexes with Oracle Coherence

class Train { String name;

Collection<String> getTrainStationsCodes() { return Collections2.transform(trainStops, ...); }

{ NamedCache trainCache = CacheFactory.getCache("train-cache");

trainCache.addIndex(new ReflectionExtractor("getName"), false, null); trainCache.addIndex(new ReflectionExtractor("getTrainStationsCodes"), false, null);}

Indexes with Websphere eXtreme Scale

@Entity(schemaRoot=true)class Train { @Index @Basic String name;

@Index Collection<String> getTrainStationsCodes() { return Collections2.transform(trainStops, ...); }

Query query = em.createQuery("select t from Train t where t.name=:name");query.getPlan();

eXtreme Scale

for q2 in Train ObjectMap using INDEX on name = ( ?name) filter ( q2.c[0] = ?name ) returning new Tuple( q2 )

This is an execution plan

More APIs

Another Java EE versus Spring battle ? JSR 347 Data Grids vs. Spring Data

Unified API ontop of NoSQL stores ?

Serialization / Object to Tuple Mapping API ?

Data Grid <-> Relational Database Interactions

Data Grid <-> Relational Database

Data Grids are “In Memory” -> we need to persist data on disk !

update / insert / delete

“select directly modified in DB”

backend DB

Highly available write behind queues+ SQL batched statements

Data Grid -> Relational Database

TrainStopdate

Traincodetype

Constrained Tree Schema <-> Relational Impedance Mismatch

Data Grid -> Relational Database

DB writes MUST succeed !

Align the database on the Data Grid model !

Denormalize the databaseRemove the foreign keys, use same PKs in DB and data gridSupport unordered SQL statements

Prefer raw SQL rather than reused business logic

backend DB

Data Grid Originated Scheduled Refresh(Oracle System Change Number, etc)

select * from train where last_modif > ?

Relational Database -> Data Grid

backend DB

Database Originated PushJMS = durable subscription(Oracle Database Change Notification, etc)

Relational Database -> Data Grid

In Memory -> prepare for reloading after maintenance operations !

Prepare consistency checkers

Need for “graceful shutdown with disk persistence”

Transactions

We didn’t have the time to talk about transaction.

Another session is planned at Paris No SQL User Group for this.

Let’s go live !

Data Grids and Operations

Standard packaging?

Limited Management

Limited debugging tools

JVM pandemia

Do It Yourself (layout, scripts, etc)

Do It Yourself (stop/start, detecting data loss, etc)

Dozens of JVM to manage !

Do It Yourself (debugging consoles, troubleshooting agents)

Data Grids and Operations

Dev / Ops collaboration is required

Experts only !

The right tool for the right job

Incredibly fast ! Even with transactions !

Scalable

Good at data replication (when it implements it)

Very geeky on both dev and ops side

“Quite” expensive

Not an enterprise grade data store

Reconciliation api, etc

Requires very skilled people + change management

If you solve the data loading issue

Questions / Answers

Paris NoSQL User Group - In Memory Data Grids in Action (without transactions chapter)

Technology

NoSQL and Big Data Analytics at NOSQL NOW! 2013

Oracle NoSQL Database – A Distributed Key-Value Store · HPTS, October 24, 2011 Agenda • Oracle and NoSQL • Oracle NoSQL Database Architecture • Oracle NoSQL Database Technical

Caching, NOSQL & Grids - GOTO Conferencegotocon.com/dl/qcon-london-2012/slides/JohnDavies... · databases - RDBs •If the data you’re storing is relational then SQL is a pretty

NoSQL or Not Only SQL - Montana Technological University · NoSQL or Not Only SQL . Reasons to go to a NoSQL database: ... SQL NoSQL ACID: • Atomic • Consistent • Isolation

Q y // NoSQL’Road’Show,’Zurich’nosqlroadshow.com/dl/NoSQL-Road-Show/slides/nosql... · NoSQL,’NewSQL’and’Beyond ... •’OrientDB ’ •’NuvolaBase ... •’ScaleBase

NoSQL Technologies from an STM Publishing Perspective (NoSQL Now 2011)

SQL vs NoSQL: The NoSQL way

IEEE TRANSACTIONS ON SMART GRID, VOL. 2, NO. 2, JUNE … · IEEE TRANSACTIONS ON SMART GRID, VOL. 2, NO. 2, JUNE 2011 399 Digital Grid: Communicative Electrical Grids of the Future

Transactions Returning to Big Data (NoSQL) OR · The rise of NoSQL & Big Data •Data explosion has caused re-evaluation of RDBMS •Initially RDBMS augmented with cache •But ultimately

Data Management in Large-Scale Distributed Systems - NoSQL ... · Introduction Why NoSQL? Transactions, ACID properties and CAP theorem Data models NoSQL databases design and implementation

Adrian Colyer - Keynote: NoSQL matters - NoSQL matters Dublin 2015

Consistent NoSQL data storage with ModeShape (NoSQL Matters 2013)

TECHNOLOGY NEWS Will NoSQL Databases Live Up to Their … · NOSQL PROS AND CONS NoSQL databases have numerous advantages and disadvantages. Advantages NoSQL databases generally pro-cess

PostSQL Using PostgreSQL as a better NoSQL - NoSQL Matters

«NoSQL benchmarking v2.0. Исследование производительности современных NoSQL-решений»

Paris NoSQL User Group - In Memory Data Grids in Action (without transactions chapter)

NoSQL. ACID Semantics Atomicity: All or nothing. Consistency: Consistent state of data and transactions. Isolation: Transactions are isolated from each

NoSQL - WordPress.com · จาก SQL สู่NoSQL • ต้องการประมวลผลข้อมูลจำนวนมากbig data) ( และรองรับผู้ใช้

NoSQL Databases for Enterprises - NoSQL Now Conference 2013

IEEE TRANSACTIONS ON SMART GRIDS, VOL. X, NO. X, XXXXX ... · IEEE TRANSACTIONS ON SMART GRIDS, VOL. X, NO. X, XXXXX XXXXX 1 Cyber-Physical Security: A Game Theory Model of Humans