44
cocktail d’expérience informatiques Genève 3 & 4 octobre 2011 Seconde édition Track Auteur Session Java Galder ZAMARREÑO Data Grids vs Databases soft -shake.ch

soft-shake.ch - Data grids and Data Grids

Embed Size (px)

DESCRIPTION

Galder ZAMARREÑO

Citation preview

Page 1: soft-shake.ch - Data grids and Data Grids

cocktail d’expérience informatiquesGenève 3 & 4 octobre 2011

Seconde édition

Track

Auteur

Session

Java

Galder ZAMARREÑO

Data Grids vsDatabases

soft-shake.ch

Page 2: soft-shake.ch - Data grids and Data Grids
Page 3: soft-shake.ch - Data grids and Data Grids

Tuesday, October 11, 11

Page 4: soft-shake.ch - Data grids and Data Grids

Tuesday, October 11, 11

Page 5: soft-shake.ch - Data grids and Data Grids

Data Grids vs Databases

Galder ZamarreñoSenior Software Engineer

Red Hat, Inc

3rd October 2011, Soft Shake

Tuesday, October 11, 11

Page 6: soft-shake.ch - Data grids and Data Grids

Galder Zamarreño

• R&D Engineer, Red Hat Inc.• Infinispan developer• 5+ years exp. with distributed data systems• Twitter: @galderz• Blog: zamarreno.com

Tuesday, October 11, 11

Page 7: soft-shake.ch - Data grids and Data Grids

Agenda

• Why do we need Data Grids?• What are exactly In-memory Data Grids?• Data Grids + Databases• Data Grids without a Database• Can Data Grids replace Databases?

Tuesday, October 11, 11

Page 8: soft-shake.ch - Data grids and Data Grids

Traditionally...

Store everything in a DB!

Tuesday, October 11, 11

Page 9: soft-shake.ch - Data grids and Data Grids

Modern requirements

DBs not particularly good at horizontal scaling...

Tuesday, October 11, 11

Page 10: soft-shake.ch - Data grids and Data Grids

One size doesn’t fill all!

DBs are not bad, but they’re not the solution to every problem either

Tuesday, October 11, 11

Page 11: soft-shake.ch - Data grids and Data Grids

Data Grids

Tuesday, October 11, 11

Page 12: soft-shake.ch - Data grids and Data Grids

Data Grids are not new

Mainstream traction only recent: vertical scaling needs, cheaper memory... and cloud!

Tuesday, October 11, 11

Page 13: soft-shake.ch - Data grids and Data Grids

Who’s offering Data Grids?

Tuesday, October 11, 11

Page 14: soft-shake.ch - Data grids and Data Grids

The Players

• Open Source: • Infinispan, EhCache, Hazelcast...

• Commercial: • Oracle Coherence, Gigaspaces, Gemfire,

IBM eXtreme Scale

Tuesday, October 11, 11

Page 15: soft-shake.ch - Data grids and Data Grids

But, what are In-memory DGs?

Tuesday, October 11, 11

Page 16: soft-shake.ch - Data grids and Data Grids

Definition

In-memory data structures that offer extremely fast access to data

Tuesday, October 11, 11

Page 17: soft-shake.ch - Data grids and Data Grids

Maps are popular!

Normally come with a Map-like API, but often come with alternatives

Tuesday, October 11, 11

Page 18: soft-shake.ch - Data grids and Data Grids

Data distribution

Store data in a subset of the grid to provide failover while being able to scale up!

Tuesday, October 11, 11

Page 19: soft-shake.ch - Data grids and Data Grids

With failure in mind

Suitable for commodity hardware because they can handle failure

Tuesday, October 11, 11

Page 20: soft-shake.ch - Data grids and Data Grids

Elastic

Remain available during topology changes

Tuesday, October 11, 11

Page 21: soft-shake.ch - Data grids and Data Grids

Durability

More durability achieved flushing to a persistent store

Tuesday, October 11, 11

Page 22: soft-shake.ch - Data grids and Data Grids

Access patterns

Embedded (client and DG in same VM) or Remote (just like DBs)

Tuesday, October 11, 11

Page 23: soft-shake.ch - Data grids and Data Grids

ACID or BASE

Transactions or Eventual Consistency?

Tuesday, October 11, 11

Page 24: soft-shake.ch - Data grids and Data Grids

DGs + DBs?

Tuesday, October 11, 11

Page 25: soft-shake.ch - Data grids and Data Grids

Caching!

Use Data Grids as caches to enhance Database access performance!

Tuesday, October 11, 11

Page 26: soft-shake.ch - Data grids and Data Grids

Can a Data Grid replace a DB?

Tuesday, October 11, 11

Page 27: soft-shake.ch - Data grids and Data Grids

Reiterating benefits

Speed, scalability, cloud-friendliness...etc

Tuesday, October 11, 11

Page 28: soft-shake.ch - Data grids and Data Grids

What are the Data Grid challenges?

Tuesday, October 11, 11

Page 29: soft-shake.ch - Data grids and Data Grids

Access patterns

Migrating from SQL to Map or alternative APIs not easy

Tuesday, October 11, 11

Page 30: soft-shake.ch - Data grids and Data Grids

Skill set

Different skill set: OO programmer vs SQL

Tuesday, October 11, 11

Page 31: soft-shake.ch - Data grids and Data Grids

Application data layer

Data layer to take data collocation into account and do more validation (less strict schema)

Tuesday, October 11, 11

Page 32: soft-shake.ch - Data grids and Data Grids

E.g. with a DB...

Tuesday, October 11, 11

Page 33: soft-shake.ch - Data grids and Data Grids

Same with Infinispan

Tuesday, October 11, 11

Page 34: soft-shake.ch - Data grids and Data Grids

Map/Reduce in detail

Tuesday, October 11, 11

Page 35: soft-shake.ch - Data grids and Data Grids

Technology to bridge gap?

Tuesday, October 11, 11

Page 36: soft-shake.ch - Data grids and Data Grids

What about JPA?

Hibernate OGM (Object/Grid Mapper) uses JPA to store in DGs as opposed to DBs

Tuesday, October 11, 11

Page 37: soft-shake.ch - Data grids and Data Grids

Most frequent use cases for DGs?

Tuesday, October 11, 11

Page 38: soft-shake.ch - Data grids and Data Grids

Use cases

• Analytic systems, i.e. financial/trading apps• XTP• Event driven apps, i.e. CEP• Clustering toolkit

Tuesday, October 11, 11

Page 39: soft-shake.ch - Data grids and Data Grids

Do I see DGs as DB replacements?

Tuesday, October 11, 11

Page 40: soft-shake.ch - Data grids and Data Grids

DBs are here to stay!

No. DBs are proven, mature, well understood plus, there are millions of systems out there!

Tuesday, October 11, 11

Page 41: soft-shake.ch - Data grids and Data Grids

One size doesn’t fill all!

DBs are not a universal data storage system any more

Tuesday, October 11, 11

Page 42: soft-shake.ch - Data grids and Data Grids

Consider Data Grids

For their speed, capabilities as data store, and cloud friendliness

Tuesday, October 11, 11

Page 43: soft-shake.ch - Data grids and Data Grids

Still some way to go

More deployments and standardization (JSR-107, JSR-347)

Tuesday, October 11, 11

Page 44: soft-shake.ch - Data grids and Data Grids

Questions

infinispan.org - @infinispan

speakerrate.com/galder

Tuesday, October 11, 11