Distributed Computing with Hazelcast - Brazil Tour

www.hazelcast.com@noctarius2k

/** * Distributed Computing * with Hazelcast */


Interactive!



Disclaimer# A fatal error has been detected by the Java Runtime Environment: # # SIGSEGV (0xb) at pc=0x00002aaaaaf6d9c3, pid=2185, tid=1086892352 # # JRE version: 6.0_21-b06 # Java VM: Java HotSpot(TM) 64-Bit Server VM (17.0-b16 mixed mode linux-amd64 ) # # An error report file with more information is saved as: # /path/to/hs_err_pid2185.log # # This presentation may include certain forward-looking statements and projections provided by the Company. Any such # statements and projections reflect various estimates and assumptions by the Company concerning anticipated # results. These statements involve risks, uncertainties and assumptions and are based on the current estimates and # assumptions of the management of the Company as of the date of this presentation and are subject to uncertainty and # changes. No representations or warranties are made by the Company as to the accuracy of any such statements or # projections. Whether or not any such forward-looking statements or projections are in fact achieved will depend upon # future events some of which are not within the control of the Company. Accordingly, actual results may vary from the # projected results and such variations may be material. Statements contained herein describing documents and # agreements are summaries only and such summaries are qualified in their entirety by reference to such documents # and agreements.


Who’s that dude?

• Chris(toph) Engelbert • Twitter fanatic: @noctarius2k • Weird 9+ Java-Years • Performance, GC, Fairytales • Apache Committer • Gaming, Travel Management, …


• Chris(toph) Engelbert • Twitter fanatic: @noctarius2k • Weird 9+ Java-Years • Performance, GC, Fairytales • Apache Committer • Gaming, Travel Management, …

The www.hazelcast.ninja

Who’s that dude?


DistributedComputing…


ToasterServer 1

TV-Server 1Power

CoffeeMakerServer 1

Kettle Server 1

StoveServer 1

Power GridServer 1, 2

DistributedComputing…


… but what meansDistributed Computing?OOO

( )



“In-Memory Computing”


‘In-Memory Computing’

L1 Cache Reference L2 Cache Reference Main Memory Reference 1K send over 1 Gbps Network 4K read from SSD Seek on Spinning Disk Send Packet CA->AMS->CA

0.5 ns 7 ns

100 ns 10,000 ns

150,000 ns 10,000,000 ns

150,000,000 ns

0.01 ms 0.15 ms

10 ms 150 ms


Evolution of:- Memory-Price- Memory-Size


Data Distribution(Partitioning / Sharding)


Data Distribution

String code = “JCP-TOUR-2015”

Map<String, TShirtDelivery> shirts = Maps.create()

shirts.put(code, new TShirtDelivery(“Your Address”))


Data Distribution

String code = “JCP-TOUR-2015”

int hashcode = hash(code)

partitionId = hashcode % partitionCount

cluster.send(partitionId, code, delivery)


Data Distribution

int hashcode = hash(code)

partitionId = hashcode % partitionCount

cluster.send(partitionId, code, delivery)

Must be a consistent hash function

Partition count must be constant

Can be calculated, smart routing possible


Demo Time


That was easy!But don’t tell anyone!


Parallelize Processing



int[] values = [0..100000].map(randomInt())

int[][] blocks = split(values, 10000)

int[] subresult = blocks.foreach(_.sum())

int result = subresult.sum()



int[][] blocks = split(values, 10000)

int[] subresult = blocks.foreach(_.sum())

int result = subresult.sum()

No mutation, only new values

Independently computable

Might split further down (fork-join, work-stealing)


Demo Time2


I told you:Easy Peasy!


Distributed Caching!


Collect, store …


…access fast when necessary!…access fast when necessary!


Caches


Evolution Of Caching

Custom


http://gadizmo.com/roll-your-own-with-the-ham-dogger.php

Roll Your Own!Evolution Of Caching

http://gadizmo.com/roll-your-own-with-the-ham-dogger.php


Caches Are Simple!public class Cache<K, V> { private final Map<K, V> cache = new CHM<>();

public void put(K key, V value) { cache.put(key, value); }

public V get(K key) { return cache.get(key); } }


Even With TTL!


And Auto-Cleanup


What Is Wrong?


What Is Wrong?

O(n)5 mio

Entries


It was a trap!



Custom


Proprietary


Custom


Proprietary


Open Source

Custom


Proprietary


Open Source

Commercial

Custom


Proprietary


Open Source

Commercial

Stupidly ExpensiveCustom


http://technoodling.net/mili-power-crystal-battery-pack-review/

Terracotta /EHcache

OracleCoherence

Infinispan

JavaCollections

API

Hazelcast

JCache

Custom

http://technoodling.net/mili-power-crystal-battery-pack-review/


Success!Feels like


Quick Shameless Plug :)Hazelcast!


Hazelcast in 5 Facts• Java Collections API• Java Concurrency API

• Transparent Data Distribution• Drop-In Replacement

• Disruptively Simple(c) Rod Johnson (@springrod)


Time to be Happy!


Time to be Happy!

Apache License 2


Time to be Happy!

www.hazelcast.org


You can do it!


Thank You!Any Questions?

@noctarius2k http://www.sourceprojects.org

http://github.com/noctarius

@hazelcast http://www.hazelcast.com http://www.hazelcast.org

http://github.com/hazelcast







Free Hazelcast T-Shirthttp://bit.ly/tshirtcast

http://bit.ly/tshirtcast







Free Hazelcast T-Shirthttp://bit.ly/tshirtcast

We’re hiring!

www.hazelcast.com/careers

http://bit.ly/tshirtcast

http://www.hazelcast.com/careers






































































Technology

Distributed Computing with Hazelcast - Brazil Tour