JAVA EE CONCURRENCY MISCONCEPTIONS Haim Yadid, Performize-IT Ltd. Daniel Pfeifer, Whistler AB

WH

O AR

E THO

SE G

UY

S? Haim Yadid

‣ Founder of Performize-IT Ltd. ‣ 18+ years in

technology ‣ Primary expertise in

performance optimization

Daniel Pfeifer ‣ Founder of

Whistler AB ‣ 10+ years in IT ‣ Primary expertise in mission-

critical business applications

JAVA EE IN THE MULTIPROCESSOR ERA

OP

TIMIS

M

Moore’s Law: The amount of transistors appx. doubles every second year

JAV

A EE

IN THE FU

TUR

E

That’s why Multi core hardware will become mainstream for JEE servers

IS MU

LTICO

RE N

EW

TO JAV

A EE

? Budget machines

0"

2"

4"

6"

8"

10"

12"

14"

16"

18"

2006" 2008" 2010" 2012"

CPUs/server"CPUs/phone"CPUs/notebook"

A FR

IEN

DLY A

DV

ICE Don’t do it yourself !!

There is an App for that Server

TO

OLS O

F TRA

DE

‣  ”Local" and cluster-wide load balancing ‣ Message driven beans for parallelization / async tasks ‣ Asynchronous annotations ‣ Automatic and bean managed concurrency ‣ Resource management

6 REAL LIFE EXAMPLES

SYNCHRONIZATION IN EJB

SY

NC

HR

ON

IZATIO

N IN EN

TER

PR

ISE B

EA

NS

Use Case You found yourself in need of sharing some data between calls so you ‣ Create a static constant pointing to a List (or perhaps your own

singleton) ‣ Add a synchronization block for your constant ‣ You operate on your instance inside the synchronize-block

SY

NC

HR

ON

IZATIO

N IN EN

TER

PR

ISE B

EA

NS

Returning unused random numbers @Stateless public class SharedStateEJB { private static final List usedNumbers = new ArrayList(); public double fetchUnusedRandomNumber() { double number = 0d; synchronized (usedNumbers) { while (true) { number = Math.random(); if (!usedNumbers.contains(number)) { usedNumbers.add(number); break; } } } return number; } }

SY

NC

HR

ON

IZATIO

N IN EN

TER

PR

ISE B

EA

NS

Problems

Sharing state like this will not work, because … ‣ Static constants are not shared among class loaders ‣ A lock will not is not propagated across machines

SY

NC

HR

ON

IZATIO

N IN EN

TER

PR

ISE B

EA

NS

What about Java EE Singleton?

SY

NC

HR

ON

IZATIO

N IN EN

TER

PR

ISE B

EA

NS

@Singleton @Singleton public class UsedNumbersBean {

private final List usedNumbers = new ArrayList();

@Lock(READ) public boolean numberExists(double num) {

return usedNumbers.contains(num);

}

@Lock(WRITE) public void addNumber(double num) {

usedNumbers.add(num);

}

}

SY

NC

HR

ON

IZATIO

N IN EN

TER

PR

ISE B

EA

NS

Suggestions “Backup” solutions ‣ Use the database and use locking => Potentially slow ‣ Use a cluster-aware cache (i.e. EHCache or JBoss Cache)

‣ Read the chapters on clusters and consistency! ‣ Terracotta “Creative” solutions ‣ Use a single EJB ‣ Use H/A Singleton (available on JBoss) ‣ Wait for the future…

ASYNCHRONICITY THROUGH THREADS

AS

YN

CH

RO

NIC

ITY THR

OU

GH T

HR

EA

DS

Use Case •  You processed an order •  You ask the bank for settlement confirmation •  You can present the customer with a result without waiting for the

bank

AS

YN

CH

RO

NIC

ITY THR

OU

GH T

HR

EA

DS

Creating a thread for async tasks @Stateless

public class OrderHandlingBean {

public String process(final Order order) {

String orderId = ...//save order to database and get id

final Thread t = new Thread() { @Override public void run() { // submit amount to bank } };

t.start(); // start “background” process

return orderId;

}

}

AS

YN

CH

RO

NIC

ITY THR

OU

GH T

HR

EA

DS

Problems

‣ Enough stuck Threads will eventually make your server go OOM. ‣ Difficult to get transaction management right.

‣ Some servers will tell you it can’t check the status, some don’t. ‣ Failed tasks in the thread need difficult manual rollback. ‣ Threads do not scale out

‣ Can’t be load-balanced to other machines. ‣ Server’s automatic resource management doesn’t cover your own

threads. ‣ Debugging and monitoring isn’t the joy it used to be.

AS

YN

CH

RO

NIC

ITY THR

OU

GH T

HR

EA

DS

Thread bugs gone wild

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

1h 4h 7h 10h 13h 16h 19h

Res

pons

e Ti

me

Runtime

Async Sync

java.lang.OutOfMemoryError: unable to create new native thread

AS

YN

CH

RO

NIC

ITY THR

OU

GH T

HR

EA

DS

Best Practice Alternatives

‣ Until Java EE 6 Message Driven Beans are used for asynchronous tasks. ‣ And if you are lucky enough to use a Java EE 6 container, you can

use the new @Asynchronous.

AS

YN

CH

RO

NIC

ITY THR

OU

GH T

HR

EA

DS

Using Queues @Stateless(name = ”QueueingEJB") public class QueueingBean {

@Resource private Queue q; @Resource private ConnectionFactory cf;

public String process(final Order order) {

// ... saving the Order to the database and fetching id

Connection cn = connectionFactory.createConnection();

Session s = cn.createSession(true, AUTO_ACKNOWLEDGE);

MessageProducer producer = s.createProducer(queue);

Message msg = s.createTextMessage(payment);

producer.send(msg);

return orderId;

}

} Don’t forget clean-up! We just want to

conserve screen estate!

AS

YN

CH

RO

NIC

ITY THR

OU

GH T

HR

EA

DS

Using Queues

@MessageDriven(name = “PayEJB”, activationProperties = { /* activation properties for your server */

})

public class PayBean implements MessageListener { public void onMessage(Message m) { // Calling bank...

}

}

AS

YN

CH

RO

NIC

ITY THR

OU

GH T

HR

EA

DS

Using @Asynchronous @Stateless(name = ”PayEJB") public class PayBean{

@Asynchronous public void callBank(final String payment) {

// Hello, ... Bank!

}

}

AS

YN

CH

RO

NIC

ITY THR

OU

GH T

HR

EA

DS

Using @Asynchronous @Stateless(name = ”AsyncCallerEJB") public class AsyncCallerBean {

@EJB private PayBean payBean;

public String process(final Order order) {

// ... saving the Order to the database and fetching id

payBean.callBank(payment);

return orderId; // <- Returns + transaction OK

}

}

That’s All!!!

AS

YN

CH

RO

NIC

ITY THR

OU

GH T

HR

EA

DS

A note for the observant We mentioned transactions… ‣ True, our examples won’t use the same Transaction, but…

‣ @Asynchronous will by spec create a Transaction (REQUIRES_NEW)

‣ MDBs will also create a transaction

PARALLELISM WITH THREAD POOLS

PA

RA

LLELIS

M W

ITH TH

RE

AD

PO

OLS

Use Case Your back-office staff fires off a bunch of invoices and you want the total as fast as possible, so you… ‣ Create a thread-pool ‣ Push all invoices for calculation to the thread-pool ‣ Wait for the answer and return it

PA

RA

LLELIS

M W

ITH TH

RE

AD

PO

OLS

Parallelizing with a ThreadPool (our Callable)

private static class OrderCallable implements Callable<Double> { private final Order order;

public OrderCallable(Order order) {

this.order = order;

}

@Override

public Double call() throws Exception { // return the total based on the order that’s been

// provided.

}

}

PA

RA

LLELIS

M W

ITH TH

RE

AD

PO

OLS

Parallelizing with a ThreadPool (using the Callable in an ExecutorService) public double processOrders(List<Order> orders) { ExecutorService executorService = Executors.newFixedThreadPool(10);

List<Future<Double>> futures = new ArrayList<>(); for (Order order : orders) { OrderCallable callable = new OrderCallable (order); callables.add(callable); futures.add(executorService.invoke (callable)); } double totalValue = 0; try { for (Future<Double> future : futures) { totalValue += future.get(10, SECONDS); } } catch (TimeoutException e) { return -1; } return totalValue; }

PA

RA

LLELIS

M W

ITH TH

RE

AD

PO

OLS

Problems

Essentially we got the same problems as previously, but… ‣ Potential for much larger thread leaks ‣ Still not covered by transaction ‣ Executors, just like Threads, won’t run tasks on other machines. ‣ There is nothing H/A about this.

‣ On crash, part or all of the workload is gone. ‣ Can’t be resumed on server restart, it’s just gone…

PA

RA

LLELIS

M W

ITH TH

RE

AD

PO

OLS

Pure Java EE alternatives

‣ Up to and including Java EE 5 you can use MDBs ‣ Queues can be clustered, so other servers can take some of the

load. ‣ @Asynchronous-annotated method returning Future

Using @Asynchronous (invokee) @Stateless @Asynchronous public class ProcessBean {

public Future<Double> getTotals(Order order) {

double value = // .. Calculate value;

return new AsyncResult<Double>(value);

}

}

PA

RA

LLELIS

M W

ITH TH

RE

AD

PO

OLS

Using @Asynchronous (invoker) @Stateless public class OrderBean { @EJB private ProcessBean processBean; public double parallelizeCalculationAsync(List<Order> orders) { List<Future<Double>> futures = new ArrayList<>(); for (Order order : orders) { futures.add(processBean.getTotals(order)); } double totalValue = 0; try { for (Future<Double> future : futures) { totalValue += future.get(10, SECONDS); } } catch (Exception e) { // ignore } return totalValue; } }

PA

RA

LLELIS

M W

ITH TH

RE

AD

PO

OLS

Using MDB (invoker) @Stateless public class OrderBean { @Resource(mappedName = “ConnectionFactory”) private ConnectionFactory cf; @Resource(mappedName = “RequestQ”) private Destination sendQueue; @Resource(mappedName = “ResponseQ”) private Destination responseQueue; @TransactionAttribute(NOT_SUPPORTED) // or UserTransaction public double handleOrders(List<Order> orders) { try { // …create a producer for (Order o : orders) { ObjectMessage outgoing = s.createObjectMessage(); outgoing.setObject(o); outgoing.setLongProperty(“RequestID”, orders.hashCode()) mp.send(outgoing); } cn.start(); // must be started, otherwise we can't receive MessageConsumer mc = s.createConsumer(responseQueue, "JMSCorrelationID = '" + orders.hashCode() + "'"); double totalValue = 0; for (int i = 0; i < orders.length(); i++) totalValue += mc.receive(10, SECONDS).getDoubleProperty(“OrderValue”); return totalValue; } catch (JMSException e) { throw new EJBException(e); } } }

Send

Receive

PA

RA

LLELIS

M W

ITH TH

RE

AD

PO

OLS

Using MDB (invokee) @MessageDriven(mappedName = “RequestQ”) public class ProcessOrderBean implements MessageListener { @Resource(mappedName = “ConnectionFactory") private ConnectionFactory cf; @Resource(mappedName = “ResponseQ") private Queue responseQueue; public void onMessage(Message message) { double orderValue = // ... Process order try { MessageProducer mp = // … create message producer Message msg = s.createMessage(); msg.setJMSCorrelationID(message.getIntProperty(“RequestID”)); msg.setDoubleProperty(”OrderValue”, orderValue); mp.send(msg); } catch (JMSException e) { throw new EJBException(e); } } }

PA

RA

LLELIS

M W

ITH TH

RE

AD

PO

OLS

A nice write up, but is it worth it?

PA

RA

LLELIS

M W

ITH TH

RE

AD

PO

OLS

Response time – Compute intensive Tasks

0

2000

4000

6000

8000

10000

12000

Serial 8 parallel

1 user 10 users 20 users

FORK AND KNIVES… UHM… JOIN

FO

RK JO

IN Introduction

‣ New to Java 7 java.util.concurrent ‣ Helps you write a parallel recursive algorithm ‣ Break a long task to Fine grained tasks ‣ Very efficient for relatively short tasks

FO

RK JO

IN Introduction

Worker

Worker

Worker

Worker

ForkJoinTask ForkJoinPool

FO

RK JO

IN Example

class PartialTask extends RecursiveAction { protected void compute() {

PartialTask pt1 = new PartialTask(firstHalf);

PartialTask pt2 = new PartialTask(secondHalf);

pt1.fork(); // fork

pt2.exec(); // execute in new thread

pt1.join(); // wait for completion

}

}

fjPool = new ForkJoinPool(para); Para.invoke(task)

FO

RK JO

IN A Benchmark K-Means Clustering Algorithm ‣ Divide n points into k clusters ‣ Based on proximity

FO

RK JO

IN Disclaimer

‣ This is not a formal benchmark ‣  I tried to do my best but it is not community criticized ‣ Ran on 8 core machine (no HT) ‣ Windows OS ‣ Warm-up was taken into account J ‣ 120K points to 50 clusters

FO

RK JO

IN Response Time – by amount of parallelism

0"

1000"

2000"

3000"

4000"

5000"

6000"

1" 2" 3" 4" 5" 6" 7" 8"

Tim

e%(sec)%

#cores%used%

Time"

FO

RK JO

IN Speedup– by amount of parallelism

0"

1"

2"

3"

4"

5"

6"

1" 2" 3" 4" 5" 6" 7" 8"

Tim

e%(sec)%

#cores%used%

Speedup"

FO

RK JO

IN Single Operation Throughput

0"

0.2"

0.4"

0.6"

0.8"

1"

1" 2" 3" 4" 5" 6" 7" 8"

Op/sec'

Op/sec"

FO

RK JO

IN FJ on a JEE Server

‣ Does your server need to do anything else ? ‣ Selfish Task Deplete all compute resources slows other task ‣ What about throughput

FO

RK JO

IN

What About Throughput ?

Lets check !

FO

RK JO

IN Throughput of 8 Concurrent Users ‣ What will be better

FO

RK JO

IN Throughput 8 concurrent users in parallel

0"0.2"0.4"0.6"0.8"1"

1.2"1.4"1.6"1.8"

Serial"Algorithm" Fork"Join"Parallelism"1"

Fork"Join"Parallelism"8"

Op/sec'

Op/sec"

FO

RK JO

IN Response Time

0"1000"2000"3000"4000"5000"6000"7000"8000"9000"

Serial"Algorithm"

Fork"Join"Parallelism"1"

Fork"Join"Parallelism"8"

TRT"of"8"Users"

TRT"of"1"User"

FO

RK JO

IN Conclusion

‣ Use FJ for the non common case ‣ Limit the number of cores do not starve other server tasks

LOAD MANAGEMENT FOR BACKEND ACCESS

LO

AD M

AN

AG

EM

EN

T FOR B

AC

KE

ND A

CC

ES

S Use Case You have a backend machine that can only handle 20 concurrent requests and you have a cluster of four servers, so you… ‣ Create a semaphore with five permits. ‣ Acquire lock on semaphore ‣ Call server ‣ Release lock

LO

AD M

AN

AG

EM

EN

T FOR B

AC

KE

ND A

CC

ES

S Problems With This Approach ‣ The backend server can be under-utilized across the cluster.

‣ First server could be full of locks while second is free. ‣ One server can stall unnecessarily long time. ‣ Recompile necessary if some parameter changes.

LO

AD M

AN

AG

EM

EN

T FOR B

AC

KE

ND A

CC

ES

S A Better Approach

Using the Request-Reply Pattern (EIP), using: ‣ Clustered queues (request + response) ‣ A caller (stateless session bean) ‣ A backend invoker bean (message-driven bean) ‣ Configuration to limit in-process messages

For the advanced: ‣ Write a resource adapter (JCA)

‣ Support load balancing ‣ Support cluster

LO

AD M

AN

AG

EM

EN

T FOR B

AC

KE

ND A

CC

ES

S Request-Reply Pattern using clustered queues (Caller SLSB)

@Stateless public class DispatcherBean { @Resource(mappedName = “ConnectionFactory”) private ConnectionFactory cf; @Resource(mappedName = “RequestQ”) private Destination sendQueue; @Resource(mappedName = “ResponseQ”) private Destination responseQueue; @TransactionAttribute(NOT_SUPPORTED) // Alternatively UserTransaction public double checkStockRate(String stockName) { try { //… create producer Message outgoing = s.createMessage(); outgoing.setStringProperty(”StockName", stockName); mp.send(outgoing); String correlationId = outgoing.getJMSMessageID(); cn.start(); // must be started, otherwise we can't receive MessageConsumer mc = s.createConsumer(responseQueue, "JMSCorrelationID = '" + correlationId + "'"); Message incoming = mc.receive(10000L); // wait no more than 10 seconds if (incoming != null) { return incoming.getDoubleProperty(”StockPrice"); } else { return Double.MIN_VALUE; } } catch (JMSException e) { throw new EJBException(e); } } }

Send

Receive

LO

AD M

AN

AG

EM

EN

T FOR B

AC

KE

ND A

CC

ES

S Request-Reply Pattern using clustered queues (Invoker MDB) @MessageDriven(mappedName = “RequestQ”) public class BackendInvokerBean implements MessageListener { @Resource(mappedName = “ConnectionFactory") private ConnectionFactory cf; @Resource(mappedName = “ResponseQ") private Queue responseQueue; public void onMessage(Message message) { double stockPrice = // ... Fetch from backend service try { // … create producer Message msg = s.createMessage(); msg.setJMSCorrelationID(message.getJMSMessageID()); msg.setDoubleProperty(”StockPrice”, stockPrice); mp.send(msg); } catch (JMSException e) { throw new EJBException(e); } } }

LO

AD M

AN

AG

EM

EN

T FOR B

AC

KE

ND A

CC

ES

S Configuration for in-process limitation Vendor-specific, but in general: ‣ You may set an upper limit on consumers

‣ Glassfish has maxNumActiveConsumers ‣ You may set an upper limit of MDBs in Pool

‣ Works on all servers regardless of vendor (i.e. set max-pool-size to 5 in glassfish-ejb-jar.xml)

LO

AD M

AN

AG

EM

EN

T FOR B

AC

KE

ND A

CC

ES

S

Isn’t MDB horribly slow?

LO

AD M

AN

AG

EM

EN

T FOR B

AC

KE

ND A

CC

ES

S Response time (in milliseconds)

0

2

4

6

8

10

12

Direct MDB

1 user 5 users 10 users

INTERFERING WITH BEAN POOL

INTE

RFE

RIN

G WITH B

EA

N PO

OL

Use Case

You occasionally suffer from resource depletion so to protect your server you: ‣ Create an EJB Interceptor that

‣  checks current beans in use (i.e. through counter or MBeans) ‣  throws an exception at chosen maximum size. ‣ Or wait until level go down ‣ Assign it to your EJB.

INTE

RFE

RIN

G WITH B

EA

N PO

OL

Problems

‣  Is there always equal load distribution across all tasks? ‣  Intelligent load balancers may continue to shoot at the wrong target

‣  It’s just an exception… not an overload ‣ DeadLock

INTE

RFE

RIN

G WITH B

EA

N PO

OL

Better approach Leave pool management to the server, pool sizes: ‣ Can be tuned on a global level

‣ Right strategy (infinite thread pools, incremental, fixed) ‣ Global default for min/max/timeout ‣ Can be set for each EJB

‣ XML ‣ Annotations (i.e. JBoss)

INTE

RFE

RIN

G WITH B

EA

N PO

OL

Leave Management to the app server It can be configured through the GUI…(Glassfish example)

INTE

RFE

RIN

G WITH B

EA

N PO

OL

Through Config Files <glassfish-ejb-jar>

<enterprise-beans>

<ejb>

<ejb-name>AnEJB</ejb-name> <bean-pool>

<steady-pool-size>10</steady-pool-size> <resize-quantity>10</resize-quantity> <max-pool-size>100</max-pool-size> <pool-idle-timeout-in-seconds> 60 </pool-idle-timeout-in-seconds> </bean-pool>

</ejb>

</enterprise-beans>

</glassfish-ejb-jar>

CONCLUSION

CO

NC

LUS

ION

‣ Most mistakes stem from misunderstanding the app server, try to understand your app server’s OOTB-capabilities… ‣ …and the Java EE specification. ‣  If you have a valid case for your own concurrency-related code,

make sure it plays well with the container. ‣ Your app server doesn’t always know best, but it often does. And if

not, you can often help it understand! ‣ Do not expect your environment to always look the same, it changes

quicker than you might think.

QUESTIONS?

CO

NTA

CT IN

FO Haim Yadid haim.yadid@gmail.com

Daniel Pfeifer danpfe@gmail.com