© 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Jason Fulghum, Development Manager –

AWS SDK

Karthi Thyagarajan, Enterprise Solutions

Architect

October 2015

DEV303

Practical DynamoDB with Java

What to Expect from the Session

REAL-TIME PROCESSING

• Lambda + DynamoDBStreams

• Traditional approaches

DEMO – Real-time processing w/ Lambda and DynamoDB

Streams

BATCH PROCESSING

• Lambda-based

• Traditional approaches

DEMO – Batch processing w/

Lambda

1 2 3 4

Scenario

- Game analytics service

- Simple scenario, but concepts carry over for other

applications

- IoT

- E-commerce

- Connect mobile games

NOTE: This session will be heavily focused on code

Lambda

No infrastructure headaches

nodeJS and Java

Trigger on events

1

DynamoDB Streams

Similar to DB triggers

Change records available up to 24 hours

Can process change records using λ

1

Near real-time processing

REAL-TIME PROCESSING

• Lambda + DynamoDBStreams

• Traditional approaches

DEMO – Real-time processing w/ Lambda and DynamoDB

Streams

BATCH PROCESSING

• Lambda-based

• Traditional approaches

DEMO – Batch processing w/

Lambda

1 2 3 4

Near real-time processing: Key challenges

Multiple views for multiple constituents

Minimize impact on performance

Near real-time availability of data in views

1

Proposed architecture

Main table – raw

scores

Player stats

“BI” table (for internal

use)

1

Satellite table –

player info

Multiple views for multiple

constituents

• Materialized views

Minimize impact on performance

• Separate tables so no

contention

Near real-time availability of data

in views

• Minimal delay

Proposed architecture (cont’d)

1

Comparison: SQL-based architecture

• Multiple views for multiple

constituents

• Joins, aggregations, group-

bys, etc.

• Minimize impact on performance

• Potential for deadlock

• Near real-time availability of

data in views

• Read replicas

1

Demo: Real-time processing

REAL-TIME PROCESSING

• Lambda + DynamoDBStreams

• Traditional approaches

DEMO – Real-time processing w/ Lambda and DynamoDB

Streams

BATCH PROCESSING

• Lambda-based

• Traditional approaches

DEMO – Batch processing w/

Lambda

1 2 3 4

Batch processing

REAL-TIME PROCESSING

• Lambda + DynamoDBStreams

• Traditional approaches

DEMO – Real-time processing w/ Lambda and DynamoDB

Streams

BATCH PROCESSING

• Lambda-based

• Traditional approaches

DEMO – Batch processing w/

Lambda

1 2 3 4

Batch processing

• Why batch processing?

• Backfill - may not have accounted for certain scenarios

• Not all views need to be real-time

3

Batch processing: Key challenges

Timely completion – even as data set grows

Must not interfere with real-time processing

3

Batch processing: Architecture

Raw scores

“BI” table

Player stats

Player stats by

date 3

Batch processing (cont’d)

Timely completion – even as data set grows

Must not interfere with real-time processing

3

Batch processing: Traditional approaches

DB: SQL

ETL

May not scale w/ data set growth

DB: NoSQL

Impala, Spark, Presto

Learning curve

3

Demo: Batch processing

REAL-TIME PROCESSING

• Lambda + DynamoDBStreams

• Traditional approaches

DEMO – Real-time processing w/ Lambda and DynamoDB

Streams

BATCH PROCESSING

• Lambda-based

• Traditional approaches

DEMO – Batch processing w/

Lambda

1 2 3 4

Summary

No infrastructure

Low barriers to entry

Force multiplier

DynamoDB

Streams λ

+

Next steps

- Links

- AWS SDK for Java: http://aws.amazon.com/sdk-for-java/

- AWS Toolkit for Eclipse: http://aws.amazon.com/eclipse/

- AWS Java Development blog: https://java.awsblog.com/

Related sessions

- ARC308 - The Serverless Company Using AWS

Lambda: Streamlining Architecture with AWS

- DAT401 - Amazon DynamoDB Deep Dive: Schema

Design, Indexing, JSON, Search, and More

Thank you!

Remember to complete

your evaluations!