View
115
Download
6
Category
Preview:
Citation preview
NVMe, Storage Class Memory, and Operational Databases:Real World Results
—Brian Bulkowski, CTO and FounderMay 24, 2016
See all the presentations from the In-Memory Computing Summit at http://imcsummit.org
Who needs a fast operational database?
3© 2016 Aerospike Inc. All rights reserved. [ ]
Identified where scale is mission critical – the AdTech Industry
1 Expanding into other use casesin BFSI, Telco, Retail, IoT
4
Built database that can scale for RTB use case for these customers and more….
2
The Scalable Operational Database Problem
3
Deployed @ Digital Enterprises that need consumer scale databases
4© 2016 Aerospike Inc. All rights reserved. [ ]
Architecture Overview
LEGACY DATABASE(Mainframe)
XDR
DATA WAREHOUSE/DATA LAKE
LEGACY RDBMSHDFS BASED
BUSINESSTRANSACTIONS
Web views
( Payments ) ( Mobile Queries )
( Recommendation ) ( And More )
High Performance NoSQL
“REAL-TIME BIG DATA”“DECISIONING”
YYYOUR APPLICATION FRAMEWORK
YY
Decisioning Engine
5© 2016 Aerospike Inc. All rights reserved. [ ]
Architecture Overview
LEGACY DATABASE(Mainframe)
XDR
Decisioning Engine
DATA WAREHOUSE/DATA LAKE
LEGACY RDBMSHDFS BASED
BUSINESSTRANSACTIONS
Web views
( Payments ) ( Mobile Queries )
( Recommendation ) ( And More )
High Performance NoSQL
“REAL-TIME BIG DATA”“DECISIONING”
500Business Trans per sec
5000 Calculations per sec
X =
2.5 M Database Transactions per sec
6© 2016 Aerospike Inc. All rights reserved. [ ]
Business Challenge• Meet payment SLA of 750 ms• Differentiate between fraudulent and legitimate
orders in real-time• Stop loss of business due to latency• Support thousands of DB reads/writes per credit
card transaction• Bring new algorithms and data
online within weeks
Prevent Only Fraudulent Transactions
7© 2016 Aerospike Inc. All rights reserved. [ ]
Bespoke Algorithms• Blend Internet behavior• Track every IP address• Recent fraudulent cart items• Machine learning generated filters• Expand to new platforms
Not business process modeling
Easier to code and validatethan SQL queries
Credit Card Processing System
Fraud Detection & Protection App
RulesRule 1Rule 2Rule 3
Historical Data
Rule 1-PassedRule 2-PassedRule 3-Failed
Account BehaviorStatic Data
AccountStatistics
Prevent Only Fraudulent Transactions
8© 2016 Aerospike Inc. All rights reserved. [ ]
Intra-day System of Record
Challenge• DB2 (RDBMS) stores positions for 10 Million customers• Must update stock prices, show balances on 300 positions,
process 250M transactions, 2M updates/day• Single view of position across
Risk, Mobile, Fraud• Enable hard-to-code Risk and Fraud
algorithms
9© 2016 Aerospike Inc. All rights reserved. [ ]
Intra-day System of Record
High performance NoSQL• Immediate consistency, no data loss• Predictable low latency at high throughput • Flash provides durability, fast restarts• Cross data center (XDR) for replication
Highly parallel account-to-trades data model
Easy-to-validate Java-based algorithms
IBM DB2(Mainframe)
Real-Time App Record App
Finance App
Real-TimeData Feed
Start of the DayData Loading
End of DayReconciliation
AccountPositions
10© 2016 Aerospike Inc. All rights reserved. [ ]
■Internet cache replacement■Telecom integrated real-time billing and routing■Retail predictive analytics■Machine learning
■Online learning updates models rapidly
■Social messaging■Gaming and gambling
Similar use cases
11© 2016 Aerospike Inc. All rights reserved. [ ]
Aerospike Architecture
■ Every node in a cluster is identical, handles both transactions and long running tasks
■ Direct attach storage for 1M+ TPS performance
■ Highly available clustering, integratedtransaction processing
OHIO Data Center
12© 2016 Aerospike Inc. All rights reserved. [ ]
■Tests simultaneous read and write performance■Small block ( 1K ) random read latency
■During continual large-block write load
■50-50 read-write load
■Defragmentation ( large blocks )
■24 hours ( variance often found at 7 to 10 hours )
■Operational databases need to accept reads and writes■Reads are not localized or predictable
Aerospike ACT certification tool for Flash
Status of Wide SATA, PCIe, NVMe,
14© 2016 Aerospike Inc. All rights reserved. [ ]
■Early SATA ( 2009, 2010 )■Intel X25M
■Samsung SS805
■Devices provide 95% < 1msfor about 2,000 IOPS
■$3 / GB
■FusionIO - 2010■PCIe, but custom driver
■CPU, bus load
■$8 / GB
Historical Perspective – Early Days
15© 2016 Aerospike Inc. All rights reserved. [ ]
■Micron proves fast PCIe possible■P320 ( SLC ) with low bus overhead, excellent driver
■Over 200,000 IOPS with 99.7% < 1ms
■( SFF-8639 hot-swap 2.5” pci-e drives )
■“Wide SATA” generally used■12 to 20 2.5” SATA drives per 2U chassis
■Intel S3700, S3500 ; Samsung 843 favored
■8 drives per controller ( many issues )
■150,000 IOPs per chassis achievable
■Violin, FusionIO troubled, DSSD sold to EMC■NVMe available but not practical
Historical Perspective – 2013, 2014
16© 2016 Aerospike Inc. All rights reserved. [ ]
■Linux, Windows drivers achieve performance■U.2 and M.2 form factors available
■Intel P3700, P3600, P3500 available – ■250k IOPs per card
■Samsung PM1735 available – ■120k IOPs per card
■Micron 9100■HGST, Toshiba – 30k to 50k per card
■SAS / SATA lingers■Samsung SM1635, PM1633; Intel S3700; Micron S600 still shipping
NVMe Arrives – 2015 to present
17© 2016 Aerospike Inc. All rights reserved. [ ]
■High, predictable performance■High transfer speed reduces jitter
■2.8uS NVMe vs 5.0uS SATA
■Better controllers
■Mature and tuned Linux driver
■U.2 front panel hot swap available ( and 24-wide )
■NVMe arrays available■Apeiron ADS1000 “direct scale-out flash”
■EMC D5, Mangstore NX63020
■All new Aerospike deployments on NVMe
NVMe’s crushing superiority
18© 2016 Aerospike Inc. All rights reserved. [ ]
1 Million TPS on 1 Server
Options for storage on a database before Aerospike:
RAM, which was fast, but allowed very limited storage Disk, which allowed for a lot of storage, but was limited in
speed
Intel achieved 1M TPS using 4 Intel P3700 SDs with 1.6 TB capacity on a single Aerospike server. The cost per GB is a fraction of the cost of RAM, while still having very high performance.
19© 2016 Aerospike Inc. All rights reserved. [ ]
■Every public cloud provider has Flash■AWS / EC2 has sophisticated offerings
■Google Compute is high performance
■Private clouds manage Flash■Docker offers storage metadata
■Pivotal manages Flash and traditional storage
Flash in the Public Cloud
Storage Class Memory( and trends )
21© 2016 Aerospike Inc. All rights reserved. [ ]
■Diverging “Drive Writes Per Day”■Low-write devices
■1~2 DWPD
■Sandisk Inifiniflash, Micron
■Increase density, lower cost
■Hadoop / Data lake “all flash” use
■High-write devices■10~15 DWPD
■P3700, Hitachi, Samsung
■Optane to the rescue
Trends in NAND
22© 2016 Aerospike Inc. All rights reserved. [ ]
■Optane this year■3D Xpoint in 2.5” NVMe package
■“7x faster” – limited by NVMe !
■Very high write durability
■Replaces SLC for some use cases
■Unknown pricing
■NVDIMM■Removes NVMe limit
■Intel cagy on delivery – “uncommitted”
■Competes with DRAM
■Really changes the world
Intel’s 3D Xpoint roadmap ( public info )
23© 2016 Aerospike Inc. All rights reserved. [ ]
■It’s not exactly like DRAM■When a system restarts, need to clear
■Slower
■Different kind of data structures required
■It’s not exactly like storage■It’s on the memory bus
■New instructions for persistance
■Think of it more like DRAM■Lower power consumption
■Much higher density ( 1T++ )
How to architect for DDR 3D XPoint
Thank YouQuestions?
Recommended