Introducing MagnetoDB, a key-value storage sevice for OpenStack

MagnetoDB Key-Value Storeservice for OpenStack

Mirantis, 2014

What is MagnetoDB?

MagnetoDB - OpenSource Amazon DynamoDB API implementation for OpenStack, the key-value database service for storing any amount of data with seamless scalability and predictable performance.

MagnetoDB key features

● Easy-to-integrate REST-like API (AWS SDK, boto clients)

● Schemaless, non-relational table-based model● Put/get/query/scan item operations● Eventual and strong consistency reads● Local Secondary indexes● Batch read/write operations● Designed to handle any amount of data and any level

of request traffic● Seamless throughput and storage scaling● Fault tolerance

MagnetoDB use cases

● High request traffic level applications

● Time series data

● Queries are distributed across the data

● Any other applications

● Migration of workloads between AWS and

OpenStack

Architecture highlights

● Layered architecture

● Horizontally scalable

● Pluggable database backend

● Uses native OpenStack infrastructure

○ OpenStack HEAT for provisioning

○ OpenStack Neutron LoadBalancer

○ Integration with OpenStack Keystone planned

Overall architecture

Integration with OpenStack services

Current status

○ Pilot implemented

○ Open source, Apache 2 license

○ Launchpad homepage: https://launchpad.net/magnetodb

○ Code published on github: https://github.

com/Mirantis/magnetodb

○ Deployable to Vanilla OpenStack

○ Compatible with existing DynamoDB clients

MagnetoDB Pilot Features

● Table CRUD API○ ListTables○ DescribeTable○ CreateTable○ DeleteTable

● Item CRUD API○ PutItem○ UpdateItem○ DeleteItem○ GetItem

● Data querying API○ Query○ Scan

● Error Handling API

MagentoDB use-case: dynamodb-geo

“Geo Library for Amazon DynamoDB allows to easily create and query geospatial data”

MagentoDB use-case: dynamodb-geo

Next steps● General

○ Move the development to OpenStack Infra○ Integration with OpenStack Keystone

● DynamoDB API○ Atomic counters○ Asynchronous table creation○ Atomic conditional write operations○ Batch operations

● Other features○ Management API○ Quota management

Summary

● The pilot implementation is available on GitHub in public

repository

● MagnetoDB is compatible with AWS SDK and boto library

● It is now possible to run applications which use DynamoDB as

a storage on top of OpenStack using MagnetoDB

Links and Resources

○ Wiki: https://wiki.openstack.org/wiki/MagnetoDB

○ Launchpad: https://launchpad.net/magnetodb/

○ Source code: https://github.com/Mirantis/magnetodb

○ IRC: #magnetodb @ FreeNode

○ AWS DynamoDB http://aws.amazon.com/dynamodb/

○ dynamodb-geo https://github.com/awslabs/dynamodb-geo