Embed Size (px)
Citation preview
gluster-swift➔ Just another application using fuse mount
➔ No libgfapi, yet
➔ Uses and “plugs into” OpenStack Swift➔ Against having to write our own object front-end➔ No catching-up to do➔ Swift is largely decoupled from OpenStack
➔ Overrides vanilla Swift’s distribution and replication
Object access trivia
➔ Object = {Data + Metadata + Identifier}➔ Abstracted ➔ Stateless (HTTP)➔ No partial PUTs➔ Ranged GETs supported➔ No renames, only COPY➔ Fixed logical hierarchy
➔ Accounts/Container/Objects➔ But you can’t mount it
Vanilla Swift
Ring
SQLite DB
SQLite DB
XFS
AccountServer
ContainerServer
ObjectServer
ProxyServer
Client
Gluster-Swift
Ring
AccountServer
ContainerServer
ObjectServer
ProxyServer
Client
Same node
FUSEmount
Proxy Server - pipeline
/etc/swift/proxy-server.conf
pipeline = catch_errors gatekeeper healthcheck proxy-logging cache container_sync bulk tempurl ratelimit tempauth copy container-quotas account-quotas slo dlo versioned_writes proxy-logging proxy-server
➔ Very similar to translators in gluster
➔ Middlewares are highly modular
➔ Many features are implemented as middlewares
Amazon S3 API Support➔ Implemented as a middleware (swift3)
➔ Converts incoming S3 requests into Swift requests
S3 Request
Swift Response
Swift3middleware
Swift Request
S3 Response
ProxyServer
S3 Client
Pluggability in Swift
• Backends implement DiskFile• Object server is pluggable officially.• Account and container code overrides aren’t.• Makes cross-version compatibility tricky.
Swift Object Ops → Fops
PUT
mkdir()*getxattr()open()write()fsetxattr()fsync()rename()close()
GET
open()fstat()fgetxattr()read()close()
POST
getxattr()stat()setxattr()
DELETE
getxattr()stat()unlink()rmdir()*
*multiple calls - recursive
HEAD
getxattr()stat()
Swift Listing Ops → Fops
Container GET(plain text) – only names
getxattr()opendir()readdir()*getxattr()1
Container GET(json/xml) – names + metadata
getxattr()opendir()getdents()*getxattr()2
*multiple calls – recursive1 only on dirs2 on every inode
# getfattr -d /mnt/gluster-object/test/c1/animals/cat/kitten
user.swift.metadata="{\"Content-Length\":\"4\",\"Etag\":\"4a4be40c96ac6314e91d93f38043a634\",\"X-Timestamp\":\"1475214280.33138\",\"X-Object-Type\":\"file\",\"X-Type\":\"Object\",\"Content-Type\":\"application/x-www-form-urlencoded\"}"
# curl -i http://host:8080/v1/AUTH_test/c1/animals/cat/kitten -X PUT -d'meow'
HTTP/1.1 201 CreatedLast-Modified: Fri, 30 Sep 2016 05:44:41 GMTContent-Length: 0Etag: 4a4be40c96ac6314e91d93f38043a634Content-Type: text/html; charset=UTF-8X-Trans-Id: txae858eb9169a4e1eb2a09-0057edfbc8Date: Fri, 30 Sep 2016 05:44:40 GMT
Object PUT
Swift API Violations
# curl -i http://localhost:8080/v1/AUTH_test/c2 -X GET
HTTP/1.1 200 OKContent-Length: 7X-Container-Object-Count: 2Accept-Ranges: bytesX-Storage-Policy: Policy-0X-Container-Bytes-Used: 0X-Timestamp: 0000000001.00000Content-Type: text/plain; charset=utf-8X-Trans-Id: tx18e21ccd03f04f4698a1a-0057ed1b84Date: Thu, 29 Sep 2016 13:47:48 GMT
HTTP Header needs to go first, then the response body.
Swift API Violations
# curl http://host:8080/v1/AUTH_test/c1/animals/cat -X PUT Now, cat is a file
# curl http://host:8080/v1/AUTH_test/c1/animals/cat/kitten -X PUT Here, cat is supposed to be directory but it’s already a file.
Backend object path on filesystem
gluster-swift:/mnt/gluster-object/test/c1/animals/cat
Vanilla Swift:/mnt/sdb1/objects/778/69f/c2b307d78b6c419c0c1b76d91c08c69f/1412628708.01757.data
TODOs➔ Bring up gluster-swift to ‘newton’➔ Centos SIG packages
➔ Zero copy and copy offload➔ Use libgfapi python bindings➔ Use hummingbird project➔ Native object ? glfs_put(..), glfs_get(..)
Ideas
Demo ?gluster-swift
