Upload
vuongdang
View
224
Download
2
Embed Size (px)
Citation preview
Copyright © 2014 Seagate Technology
Case Study: Seagate’s OpenStack Swift Security
Presentation for Cloud Security Alliance (CSA) Congress
Rodney Beede, Seagate Technology
September 2014
Copyright © 2014 Seagate Technology
Introduction● Rodney Beede
○ Cloud Security Engineer○ Seagate Technology
● M.S. in Computer Science○ University of Colorado○ “A Framework for Benevolent Computer Worms” 2012
● Doing computer security since 2001○ Primary interests are web and cloud security
● Tech blog○ https://www.rodneybeede.com/○ The views expressed in this blog are my personal view and have not been reviewed or
approved by Seagate.
Copyright © 2014 Seagate Technology
What Is OpenStack Swift?● Object Storage
○ HTTP REST API web service○ Stores objects
Object Storage API example
PUT /v1/my_account/container/example_obj.txt HTTP/1.1User-Agent: curl/7.32.0Host: localhost:8080Accept: */*X-Auth-Token: authtokenhereContent-Length: 38Expect: 100-continue
------------------------------------------------------
HTTP/1.1 201 CreatedContent-Length: 118Content-Type: text/html; charset=UTF-8Etag: f7d40eceffdd9c2ecab226105737b2a6
Image from “OpenStack Installation Guide for Red Hat Enterprise Linux, CentOS, and Fedora” and licensed under the Apache License, Version 2.0
Copyright © 2014 Seagate Technology
What Do We Use It For?● Testing hard drives
○ Benchmarking○ Measuring drive temperature○ Measuring drive vibration
● IT uses○ Backups
■ Petabytes of storage space
● EVault○ Long term storage (tape replacement)
Copyright © 2014 Seagate Technology
Some Supporting Technologies
● Linux
● Puppet - puppetlabs.com○ Centralized automated config
● Center for Internet Security (CIS)○ Security hardening benchmarks
Copyright © 2014 Seagate Technology
The Short Version - Swift Security● Lots of components to secure
○ OS○ Web server○ Authentication store○ Network traffic○ Remote console interface
● Lack of guidelines (until now)○ http://docs.openstack.org/sec/○ …”we do not have specific
guidance related to configur-ation of the storage projects…”
○ So I wrote my own
Copyright © 2014 Seagate Technology
Copyright © 2014 Seagate Technology
Copyright © 2014 Seagate Technology
Question: Network Protocol● Which
services have encrypt-ion over the wire?
Copyright © 2014 Seagate Technology
Copyright © 2014 Seagate Technology
Remote Consoles - Overlooked● Commonly known as
○ IPMI○ BMC○ iLO○ DRAC
● Security problem○ Default password○ IPMI protocol insecure
● A Penetration Tester's Guide to IPMI and BMCs - Rapid 7○ HD Moore, Metasploit, July 2, 2013 12:22:49 PM○ Unpatched firmware - admin access○ Admin hash vulnerability unpatchable
Image care of https://flic.kr/p/trJkJ Licensed under CC BY-SA 2.0. Modified with red highlight. Original by Cloned Milkmen and titled “T2000 USB and LED Close-up” 2006.
Copyright © 2014 Seagate Technology
Remote Consoles - Remediation1. Network segregation
2. Patch to latest firmware
3. Rotate your passwords often
a. Typically manual process
b. IPMI has standard protocol for user management
i. Hint: Linux ipmitool command
c. We use CyberArk with custom plugin
Copyright © 2014 Seagate Technology
Securing Services - General● Standard OS configuration file permissions
# chown -R root:swift /etc/swift/
# find /etc/swift/ -type f -exec chmod 640 {} \;
# find /etc/swift/ -type d -exec chmod 750 {} \;
● Run services as “swift”○ Not root○ You won’t need TCP ports < 1024 either
Copyright © 2014 Seagate Technology
Real World Mistake● Question: What are these files?
[user@host] $ ls /etc/certs/ca.crtintermediate.crtserver.crtserver.csrserver.pem
Copyright © 2014 Seagate Technology
Real World Mistake - Problem● Question: What is the matter here?
[user@host] $ ls -la /etc/certs/
drwxrwxr-x 2 www www 4096 Jul 22 12:15 .-rw-r--r-- 1 www www 2110 Jan 2 2014 server.crt-rw-r--r-- 1 www www 1813 Jan 2 2014 server.csr-rw-r--r-- 1 www www 3243 Jan 2 2-14 server.pem
Copyright © 2014 Seagate Technology
Real World Mistake - Correct Security● Question: Which is correct?
A. -r--r----- 1 www www server.pem
B. -rw-r----- 1 root www server.pem
C. -rw-r----- 1 root root server.pem
D. ---------- 1 root root server.pem
Copyright © 2014 Seagate Technology
Real World Mistake - Answer● Answer: All of them can be
A. -r--r----- 1 www www server.pem
B. -rw-r----- 1 root www server.pem
C. -rw-r----- 1 root root server.pem
D. ---------- 1 root root server.pem
Copyright © 2014 Seagate Technology
Securing Storage ServicesThe following are the default listening ports for the various storage services:
Service name Port Type
Account service 6002 TCP
Container service 6001 TCP
Object service 6000 TCP
Rsync 873 TCP
Copyright © 2014 Seagate Technology
Object Storage "Account" TerminologyAn object storage "account" isn’t what you think it means
OpenStack Object Storage Account
Collection of containers; not user accounts.
Supports ACLs to associate “users” with the account.
OpenStack Object Storage Containers
Collection of objects.
Supports ACLs.
OpenStack Object Storage Objects
The actual data objects.
Supports ACLs.
Copyright © 2014 Seagate Technology
Another way of thinking about it:
● A single shelf (Account) holds zero or more
○ Buckets (Containers)
● Buckets (Containers) each hold zero or more
○ Objects
● A garage (Object Storage cloud environment) may have
○ Multiple shelves (Accounts)
○ Each shelf may belong to zero or more users.
Object Storage "Account" Terminology
Copyright © 2014 Seagate Technology
Securing Proxy Services● This is the consumer facing service
Copyright © 2014 Seagate Technology
Securing Proxy Services1. Use SSL/TLS
a. Not the built-in web serverb. Apache, Nginx, etc. with mod-wsgi
2. Run web server as non-roota. “swift” system account is one optionb. Listen on port > 1024
i. URL is typically not typed anyway
https://swift.cloud.example.org:44443/v1/AUTH_1234
3. SSL Load Balancera. Common pitfall - early termination
Copyright © 2014 Seagate Technology
Load Balancerwith SSL offload
Copyright © 2014 Seagate Technology
Identity (Keystone)● Formally used SWAuth
○ Stored all info as Swift objects
● Now using Identity Service API ver 2.0○ Custom implementation versus Keystone○ Going to add LDAP for internal use
● Basic principles○ Manage credentials
■ Password length, complexity■ Account termination
○ Audit your logs for policy conformity
Copyright © 2014 Seagate Technology
Security Testing● CVE list
○ http://www.cvedetails.com/vulnerability-list/vendor_id-11727/Openstack.html
● Authentication token validation○ Lots of CVEs for invalid tokens○ Distributed clustered system causes auth delays
● Check for exposed storage node ports○ TCP ports 6000-6002 (default)○ Also think about packet sniffing internally
■ Network switch != secure (think ARP flood)
Copyright © 2014 Seagate Technology
Security Testing● It’s a web service
○ Similar approach as other web services○ WSDL is optional, Swift is REST
● Encoding attacks in URIs○ Ex: Third party web applications tend to “trust” the web
service data without proper output encoding
○ Create an object with a name like
“test<script>alert(‘xss’)</script>me”
■ Valid name to Swift■ Can trip up third party web application consumers
Copyright © 2014 Seagate Technology
Interesting Problem & Solution
Copyright © 2014 Seagate Technology
Interesting Problem & Solution
These nodes have no Internet or corporate network access
Copyright © 2014 Seagate Technology
Options● NAT
○ Requires more resources○ Additional security rules to manage
● HTTP Proxy○ Must configure each node to use it
■ Puppet module can do this○ Possibility of caching is a bonus
● Local package repo○ Provides local mirror
■ Faster■ Available even if Internet is not
Copyright © 2014 Seagate Technology
Trial and Error● NAT
○ Worked for small scale○ Security misconfiguration
● HTTP Proxy○ Wrote Puppet module
■ Provided auto-configuration○ Not helpful if Internet goes down
Copyright © 2014 Seagate Technology
Solution● Local mirror repository
○ Controlled package versions
○ No constant Internet connection required
○ Use Puppet for client configuration
Copyright © 2014 Seagate Technology
Puppet● Automation is the key
○ Account management■ Ex: “swift” OS system account
○ Cloud installation■ Public (not Seagate developed) modules
● puppetlabs/swift● puppetlabs/apache
■ Private (Seagate developed) modules● HP OpenView● Security hardening● SSH key management
● Configuration○ Hiera driven (Hierarchical Database)○ Allows separate environments
● Con: Mistake in Puppet config can take down entire environment○ Although usually can recover via Puppet too
Copyright © 2014 Seagate Technology
Puppet - Exampleuser { ‘swift’:
ensure =>present,comment =>‘OpenStack’,expiry =>absent,forcelocal =>true,password => ‘*’, # no password loginshell =>‘/sbin/nologin’,system =>true,
}
Copyright © 2014 Seagate Technology
Future● Detailed object reporting
○ What ACLs exist on objects■ Something open to anonymous read-only■ Document marked confidential?
● Account management○ Orphaned accounts○ Verification of terminated users
● Automated unit tests for API security○ Verification of token life cycle○ ACL’s on objects○ Malicious character names (persistent XSS)
Copyright © 2014 Seagate Technology
References● OpenStack Security Guide
○ http://docs.openstack.org/sec/
● Learning Puppet
● Kinetic○ New Seagate Breakthrough Fundamentally Alters Cloud Economics
- Oct 22, 2013
○ Seagate Kinetic Open Storage Platform
○ Kinetic Open Storage Documentation Wiki
Copyright © 2014 Seagate Technology
Thank You
Copyright © 2014 Seagate Technology
BACKUP MATERIAL
Copyright © 2014 Seagate Technology
Kinetic● Internet has published information
Copyright © 2014 Seagate Technology
Kinetic - OpenStack Swift
Copyright © 2014 Seagate Technology
Kinetic - SecurityFrom “Kinetic Open Storage Value Proposition”:Security
The security of storage services within the cloud datacenter is a difficult task. The Kinetic drive and interface library supports:
● Authentication - A full cryptographic authentication of servers that have access permission to the drive.
● Integrity - Full integrity check of the command and the data.● Authorization - A clear set of roles by server as to what the application is
allowed to do. Typical roles are read, read/write, management of the drive and management of the security in the drive.
● Transport Layer Security - For the security of very sensitive data and/or management commands, a full industry standard TLS suite is also provided.