Upload
april-elderkin
View
215
Download
0
Tags:
Embed Size (px)
Citation preview
The Consequences ofThe Consequences ofDecentralized Security in a Decentralized Security in a
Cooperative Storage SystemCooperative Storage System
Douglas Thain, Chris Moretti, Paul Madrid,Douglas Thain, Chris Moretti, Paul Madrid,Phil Snowberger, and Jeff HemmesPhil Snowberger, and Jeff Hemmes
University of Notre DameUniversity of Notre Damehttp://www.cse.nd.edu/~cclhttp://www.cse.nd.edu/~ccl
IEEE Workshop on Security in Storage 2005IEEE Workshop on Security in Storage 2005
AbstractAbstract
Suppose that security in storage has been Suppose that security in storage has been deployed at all endpoints.deployed at all endpoints.How does this affect the design of How does this affect the design of distributed storage systems that rely upon distributed storage systems that rely upon these devices?these devices?Clients must become much more:Clients must become much more:– Fault tolerant, adaptive, and self reliant.Fault tolerant, adaptive, and self reliant.– Aware of resource allocation issues.Aware of resource allocation issues.– Helpful to the end user!Helpful to the end user!
Environment: Storage Pool at Notre DameEnvironment: Storage Pool at Notre Dame
Traditional File System SecurityTraditional File System Security
applappl appl
Security Interface
File System Abstraction
disk disk diskfile
Owner ofInode 9842is UID 56
trusted network:PCI
RAIDSAN
MyrinetEthernet
untrusted network:EthernetInternet
placement,replication,reliability
I amJohn Doe!
Decentralized SecurityDecentralized Security
applappl appl
Abstr.
disk disk diskfile
untrusted network:EthernetInternet
Security Security Security
Abstr. Abstr.
placement,replication,reliability
Owner ofFile /foo/baris John Doe
I amJohn Doe!
What is Cooperative Storage?What is Cooperative Storage?
Many devices bound together that can Many devices bound together that can accomplish more than one device alone.accomplish more than one device alone.– Improve capacity, reliability, performance...Improve capacity, reliability, performance...– Could be one person, or many cooperating users.Could be one person, or many cooperating users.
Key property:Key property:– Each person retains absolute control of their own Each person retains absolute control of their own
resources by setting local policies.resources by setting local policies.– People share and collaborate with others that they People share and collaborate with others that they
know and trust. No free love! No central control!know and trust. No free love! No central control!– However, some resources are set up for the common However, some resources are set up for the common
good by an good by an authorityauthority. (CS workstations usable by . (CS workstations usable by any member of the CS department, says the chair.)any member of the CS department, says the chair.)
file transfer
filesystem
filesystem
filesystem
filesystem
filesystem
filesystem
filesystem
CentralFilesystem
App
Distributed Database Abstraction
Adapter
App
Distributed Filesystem Abstraction
Adapter
App
Cluster administrator controlspolicy on all storage in cluster
UNIX UNIX UNIX UNIX UNIX UNIX UNIX
Workstations owners controlpolicy on each machine.
fileserver
fileserver
fileserver
fileserver
fileserver
fileserver
fileserver
UNIX UNIX UNIX UNIX UNIX UNIX UNIX
???Adapter
3PT
CFS: Central File SystemCFS: Central File System
fileserver
adapteradapter adapter
appl appl appl
file file
file
CFSCFSCFS
ptr ptr
ptr
DSFS: Dist. Shared File SystemDSFS: Dist. Shared File System
fileserver
appl appl
fileserver
fileserver
file file
filefilefile
file filefile
filefile
adapter adapterDSFSDSFS
lookupfile
location
accessdata
DSDB: Dist. Shared DatabaseDSDB: Dist. Shared Database
adapter adapter
appl appl
fileserver
fileserver
file file
filefilefile
file filefile
filefile
databaseserver
file index
query
directaccess
insert
create
file
DSDBDSDB
ApplicationsApplications
Simple and Secure Remote AccessSimple and Secure Remote Access– CDF: Remote Dynamic LinkingCDF: Remote Dynamic Linking– BaBar: Remote Database AccessBaBar: Remote Database Access– LHC: Semantic Remote FilesystemsLHC: Semantic Remote Filesystems
Distributed File SystemsDistributed File Systems– GRAND: Scalable Archive for Online DataGRAND: Scalable Archive for Online Data
Distributed DatabasesDistributed Databases– GEMS: Molecular Dynamics SimulationGEMS: Molecular Dynamics Simulation– CVRL: Biometric Image Storage/AnalysisCVRL: Biometric Image Storage/Analysis
Challenges of DecentralizationChallenges of Decentralization
Unbounded Set of UsersUnbounded Set of Users– There is no global /etc/passwd or /etc/group!There is no global /etc/passwd or /etc/group!
Multiple Identities per UserMultiple Identities per User– Kerberos creds from Notre Dame / Wisconsin.Kerberos creds from Notre Dame / Wisconsin.– GSI creds from ND/UW/DOE/NCSA.GSI creds from ND/UW/DOE/NCSA.
New Decision PointsNew Decision Points– Placement decision made, but action fails!Placement decision made, but action fails!– Directory op succeeds, but file creation fails!Directory op succeeds, but file creation fails!
Unexpected Policy CouplingUnexpected Policy Coupling– Data placement may affect access control!Data placement may affect access control!
Outline of PaperOutline of Paper
Centralized vs Decentralized SecurityCentralized vs Decentralized SecurityArchitecture of Cooperative StorageArchitecture of Cooperative StorageBasic Security MechanismBasic Security Mechanism– Problem: Complexity Confuses!Problem: Complexity Confuses!– Detail: Reservation RightDetail: Reservation Right
ChallengesChallenges– Authorization in Distributed File SystemsAuthorization in Distributed File Systems– Logistics of Third Party TransferLogistics of Third Party Transfer– Mechanisms for Active StorageMechanisms for Active Storage– Semantics of Distributed Group ManagementSemantics of Distributed Group Management
Basic Security MechanismBasic Security Mechanism
Negotiate an Authentication MethodNegotiate an Authentication Method– Client proposes, server agrees/disagrees.Client proposes, server agrees/disagrees.– Default ordering works for most + manual override.Default ordering works for most + manual override.– Different servers/clients may support diff subsets.Different servers/clients may support diff subsets.
Then, Authenticate via Chosen MethodThen, Authenticate via Chosen Method– May involve challenges, cert exchange, etc...May involve challenges, cert exchange, etc...
Yields a Subject Name for the Session:Yields a Subject Name for the Session:– kerberos:[email protected]:[email protected]– globus:/O=NotreDame/CN=DouglasThainglobus:/O=NotreDame/CN=DouglasThain– hostname:hedwig.cse.nd.eduhostname:hedwig.cse.nd.edu– unix:dthainunix:dthain
Authorization MechanismAuthorization Mechanism
Unix Access Controls Are Not SufficientUnix Access Controls Are Not Sufficient– Integer UIDs are not sufficient for principals.Integer UIDs are not sufficient for principals.– Nine owner/group/others bits are restrictive.Nine owner/group/others bits are restrictive.– Mapping from subjects to Unix is a mess.Mapping from subjects to Unix is a mess.
Place Variable Length ACLs on dirs:Place Variable Length ACLs on dirs:globus:/O=NotreDame/CN=DThain RWLAXglobus:/O=NotreDame/CN=DThain RWLAX
kerberos:[email protected] RWLkerberos:[email protected] RWL
hostname:*.cs.nd.edu RLhostname:*.cs.nd.edu RL
globus:/O=NotreDame/*globus:/O=NotreDame/* RL RL
Problem: Complexity Confuses!Problem: Complexity Confuses!
For beginning users:For beginning users:– Negotiated authentication makes life easy.Negotiated authentication makes life easy.– Everybody can authenticate in some way.Everybody can authenticate in some way.– Most users don’t think about it first.Most users don’t think about it first.
For advanced users:For advanced users:– Negotiation has unexpected effects.Negotiation has unexpected effects.– What happens when credentials expire?What happens when credentials expire?– For long running / large tasks, better to manually For long running / large tasks, better to manually
specify the authentication mode.specify the authentication mode.– AuthN failure is easier to retry than authZ failure!AuthN failure is easier to retry than authZ failure!
Unexpected authentication is hard to debug.Unexpected authentication is hard to debug.– Full detail logging mode reveals auth algorithm.Full detail logging mode reveals auth algorithm.– Always prominently display subject name in all tools!Always prominently display subject name in all tools!
Problem: Shared NamespaceProblem: Shared Namespacefile
server
globus:/O=NotreDame/* RWLAX
a.out
test.c test.dat
cms.exe
Solution: Reservation (V) RightSolution: Reservation (V) Rightfile
server
O=NotreDame/CN=* V(RWLA)
/O=NotreDame/CN=Monk RWLA
mkdir
a.outtest.c
/O=NotreDame/CN=Monk
mkdir
/O=NotreDame/CN=Ted RWLA
a.outtest.c
/O=NotreDame/CN=Tedmkdir only!
Outline of PaperOutline of Paper
Centralized vs Decentralized SecurityCentralized vs Decentralized SecurityArchitecture of Cooperative StorageArchitecture of Cooperative StorageBasic Security MechanismBasic Security Mechanism– Problem: Complexity Confuses!Problem: Complexity Confuses!– Detail: Reservation RightDetail: Reservation Right
ChallengesChallenges– Authorization in Distributed File SystemsAuthorization in Distributed File Systems– Logistics of Third Party TransferLogistics of Third Party Transfer– Mechanisms for Active StorageMechanisms for Active Storage– Semantics of Distributed Group ManagementSemantics of Distributed Group Management
ptr ptr
ptr
DSFS: Dist. Shared File SystemDSFS: Dist. Shared File System
fileserver
appl appl
fileserver
fileserver
file file
filefilefile
file filefile
filefile
adapter adapterDSFSDSFS
lookupfile
location
accessdata
DSFS LogisticsDSFS Logistics
Consider Creating a File:Consider Creating a File:– Fetch list of resources:Fetch list of resources:
online catalog / static list / user selectedonline catalog / static list / user selected
– Make placement decision:Make placement decision:random / fill in order / user selectedrandom / fill in order / user selected
– Create stub file on dir server. (fail?)Create stub file on dir server. (fail?)– Create actual file on data server. (fail?)Create actual file on data server. (fail?)
Note that two access controls are in play:Note that two access controls are in play:– One controls access to the namespace.One controls access to the namespace.– Another controls access to the data storage.Another controls access to the data storage.
DSFS ApplicationsDSFS Applications
Personal Mass StoragePersonal Mass Storage– Expand your local filesystem to include all the disks Expand your local filesystem to include all the disks
available in a cluster / lab / basement.available in a cluster / lab / basement.
Distributed /tmp for Cluster ComputingDistributed /tmp for Cluster Computing– Harness remote cluster for the duration of a job.Harness remote cluster for the duration of a job.
Multi-User Scalable StorageMulti-User Scalable Storage– Department provides directory, but no space.Department provides directory, but no space.
/O=NotreDame/O=CSE/CN=* RWL/O=NotreDame/O=CSE/CN=* RWL
– Participants provide their own data servers.Participants provide their own data servers./O=NotreDame/O=CSE/CN=JohnDoe RWLA/O=NotreDame/O=CSE/CN=JohnDoe RWLA
– Separates Separates provisioningprovisioning from from accessaccess!!
Dealing with FailureDealing with Failure
Failure to place data is very common!Failure to place data is very common!– Unexpected access controls on device.Unexpected access controls on device.– Device is temporarily unavailable. (reboot?)Device is temporarily unavailable. (reboot?)– Device is newly installed or creds expired.Device is newly installed or creds expired.– Owner changed the sharing policy.Owner changed the sharing policy.
Soln: Client Needs to Model the SystemSoln: Client Needs to Model the System– Track successes and failures on each device.Track successes and failures on each device.– Failed devices are not tried again for a time.Failed devices are not tried again for a time.– Of course, cannot avoid a device forever...Of course, cannot avoid a device forever...
Outline of PaperOutline of Paper
Centralized vs Decentralized SecurityCentralized vs Decentralized SecurityArchitecture of Cooperative StorageArchitecture of Cooperative StorageBasic Security MechanismBasic Security Mechanism– Problem: Complexity Confuses!Problem: Complexity Confuses!– Detail: Reservation RightDetail: Reservation Right
ChallengesChallenges– Authorization in Distributed File SystemsAuthorization in Distributed File Systems– Logistics of Third Party TransferLogistics of Third Party Transfer– Mechanisms for Active StorageMechanisms for Active Storage– Semantics of Distributed Group ManagementSemantics of Distributed Group Management
PINS: Processing in StoragePINS: Processing in Storage
Observation:Observation:– Traditional clusters separate CPU and storage into Traditional clusters separate CPU and storage into
two distinct systems/problems.two distinct systems/problems.– Distributed computing is always some direct Distributed computing is always some direct
combination of CPU and I/O needs.combination of CPU and I/O needs.
Idea: PINSIdea: PINS– Cluster HW is already a tighly integrated complex of Cluster HW is already a tighly integrated complex of
CPU and I/O. Make the SW reflect the HW.CPU and I/O. Make the SW reflect the HW.– Key: Always compute in the same place that the data Key: Always compute in the same place that the data
is located. Leave newly created data in place.is located. Leave newly created data in place.
Compute via Passive StorageCompute via Passive Storage
file server file server file server file server
A B C D (X 200)
S1 S2 S3 S4
Compute Y=F(X)where X={A,B,C,D} F Y1 Y2 Y3 Y4
Compute via Active StorageCompute via Active Storage
file server file server file server file server
A B C D (X 200)
S1 S2 S3 S4
Compute Y=F(X)where X={A,B,C,D} F
Y1 Y2 Y3 Y4
FFF
Technique: Identity BoxingTechnique: Identity Boxing
/ directory ACL:hostname:*.cse.nd.edu RWLXglobus:/O=NotreDame/* RWLX
fileserver
sim.exe
storage owner
in.datIdentity Box:/O=NotreDame/CN=Monk
sim.exeout.dat
client
> open x.nd.edu
> put sim.exe
> put in.dat
> exec sim.exe
> get out.dat
Unified SemanticsUnified Semantics
Same Identity for Exec and Data AccessSame Identity for Exec and Data Access– Stage in data as user X.Stage in data as user X.– Program runs as user X, data is protected.Program runs as user X, data is protected.– Access results as user X.Access results as user X.
Same ACLs for Exec and Data AccessSame ACLs for Exec and Data Access– Need the X right to run a program.Need the X right to run a program.– RX rights – given user can run fixed F(X).RX rights – given user can run fixed F(X).– WX rights – given user can stage in any F(X).WX rights – given user can stage in any F(X).
Outline of PaperOutline of Paper
Centralized vs Decentralized SecurityCentralized vs Decentralized SecurityArchitecture of Cooperative StorageArchitecture of Cooperative StorageBasic Security MechanismBasic Security Mechanism– Problem: Complexity Confuses!Problem: Complexity Confuses!– Detail: Reservation RightDetail: Reservation Right
ChallengesChallenges– Authorization in Distributed File SystemsAuthorization in Distributed File Systems– Logistics of Third Party TransferLogistics of Third Party Transfer– Mechanisms for Active StorageMechanisms for Active Storage– Semantics of Distributed Group ManagementSemantics of Distributed Group Management
Fully Decentralized User GroupsFully Decentralized User Groups
Distributed Orgs Have Complex NeedsDistributed Orgs Have Complex Needs– CMS Collaboration: 10s of institutions, 100s of PIs, CMS Collaboration: 10s of institutions, 100s of PIs,
1000s of graduate students staff.1000s of graduate students staff.– There is no centralized database for CMS.There is no centralized database for CMS.– Local managers add/remove members locally.Local managers add/remove members locally.
Want Storage Systems that Allow Reference to Want Storage Systems that Allow Reference to Groups Managed by Others:Groups Managed by Others:– Allow access to all staff involved in CMS.Allow access to all staff involved in CMS.– Allow access to any NSF program manager.Allow access to any NSF program manager.– Allow access to all CS faculty at ND/Purdue.Allow access to all CS faculty at ND/Purdue.
Fully Decentralized ACLsFully Decentralized ACLs
univ.edu
members
univ.edu
members
filesystem
fileserver
file
client
read
Access Control Listgroup:ccl.nd.edu/faculty RWLgroup:serv.nsf.gov/managers RLgroup:ftp.cern.org/members RL
checkACL
ccl.nd.edu serv.nsf.gov
faculty managers
ftp.cern.org
members
group lookups
group lookups
Challenges of ACLsChallenges of ACLsPerformance / Availability / ConsistencyPerformance / Availability / Consistency– Give the group/ACL owner control.Give the group/ACL owner control.– Specify maximum time for stale data.Specify maximum time for stale data.
Implemented, but continuing experience leads to Implemented, but continuing experience leads to reflection on the semantics.reflection on the semantics.Example: What to do under failures?Example: What to do under failures?– Partial answer: servers fail quickly, client retries up to Partial answer: servers fail quickly, client retries up to
a user-controlled limit.a user-controlled limit.– Consider: Group A gives W access, group B gives R.Consider: Group A gives W access, group B gives R.– What happens when group A is unavailable?What happens when group A is unavailable?– Two very different questions:Two very different questions:
What rights does user X have?What rights does user X have?Can user X perform a read? Can user X perform a read?
Outline of PaperOutline of Paper
Centralized vs Decentralized SecurityCentralized vs Decentralized SecurityArchitecture of Cooperative StorageArchitecture of Cooperative StorageBasic Security MechanismBasic Security Mechanism– Problem: Complexity Confuses!Problem: Complexity Confuses!– Detail: Reservation RightDetail: Reservation Right
ChallengesChallenges– Authorization in Distributed File SystemsAuthorization in Distributed File Systems– Logistics of Third Party TransferLogistics of Third Party Transfer– Mechanisms for Active StorageMechanisms for Active Storage– Semantics of Distributed Group ManagementSemantics of Distributed Group Management
Practical LessonsPractical Lessons
In a system with decentralized security...In a system with decentralized security...
Users need debugging tools!Users need debugging tools!– Simple examples: whoami, rwhoamiSimple examples: whoami, rwhoami
Client software must become “heavier”Client software must become “heavier”– Must carefully parse a vast array of errors.Must carefully parse a vast array of errors.– Must maintain a model of remote devices.Must maintain a model of remote devices.
High level names must be used deep within the High level names must be used deep within the system software stack.system software stack.– Run processes with subject name, not Unix UID.Run processes with subject name, not Unix UID.
For more information...For more information...
Cooperative Computing LabCooperative Computing Lab
http://www.cse.nd.edu/~cclhttp://www.cse.nd.edu/~ccl
Cooperative Computing ToolsCooperative Computing Tools
http://http://www.cctools.orgwww.cctools.org
Douglas ThainDouglas Thain– [email protected]@cse.nd.edu– http://http://www.cse.nd.edu/~dthainwww.cse.nd.edu/~dthain