Permissions Mapping in the Isilon OneFS File System

Storage Developer Conference 2009 © 2009 Isilon Systems. All rights reserved.

Permissions Mapping in the Isilon OneFS File System

NTFS ACLs, NFSv4 ACLs, and POSIX Mode BitsSteven Danneman and Zack Kirsch


Agenda

What is OneFS?POSIX, NTFS and NFSv4 Permission OverviewIsilon’s Permission Implementation

SettingRetrievalEnforcement

Advanced Permission ImplementationSpecial IdentitiesInheritanceCanonical Order

2


Isilon OneFS Cluster

NAS file serverScalable

Add more storage in 5 minsReliable

8x mirror / +4 parityStriped across nodes

Single volume file system (5.2 PB)3 to 144 nodesFully symmetric peers

No metadata servers Commodity hardware

CPU, Mem, Disks (12 to 36)

3


Isilon OneFS File System

Concurrent access to all files with all protocols

CIFS/SMBNFSv3SSHHTTP/FTP

Coming SoonNFSv4SMB2

4


Permission Basics

5


Unix Permissions

Mode bitsrwxrwxrwxRead / Write / ExecuteOwner / Group / Other

POSIX ACLsGive rwx permission to other users & groupsCloser to NTFS ACLs, but less expressiveReplaced in OneFS by NTFS ACLs

6


NTFS Access Control List

Approximately 15 rights vs 3 rwx rights.Security Descriptor (SD)

Owner, GroupDiscretionary ACL (ACL)

List of Access Control Entries (ACE)

System ACLACE

User / Group Identifier (UID/GID in OneFS)Allow & DenyList of rightsInheritance

7


Comparison

POSIX modes are a complete subset of NTFS rightsMinus the top 3 bits

SetUID, SetGID, Sticky

Order of enforcement is differentPOSIX

1. Determine identity2. Check 1 of 3 possible lists

NTFS1. Determine identity2. Check 1 list

8


Permission Modification

POSIX semantics:chmod: Only owner/rootchown: Only rootchgrp: Only owner/root, only to groups they are part of

NTFS semantics:chmod: Needs WRITE_DAC; owner can always change permissionschown: Needs WRITE_OWNER; cannot give away a filechgrp: Needs WRITE_OWNER; can change to any group

OneFS: Global Policy dictates behavior regardless of protocol

9


NFSv4 Access Control List

Small DifferencesUses principals instead of IDs, e.g. “user@domain”uid/gid allowed for backwards compatibility

New RightsACE4_WRITE_RETENTION / ACE4_WRITE_RETENTION_HOLDMappable to ACE4_WRITE_ATTRIBUTES

Mostly Identical to NTFS ACL

10


Isilon Implementation

11


Design Goals

Store one authoritative set of permissions per filePreference NTFS ACL over mode bits

Enforce identical permissions for all protocolsProvide view of alternate permission type:

NFS is returned approximated mode bitsSMB is returned a SYNTHETIC ACL

Provide configuration through global permission policyExtend standard Unix tools for all permission management

ls, chmod, chown, chgrp

12


SMB File Creation

Store ACL1) SD sent with create : Store provided ACL2) Inheritable ACL exists on parent : Store Inherited ACL3) No Inheritable ACL exists : Store Default ACL

Store approximated mode bitsGive NFS clients a view of the permissionsStored mode bits are not used for enforcementPermissive enough to trick client access evaluation

13


NFS File Creation

No inheritable ACL existsStore mode bits only

Inheritable ACL exists on parentApply inheritable ACL onlyStore approximated mode bits

14


Permissions Setting

chmod w/ ACL (SMB or local)Store ACLStore approximated mode bits

chmod w/ mode bits (NFS or local)No ACL exists

Store mode bits

ACL existsMerge mode bits with ACL

Add/modify ACEs for three identities: owner, group, everyoneLeave other identities unchangedAdd deny ACEs for bits that are not presentInheritance hierarchy remains

15


Permissions Retrieval

SMBIf ACL, ACL is returnedIf mode bits, return SYNTHETIC ACL

Not stored on disk, translated on demand

NFSAlways show stored mode bits

16


Basic Permission Enforcement

Goal: Enforce the same access on all files, from all protocols.

SMB access on file with ACL Scan through ACL, until desired rights are allowed or denied

NFS access on file with mode bitsSimple comparison against owner, group or other

Algorithm:1. Convert desired rights / access mask to file’s permission type2. Basic permission enforcement

17


Advanced Enforcement I

SMB access on file with mode bitsConvert desired rights to Unix permissions

List Folder -> Unix READCreate Files or Create Folders or Delete Subfolders/Files -> Unix WRITETraverse Folder -> Unix EXECUTE

Change Permissions, Take Ownership and Delete do not mapACL Policy: rwx = Full Control

18


Advanced Enforcement II

NFS/Local access on file with ACLConvert desired access mask to ACL rights

Unix READ -> List FolderUnix WRITE -> Create Files AND Create Folders AND Delete Subfolders/FilesUnix EXECUTE -> Traverse Folder

NFS Server uses Windows rightsE.g. Asks for Create Files access instead of WRITE access

NFS Access Request needs approximationUnix WRITE -> Create Files OR Create Folders OR Delete Subfolders/Files

19


Advanced Implementation

20


SMB Special Identities

Changed UID/GID to struct identityType / ID

EveryoneNull

Used only for owner or groupGroup owner

Used only for owner attributeCREATOR OWNER / CREATOR GROUP

Inherit_only ACE on directory

21


Inheritance

Auto Inheritance vs. Dynamic inheritanceAuto - provide client with info to propagate ACLsDynamic - file system handles ACL propagation

Necessary for local inheritance propagation

22

1

2

3

Auto Inheritance

1

2

3

Dynamic Inheritance


Canonical Order

Canonical order:Explicit DenyExplicit AllowInherited DenyInherited Allow

Enforced by Windows GUIMoves deny ACEs up to the top

Windows API allows setting ACEs in any order

Problem: Out of order ACLs are necessary to represent POSIX ACLsr-- : Allow read, deny write, deny execute

23


# chmod 754 file.txt# ls -le file.txt-rwxr-xr-- 1 test-user test-group 0 Sep 1 02:04 file.txtSYNTHETIC ACL0: user:test-user allow full_control1: group:test-group allow read, execute2: group:test-group deny write3: everyone allow read4: everyone deny write, execute

Canonical Order - Example 1

24

Mode 754 with deny ACEsSimplified output:


# ls -le file.txt-r-xr-xr-x 1 test-user test-group 0 Sep 1 02:04 file.txt0: group:test-group deny write1: everyone deny write 2: user:test-user allow full_control3: group:test-group allow read, execute4: everyone allow read, execute


25

After adding “execute” rights for Everyone via Windows GUI:Mode changed from 754 to 555, instead of 755


# chmod 754 file.txt# ls -le file.txt-rwxr-xr-- 1 test-user test-group 0 Sep 1 02:04 file.txtSYNTHETIC ACL0: user:test-user allow full_control1: group:test-group allow generic_read, generic_execute2: everyone allow generic_read


26

Mode 754 without deny ACEsSimplified output:

Configurable ACL policies for dealing with deny ACEs


Configurable Permission Policies

Disallow ACL creationDisallow chmod from NFS

Chown: Modify the owner/group permissions?

Owning group on file creationBSD -> parent folder’s owning groupWindows/Linux -> user’s primary GID

27


Lessons Learned

Mixed permissions are challenging, but possible

Some decisions must be left up to policy

Best practice is to choose a default and document

Call to Arms: ACL Interop Spec

28


Questions?

Zack [email protected]

Steven [email protected]

29

Documents

Permissions Mapping in the Isilon OneFS File System