Embed Size (px)
DESCRIPTION
Citation preview
Beolink.org
Introduction to OpenAFS
Fabrizio Manfred Furuholmen
Beolink.org
09/04/2023
2
Agenda
Introduction
Architecture
Client
Administration tasks
Setup your Cell
Beolink.org
09/04/2023
3
Introduction
What is a Distributed File system ?
“A distributed file system takes advantage of the
interconnected nature of the network by storing
files on more than one computer in the network
and making them accessible to all of them..”
Beolink.org
Andrew File System is a distributed file system
designed to:
handle terabytes of data handle thousands of users working in WAN environment
Andrew File System
Introduction
4
Beolink.org
AFS is 25 years old !
Introduction
5
Beolink.org
Brief history of a AFS
1983 Andrew Project started at Carnegie Mellon University (CMU) 1987 Coda research work begun (based on AFS) 1988 First use of AFS version 3 (First use of AFS outside CMU) 1988 Institutional File System project at University of Michigan 1989 Transarc Corporation founded to commercialize AFS 1993 Arla project started at Kungliga Tekniska Högskolan 1998 Transarc Corporation becomes wholly owned subsidiary of IBM 2000 IBM releases OpenAFS as OpenSource (IBM License) 2000 OpenAFS release version 1.0 based on Transarc 3.6 2001 OpenAFS release version 1.2 first release with better support of new
operating system and fix several memory leak 2005 OpenAFS release version 1.4 with a lot of new feature 2005 AFS was discontinued from IBM 2008 U.S. Department of Energy Funds OpenAFS Development 2010 OpenAFS release version 1.6 (?)
Introduction
6
Beolink.org
Location independenceUser does not need to know which fileserver holds the file, the user only needs to know the pathname of a file.
ScalabilityAn architectural goal of the AFS designers was client/server ratios of 114.000:1 A ratio of 2000:1 has been successfully exceeded at some sites.
SecurityAFS makes use of Kerberos for mutual authentication, both the service provider and the requester prove their identitiesAFS uses access control list (ACLs) to enable users to restrict access to their own directories, users can also create groupsAFS Federation with inter cell grant
Uniform NamespaceNo matter where users are logged in, they see the same files
Replicates AFS VolumesFrequently accessed data can be read-only replicated on several servers (rw with osd version). Client will access the closest volume copy or load balance from a different replica
7
Benefits of using AFS
Beolink.org
Improved robustness to server crashClients maintain Local copies of accessed files, replicated read-only volumes on alternate fileservers can satisfy requests for a files
Wide Area NetworkAFS communications protocols is optimized for WAN. Retransmitting only the single bad packet in a batch of packets (RPC)
Improve system management capabilityConfiguration changes can be made from any client in the AFS cellAFS volumes can move from one server to another without users noticing it
Operating system independentAFS client software runs on many systems (12 platforms)
8
Benefits of using AFS
Beolink.orgElements
Server A
Server A+B
Server C
9
Beolink.org
09/04/2023
10
Architecture
Beolink.org
“..That notion of callbacks gives OpenAFS a much stronger consistency guarantee than most other distributed filesystems.”
Cache ManagerClient-side caching lets clients access data from their local cache without going across the network for every access.
CallbacksOpenAFS uses callbacks, which are a promise from the file server to the client that if the file changes, the server will contact the client to tell the client to invalidate the cached contents.
Consistency
11
Beolink.org
Example write operation client side
1. create file rpc
2. write chunks into cache (interrupted by store_data RPC)
3. read from cache
4. transfer over network
5. write to /vicepXX
Write operation
12
Beolink.org
Example write operation server side
1 Create file
2 Check metadata, permission, quota and return file path
3 write file into /vicepXX
4 Update meta data on server
5 Update db
Write operation
13
Beolink.org Client side
Client
14
Beolink.org
AIX 5 and 6 (though 6.3) FreeBSD 7, 8 and current HP-UX 11.0, 11i v1 and v2 Irix 6.5 Linux 2.2, 2.4, 2.6 (ia32, ia64, x86_64, ppc, ppc64, arm,
sparc, sparc64, s390, s390x) MacOS 10.3, 10.4, 10.5, 10.6 (including 64 bit). OpenBSD 4.4, 4.5, 4.6, 4.7. Solaris 2.6, 7, 8, 9, 10, 11 (and OpenSolaris) Also Windows ...
Installation
Supported clients
15
Beolink.org
Download and install client package and kernel module
Configure krb5 if you use it
Configure AFS Files ThisCell : the name of your cell CellServDB : cell list ( of the world) cacheinfo : cache configuration (dimension and location)
Installation
Configuration
16
Beolink.org
Authentication Kerberos 5 kinit, retrieve a kerberos ticket aklog, convert the krb5 ticket in afs token
Authentication Kaserver klog, retrieve a afs token
Token operations klist, list tikets
tokens, list afs token
kdestroy, ticket destroy unlog, token destroy
Don’t Forget Credentials expire after some time AFS service ticket is in the kernel memory
Authentication
17
Ticket cache: FILE:/tmp/krb5cc_0Default principal: manfred/[email protected]
Valid starting Expires Service principal08/16/10 16:03:46 08/17/10 16:03:46 krbtgt/[email protected]/16/10 16:03:54 08/17/10 16:03:46 afs/[email protected]
Tokens held by the Cache Manager:
User's (AFS ID 15) tokens for [email protected] [Expires Aug 17 16:03]
Beolink.org
ACLs are only for directories ! (Files soon)
ACL inheritance, AFS copies ACL on a parent directory over to a new subdirectory at the time of creation
ACL awareness, not many commands are aware of ACLs (copy)
Access rights
ACL Permission
lookup (l) List contents of directory
insert (i) Add Files or directories
delete (d) Delete entries in directory
administer (a) Manipulate ACL for directory
read (r) Read file content, query file status
write (r) Write file content, change Unix permissions
look (k) Full file advisory lock
18
Beolink.org Server side
Servers
19
Beolink.org
09/04/2023
20
Architecture
Beolink.org Process
21
Server Process Function
bosserver Basic OverSeer Server
fileserver Serves the files
volserver Serves volume data
vlserver Volume location server
ptserver Protection server
buserver Backup server
upserver Update server
upclient Update client
Beolink.org
Fileserver machine file storage
Database server machine File and Volume localization Groups administration Authentication provider Backup database
Binary distribution Master server for afs binary (specific
architecture)
System control machine Time server AFS configuration master
AFS Servers Types
Architecture
Beolink.org Commands
Administration Commands
23
Beolink.org
fs command
Cache management administration Quota management ACLs management Mount management on the AFS path
FS
24
fs: Commands are:apropos search by help textcheckservers check local cell's serverscheckvolumes check volumeID/name mappingscleanacl clean up access control listcopyacl copy access control listdiskfree show server disk space usageexamine display file/volume statusexportafs enable/disable translators to AFSflush flush file from cacheflushmount flush mount symlink from cacheflushvolume flush all data in volumegetcacheparms get cache usage infogetcalleraccess list callers accessgetcellstatus get cell statusgetclientaddrs get client network interface addressesgetcrypt get cache manager encryption flaggetfid get fid for file(s)getserverprefs get server rankshelp get help on commandslistacl list access control listlistaliases list configured cell aliaseslistcells list configured cellslistquota list volume quotalsmount list mount pointmessages control Cache Manager messagesmkmount make mount pointnewalias configure new cell aliasnewcell configure new cellquota show volume quota usagermmount remove mount pointrxstatpeer Manage per peer RX statisticsrxstatproc Manage per process RX statisticssetacl set access control listsetcachesize set cache size
setcachesize set cache sizesetcbaddr configure callback connection addresssetcell set cell statussetclientaddrs set client network interface addressessetcrypt set cache manager encryption flagsetquota set volume quotasetserverprefs set server rankssetvol set volume statusstorebehind store to server after file closesysname get/set sysname (i.e. @sys) valueuuid manage the UUID for the cache managerwhereis list file's locationwhichcell list file's cellwscell list workstation's cell
Beolink.org
BOS Command
Process creation Process administration (start, stop, status ...)
Manage Users Administrator for process
Volume check
Administration
25
Beolink.org
VOS Command
Create volume Volume Replication Volume Information
Move volume
Administration
26
DON’T FORGET THERE ARE DIFFERENT PATH FOR RW and R0
RW = /afs/cell/.mount_pointRO = /afs/cell/mount_point
Beolink.org
PTS Command
Create id for users and groups Users have a positive number Groups use negative number
Management of Group/User membership
Management metadata (group quota, flags)
IMPORTANT Users can create their own groups Special groups
system:anyuser system:authuser system:administrators
Administration
27
Beolink.org Limits
Limits
28
Beolink.org
General Limits
OpenAFS can support a maximum of 114.000 clients per server
tmpfs no work as AFS Cache, (ramdisk work) Max 255 partition per server (/vicepa-/vicepiv), no limits in
partition size Max 4,294,967,295 volumes per partition (this a limit of VLDB), Max file limit per directory is 64,000 files (less than 16
characters).
Windows Limits
No integration on Microsoft DFS No native implementation
Limits
29
Beolink.org
AFS does not allow certain type of files: Pipes Device files Socket
AFS cannot do byte range locking on all platform Client has working byte rage locks Full file locks on the server
ACLs works on directories not files (yes)
AFS does not support mandatory file locks
DES Encryption for file transport
Limits
30
Beolink.org
AFS is not so well suited for these situation
No reuse of read data
Access to file larger than cache
Mostly write access
Larger numbers of directory entry changes from multiple clients
Weaknesses
31
Beolink.org
Full circle
Introduction
32
Storage is cheap.
Managing storage is more expensive.
Wide access to data is still critical.
Today and into the future.
Beolink.org Thanks to...
Alf Watchsmann for usage of “Introduction to AFS and its Best Practices” Please read the original presentation for a complete overviewhttp://workshop.openafs.org/afsbpw10/
For more information read Documentation on www.openafs.org
Other presentation are available on www.beolink.org
33
Beolink.orgI look forward to meeting you…
XVII European AFS meeting 2011 HAMBURG – GERMANY
Who should attend: Everyone interested in deploying a globally accessible
file system Everyone interested in learning more about real world
usage of Kerberos authentication in single realm and federated single sign-on environments
Everyone who wants to share their knowledge and experience with other members of the AFS and Kerberos communities
Everyone who wants to find out the latest developments affecting AFS and Kerberos
More Info: http://www.openafs.org/
09/04/2023
34
