Coda File System.ppt

8/14/2019 Coda File System.ppt

1/45

The Coda File System

Imagine, if you can, a network filesystem so realiable that the clients

keep working even if the server falls

over. Imagine using it at home without

dialling up, or connecting once a day

for a minute to syncronise with the

server. That's Coda; not vapourware,

but something that exists now.

I'll discuss the Coda logo later.


2/45

Understanding Coda

Should I use it?

How does it work?

How do I set it up?

How do I manage it?

After reading this, I hope you can answer the question `Should I use Coda?' And to help you answer it, I'll try toexplain what it does and how it works, what's involved in setting it up, and what's involved in keeping it going.

I'll begin with an overview of Coda, continue with a deeper look at certain points, and then give a summary and anevaluation of Coda and its current state of development and usefulness.


3/45

Concepts and Features

We'll start with a little history, some concepts,

and a quick look at a few big features of Coda.


4/45

Network File Systems

NFS

AFS and relatives

Sprite, GFS, Novell, Samba, and others

I've found it surprisingly difficult to pin down the early history of distributed file systems. The firstreference I can find to Sun's NFS is a paper presented in 1985, when the software was alreadyworking and deployed. A few years later, when we started buying Hewlett Packard workstationsin Manchester, Sun was urging other Unix vendors to include NFS with their Operating Systems,

and HP were encouraging their customers to switch to NFS from the HP system, which I nevertried myself. (It was called Remote File Access, or something like that.) The success of NFS is oneexample of the success of open standards in software.

In the same year 1985 I find the first paper referring to the AFS file system, the immediateancestor of Coda and several other closely related file systems. AFS was developed (under adifferent name) at Carnegie Mellon University in Pittsburgh, and soon after 1985 its name waschanged to 'the Andrews File System' (after Andrew Carnegie and Andrew Mellon, the twopatrons of Carnegie Mellon University). CMU spawned the Transarc Company to develop andmarket AFS.

I don't know exactly when the other network file systems arose, but I doubt any of them wereearlier than NFS or AFS; some of them of course arrived on the scene much later.


5/45

AFS and its relatives

AFS

Arla

Coda

DFS

Coda is one of several relatives of AFS, which resemble each other in many respects, and which differ from other shared files systems.

AFS itself has been a commercial product since about 1987. It still exists, and it got a new lease on life when IBM bought Transarc. AFS is very welldocumented, and the documents are all available free on line; it's also discussed in a number of books about file systems. If you're interested inCoda, some of this information may be relevant. I'll say something about Coda's own documentation later.

The Arla projectat the Royal Technical Hochschule (KTH) in Stockholm is an attempt to create a free clone of AFS that interoperates with it. Arlahas concentrated on duplicating the AFS client software and the user interface, though some work is now being done on Arla servers. This projectis being encouraged by IBM and Transarc, who have donated some code to it, and some of the early AFS developers are helping out in small ways,

at least. Coda is a research project that began at CMU about 1987. It was designed as an AFS-like file system which supports mobile computing, and which

is more robust than AFS when it faces network problems and server failures. A paper which appeared in 1987 says that the design of Coda iscomplete, but that no code has yet been written. The file system has been developed at CMU since then by the systems group of M.Satyanarayanan.

DFS is a file system that is based on AFS and was originally intended to supersede it. Several major computer manufacturers, including IBM, DEC,and HP, put together a 'Distributed Computing Environment' which they hoped would become a sort of industry standard; the 'Common DesktopEnvironment' CDE is perhaps the most successful piece of DCE. DFS itself has never really caught on, partially because of some early difficulties ingetting it working, and partially, in my view, because of the excessive licence fees.

I have had some experience with all four of these systems, though I have seen relatively little of DFS, because we in Manchester decided not tobuy it.

These file systems are quite different from NFS or the other systems I have named. In some ways, the structure of data in AFS and DFS is managed

globally, so that all AFS or DFS users in the world see the same file system. If I make a change to my AFS cell, it is visible immediately not only toall the machines in my cell, but to everyone else as well. Coda lacks the global aspect; so far as I can see, I can't make two Coda cells visible on oneclient (at least not without running two independent Coda client processes for two different mount points). But Coda management is cell-based,like that of AFS and DFS. In other words, if I mount something in the file system, it immediately appears everywhere in the cell.
http://www.stacken.kth.se/project/arla/http://www.stacken.kth.se/project/arla/http://www.stacken.kth.se/project/arla/http://www.stacken.kth.se/project/arla/


6/45

Kinds of machine

Clients

File and Data Servers

System Control Machine (SCM)

Backup machines

A cell contains several kinds of machine, including, of course, the clients. A client is any machinewhich can see a cell's Coda file system.

In addition to clients, there is another kind of machine which we may call a file and data server,

since Coda deals with a lot of data besides just files. One of the file and data servers plays aspecial role in a cell: the System Control Machine, or SCM. It is the master server, and needs to beinstalled first. And if you plan to backup your Coda system, you will need a backup machine.

A single machine may perform all four functions, so that you can have a cell with only onemachine in it. Servers do not need to be clients in a Coda cell, nor do backup machines. But youwill need more than one file and data server to take advantage of Coda's best features.

The organisation of a Coda cell has some implications for security: you need to protect theintegrity of your file and data servers. In general, a hacker who breaks into a Coda client cannot

compromise the security of the servers. Of course, if he lurks undetected, with root access, theblackguard may learn something compromising from another user of the cell.


7/45

Clients: the Cache

In (virtual) memory or on disk

100 - 200 MB?

Cache manager (venus)

A client machine has an area called `the cache', where file system data staystemporarily. This cache is normally on disk, though in AFS at least it can be invirtual memory. The advantage of having all (or part) of your cache in memory is

that, under optimal conditions, you can read files and directories faster than ifthey were on local disks. (The Linux buffer cache has something of the sameeffect.)

The cache needs to be big enough to hold the biggest file you wish to read orwrite, as well as some other data. So if I have a cache of 200mb, I won't be ableto read a file larger than this. Coda documents recommend a cache of about20mb, but I prefer to use a much larger one. It depends on the specific use you

make of Coda. AFS and, I think, DFS allow you to cache only parts of a large file,and so get round the size limitation, though with some problems.

A local process called the cache manager manages the data in the cache; on Codaclients it is named `venus'.


8/45

Cache manager (venus)

Program calls open()

Kernel VFS layer

Kernel Coda module

Venus process

Contact file server

Return location of data to kernel

Suppose a program running on my machine tries to open a file in the Coda file system. Theprogram calls the function open(), which contacts the kernel's virtual file system layer. Thisnotices that the file is in the Coda file system, and passes the request to the kernel's Codamodule. This contacts the venus process through a character device in /dev.

The venus process is responsible for getting the requested data into the cache, and for makingsure it is up-to-date. It may contact the file server to do this, and it may need to flush older dataout of the cache to make room for the file. If it already has a copy of the file locally, it may ask thefile server whether this is current, to avoid having to recopy it. Files can persist in the cache evenacross reboots, and of course the cache manager doesn't contact the server during disconnected

operation. Incidentally, if you have read or written a file, and you want to be sure it isn't in the cache for

security reasons, you can force the cache manager to get rid of it; but the data in the file willprobably remain on disk.

When the cache manager has a current copy of the file in the cache, it returns this information tothe kernel, which then opens it like any other file.


9/45

Volumes

Virtual partition

Mounted globally

Quota

Cloning

Different kinds of volume

The building block of a Coda file system is the volume. A volume issomething like a virtual partition. It may exist physically on one server, orit may have several identical copies on two or more servers. It must bemounted cell-wide if it is to be read or written, but it can be backed upwithout being mounted. Quota is managed by setting a limit on eachvolume. A volume may have no quota set, in which case it can grow until

one of the physical partitions where it lives fills up. Coda does something to volumes called cloning, which we need to look

at more closely. Also there are several different kinds of volume.


10/45

Cloning

Like fork: copy on write

Occupies same space (initially)

Like a collection of hard links

One copy is read-only

Helps backup

Used to replicate (copy) volumes

Cloning does to a volume something like what fork does to a Unix process. The original volumeand the clone share the same disk space, which in the clone is read-only, and in the originalvolume is copy-on-write. The original and its clone are rather like two identical collections of hardlinks to the same files. When you change a file in the volume, the link to the read-only copy ofthat file is broken, and a new file is created. So the clone is like a backup of a volume at the timeit was taken. It requires very little disk space unless the volume changes substantially.

The clone of a volume can itself serve as a limited backup of the volume. In one of our cells, weclone all user volumes every night, and if you accidentally delete or mangle a file, you can read orcopy the file from the clone without bothering the system administrator. When you backup a

Coda system, you don't need to stop using it; instead, you clone each volume to be backed up,and you dump the clone to tape, or to a staging disk from which you can do something else withit.

Cloning was used to produce read-only replicated volumes, but Coda seems to be droppingsupport for these.


11/45

Types of volume

Simple (read-write)

Read-only (e.g., backup)

Replicated (read-only)

Replicated (read-write) (Coda only)

We have already seen two types of volume, a simple read-write volume, and a read-only volumesuch as a clone. When you create a full dump of a Coda volume, you can restore this as a read-only volume. Read-only volumes are also useful because you can make identical copies on several

servers, and access to these read-only replicas may be much more efficient than access to a read-write volume. A replicated volume is fully usable as long as at least one running server has a copyof it; this may improve the availability of the data. (Coda seems to be dropping support for read-only replicas. It seems that a slight gain in efficiency is offset by the increased complexity of thecode.)

Besides volumes of these kinds, which all Coda's cousins support, Coda also has read-writereplicas. These are convenient because they give the advantage of high availability, thoughwriting to them may be less efficient than writing to simple read-write volumes. Coda classessome volumes as read-write replicas even though they exist on only one server. Read-write

replicas, whether they live on one server or on several, are the normal kind of volume in Codacells, and you should plan your cell structure with this in mind.

Replication is something like having a highly configurable, flexible RAID system in software. It istrue that there is some management overhead, but we find that hardware RAID systems can bedown for a long time if the controller breaks, which has happened to us several times now.


12/45

Disconnected operation

Mobile computing

Accidental disconnection

High availability

Deliberate disconnection

Hoarding (selection and priority)

Reconnecting

Fixing conflicts

The idea of disconnected operation is central to Coda, and was part of its design from thebeginning; the earliest Coda papers speak of `mobile computing'. A client may lose contact withthe cell's servers accidentally, whether through network failure or because servers go down, ordeliberately, because you are removing the client, or because networking over a phone line is tooexpensive to do continuously.

Besides files that may happen to be in the cache, Coda lets you select portions of the file systemwhich the cache manager tries (if possible) to `hoard'; that is, to keep current. You can setpriorities to guide the cache manager in this. Reconnecting is easy if you haven't changed

anything. If you have, if you created or edited files while the system was disconnected, you needto tell the system to reintegrate your changes.

Of course, it is possible while your system is disconnected, for two people to change the same fileor directory independently. In that case, Coda provides a mechanism for comparing the differentversions, harmonising them, and resolving the conflict.


13/45

Coda Logo

Artist Gaich Muramatsu

The Japanese artist Gaich Muramatsu has created several entertaining drawings about Coda, which illustrate its

various features. The Coda logo derives from a pair of pictures which contrast AFS users, who are unhappy

when they lose their network connections, and Coda users, who continue to work without a problem in similar

circumstances.

What happens with AFSWhat happens with Coda


14/45

Supported Platforms

Linux (all platforms?)

FreeBSD

NetBSD (all platforms?)

Solaris (sparc and i386)

Windows 95/98

NT (Coda server only)

Coda runs under Linux on all platforms, though I suspect it has been well testedonly on the i386 and perhaps the sparc Linux. It was developed originally on BSDsystems based on the Mach microkernel, so it has been ported to FreeBSD andNetBSD; in the latter case again I suspect it hasn't been tested as thoroughly onsome of the less common platforms.

Recently Solaris ports were announced, for the sparc and i386 platforms; these

are fairly new, and not as well-used as the others. Coda can be compiled in the CygWin environment from Cygnus, and the resulting

client and server does work under Windows 95 and 98, though there are knownto be some problems with this. The Coda server works under NT as well, thoughthe Coda client does not.


15/45

Features

I'd like to look next at some of Coda's

features, particularly those in which it differs

significantly from NFS. These cause some

slight limitations, which you need to know ifyou plan to use Coda.


16/45

Users and Groups

Internal to Coda

pdbtool

groups have owners

users or groups may own groups

owners should be able to manage their groups

but in Coda this does not work

Coda has its own users and groups. These are internal to Coda; they are not thesame as Unix users or groups. Coda users correspond to positive integers, whilegroups correspond to negative integers. Your life will be easier if most Coda usersand Unix users have the same names and ID numbers.

The program `pdbtool' manages users and groups. It must be run by root on theSystem Control Machine.

Coda groups have owners, and both users and groups may own groups. UnderAFS and DFS, any user may create a group, delete the groups he owns, and addother users to his groups. None of this works in a Coda cell, and this is in myopinion something that needs to be fixed. At present, a cell with many users willprobably be more of a burden to its administrator because of this.


17/45

Passwords and Authentication

Choice of three systems:

Coda internal system

Kerberos 4

Kerberos 5

MIT Kerberos and KTH Kerberos supported

When you set up a Coda cell, you can choose from one of several authenticationsystems: Coda's internal system, Kerberos 4, or Kerberos 5. Coda's currentinternal authentication system is very weak and should not be used in aproduction cell. Coda can use both authentic MIT Kerberos or the versions ofKerberos 4 and 5 available from KTH in Stockholm, the same place where theArla project is being developed.

If you use KTH Kerberos, you need to make some small changes to the Codasource if you compile it yourself. Patches are available for this; you'll need tochange a couple of Makefiles and about 11 lines in one source file.

Once you have a Kerberos ticket, the program 'kclog' gets a Coda token for youwithout further interaction, so it could in principle be placed in the profile of aUnix machine which uses Kerberos authentication in its login procedure.


18/45

Access Control Lists

On Directories

Not on Files

Give permissions to Coda users and groups

Effect modified by mode bits on files

Hard links allowed only in the same directory

Permissions may be positive or negative

A major feature of the way objects are protected in Coda is the use of Access Control Lists on

directories. An Access Control List, or ACL, is attached to each directory in Coda, and determines to a

considerable extent who can do what in that directory. You cannot attach a Coda ACL to a file,unlike DFS ACLs. Each item in an ACL gives or takes away some permission to or from a Coda useror group. As we shall see, the effect of a directory ACL is modified slightly by the mode bits onfiles in that directory.

Because the ACL on a directory has much more influence on its contents than Unix directorymodes have ordinarily, Coda does not allow you to make a hard link in one directory to a file in

another directory. This occasionally may cause problems when you extract tar files inside Coda,or when some Makefile assumes you can create hard links between files in different directories.

Coda ACLs are not the same as Posix ACLs. Also, some Unix programs may have problems on aCoda client if they make 'illegitimate' use of mode bits instead of calling access() to determinewhat they are able to do.


19/45

Access Control

R

Read files in this directory

L

Attach to and list this directory

I

Insert a file into this directory

D

Delete a file from this directory

K

Lock files in this directory (not Coda)

W

Write or append to files in this directory

A

Change ACLs on this directory

These are the seven access permissions which may appear in an ACL. R or read access lets you read files in a directory. L, li st, or lookup access letsyou attach to a directory and list its contents. Coda does not support `hidden' directories, which a user can access without being able to do adirectory listing. None of the other permissions actually works unless you have L access to the directory as well.

I or insert access, and D or delete access let you insert or delete files in a directory respectively, or create or delete a subdirectory respectively, ormount or unmount a volume there.

The K or lock access is intended to control the ability to lock files in a directory, but this has no effect in a Coda system.

W or write access lets you write or append to files in a directory, and A or admin access lets you change a directory's Access Control List.


20/45

Mode Bits

Ignored for directories

For files, group and other mode bits ignored

R mode bit must be set to read a file

R and X bits must be set to execute a file

W bit must be set to write to or delete a file

Suid and sgid may not do what you expect

In Coda, the Unix mode bits on directories are ignored. On files, the group and other mode bits

are ignored; only the user bits have any effect. To read a file you need read and lookup access tothe directory through its ACLs, and the file needs its R mode bit set. (While other Unix filesystems allow you to read files in `hidden' directories if you know their names, Coda does not.)

To execute a file, you need read and lookup ACLs, and you need both R and X mode bits set.(While some other Unix file systems allow you to execute a file which you cannot read, Coda doesnot.) And you cannot write or delete a file unless its W bit is set.

Coda remembers the Coda user ID of a file, the ID of the Coda user who created it. But the ownerand group of a file have no influence on who can access it. The Coda system administrator can

make an executable file in Coda SUID, in which case the command gets the effective Unix user IDof the file owner. Note that this does not give the program the access which that Coda user has; itdoes not give a running SUID program an effective Coda ID. This may cause problems if you wantcron jobs to write into Coda.

All files and directories in Coda appear to have the Unix group `nogroup', so SGID programs arenot useful.


21/45

Warnings

Flush() does not send data to fileservers:

you must close() file before data gets written.

Coda is not suitable for many log files.

Commands like 'tail -f' may not work.

When a process on one machine is writing,

it may not be visible on another machine. Besides the above-mentioned problems related to ACLs and mode bits,

you should be aware that a Coda client does not send data to fileserverswhen a running program calls flush(). You have to close() a file before itgets sent to the fileserver, and even then it doesn't happen if you aredisconnected. After disconnected operation, you can't reintegrate

without authenticating first. For this reason, Coda may not be a good place to write log files.

Commands like 'tail -f' don't work unless you are on the same clientmachine. A file being written by a process on one machine cannot beread on another machine until the writing process closes the file.


22/45

More Warnings

Databases should not have data files in Coda.

Data may be lost if the writing process fails.

Processes on other machines may not see data.

But this warning applies only to databasesmanaged by daemons which leave them open.

For this reason, Coda is not in general a good place to storedatabases. If the writing process aborts, or the client reboots,data that has been written may be lost. If you intend to share dataacross machines, you may have problems when a file is open.

Note that this applies only to data base files which get left open

by their managers; Berkeley DB-style databases are usually OK, asare others which get closed after being changed.


23/45

Looking Deeper

I hope by this time you have some idea of

what Coda can do and how it works. Now I'd

like to look a little more deeply into system

administration and disconnected operation.


24/45

Internal Data

Files: /vicepa

Data about volumes (kind, where, backed up)

Data about protection

Data for encryption (shared keys)

Data about backup

Data about cell

Recoverable virtual memory

Coda file and data servers store several kinds of data.

The data which appears on clients as files, directories, and so on is stored on the file server, of course, but not inthis format. Each Coda file server has at least one physical partition, usually named '/vicepa', '/vicepb', and soforth, where some of the contents of the Coda file system are stored in a binary format. There is approximatelyone file per file, but they are arranged in a hash tree.

The file servers store information about the cell's volumes (what kind they are, where they are, and so on), themapping between Coda users and groups and their ID numbers, the list of groups to which each user belongs,the list of members for each group, the shared secrets which allow servers to communicate with each other,information about backup (whether each volume has been or will be backed up, when, so that you can doincremental backups); information about the cell (which machine is the master SCM server, what other serversthere are, and so on).

Data on its way to its final home in the vice partitions gets stored in 'recoverable virtual memory' on that fileserver, to protect its integrity. Normally RVM requires two raw partitions on each file server, in addition to thepartition(s) used for /vicepa, (/vicepb,...).

That is, if you plan to make a machine a Coda file server, you should have at least three physical partitions to bedevoted to Coda: two raw partitions to be used by RVM, and one file system mounted on /vicepa to containCoda files.


25/45

Servers

codasrv: the workhorse

updateclnt and updatesrv

rpc2portmap

kauth2

Now let's look at the processes that run on a Coda server.

A file server always has a process called `codasrv', which is the workhorse: itserves files to Coda clients on other machines, and it keeps the vice partitions up-to-date.

Processes called `updateclnt' and `updatesrv' transfer between the variousservers in a Coda cell the other data which Coda keeps. This data is kept in onedirectory, /vice/db, on each server. Every 30 seconds these processes check tosee whether the data is the same on all servers, and if not, they copy it across.This may be very inefficient if you have many volumes or many users, at least ifthis data changes frequently.

The rpc2portmap daemon listens for incoming requests for Coda services andpasses them on.

kauth2 receives requests for Coda tokens, and passes them out when the clienthas a valid Kerberos ticket.


26/45

Server Security

If you are a hacker who breaks into aCoda client, you can't access protected

data in Coda without a Coda token.

You can, of course, read the contents

of the local cache if you have root

access. You can't change Coda files

unless they are world writable; that is,

unless the Coda group System:AnyUser

has write and lookup access to thatdirectory.

On the other hand, a hacker who

breaks into a server can run pdbtool to

create users, delete data, fake a Coda

token, and do a great deal of damage.

We may say that Coda affords some

protection to the clients and the data,but you must take pains to guard the

Coda server against attack.


27/45

Backing Up

backup program backup.sh script

Requires physical backup partition

well documented (with exceptions)

configured by dumplist file:

7F000002

IIIFIII

coda.proj

7F000003

IFIIIII

coda.user Coda comes with a backup program `backup' and a shell script `backup.sh' which runs it. Together they read a list of volumes to be backed up, and

do this (if possible). Backup clones the volume to be backed up, then dumps the clone to a file. It supports both full and in cremental backups, andis designed to be run daily or weekly.

There is a script `backup.sh' is designed to be run by cron. It is the script used at CMU, and the installation does not attempt to edit it for your cell.

The backup program must backup volumes to a physical partition, since it is intended to be used with some utility like 'dump' to copy the wholepartition to tape. The backup procedure is fairly well documented, though the documents do not say that you must customise backup.sh to makeit work in your cell. Here are a couple of lines from a sample dumplist file:

7F000002 IIIFIII coda.proj 7F000003 IFIIIII coda.user

These mean that the volume with groupID 7F000002, named coda.proj, should have a full dump on Wednesdays and incremental dumps everyother day, while the other volume has its full dump on Mondays. Dumplist is updated automatically when you create a read-write replicatedvolume; otherwise you need to edit it by hand.


28/45

Restoring from backups

Documentation is weaker than for backup.

merge combines full and incremental dumps.

No program or script

The procedure produces a read-onlyvolume, a copy of the clone.

You must copy files and ACLs manuallyif you wish to restore a whole volume.

The Coda system for restoring backed-up volumes is less satisfactory than the backup system. Inparticular the documentation is rather incomplete and contains some mistakes. You need toexperiment with restoring volumes to understand exactly what you need to do.

The program `merge' combines one full and one incremental dump file to create a single dumpfile for the volume to be restored. If you have create several incremental dumps, you need to runmerge several times. There is no script to restore a volume; you must figure out the correctcommands and arguments to use. This procedure creates a new read-only volume, which youneed to mount somewhere.

If you wish to restore a whole volume from its backup, you need to create a new read-writevolume, and manually to copy the files, mount points, and ACLs from the restored volume. Thenotes say that restoring the whole system after a crash is difficult; it is in fact a very tedious job.


29/45

Disconnected operation

Hoard files

clear add /coda/projects 100:c+

priority: 1 -- 1000

optional codes for directories: c, c+, d, d+

volume mounted in hoarded volume doesnot get hoarded; need explicit reference

spy program

Coda keeps on each client a list of instructions about what (if anything) it should try to cache, and with what

priority. You configure this list with the hoard command, which normally takes its instructions from a hoard file.That is, you could give it a lot of commands manually, but it might get rather tedious.

Here is a simple example of a hoard file:

clear add /coda/projects 100:c+ It says to clear the stored hoard list, and to add the directory /coda/projectswith a priority of 100. Files and directories with larger priority get preference over others. The code `c+' says toinclude with this directory all the files and directories in it (though not their subdirectories), as well as files anddirectories that might get added to /coda/projects in the future. The optional directory codes are c for childrenand d for descendents, where d includes all subdirectories in a volume. A `+' on c or d means include not justthe present contents but also future contents. Note that clear and add can be abbreviated to c and a.

The children or descendents of a directory do not include volumes mounted in the directory, since a mountmount is something like a symbolic link. If you wish to hoard files or directories which are in another volume,you must specify at least one path to that volume in the hoard file.

Coda comes with a program called spy, which is designed to help you construct hoard files. Suppose you intendto use some program inside Coda which has a lot of subdirectories and files, only some of which you may need;for example, suppose you intend to use emacs. You start spy, redirecting its output to a file, Run emacs, doingthe sort of thing you expect to be doing. Spy records all the bits of emacs inside Coda that you have used.


30/45

Hoard command

hoard -f hoard clear

hoard walk

hoard list

hoard off

hoard on

hoard delete

The hoard command configures the way the cache manager keeps files and directories in thecache. The command

hoard -f reads a hoard file and sends the commands in it to the cache manager. Venus remembersthese commands and checks the cache every 10 minutes to see that these files are there. Notethat you cannot tell hoard to save files or directories which you have no right to access; you musthave a Coda token to run most hoard commands.

`clear' tells the cache manager to forget its hoard list. `walk' forces venus to check the cache nowand add files and directories if necessary. `list' prints a list of the cached objects; it reassures me

that hoarding is going on. There are also `off' and `on' commands which let you switch the automatic hoard monitoring

function off and back on without actually clearing the hoard list, and a `delete' command whichremoves a file or directory from the hoard list.


31/45

Start disconnected server

venus -h 130.88.200.1 -r coda.root &

IP address of the SCM

name of the Coda root volume,the one to be mounted on /coda

This command starts the cache manager indisconnected mode:

venus -h 130.88.200.1 -r coda.root & The

arguments on the command line are the IPaddress of the SCM, which is the main Coda dataserver, and the name of the volume which is tobe mounted as /coda, the `Coda root volume'.


32/45

Authenticating

kclog kclog -tofile tokenfile

kclog -fromfile tokenfile

ctokens

cunlog

By `authenticating' I mean proving to Coda that I am the person who is allowed to do certain things in the filesystem. The Coda authentication program is `kclog'. Before you run it, you need the Kerberos tickets for a Codauser.

On its own, `kclog' or `kclog [username]' checks that your Kerberos tickets are valid, and if so, it gives you a Codatoken for a limited period. With the arguments `-tofile [filename]', it also writes the token to a file, so that youcan use it in disconnected mode. With the arguments `-fromfile [filename]' it reads and sets the Coda tokenfrom a file which you created earlier, without trying to verify your Kerberos credentials.

The command `ctokens' lists your Coda tokens, and the command `cunlog' destroys them.

The default time limit for Coda tokens is 25 hours. I find this uselessly short for disconnected operation, but theonly way to get a longer token is to edit the source of the kauth2 daemon and recompile it. I think there shouldbe some way to specify the lifetime of a token.

Getting a very long token, one that lasts weeks or months, is something of a security risk, as is saving it to a file.On the other hand, it is difficult to see how this can be avoided. As far as I can tell, Coda associates the user'sCoda token with the Unix user ID which requested it, which is not the safest policy. It seems to me that Codaneeds some idea like the Program Authentication Group (or PAG) of AFS, which lets you control this a littlebetter.


33/45

Shut down a client

vutil shutdown umount /coda

venus.init stop

There are two ways to shut down a client safely. Oneis to use the command `vutil shutdown', then

`umount /coda'. The other is to use the suppliedscript `venus.init' with the `stop' argument. If youdon't shut down a client properly, you risk losingdata.


34/45

Reintegrate

Start venus (venus &) Get Coda token.

cfs fr /coda/myfiles/mountpoint

Resolve any conflicts.

To reintegrate after disconnected operation, start venus as usual, and get a Codatoken, since without a token, you won't be allowed to write changes to the file

server. Give the command cfs fr /coda/myfiles/mountpoint (which means, Coda file system, force

reintegration on this volume) specifying the mount point of each volume youwish to reintegrate. (This is a minor nuisance if you forget that you have changedsome other volume.) If conflicts are reported, you need to resolve them as soonas you can.

Conflicts are handled by a process called `repairing', and there are several

commands that help you do this. When you begin a repair, the volume is locked,and the inconsistent files are `spread out' as distinct objects in the file system.You can then edit the files and harmonise them before ending the repair andreintegrating the file system again.


35/45

Security

Use Kerberos 5. Guard your file servers.

Not for highly sensitive data

possible security holes becausemost of Coda has not been stressed

Now I'd like to make some points about security. I think you should inany case use Kerberos 5 with Coda, because the default internalauthentication is so poor. You need to guard your file servers, since theyare by far the most vulnerable point in the cell. And there are potentialsecurity holes because most of Coda has not been the subject of serioussecurity testing. Despite this, I suspect that Coda may be more securethan NFS, and certainly not substantially less secure than AFS or DFS,

bearing in mind its inadequate encryption. The Coda developers are well aware of the limitations of Coda's

encryption, and intend to replace it with something more serious.


36/45

Planning and Installing

If you should decide to use Coda, what do

you need to know before you install it? How

many machines do you need, and what

resources should you have on each of them?What should you read to learn more about

the system?


37/45

Installing Coda

Plan carefully; have one or two trials Many scripts may need to be edited.

Join the mailing list, orfind a friendly expert.

Beware prepackaged binaries.

Read through documentation andlook at the mailing list archives.

It's easier to install Coda when you have a little experience in doing so, but that's true of a lot of software. Beginby doing a one-server cell with one additional client. This will give you the feel of the software and remind youof some critical issues. Next try a small, real cell with two or three other users. To learn the backup system, get itworking, then do a full backup and two incrementals on successive days, then delete the whole cell and restoreit. This will be very painful.

You will probably need to edit many of the scripts that come with Coda; I had to change seven of them for myuse, but then I don't like to install every executable in /usr/bin.

If you use Coda, you ought to join the Coda mailing list, or at least find an expert to help you over the roughspots. The mailing list is very friendly and (within reasonable bounds) very tolerant of newcomers.

If I say `beware of prepackaged binaries', I don't mean that you shouldn't use them, but that they may makechoices which you ought to change. I suspect most people will find a prepackaged binary is the best way to try

Coda for the first time, but I don't think any of them should be used in a production cell. It is worth reading through the main documentation carefully. Also look at recent archives of the mailing list;

the problems you face have probably bothered someone else, and the solutions probably exist already.


38/45

Documentation

9 manuals Coda Administration and User Manual

Notes and Explanation about Coda Security

man pages

The Coda documentation, which incidentally is not included withthe Coda source, contains 9 manuals. The most useful of these isthe Coda Administration and User Manual, though you shouldalso read the Notes and Explanations about Coda Security.

TheAdministration and User Manualcontains man pages, thoughthey are (I think) in HTML or PostScript format rather than in troff

source format. (Someone has put out a collection of plain manpages, but, the last time I looked, they were a different versionfrom those in the Admin manual.)


39/45

Good Points about Coda Docs

Usually very full

Many clear explanations

Many examples Coda documentation is, for

the most part,

comprehensive and clear. It

contains many examples,which are quite helpful to a

new Coda administrator.

Bad Points about Coda Docs

Out of date (until recently):

Nothing about new commands or options

Sometimes completely misleading:

(1) Read-only root volume

(2) Disconnected operation

Until recently, Coda's documents were seriously out of date. Theydidn't cover new commands and new options which had beenadded to the system in the past two years. They referred to anddescribed at length several procedures which dependedessentially on commands which no longer exist. And sometimesthey were completely misleading. Since I gave the first version ofthis talk, this has improved substantially; a new version of theAdmin manual has been produced which corrects many of thesefaults.

For example, the Admin manual in several places recommendedthat you create a replicated read-only root volume for your cell,and gave details of how to do this. The procedure didn't work, andis not supported in the current version of Coda.

Disconnected operation involves running hoard walk'; thisappears in the manual, but not in the man page for hoard, whichis quite incomplete. Saving Coda tokens for disconnectedoperation does not appear in the documentation, nor are thereinstructions for starting venus in disconnected mode. The cfssubcommand fr' or forcereintegrate' does not appear in thedocumentation.

In short, the documentation has flaws and omissions; it still needssome work.


40/45

Summary

To sum up, I shall give my evaluation of some

of Coda's functionality, and suggest work I

think it is ready to do, though with some

cautions and qualifications. I shall alsosuggest possible future developments.


41/45

Core Functions

File serving Mobile operation

Resilience

High availability

The main thing Coda does is to serve files. The Codafiles works extremely well under stress and does notseem to fail.

Coda is resilient, whether the network fails, the file

server goes down, or you do something unexpected.It is reliable. If you are looking for high availability,Coda is certainly a serious option.


42/45

Attractive Applications

Web server Shared files:

in, e.g., Beowulf clusteror across an enterprise

especially where NFS is inadequate

mobile/disconnected operation

There are some applications which Coda is particularly suited to support. One of those is webserving. If you need to share web pages across several servers, as in a Linux Virtual Server cluster,or if you need extra security, Coda may be just what you need, and its caching could have beendesigned with web serving in mind. For the same reasons Coda's cousin AFS got its lifetimeextended and found new customers.

Coda may be a good choice wherever you need to share files across several machines, such as ina Beowulf cluster, or across a department or a larger enterprise; this is especially the case whereNFS is not an appropriate solution because of its poor functionality and limited performance.

Coda was designed with mobile, disconnected operation in mind, It works well with slowconnections, it allows you to connect briefly and update your hoard, then to disconnect and work

for hours, then to reconnect again briefly and reintegrate again.


43/45

User Issues

Few users = few problems especially for system administrators

There is not much for users to learn

Beware of risky acts by users on clients.

The difficulties of users of Coda will depend on how many they are, and

on how complex their usage is. If you have many users, especially if theywant their own groups, if their file store has to be backed up, and so on,it would be quite a headache for the system administrator.

Because Coda's file semantics differ slightly from those of plain Unix filesystems, you might think users need to learn a lot before they are turnedloose on the system. But this is not so: beginners do not need to learneverything, unless perhaps they want to use disconnected operation.The number of utilities which will have problems because of Coda's non-classical-Unix semantics is surprisingly small.


44/45

Stored Data

Not too much hard to administer

hard to restore from backups

No sensitive data

limited security, limited stress

The problem of adapting to large scale usage affects the amount of data as well

as the number of users. Coda's protection and volume databases will not workwell for very large cells in their present form. But the larger problem lies inadministering, backing up, and restoring all that data, and not in Coda's internalfunctions.

Also, I have said that Coda is not suited to holding really sensitive data, becauseof its limited security at present; that is, because its internal encryption routinesare crippled, and because things may remain in a client's cache for a long time.

Moreover, Coda's constituent parts have not had a serious security review, orbeen exposed yet to systematic attack or exploitation attempts. On the otherhand, you should probably not serve secure data using NFS or any other sharedfile system. Of course, you can always use encrypted loopback on Coda files on aLinux box.


45/45

Needs Work

Documentation

Better administration

Remove dependence on SCM

Coda needs a lot of work. The documents

need to be brought up to date and expanded,

especially to explain things clearly to

ordinary users.

Documents

Coda File System.ppt