55
NFS 1-1 Sharing Unix File Systems NFS (Network File System ) allows you to share file systems Reasons for sharing file system Transparent to user User can keep use their familiar commands Access the same file from multiple nodes. To provide disk space to diskless clients To prevent duplication To provide centrally supported programs and data To share data among users Simplifies central support tasks Such as backup. Space usage monitor

NFS and NIS

Embed Size (px)

Citation preview

Page 1: NFS and NIS

NFS 1-1

Sharing Unix File Systems NFS (Network File System ) allows you to share

file systems Reasons for sharing file system

Transparent to user• User can keep use their familiar commands• Access the same file from multiple nodes.

To provide disk space to diskless clients To prevent duplication To provide centrally supported programs and data To share data among users Simplifies central support tasks

• Such as backup.• Space usage monitor

Page 2: NFS and NIS

NFS 1-2

NFS

Candidates could be: Home dirs, Web page, /usr/local and other common utilities Documentations Temporary huge space request Source code repository … …

Page 3: NFS and NIS

NFS 1-3

NFS protocol versions

NFS was introduced by Sun Microsystems in 1985 The original public release of NFS was version 2 In 1990s, version 3

Increases performance• Makes writes safely asynchronous

Better support for large files version 4

No ancillary protocols – integrated locking and mount Compound operations – bundle multiple RPC together in

a single exchange Strong security – uses RPCSSEC_GSS API Require use of transport protocols that offer congestion

control – hence NFS v4 will not support UDP transport.

Page 4: NFS and NIS

NFS 1-4

More about NFS

NFS run on top of Sun’s RPC (Remote Procedure Call) protocol RPC provides a system-independent way for

processes to communicate in a client-server fashion over a network.

Some NFS versions run on top of UDP and TCP Why?

WebNFS Promote the use of NFS over the Internet A extention of NFS protocol that allow easier access

to servers and clients through Internet firewalls A WebNFS-enhanced web browser can use an NFS

URL to access data directory from the server– nfs:/www.YourCompany.com/

Page 5: NFS and NIS

NFS 1-5

Security and NFS

NFS protocol was originally designed with essentially no concerns for security

Tight control over shared filesystems Don’t export any filesystems if anyone that

you don’t trust has root access on a client host

If you have a firewall, block port 2049, block SunRPC portmap deamon port 111

Kerberos to authenticate to ensure that remote users really are who they say they are.

Ipsec to encrypt the data

Page 6: NFS and NIS

NFS 1-6

Network File System

NFS consists of a number of components Mounting protocol Mount server Daemons that coordinate basic file service Several diagnostic utilities

Basically two parts: Client

• Mount a directory Server

• Export a directory

Page 7: NFS and NIS

NFS 1-7

Server-side NFS

Mount a filesystem Daemon: mountd

Access files Daemon: nfsd

Both daemon started at boottime Mountd and nfsd use the same database

( (xtab on most systems, sharetab on Solaris) There are commands (exportfs, share) to

add/remove the entries in the database Exportfs reads /etc/exports file

• Run exportfs –a• To remove entries, run exportfs –u

Page 8: NFS and NIS

NFS 1-8

Server-side NFS

On Solaris, /etc/dfs/dfstab is a shell script• Run shareall command, which equivalent to sh

/etc/dfs/dfstab To remove entries, run unshare Solaris also provides command “exports”, which

translates export options to share/unshare commands Any directory can be exported

Client can mount a subdirectories of an exported directory

• Example: On HP-UX 11iYou can do above, but you may not to mount another sub dir from the

same exported directory.

Each device to be exported separately For example

• /user is a separate partition, / can be exported without exporting /users

Page 9: NFS and NIS

NFS 1-9

NFS Daemons on Solaris The following daemons are running on server side

• nfsd [nservers]• mountd• nfslogd

The following daemons are running on both server side and client side

• rquotad• lockd• statd

How to start the daemons:• Daemons are under /usr/lib/nfs • Daemons are started at boot time by

/etc/init.d/nfs.client and nfs.server• nfs.client starts statd and lockd• nfs.server starts nfsd (16 copies default), mountd,

nfslogd• inetd start rquotad.

Page 10: NFS and NIS

NFS 1-10

Configure NFS on Solaris The share command to export a dir

share –F nfs [-o options] pathname Options are:

rw • Export read-write to the entire world

rw=accesslist• Export read-only with access only by listed hosts

ro ro=accesslist root=list

• Lists hosts permitted to access this filesystem as root• Otherwise, root access from a client is equivalent to

access by “nobody” (UID –2)

Page 11: NFS and NIS

NFS 1-11

Configure NFS on Solaris

anon=uid • defines the uid used for users who do not provide a valid user ID.

nosub• Forbids clients to mount subdirectories of the exported

directory nosuid

• Prevents setuid and setgid files from being created via NFS

Access list format Separated by : Identify computers by

• individual hostnames/IP• Domain, .domain• NIS netgroup, groupname• Network, @network[/prefix]

Page 12: NFS and NIS

NFS 1-12

Configure NFS on Solaris Examples: on Colossus /etc/dfs/dfstab

share -F nfs -o root=aa.aaa.aarw=keeper.csl.mtu.edu:csl:scifi:language:pizza:icu:cs:fishlab:cec /home/csdept

share -F nfs -o root=xx.xxx.xx:yy.yy.yyyrw=csl:language:scifi:pizza:icu:cs:fishlab:physpecial:cec:cs.mtu.edu

/export/major

Page 13: NFS and NIS

NFS 1-13

Configuring NFS on Linux /etc/exports

The NFS server configuration file Controls which files/dirs are exported Access control

The general format of entries: directory [host(option)] Host can be

• empty, means every host• individual host, name or IP address• Domains, *wrotethebook.com• Networks, IP address/mask(prefix length)• Netgroups, @group1

Option can be• ro• rw

Page 14: NFS and NIS

NFS 1-14

LINUX NFS Permission Control

NFS server trusts local authentication USE GID/UID to do regular Unix permission

control. Mismatch uid/gid will be a unexpected problem What about root?

• Do you want root users on NFS client to act like root users on NFS servers? Probably Not.

• NFS prevents by default setting root_squash– Mapping root to nobody UID/GID

• Set no_root_squash to allow it. Option all_squash

• Map all uids and gids to the anonymous user. Useful for NFS-exportd public FTP directories, news spool directories, etc.

Page 15: NFS and NIS

NFS 1-15

LINUX NFS Assign anonymous a UID/GID using option

anonuid=XXX, anongid=XXXThis option is primarily useful for PC/NFS clients, where you might want all requests appear to be from one user.

UID/GID mapping file using option map_static=filename

Command exportfs, build /var/lib/nfs/xtab export all in /etc/exports file using “exportfs –a “ Sync /etc/exports and /var/bin/nfs/xtab “exportfs –r” Temporarily export a filesystem

• exportfs fox:/usr/local –o rw remove from export list “exportfs –u fox:/usr/local” remove all export list “exportfs –ua “

Page 16: NFS and NIS

NFS 1-16

Client-side NFS

Mounting remote filesystems Use mount command to establish temporary

network mounts List in /etc/fstab or /etc/vfstabb Automatic mounting services such as

automount what are exported or where

Ask administrator Find out yourself:

• what have been exported from a host: showmount –e hostname

Page 17: NFS and NIS

NFS 1-17

Mounting Remote Filesystems Using mount command

Basic format: mount hostname:remote-dir local_dir

• hostname is the NFS server• local_dir must exist already

Using umount command umount local_dir[remote_dir]

Page 18: NFS and NIS

NFS 1-18

Mounting NFS

On Solaris, entries in /etc/vfstab are mounted by mountall during system startup

On Linux, entries in /etc/fstab are mounted via mount –a in startup files.

NFS mount flags ( page 502) Options:

• rw• ro• bg• hard

– cause the operations that try to access it to block until the server comes backup

• Soft– What about a job that has been running 18 hours and will be done in an

hour and aborted?

Page 19: NFS and NIS

NFS 1-19

Mounting NFS More mounting options

• intr– Allow user to interrupt blocked operations

• nointr• retrans=n

– Specifies the number of times to repeat a request before returning an error on a soft-mountd filesystem

• timeo=n– Set the timeout period for requests

• rsize – 8K for the same network

• wsize • vers• tcp

Page 20: NFS and NIS

NFS 1-20

Common mount options on LinuxOption Function

async

auto When –a is used

dev Allow character and block special files on this file system

exec

noauto Don’t mount even with -a

noexec

nosuid Don’t allow setuid, setgid

nouser Only root can mount

remount

ro

rw

suid Allow programs to run setuid or setgid

sync

user Ordinary users to mount the file system

atime Update inode

noatime

defaults Rw, suid, dev, exec, auto, nouser, and async

Page 21: NFS and NIS

NFS 1-21

Dedicated NFS file Servers

Fast, reliable file service is one of the most important elements

Dedicated NFS servers Optimized for file service Storage scale smoothly Reliable, simplified software Redundant hardware Support different file formats for different

platforms, such as NFS, CIFS Easier to administer than UNIX file servers Good backup and checkpoint facilities

Page 22: NFS and NIS

NFS 1-22

NFS Automounter

Automatically mounts NFS when needed Why automount?

• Maintaining /etc/fstab can be tedious• Minimizes the number of active mount points to

reduce/Avoid chaos when servers crashes How?

• Mount a virtual filesystem driver on the directories for automatic mount to occur

– Kernel-resident filesystem driver called autfs is used

• When the user references a directory withing the virtual filesystem, the automounter intercepts the reference and mount the actual filesystem the user is trying to reach.

Page 23: NFS and NIS

NFS 1-23

NFS Automounter

Daemons automountd and automount are started

by /etc/init.d/autofs• Automount reads the configuration file, setup

autofs mounts • References to automounted filesystems are

handled by a separate daemon automountd On solaris 10, you can start automounter

• #svcadm enable system/filesystem/autofs• The actual script is under /lib/svc/method/svc-

autofs

Page 24: NFS and NIS

NFS 1-24

automount

automount Basic configuration files

• Master map• Direct maps• Indirect maps

Direct and indirect maps provide information about filesystems that are to be automounted

A master map lists direct and indirect maps that automount should pay attention to.

• Only one master map that can be active• The default one is /etc/auto_master

Page 25: NFS and NIS

NFS 1-25

automount

Indirect maps• Automount several filesystems underneath a

common directory• The path is specified in the master maps, not in

the indirect map itself• The name indirect map will be used as

subdirectory to install the mount– Example, indirect map

Info –ro chimchimL/chimchim/info• Using ls a directory full of indirect mounts can be

confusing » automount does not show the subdirectories until

their contents have been accessed» What does ls do?

Page 26: NFS and NIS

NFS 1-26

automount

Direct maps• List the maps that do not share a common prefix

– Such as:

/cs/tools –ro anchor:/cs/tools

/usr/src chimchim:/usr/src

• Each implemented with a separate autofs mount• Require slightly overhead• But mount point and directory structure are

always accessible by commands such as ls

Page 27: NFS and NIS

NFS 1-27

automount

master map• format: mount-point map-name option

– List the direct and indirect map– For indirect map, it specify root directory used by the

mounts defined in the map.– /- is for direct maps– Option set the defaults for all mounts within the map

[ruihong@cslserver nis]$ more /etc/auto_master# cs and csl don't use auto_home and auto yet.#/net -hosts -nosuid,nobrowse#/home auto_home -nobrowse#/local auto_local -nobrowse/fisher auto_fisher/- auto_direct

Page 28: NFS and NIS

NFS 1-28

automatic automount

Have automount to figure it out itself Query mountd running on a remote server to find out

what filesystems the server exports Use –host as a map name, automount will map

remote hosts’exports into the specified automount directory

• ExampleThe serer chimchim exported /usr/share/manThe client’s master map has a line

/net -hosts –nosuid,softThe mount point will be

/net/chimchim/usr/share/man• It does enumerate all possible hosts• It waits for individual subdirectory names to be

references,then runs off and mounts the exported filesystem

Page 29: NFS and NIS

NFS 1-29

automatic automount

Key substitutions• Impress a degree of regularity on the

automounter maps• The ampersand (&) expands to the matched key

value in a map• The asterisk(*) is a default case. • Example: a indirect map:

usr1 -rw thud:/export/home/usr1usr2 -rw thud:/export/home/usr2usr3 -rw thud:/export/home/usr3usr4 -rw thud:/export/home/usr4Can be rewritten as only one line* -rw thud:/export/home/&

Page 30: NFS and NIS

NFS 1-30

NFS Automounter Automount can use maps from NIS

server. Other type maps, like auto_home mount

# Master map for automounter# #+auto_master #/xfn -xfn /net -hosts -nosuid /home auto_home /- auto_direct

# Home directory map for automounter # #+auto_home craig almond:/export/home/craig * pecan:/export/home/&

Page 31: NFS and NIS

NFS 1-31

Practice

In Lab4, we created a dir called /research. As a group, let’s try the following tasks

Export /research, so some hosts in your group as rw some hosts in your group as ro one extra host have root mapping Access the exported from /net/hostname/home

Mount /research from some hosts in your group

Automount the exported /research to /research from some hosts in your group

Page 32: NFS and NIS

NFS 1-32

Network Information Service

NIS Was originally called the Sun Yellow Pages Is an administrative database

• What information does database store? Provides central control and automatic

dissemination of important files.

Page 33: NFS and NIS

NFS 1-33

Network Information Service

The unit of sharing in NIS is the record, not the file. A record is usually corresponds to one line

Mast server maintains the authoritative copies of system files Are kept in their original locations and formats Edited with a text editor before

A server process makes the contents of the file available over the network.

A server and its clients constitute an NIS “domain” associated with A set of maps Different from DNS domain

Page 34: NFS and NIS

NFS 1-34

Network Information Service

On the master server Data file are preprocessed into database

files (called maps) by a hashing library• One key associated with each entry• A system file may have to be translated into

several NIS maps– Example:

» /etc/passwd is translated into two different NIS Maps

passwd.byname

passwd.byuid

• The record order is not preserved.

Page 35: NFS and NIS

NFS 1-35

Network Information Service

On the slave server Relieve load on the master and keep clients

functioning when master is down. Replicate the network maps from master server

On the client Does not distinguish between the master server and

slaves Traditional

• Use IP broadcasting to locate servers• Place at least one NIS server on every physical network

Some implementation allows to specify the server name instead of broadcasting

Page 36: NFS and NIS

NFS 1-36

Network Information Service

Databases are called NIS maps /etc/passwd /etc/group /etc/netgroup /etc/hosts /etc/networks /etc/protocols /etc/services /etc/aliases /etc/auto_master … …

Page 37: NFS and NIS

NFS 1-37

Network Information Service

Netgroups Sets of users, machines, and nets for easy reference in

other system files The format of netgroup entry is

Groupname list-of-members Members are separated by whitespace. A member is

• Either a netgroup name• Or a tripnet of the form

(hostname, username, nisdomainname) a empty value in a field matching any value in that field

Netgroup can be used in several system files that define permissions

• Configuring NFS export ( /etc/exports file)• Restrict remote logins and shell access ( /etc/hosts.equiv

file)• Local login access ( /etc/passwd file)

Page 38: NFS and NIS

NFS 1-38

Network Information Service Exmaples

/etc/netgroupgrouplab \(goofy.csl.mtu.edu,,) (mickey.csl.mtu.edu,,) \(minnie.csl.mtu.edu,,) (donald.csl.mtu.edu,,) \(simba.csl.mtu.edu,,) (nala.csl.mtu.edu,,) \(tramp.csl.mtu.edu,,)

noncs1121d \(,msbenson,) (,behart,) (,adrogers,) (,agwalthe,) (,adpeters,) (,ammurrel,) \(,aeblechi,) (,jaeul,) (,srkelley,) (,daperry,) (,anlevend,) (,frtuntla,) \(,cwminnic,) (,rrmilkov,) (,iatrifon,) (,ksrivast,) (,talange,) (,mtnielse,) \(,csdummy,)

/etc/dfs/dfstab

share -F nfs -o root=aa.aaa.aarw=keeper.csl.mtu.edu:csl:scifi:language:pizza:icu:cs:fishlab:cec /home/csdept

/etc/passwd+@nonenrolled::::::/usr/local/shells/gone+@cs1121nons::::::/usr/local/shells/limitaccess

Page 39: NFS and NIS

NFS 1-39

Network Information Service

How NIS works NIS’s data files are stored in one directory

• Usually /var/yp Each NIS map is stored in a hashed format in a

subdirectory named for the NIS domain• Exact Map files names depends on the hashing library

being used.• For example:

– On cslserver, under /var/yp/csl.mtu.edu, there are ndbm files:

password.byname.dir

password.byname.pag

auto.direct.dir

auto.direct.pag

Page 40: NFS and NIS

NFS 1-40

Network Information Service

There is a makefile under /var/yp• Which calls makedbm to generate NIS maps from flat

files. • After you modify a system file, cd to NIS dir which is

/var/yp, and run make. – Make checks the modification date and rebuild some maps.

Slave server run ypxfr command regularly as cron to pull the map copies

Server runs yppush to instruct each slave server to execute ypxfr.

• Yppush is used in makefile after rebuilding a updated map.

• a special map called ypservers – no correspond flat file

– constructed automatically

Page 41: NFS and NIS

NFS 1-41

Network Information Service

After the initial configuration Deamon ypserv running only on the servers

• Accept queries from clients and answers them by looking up information in the hashed map files

Deamon ypbind running on every machine, include servers

• C library contacts the local ypbind to answer a query

– provided /etc/nsswitch.conf says to check with nis

• Ypbind locates a ypserv and returns its identify to the C library

• C library contacts the server directly

Page 42: NFS and NIS

NFS 1-42

NIS commands and daemons (page 527) ypserv NIS server daemon ypbind NIS client daemon domainname Sets the NIS domain a machine is in ypxfr Downloads current version of a map from master ypxfrd Serves request from ypxfr (runs on master server) yppush Makes slave servers update their versions of a map makedbm builds a hashed map from a flat file ypmake rebuild map ypinit Configure the host as a master or slave server ypset Makes ypbind connect to a particular server ypwhich find out which server the current host is

using ypcat Prints the values contained in an NIS map ypmatch prints map entries for a specified key yppasswd Changes a password on the NIS master

server ypchfn Changes GECOS info on the NIS master server ypchsh Changes a login shell on NIS master server yppassdd Server for yppasswd, ypchsh, and ypchfn

Page 43: NFS and NIS

NFS 1-43

Set UP NIS Procedure

Prepare the source file Initialize master server

• Run ypinit -m• Starting NIS service

– Run ypserv Initialize slave server

• Set domainname• Run ypinit command with –c and master serve ip.

Setup NIS clients• Setup default domainname • /etc/nsswitch.conf• Run ypinit –c• Start ypbind daemon

– Run ypbind

Page 44: NFS and NIS

NFS 1-44

NIS

Case study: Let’s take a look at where the startup files

for starting NIS server and NIS client.• /etc/init.d/rpc• /usr/lib/netsvc/yp/ypstart (ypstop)

Maps• /var/yp/csl.mtu.edu

Let’s take a look at the make file

Page 45: NFS and NIS

NFS 1-45

NIS

Work with DNS Hostname can be converted to IP address by DNS,

NIS, and the host file. /etc/nsswitch.conf

Define the order in which the sources are searched. Choices are:

• Dns• Nis• FilesFor example:hosts: dns nis files networks: nis [NOTFOUND=return] files services: nis files protocols: nis files

Page 46: NFS and NIS

NFS 1-46

NIS+

NIS+ is a completely new software product and structured entirely differently from NIS Sun’s new administrative database released in early

90s Provides all the functionality of NIS Improved security

• NIS does not authenticate servers or clients• NIS+ use secure DES• NIS provide same access to everyone in the domain• NIS+ provide access levels to different users

NIS+ is a hierarchical decentralized architecture. NIS+ has enhanced data structures.

• Multiple columns table and relation query between tables

NIS+ is complex and not as popular as NIS

Page 47: NFS and NIS

NFS 1-47

NIS practice

Refer to the Sun document “System Administration Guide: Naming and

Directory Service “• section “ NIS Setup and Administration”

As a group Set up a NIS server and a slave server On the NIS server

• create a user account named visitor1 with home dir /research/visitor1

Set up a NIS client• Modify /etc/nsswitch.conf

– To have passwd line to include nis

Page 48: NFS and NIS

NFS 1-48

Copying files around

Not elegant solution, but it works Example:

• A change to a startup script • A new version of software

File copying systems can use “push” model or “pull” model rdist: push files rsync: push files more securely expect: pull files

Page 49: NFS and NIS

NFS 1-49

Copying files around

Rdist The easiest way to distribute files from a

central server. • Sort of like make

– A text specification of the files to be distributed

– Rdist copies file when they are out of date.

• Preserves owner, group, mode and modification time of files.

• When updating an existing file, it first deletes the old version before installing the new

– Makes rdist suitable for transferring executables that might be in use during the update.

Page 50: NFS and NIS

NFS 1-50

Copying files around

It runs on rsh and relies on rsh-style authentication to gain access to remote system

• The protocol can also be changed from command line rdist -P /usr/local/bin/ssh -f myDistfile

Control file has the statements of the formLabel: pathnames -> destination commands

Where:

» label: associate a name with the statement

» pathname: lists of files to be copied

» destination: list of hosts to copy files to

» commands: modify the behavior ( default is to copy files to the equivalent paths)

Page 51: NFS and NIS

NFS 1-51

Copying files around

EXAMPLE:SYS_FILES = (/etc/passwd /etc/group /etc/mail/aliases)GET_ALL = (chimchim lollopop barkadon)

GET_SOME = (whammo spiff)

all: ${SYS_FILES} -> ${GET_ALL}notify barb;special /etc/mail/aliases “/usr/bin/newaliases”

some: ${SYS_FILES} -> $ {GET_SOME}except /etc/mail/aliases;notify eddie@spiff;

Page 52: NFS and NIS

NFS 1-52

Copying files around

Rsync: push files Flexiblie and faster replacement of rcp

• Only transmit the differences • Support for copying links, devices,owners, groups and

permissions• Can use any transparent remote shell rsh, ssh• Can use rsync daemon mode

No controlfile Must run repeatedly to transfer a set of files to

multiple hosts. Example:

• #rsync –gopt –password-file=/etc/rsync.pwd /etc/passwd lollipop::sysfiles

• #rsync –gopt –e ssh /etc/passwd /etc/shadow lollipop:/etc

Page 53: NFS and NIS

NFS 1-53

Copying files around

Expect: pull files Expect is a TCL extension scripting language

• Allows to you write control scripts for interactive programs

– The output produced by each operation is examined to determine what input should be send next

Use expect to retrieve and install them• Fundamental expect commands are:

– spawn : start up a subprocess to control– send: feed input to a subprocess– expect: take action depending on a subprocess’s output

Example: Make system files available via FTP from central server

Page 54: NFS and NIS

NFS 1-54

Copying files aroundExample: ftps the /etc/passwd file from the machine netserver

spawn /usr/bin/ftp netserverwhile 1 { expect {

“Name *:” {send “netclient\r”}“Password:” {send “netclientpassword\r”}“ftp> “ {break}“failed” {send_user “Can’t log in .\r”; exit 1}timeout {send_user ‘Timeout problem. \r”; exit

2}}}send “lcd /etc\r”expect “ftp>” {send “cd pub/sysfiles\r”}expect “ftp>” {send “get passwd\r”}expect “ftp>” {send “quit\r”, send_user “\r”}exit 0

Page 55: NFS and NIS

NFS 1-55

Summary

NFS NIS Copying file around