Network File Systems (NFS) and Remote Procedure Calls …bill/cs560/Nfs_Slides.pdf · 1 1 Network File Systems (NFS) and Remote Procedure Calls (RPC) Chavalit Srisathapornphat CISC856

1

1

Network File Systems (NFS)Network File Systems (NFS)

andand

Remote Procedure Calls (RPC)Remote Procedure Calls (RPC)

Chavalit SrisathapornphatCISC856

April 22, 1998

Good afternoon everybody. My name is Chavalit Srisathapornphat. Today Iím going to talk about the NFS, the Network file system, which is the transparent file access to the remote storage and the RPC, the Remote Procedure Call, the other way to do network programming by calling the procedure in the remote host. At the end of the presentation we will look at the example of the NFS operations from tcpdump output. I plan to spend only 60 minutes for the whole talk but Iím not sure I can do in on time. So, I think it is better to start now with the NFS on my next slide. If you have any problems, including problems with my pronunciation, please feel free to stop me at any time.

2

2

Network File Systems (NFS)Network File Systems (NFS)

• What is NFS ?

• Why do we need NFS ?

• Advantages of NFS

• NFS protocol and OSI model

• How does NFS work ?

• File handles

• Statelessness & Idempotence

• TCP or UDP

The first part of my talk is on NFS. Iíll tell you about what the NFS is, what the problem is when we work on remote files without NFS, what the advantages of NFS are, what the NFS protocol looks like and how it compares to the OSI model, how the NFS works, what is the file handle and its importance, what are statelessness and idempotence procedures and how they are important in NFS Finally, we will see the different between NFS using TCP and UDP. Iíll postpone the sample tcpdumpoutput until we finish talking about RPC.

3

3

Problem: Diskless workstationsProblem: Diskless workstations

Ethernet

pooh.ecl.udel.edu rabbit.ecl.udel.edu piglet.ecl.udel.edu

tigger.ecl.udel.edu

NFS Server

DISK

Solution: NFS

Letís see how NFS works. When a user is accessing a file, the kernel determines whether the file is a local file or an NFS file. The kernel passes all references to local files to the local file access module and all references to the NFS files to the NFS client module, as indicated as number 1 in the figure.

The NFS client sends RPC requests to the NFS server through its TCP/TP module, as shown in number 2. Normally, NFS is used with UDP, but newerimplementations can use TCP.

Then the NFS server receives the requests on port 2049.

Next, the NFS server passes the request through its local file access routines, which access the file on serverís local disk.

After the server gets the results back from the local file access routines, the NFS server sends back the reply in the RPC reply format to the client.

Some subtle points to mention here are first, while the NFS server is handling the clientís request, the local file system needs some amount of time to return the results to the server. During this time the server does not want to block other incoming client requests. To handle multiple client requests, NFS servers are multithreaded or there are multiple servers running at the same time.

Second, the same situation occurs in the clientís side. Some Unix systems often use a technique similar to the NFS server: there are multiple biodís running on the client side to provide more concurrency of NFS requests.

4

4

Solution: Diskless workstationsSolution: Diskless workstations(Logical view)(Logical view)

pooh:/ rabbit:/ piglet:/

tigger:/

NFS Server's disk:/

root_tigger root_pigletroot_rabbitroot_pooh

.

.

.

.

.

.

.

.

.

.

.

.








5

5

Problem: Several workstations without NFSProblem: Several workstations without NFS

Ethernet


tigger.ecl.udel.edu








6

6

Solution: Several workstations with NFSSolution: Several workstations with NFS

Ethernet


tigger.ecl.udel.edu

NFS Server

DISK








7

7

Solution: Several workstations with NFSSolution: Several workstations with NFS(Logical view)(Logical view)

pooh:/

etc home usr ...

rabbit:/

etc home usr ...

piglet:/

etc home usr ...

NFS Server's disk:/

etc home usr ...

amer pakala zshao ...

tigger:/

etc home usr ...








8

8

OSI v.s. NFSOSI v.s. NFS

Application

Presentation

Session

Transport

Network

Link

Physical

NFSMOUNT

PORT MAPPERNIS(Network Information System)

XDR (eXternal Data Representation)

RPC (Remote Procedure Call)

TCP, UDP

IP

Ethernet

OSI Model NFS Protocol Layers

Letís look at the detail of the NFS protocol compare with the OSI model. NFS is a protocol in the application layer. It works with some protocols. The mount protocol provides the method of validation and permission checking and initiates the root file handle for client. The port mapper protocol provides the current server port number to the client that needs to access the specific server program. We will see more details on port mapper when we talk about RPC in a few minutes. NIS is usually implemented with NFS. It provides a convenient way user can login with the same user name and password to all the machines in the same NIS group. We wonít talk about NIS in this presentation but you can find the information about NIS from this book. NIS was formerly called YP or Yellow page.

NFS and all related protocols are using the service provided by RPC. All NFS requests and replies are in the format specified by RPC. XDR is the standard for encoding data in RPC. We will concentrate only on NFS and RPC in this talk.

From this figure, you can see that NFS and RPC requests can be used with both UDP and TCP, I will point out the differences between using TCP and UDP after we know how NFS works. NFS was designed to beindependent from transport layer. That means NFS can use on top of many transport protocols. However, in this class we interested in TCP and UDP only.

9

9

How does NFS work ?How does NFS work ?

localfile

access

NFSclient

TCP/UDPIP

userprocess

NFSserver

TCP/UDPIP

localfile

accessport 2049

client kernel server kernel

localdisk

localdisk

¶

· ¸¹

RPC RPC

Client wants to access a file from server








10

10

File HandlesFile Handles

• How does a server know which file/directory the client needs to access?– At first, client obtains a file handle for root of the

file system– File handle is opaque to the client– Client sends file handle to server when

referencing a file/directory– No need to use the full path names

• “The file handle can contain whatever information the server needs to distinguish an individual file”

At this point, you may be wondering that how the NFS server knows which file the client needs to access.

There is a data structure that is called the File Handle. The File handle is created by the NFS server and it is a unique reference to the specific file or directory on the NFS server itself. This FH is passed to the client at the first time the NFS client contacts the NFS server. The process of first contact is called the Mounting process. It is described in the mount protocol that we have already mentioned. The top directory of the NFS server file system is called the root of the mounted file system. So, when the client mounts the server file system, the client will get a file handle of the root file system from the server. FH is opaque to the client. This means the client does not do anything with the FH. The client only sends it back to the server when it wants to access that file/directory. And the server can know from the file handle which file/dir the client needs to access. One advantage of FH is the client needs not to specify the full path name. This is important because NFS is used across many different kinds of file systems. With the FH, the client does not need to know how the NFS server specifies the path name. And the other important point is that the server doesnít need to keep track of what is the current access point of the client. This is the main idea of a stateless server that we will talking about in the next few slides.

11

11

Example of File HandlesExample of File HandlesSuppose : (tcpdump-C)client needs to cat the file sub2/myname.txt under the current directory (~/srisatha)

What is the attributeof current

dir(9925949) ?These are attributesof 9925949

What is FH of "856"in 9925949 ?

FH of 856 is 7656838

What is FH of "myname.txt"in 7656838 ?

FH of myname.txtis 7657235

What is the attribute of7657235 ?

.

.

.

NFS Client NFS Server

¶

¸

·

º

¹

»

¼

Letís look at the draft scheme that I modified from the tcpdump output. We will see the detailed tcpdump output at the end but for now letís take a close look at the function of FH. Suppose that the client needs to access the file myname.txt that is in the directory sub2 under the root directory. Please realize that the word root here doesnít mean the root of the NFS serverís file system. It means the root of the mounted point.

At first, when the client mounts the NFS server file system, the client asks for the FH of that file system and the server sends back the FH (which for our example is 4996983). Sometime later, the client needs to access the file sub2/myname.txt. The Client has to send the FH to the server and specifies the name of the file/directory needs to access. After the server gets this FH, it checks with its local file system to see if it has that name in the specified FH. If so, the server sends the FH of that file/directory back to the client again. In this example, the client needs to know if it has a directory sub2 under the FH 4996983, so it sends the FH and specifies the name ìsub2î to the server.

The server returns the FH of sub2 which is 8143290 to the client. The client knows that this ìsub2î exists on the server with the FH it just received.

The process continues. The client asks for the FH of the file myname.txt in the FH 8143290, which is ìsub2î. The server sends back the FH of myname.txt, which is 8143365. Now, the client can ask for other operations with this myname.txt file by specifying the FH of this file.

You will see that client doesnít touch the content in the FH : it just sends the FH back to the server when it needs to access that file/dir.

12

12

StatelessnessStatelessness

• What is statelessness ?– Server does not need to maintain protocol state

about it’s client– Server does not keep previous request information– Client keeps track of all information required to

send requests to the server

• Advantage :– If server crashes, no state information lost– Client needs only retransmit a request until the

server responds

RFC1813 states that NFS assumes a stateless server implementation. That means that the server does not need to maintain state about any of its clients in order to function correctly. Or we can say in another way: the server does not keep the previous clientís request information. The server does not keep track of which clients accessing which files or directories. Because of this server requirement, the client needs to give complete information to the server when the client sends a request. The advantage of a stateless server is that if the server crashes, the client only retransmits until the server responds. There is no state information to lose.

13

13

Why isWhy is idempotentidempotent important ?important ?Suppose : (tcpdump-D)client needs to remove the file sub2/myname.txt

Remove OK

Here is the attr andcontent of 7656838

Remove "myname.txt"from 7656838

NFS Client NFS Server

Remove "myname.txt" from7656838 (retransmitted)

Error : No such file or dir

What is the attribute of7656838 (dir 856) ?

¶

¸

º

¹

·

»

Letís look at the example that continues from the last example. Suppose the client needs to remove the file myname.txt from directory sub2, which itís file handle is 8143290. The client sends an RPC request to remove that file. After the server receives the request, it removes that file and sends the RPC result back to the client. But for some unknown reason, the reply message gets lost in the network. After the timeout period, the client retransmits that removal request again. If the server doesnít keep track of anything about the previous request, the server will process the duplicated request again. This time, the result will be an error.

For this reason, the remote procedure call from the client must be an idempotent procedure.

14

14

Idempotent Idempotent proceduresprocedures

• Can be executed more than once by the server and still return the same result

• Stateless protocol requires idempotent operation

• How to makes all NFS requests idempotent:– Server records recently performed operations in

cache– Server checks in cache for duplicate requests– Server returns the previous result if it is a duplicate

Idempotent procedure means that it can be executed more than once by the server and still return the same result. This is required for the stateless server as we have seen from the example. Some NFS requests are idempotent, but some are not. The example of the NFS requests that are idempotent are getattr, read, readlink, readdir. But the question is how to makes the other requests, that are not idempotent by nature, to be idempotent. (e.g. remove, create, mknod, mkdir, rmdir,...) The solution is that the server records recently performed operations in the cache and checks in the cache for duplicate requests. If there is one, the server returns the previous result.

15

15

Should NFS use TCP or UDP ?Should NFS use TCP or UDP ?

• From the beginning, NFS used UDP– Most NFS systems were on LAN– High overhead if using TCP

• Currently, NFS across WAN needs TCP– Reliability and congestion control– Both sides set TCP’s keep alive option– If server crashes, client opens new TCP connection– If client crashes, server will terminate the connection

after the next keep alive probe

We have reached the last slide for NFS. When do we use NFS over TCP or UDP ? First time NFS was developed, it was intended to be used on LAN and because of LAN was very reliable. So, using UDP can save more overhead than TCP. However, time changes everything. Nowadays, NFS is no longer restricted to LAN. Some NFS systems are used across WAN. For this case, TCP is chosen instead of UDP. Everybody already has had a good knowledge in how TCP is better than UDP. TCP provides reliability and congestion control. The implementation in both client and server sets the TCPís keep alive option. Then if the server crashes, the client only opens a new TCP connection to the server (after the server reboots). If the client crashes, all applications in the clientís side also crash. The server only terminates the half open connection after the next TCPís keep alive probe.

16

16

Remote Procedure Call (RPC)Remote Procedure Call (RPC)

• What is RPC ?

• How does RPC work ?

• How does RPC differ from Local procedure call ?

• List of RPC programs

• Port Mapper/RPCBIND

• tcpdump output (NFS)

• Summary

Now Weíre looking at the RPC. Weíll see that what the RPC is, how it works, how it different from normal procedure call, the format of RPC call and reply. We will look at some RPC program numbers and the concept of port mapper or RPCBIND. Finally, we will examine some real examples from tcpdump outputs.

17

17

How does RPC works ?How does RPC works ?

Client Process

Clientexecutes

ServerwaitsRPC

messageServer starts

Procedure call

Procedure return

Server executesprocedure

Call terminatesRPC returnmessage

Clientwaits

Clientcontinues

¶

·

¸

¹

Server Process

Looking at the process of one RPC call, start from number 1, when the client calls the remote procedure, itís really calling a local function in the RPC package. This function is called the client stub. The client stub packs the procedure arguments into the network message and sends it to the server

On the server side, the server stub, or the function in the RPC package, receives the network message from the transport layer. It takes the arguments from the network message, and calls the server procedure that the programmer specifies in the message.

The procedure returns to the server stub. The server stub packs the result values into a network message, and sends it back to the client.

The client stub unpacks the return values from the network message and returns to the normal application.

In the next slide, we will see the differences between RPC and local procedure call.

18

18

How does RPC different from How does RPC different from local procedure call ?local procedure call ?

• Error handling:– failures of the server or network must be handled

• Global variables:– arguments cannot be passed as global variables

• Performance:– slower than local procedure calls

• Authentication:– RPC can be transported over insecure networks

We can summarize some different points between RPC and LPC. First, for error handling, the RPC client has to be concerned about network failure or server crash. Next, we cannot use any kind of global variables in RPC. Moreover, the RPC usually takes a longer time in each call than local procedure call because the request and reply have to pass through the network. The last point is that RPC has to have some kind of authentication because the request and reply can be transported over the insecure networks.

19

19

List of RPC ProgramsList of RPC Programs

pooh[12:06am][~/]>rpcinfo -pprogram vers proto port service100000 4 tcp 111 portmapper100000 3 tcp 111 portmapper100000 2 tcp 111 portmapper100000 4 udp 111 portmapper100000 3 udp 111 portmapper100000 2 udp 111 portmapper100003 2 udp 2049 nfs100003 3 udp 2049 nfs100003 2 tcp 2049 nfs100003 3 tcp 2049 nfs100005 1 udp 32890 mountd100005 2 udp 32890 mountd100005 3 udp 32890 mountd100005 1 tcp 32870 mountd100005 2 tcp 32870 mountd100005 3 tcp 32870 mountd

I have selected some parts of the RPC program that are provided on pooh to show here. You will notice that each program will be assigned a unique program number, such as port mapper is 100000, nfs is 100003.

One program can provides service for more than one version, such as NFS provides services for both version 2 and 3.

Each version can receive requests from both UDP and TCP. Some programs do the passive opening on the same port number for both TCP and UDP, such as NFS, but some programs use the different port number, such as mountd. It depends on the implementations.

If you look at this list carefully, you will notice that every program, except the port mapper, uses the ephemeral ports, not the well known port numbers. Do you wonder how the client knows which port number the server uses?

20

20

PortPort MapperMapper : Analogy: Analogy

Airport Entrance

Terminal A

Terminal BTe

rmin

al C

Term

inal

D

Terminal E

Terminal F

US109 to Akron

Suppose you know that you are going to Akron by flight US109 and you have just arrived the Philadelphia International Airport. You donít know from which gate your plan will depart. What should you do?

21

21

PortPort MapperMapper : Analogy: AnalogyAirport Entrance

Terminal A

Terminal B

Term

inal

C

Term

inal

D

Terminal E

Terminal FFlight schedule

US109 to AkronFlight Departure timeDestination Gate

DE427 6:15 AMCincinnati E8US109 7:40 AMAkron OH B5US278 4:35 PMDetroit MI C9UA0097 6:00 PMLAX CA D12

You should look at the flight schedule. How do you know where the flight schedules are? Everybody knows that you just look on the big board that may be hung from the ceiling or on a monitor. This is the way RPC client knows the port number.

22

22

Port Port MapperMapper/RPCBIND/RPCBIND

ClientProcess

Client Kernel Server Kernel

Port Mapper

userprocess

userprocess

ServerProcess

userprocess

(1)register

at start

(2) get port# RPC request

(3) RPC reply with port#

(4) RPC call (request)

(5) RPC reply message

The port mapper acts as a flight schedule. It does the passive opening with port 111, the well known port. Every time the server programs start up, they register their port number with the port mapper. Just as when the flights are ready to depart or the time to depart is known exactly, the departure details of that flight will be shown on the flight schedule. Everyone can look at it. So can the clients. The clients just send a request asking for the port number of the server program from the port mapper.

23

23

SummarySummary

• NFS provides transparent file accesses

• NFS sends messages in XDR format using RPC

• NFS server is stateless

• NFS procedures are idempotent

• RPC executes remote procedure for clients

• Most RPC programs use ephemeral ports

• Port Mapper maps RPC programs and version numbers to port numbers

24

24

NFS DocumentsNFS Documents

• RPC version 2 (RFC 1831)

• XDR (RFC 1832)

• NFS version 3 (RFC 1813)

• Port mapper/RPCBIND (RFC 1833)

• Mount protocol (In the appendix of RFC 1813)

25

25

NFS Example : tcpdumpNFS Example : tcpdumpbifur[10:46pm][~/]>tcpdump -s 192 host nori and nfstcpdump: listening on le0

A nori[10:46pm][~/]>mkdir sub2

1 693259 nori.10497c > dilbert.nfs: 116 getattr fh 85,10/99259492 694011 dilbert.nfs > nori.10497c: reply ok 96 getattr DIR 40711 ids 5564/5564 sz 15363 695424 nori.10497d > dilbert.nfs: 124 lookup fh 85,10/9925949 "sub2"4 696460 dilbert.nfs > nori.10497d: reply ok 128 lookup ERROR: No such file or directory [|nfs]5 697719 nori.10497e > dilbert.nfs: 156 mkdir fh 85,10/9925949 "sub2"6 746166 dilbert.nfs > nori.10497e: reply ok 128 mkdir fh 85,10/10087924

=====================================================================================================B nori[10:47pm][~/]>echo Chavalit Srisathapornphat >sub2/myname.txt

1 264571 nori.104980 > dilbert.nfs: 116 getattr fh 85,10/99259492 265327 dilbert.nfs > nori.104980: reply ok 96 getattr DIR 40711 ids 5564/5564 sz 15363 293475 nori.104982 > dilbert.nfs: 124 lookup fh 85,10/9925949 "sub2"4 294169 dilbert.nfs > nori.104982: reply ok 128 lookup fh 85,10/100879245 295823 nori.104983 > dilbert.nfs: 132 lookup fh 85,10/10087924 "myname.txt"6 296731 dilbert.nfs > nori.104983: reply ok 128 lookup ERROR: No such file or directory [|nfs]7 304163 nori.104984 > dilbert.nfs: 132 lookup fh 85,10/10087924 "myname.txt"8 305073 dilbert.nfs > nori.104984: reply ok 128 lookup ERROR: No such file or directory [|nfs]9 306429 nori.104985 > dilbert.nfs: 132 lookup fh 85,10/10087924 "myname.txt"10 307262 dilbert.nfs > nori.104985: reply ok 128 lookup ERROR: No such file or directory [|nfs]11 308601 nori.104986 > dilbert.nfs: 164 create fh 85,10/10087924 "myname.txt"12 335731 dilbert.nfs > nori.104986: reply ok 128 create fh 85,10/1008799113 336947 nori.104987 > dilbert.nfs: 116 getattr fh 85,10/1008799114 337697 dilbert.nfs > nori.104987: reply ok 96 getattr REG 100640 ids 5564/5564 sz 015 342637 nori.104988 > dilbert.nfs: 160 write fh 85,10/10087991 26 (26) bytes @ 0 (0)16 358635 dilbert.nfs > nori.104988: reply ok 96 write

26

26

NFS Example : tcpdumpNFS Example : tcpdump

C gloin[8:25pm][~/]>cat 856/myname.txtchavalit srisathapornphat

1 373176 gloin.170291 > dilbert.nfs: 116 getattr fh 85,10/99259492 373722 dilbert.nfs > gloin.170291: reply ok 96 getattr DIR 40711 ids 5564/5564 sz 15363 375406 gloin.170292 > dilbert.nfs: 124 lookup fh 85,10/9925949 "856"4 375944 dilbert.nfs > gloin.170292: reply ok 128 lookup fh 85,10/76568385 377517 gloin.170293 > dilbert.nfs: 132 lookup fh 85,10/7656838 "myname.txt"6 378152 dilbert.nfs > gloin.170293: reply ok 128 lookup fh 85,10/76572357 379663 gloin.170294 > dilbert.nfs: 116 getattr fh 85,10/76572358 380235 dilbert.nfs > gloin.170294: reply ok 96 getattr REG 100640 ids 5564/5564 sz 269 385501 gloin.170295 > dilbert.nfs: 128 read fh 85,10/7657235 4096 bytes @ 010 386208 dilbert.nfs > gloin.170295: reply ok 128 read

=====================================================================================================D bifur[9:24pm][~/]>rm 856/myname.txtbifur[9:25pm][~/]>rm 856/myname.txt

1 645498 bifur.251360 > dilbert.nfs: 116 getattr fh 85,10/99259492 646088 dilbert.nfs > bifur.251360: reply ok 96 getattr DIR 40711 ids 5564/5564 sz 15363 647881 bifur.251361 > dilbert.nfs: 116 getattr fh 85,10/76568384 648435 dilbert.nfs > bifur.251361: reply ok 96 getattr DIR 40750 ids 5564/5564 sz 5125 652008 bifur.251362 > dilbert.nfs: 132 remove fh 85,10/7656838 "myname.txt"6 657761 dilbert.nfs > bifur.251362: reply ok 28 remove7 556377 bifur.251397 > dilbert.nfs: 132 lookup fh 85,10/7656838 "myname.txt"8 557409 dilbert.nfs > bifur.251397: reply ok 128 lookup ERROR: No such file or directory

27

27

NFS proceduresNFS procedures

NFSProcedures

Functions

LOOKUP Returns a file handle and attribute corresponding to a file name in a specified directory.MKDIR Create a directory.RMDIR Delete a directory.READDIR Read a directory. Used by the Unix ls command, for example.RENAME Rename a file.REMOVE Delete a file.CREATE Create a file.READ Read from a file, by specify the file handle, starting offset and max. no. of bytes to read

(up to 8192).WRITE Write to a file.GETATTR Returns the attributes of a file: type of file, permissions, size, owner, last-access time,

and so on.SETATTR Set the attributes of a file: permissions, owner, group, size,and last-access and last-

modification time.LINK Create a Unix hard link to a file.SYMLINK Create a symbolic link to a file.READLINK Returns the name of the file to whidh the symbolic link points.STATFS Returns the status of a file system. Used by the Unix df command, for example.

28

28

Format of RPC callFormat of RPC call

IP header 20 bytes

UDP header 8

Transaction ID (XID) 4Send direction (0) 4RPC version (2) 4Program number 4Version number 4

Procedure number 4

Credentials up to 408 bytes

Verifier up to 408 bytes

Procedure call parameters N

common for all Sun RPC

procedure call

depends on specific procedure being

called

29

29

Format of RPC replyFormat of RPC reply

IP header 20 bytes

UDP header 8

Transaction ID (XID) 4Send direction (1) 4

Status (0=accepted) 4

Verifier up to 400 bytes

Accept status (0=success) 4

Procedure results N

common for all Sun RPC procedure

depends on specific procedure

30

30

Before running Before running tcpdumptcpdump

Please let me mountyour/home directory.

These are attributesof 3625360

What is FH of "srisatha"in 3625360 ?

FH of srisatha is9925949

.

.

.

NFS Clientmachine

NFS Servermachine

¶

¸

·

¹

Mount : When NFS client starts up and mounts home directory

NFS :When a user login

NFS

Mount