SAJENDRA KUMARM.Tech-CS- Shobhit Univerty, Meerut

LESSON – 2: INTRODUCTION TO PROCESSCONTENTS2.1 Aims and Objectives2.2 Introduction to process2.3 Process states2.4 Process state transitions2.5 Operations on process2.5.1 Suspend and resume2.5.2 Suspending a process2.6 Let us sum up2.7 Points for Discussion2.8 Model answers to “Check your Progress”2.9 Lesson end activities2.10 References2.1 Aims and ObjectivesIn this lesson we will learn about the introduction of process, various states of theprocess, and process transitions. The objectives of this lesson are to make the student awareof the basic concepts process and its behaviors in an operating system.2.2 Introduction to ProcessProcess is defined as a program in execution and is the unit of work in a modern timesharingsystem. Such a system consists of a collection of processes: Operating-systemprocesses executing system code and user processes executing user code. All these processescan potentially execute concurrently, with the CPU (or CPUs) multiplexed among them. Byswitching the CPU between processes, the operating system can make the computer moreproductive.A process is more than the program code, it includes the program counter, the processstack, and the contents of process register etc. The purpose of process stack is to storetemporary data, such as subroutine parameters, return address and temporary variables. Allthese information will be stored in Process Control Block (PCB). The Process control blockis a record containing many pieces of information associated with a process including processstate, program counter, cpu registers, memory management information, accountinginformation, I/O status information, cpu scheduling information, memory limits, and list ofopen files.112.3 Process StatesWhen a process executes, it changes the state, generally the state of process isdetermine by the current activity of the process. Each process may be in one of the followingstates.New-------------: The process is being created.Running--------: The process is being executedWaiting---------: The process is waiting for some event to occur.Ready-----------: The process is waiting to be assigned to a processor.Terminate------: The process has finished execution.The important thing is only one process can be running in any processor at any time.But many processes may be in ready and waiting states. The ready processes are loaded in toa “ready queue”. A queue is one type of data structure. It is used here to store process.The operating system creates a process and prepares the process to be executed thenthe operating systems moved the process in to ready queue. When it is time to select aprocess to run, the operating system selects one of the jobs from the ready queue and movesthe process from ready state to running state. When the execution of a process has completedthen the operating system terminates that process from running state. Some times operatingsystem terminates the process for some other reasons also which include time limit exceeded,memory unavailable, access violation, protection error, I/O failure, data misuse and so on.When the time slot of the processor expires or if the processor receives any interruptsignal, then the operating system shifts running process to ready state. For example, letprocess P1 be executing in the processor and in the mean time let process P2 generates aninterrupt signal to the processor. The processor compares the priorities of process P1 and P2.If P1>P2 then the processor continue the process P1. Otherwise the processor switches toprocess P2 and the process P1 is moved to ready state.A process is put into the waiting state, if the process need an event to occur or an I/Otask is to be done. For example if a process in running state need an I/O device, then the

process is moved to blocked (or) waiting state.A process in the blocked (waiting) state is moved to the ready state when the event forwhich it has been waiting occurs.The OS maintains a ready list and a blocked list to store references to processes notrunning. The following figure shows the process state diagramNew Ready running Terminatedwaiting12The new and terminated states are worth a bit of more explanation. The former referto a process that has just been defined (e.g. because an user issued a command at a terminal),and for which the OS has performed the necessary housekeeping chores. The latter refers to aprocess whose task is not running anymore, but whose context is still being saved (e.g.because an user may want to inspect it using a debugger program).A simple way to implement this process handling model in a multiprogramming OSwould be to maintain a queue (i.e. a first-in-first-out linear data structure) of processes, put atthe end the queue the current process when it must be paused, and run the first process in thequeue.However, it's easy to realize that this simple two-state model does not work. Giventhat the number of processes that an OS can manage is limited by the available resources, andthat I/O events occur at much larger time scale that CPU events, it may well be the case thatthe first process of the queue must still wait for an I/O event before being able to restart; evenworse, it may happen that most of the processes in the queue must wait for I/O. In thiscondition the scheduler would just waste its time shifting the queue in search of a runnableprocess.A solution is to split the not-running process class according to two possibleconditions: processes blocked in the wait for an I/O event to occur, and processes in pause,but nonetheless ready to run when given a chance. A process would then be put from runninginto blocked state on account of an event wait transition, would go running to ready state dueto a timeout transition, and from blocked to ready due to event occurred transition.This model would work fine if the OS had an very large amount of main memoryavailable and none of the processes hogged too much of it, since in this case there wouldalways be a fair number of ready processes. However, because the costs involved thisscenario is hardly possible, and again the likely result is a list of blocked processes all waitingfor I/O.2.4 Process State TransitionThe various process states, displayed in a state diagram, with arrows indicatingpossible transitions between states. Processes go through various process states whichdetermine how the process is handled by the operating system kernel. The specificimplementations of these states vary in different operating systems, and the names of thesestates are not standardized, but the general high-level functionality is the same.When a process is created, it needs to wait for the process scheduler (of the operatingsystem) to set its status to "waiting" and load it into main memory from secondary storagedevice (such as a hard disk or a CD-ROM). Once the process has been assigned to aprocessor by a short-term scheduler, a context switch is performed (loading the process intothe processor) and the process state is set to "running" - where the processor executes itsinstructions. If a process needs to wait for a resource (such as waiting for user input, orwaiting for a file to become available), it is moved into the "blocked" state until it no longerneeds to wait - then it is moved back into the "waiting" state. Once the process finishesexecution, or is terminated by the operating system, it is moved to the "terminated" statewhere it waits to be removed from main memory. The act of assigning a processor to the first13process on the ready list is called dispatching. The OS may use an interval timer to allow aprocess to run for a specific time interval or quantum.2.5 Operations on ProcessThere are various operations that can be performed on a process and are listed below.a) createb) destroyc) suspendd) resumee) change priorityf) blockg) wake uph) dispatchi) enable2.5.1 Suspend and Resume

The OS could then perform a suspend transition on blocked processes, swapping themon disk and marking their state as suspended (after all, if they must wait for I/O, they mightas well do it out of costly RAM), load into main memory a previously suspended process,activating into ready state and go on. However, swapping is an I/O operation in itself, and soat a first sight things might seem to get even worse doing this way. Again the solution is tocarefully reconsider the reasons why processes are blocked and swapped, and recognize thatif a process is blocked because it waits for I/O, and then suspended. The I/O event mightoccur while it sits swapped on the disk.14Process state transitions with suspend and resumeWe can thus classify suspended processes into two classes: ready-suspended for thosesuspended process whose restarting condition has occurred, and blocked-suspended for thosewho must still wait instead. This classification allows the OS to pick from the good pool ofready-suspended processes, when it wants to revive the queue in main memory. Provisionsmust be made for passing processes between the new states. This means allowing for newtransitions: activate and suspend between ready and ready-suspended, and between blockedsuspendedand blocked as well, end event-occurred transitions from blocked to ready, andfrom blocked-suspended to ready-suspended as well.2.5.2 Suspending a process• Indefinitely removes it from contention for time on a processor without beingdestroyed• Useful for detecting security threats and for software debugging purposes• A suspension may be initiated by the process being suspended or by another process• A suspended process must be resumed by another process2.6 Let us Sum UpIn this lesson we have learnt abouta) the Processb) the process statesc) the process control blockd) the process state transitions152.7 Points for Discussiona) Discuss about process control blockb) Discuss about the process transition diagram2.8 Model answers to “Check your Progress”A process is more than the program code, it includes the program counter, the processstack, and the contents of process register etc. The purpose of process stack is to storetemporary data, such as subroutine parameters, return address and temporary variables. Allthese information will be stored in Process Control Block (PCB). The Process control blockis a record containing many pieces of information associated with a process including processstate, program counter, cpu registers, memory management information, accountinginformation, I/O status information, cpu scheduling information, memory limits, and list ofopen files.2.9 Lesson end ActivitiesAfter learning this chapter, try to discuss among your friends and answer thesequestions to check your progress.a) Define Processb) What are the various possible states of a process2.10 Referencese) Charles Crowley, Chapter 5, 8 of “Operating Systems – A Design-OrientedApproach”, Tata McGraw-Hill, 2001f) H.M. Deitel, Chapter 3, 4 of “Operating Systems”, Second Edition, PearsonEducation, 2001g) Andrew S. Tanenbaum, Chapter 2 of “Modern Operating Systems”, PHI, 1996h) D.M. Dhamdhere, Chapter 10 of “Systems Programming and Operating Systems”,Tata McGraw-Hill, 199716LESSON – 3: INTERRUPT PROCESSING AND CONTEXTSWITCHINGCONTENTS3.1 Aims and Objectives3.2 Interrupt Processing3.2.1 Identifying the High Level Routine3.2.2 Interrupt Dispatch Table3.2.3 Rules for Interrupt Processing

3.2.4 Rescheduling while Processing an Interrupt3.2.5 Interrupt Classes3.3 Context Switching3.3.1 Context Switches and Mode Switches3.3.2 Cost of Context Switching3.4 Let us Sum Up3.5 Points for Discussion3.6 Model answers to “Check your Progress”3.7 Lesson end Activities3.8 References3.1 Aims and ObjectivesThis lesson focuses on the following conceptsa) Introduction to interrupt processingb) Interrupt classesc) Context switchingThe main objective of this lesson is to make the student aware of the interruptprocessing, classes and context switching.3.2 Introduction to Interrupt ProcessingAn interrupt is an event that alters the sequence in which a processor executesinstructions and it is generated by the hardware of the computer system.Handling interrupts• After receiving an interrupt, the processor completes execution of the currentinstruction, then pauses the current process• The processor will then execute one of the kernel’s interrupt-handling functions• The interrupt handler determines how the system should respond• Interrupt handlers are stored in an array of pointers called the interrupt vector17• After the interrupt handler completes, the interrupted process is restored and executedor the next process is executedInterrupt handlers should be written in high-level languages so that they are easy tounderstand and modify. They should be written in assembly language for efficiency reasonsand because they manipulate hardware registers and use special call/return sequences thatcannot be coded in high-level languages.To satisfy both goals certain OS employs the following two-level strategy. Interruptsbranch to low-level interrupt dispatch routines that are written in assembly language. Thesehandle low-level tasks such as saving registers and returning from the interrupt when it hasbeen processed. However, they do little else, they call high-level interrupt routines to do thebulk of interrupt processing, passing them enough information to identify the interruptingdevice. The OS provides three interrupt dispatchers: one to handle input interrupts, one tohandle output interrupts, and one to handle clock interrupts. Input and output dispatchers areseparated for convenience, and a special clock dispatcher is provided for efficiency reasons.NT provides an even more modular structure. A single routine, called the trap handler,handles both traps (called exceptions by NT) and interrupts, saving and restoring registers,which are common to both. If the asynchronous event was an interrupt, then it calls aninterrupt handler. The task of this routine is to raise the processor priority to that of the deviceinterrupting (so that a lower-level device cannot preempt), call either an internal kernelroutine or an external routine called an ISR, and then restore the processor priority. Thesetwo routines roughly correspond to our high-level interrupt routine and the trap handlercorresponds to our low-level routine. Thus, NT trades off the efficiency of 2 levels for thereusability of 3 levels.The device table entry for an input or output interrupt handler points at the high-levelpart of the interrupt handler, which is device-specific, and not the low-level part which isshared by all devices (except the clock).18Handling interrupts3.2.1 Identifying the High-Level RoutineIf all input (output) interrupts branch to the same input (output) dispatch routine, howdoes the dispatcher know which device-specific interrupt routine to call? The input (output)dispatch routine needs some way to discover the device that interrupted it, so that it can usethis information to call the appropriate high-level routine. There are several ways to identifyan interrupting device. Here are two of them:The dispatcher may use a special machine instruction to get either the device addressor the interrupt vector address of the device. Not all machines have such instructions. Thedispatcher may poll devices until it finds one with an interrupt pending.The following `trick' is used, which is common to operating systems, to help identify

the interrupting device. The device descriptor (not the device address) is stored in the secondword of the interrupt vector. Recall that this word stores the value to be loaded into the PSregister when the interrupt routine is called. Certain OS uses the lower order 4 bits, which areused for condition codes, to store the descriptor of the device. These four bits are then used toidentify the high-level routine.193.2.2 Interrupt Dispatch TableAn interrupt dispatch table is used to relate device descriptors with (high-level)interrupt routines. The table is indexed by a device descriptor and each entry contains thefollowing information:• The address of the input interrupts routine• An input code which is passed as an argument to the input interrupts routine• The address of the output interrupts routine• An output code which is passed as an argument to the above routineThe input (output) dispatch routine uses the device descriptor to access the appropriatedispatch table entry and calls the input (output) interrupt routine with the input (output) codeas an argument code. The input and output codes can be anything the high-level routinesneed. In certain OS, they are initially the minor number of the device. Thus only one interruptroutine is needed for all devices of the same type. The minor number is used to distinguishbetween these devices.3.2.3 Rules for Interrupt ProcessingThere are several rules for interrupt processing: First, they should ensure that shareddata are not manipulated simultaneously by different processes. One way to do so is to makeinterrupt routines uninterruptible. Thus the PS value stored in the interrupt vector of a devicedisables interrupts. This value is loaded when the interrupt handler is invoked. As a result,the interrupt routine is uninterruptible while the PS maintains this priority.However, the PS may be changed while the interrupt routine is executing if it callsresched, which may switch to a process that has interrupts enabled. Therefore, the interruptroutine has to ensure that it completes changes to global data structures before it makes anycall that results in context switching. An interrupt routine should also make sure that it doesnot keep interrupts disabled too long. For instance, if the processor does not accept acharacter from an input device before another arrives, data will be lost.Finally, interrupt routines should never call routines that could block the currentprocess (that is the process executing when the interrupt occurred) in some queue. Otherwise,if the interrupt occurs while the null process is executing, the ready list will be empty.However, resched assumes that there is always some process to execute! Some processshould always be runnable to that interrupts can be executed. Thus interrupt routines need tocall only those routines that leave the current process in the ready or current state, and maynot call routines such as wait.3.2.4 Rescheduling while Processing an InterruptWe assumed above that interrupt routines could call resched. We now answer thefollowing questions: First, is it useful to do so? Second, is it safe?It is useful to call resched from an interrupt routine. An output routine after removinga character from a buffer may signal a semaphore to allow another process to write data to thebuffer space that it makes available. Similarly, an input routine might send data it obtainsfrom the device to a process. In each case, the routine resched is called.20It is also safe to call resched. Intuitively it may not seem so, because switching to aprocess that has interrupts enabled could lead to a sequence of interrupts piling up until thestack overflowed. However, such a danger does not exist for the following reason: A processthat is executing an interrupt handler cannot be interrupted again. Some other process,however, can be. Thus a process's stack will hold the PS and PC value for only one interruptand there will never be more interrupts pending than the number of processes in the system.3.2.5 Interrupt ClassesSVC (supervisor call) interrupts: - They enable software to respond to signals fromhardware. These are initiated by a running process that executes the SVC instruction. AnSVC is a user generated request for a particular system service such ad performinginput/output, obtaining more storage, or communicating with the system operator. A usermust request a service through an SVC which helps the OS secure from the user.I/O interrupts: - These are initiated by the input/output hardware. They signal to the CPUthat the status of a channel or device has changed. For example, I/O interrupts are causedwhen an I/O operation completes, when an I/O error occurs, or when a device is made ready.External interrupts: - These are caused by various events including the expiration of aquantum on an interrupting clock, the pressing of the console’s interrupt key by the operator,or the receipt of a signal from another processor on a multiprocessor system.

Restart interrupts: - These occur when the operator presses the console’s restart button, orwhen a restart SIGP (signal processor) instruction arrives from another processor on amultiprocessor system.Program check interrupts: - These occur as a program’s machine language instructions areexecuted. These problems include divide by zero, arithmetic overflow, data is in wrongformat, attempt to execute an invalid operation code, attempt to reference beyond the limitsof real memory, attempt by a user process to execute a privileged instruction and attempts toreference a protected resources.Machine check interrupts: - These are caused by malfunctioning hardware.3.3 Context SwitchingA context switch (also sometimes referred to as a process switch or a task switch) isthe switching of the CPU (central processing unit) from one process or thread to another. Aprocess (also sometimes referred to as a task) is an executing (i.e., running) instance of aprogram. In Linux, threads are lightweight processes that can run in parallel and share anaddress space (i.e., a range of memory locations) and other resources with their parentprocesses (i.e., the processes that created them).A context is the contents of a CPU's registers and program counter at any point intime. A register is a small amount of very fast memory inside of a CPU (as opposed to theslower RAM main memory outside of the CPU) that is used to speed the execution ofcomputer programs by providing quick access to commonly used values, generally those inthe midst of a calculation. A program counter is a specialized register that indicates theposition of the CPU in its instruction sequence and which holds either the address of the21instruction being executed or the address of the next instruction to be executed, depending onthe specific system.Context switching can be described in slightly more detail as the kernel (i.e., the coreof the operating system) performing the following activities with regard to processes(including threads) on the CPU: (1) suspending the progression of one process and storing theCPU's state (i.e., the context) for that process somewhere in memory, (2) retrieving thecontext of the next process from memory and restoring it in the CPU's registers and (3)returning to the location indicated by the program counter (i.e., returning to the line of code atwhich the process was interrupted) in order to resume the process.A context switch is sometimes described as the kernel suspending execution of oneprocess on the CPU and resuming execution of some other process that had previously beensuspended. Although this wording can help clarify the concept, it can be confusing in itselfbecause a process is, by definition, an executing instance of a program. Thus the wordingsuspending progression of a process might be preferable.3.3.1 Context Switches and Mode SwitchesContext switches can occur only in kernel mode. Kernel mode is a privileged mode ofthe CPU in which only the kernel runs and which provides access to all memory locationsand all other system resources. Other programs, including applications, initially operate inuser mode, but they can run portions of the kernel code via system calls. A system call is arequest in a UNIX-like operating system by an active process (i.e., a process currentlyprogressing in the CPU) for a service performed by the kernel, such as input/output (I/O) orprocess creation (i.e., creation of a new process). I/O can be defined as any movement ofinformation to or from the combination of the CPU and main memory (i.e. RAM), that is,communication between this combination and the computer's users (e.g., via the keyboard ormouse), its storage devices (e.g., disk or tape drives), or other computers.The existence of these two modes in Unix-like operating systems means that a similar,but simpler, operation is necessary when a system call causes the CPU to shift to kernelmode. This is referred to as a mode switch rather than a context switch, because it does notchange the current process.Context switching is an essential feature of multitasking operating systems. Amultitasking operating system is one in which multiple processes execute on a single CPUseemingly simultaneously and without interfering with each other. This illusion ofconcurrency is achieved by means of context switches that are occurring in rapid succession(tens or hundreds of times per second). These context switches occur as a result of processesvoluntarily relinquishing their time in the CPU or as a result of the scheduler making theswitch when a process has used up its CPU time slice.A context switch can also occur as a result of a hardware interrupt, which is a signalfrom a hardware device (such as a keyboard, mouse, modem or system clock) to the kernelthat an event (e.g., a key press, mouse movement or arrival of data from a networkconnection) has occurred.Intel 80386 and higher CPUs contain hardware support for context switches.However, most modern operating systems perform software context switching, which can be

22used on any CPU, rather than hardware context switching in an attempt to obtain improvedperformance. Software context switching was first implemented in Linux for Intel-compatibleprocessors with the 2.4 kernel.One major advantage claimed for software context switching is that, whereas thehardware mechanism saves almost all of the CPU state, software can be more selective andsave only that portion that actually needs to be saved and reloaded. However, there is somequestion as to how important this really is in increasing the efficiency of context switching.Its advocates also claim that software context switching allows for the possibility ofimproving the switching code, thereby further enhancing efficiency, and that it permits bettercontrol over the validity of the data that is being loaded.3.3.2 The Cost of Context SwitchingContext switching is generally computationally intensive. That is, it requiresconsiderable processor time, which can be on the order of nanoseconds for each of the tens orhundreds of switches per second. Thus, context switching represents a substantial cost to thesystem in terms of CPU time and can, in fact, be the most costly operation on an operatingsystem.Consequently, a major focus in the design of operating systems has been to avoidunnecessary context switching to the extent possible. However, this has not been easy toaccomplish in practice. In fact, although the cost of context switching has been decliningwhen measured in terms of the absolute amount of CPU time consumed, this appears to bedue mainly to increases in CPU clock speeds rather than to improvements in the efficiency ofcontext switching itself.One of the many advantages claimed for Linux as compared with other operatingsystems, including some other Unix-like systems, is its extremely low cost of contextswitching and mode switching.Context switches are– Performed by the OS to stop executing a running process and begin executinga previously ready process– Save the execution context of the running process to its PCB– Load the ready process’s execution context from its PCB– Must be transparent to processes– Require the processor to not perform any “useful” computation• OS must therefore minimize context-switching time– Performed in hardware by some architectures3.4 Let us Sum UpIn this lesson we have learnt abouta) the Interrupt Processingb) the interrupt classesc) and the context switching233.5 Points for Discussiona) Discuss about context switchingb) Discuss about the interrupt classes3.6 Model answers to “Check your Progress”A context switch (also sometimes referred to as a process switch or a task switch) isthe switching of the CPU (central processing unit) from one process or thread to another. Aprocess (also sometimes referred to as a task) is an executing (i.e., running) instance of aprogram. In Linux, threads are lightweight processes that can run in parallel and share anaddress space (i.e., a range of memory locations) and other resources with their parentprocesses (i.e., the processes that created them).3.7 Lesson - end ActivitiesAfter learning this chapter, try to discuss among your friends and answer thesequestions to check your progress.a) Discuss about the interruptb) Discuss about various types of interrupt processing3.8 Referencesa) Charles Crowley, Chapter 5, 8 of “Operating Systems – A Design-OrientedApproach”, Tata McGraw-Hill, 2001b) H.M. Deitel, Chapter 3, 4 of “Operating Systems”, Second Edition, PearsonEducation, 2001c) Andrew S. Tanenbaum, Chapter 2 of “Modern Operating Systems”, PHI, 1996d) D.M. Dhamdhere, Chapter 10 of “Systems Programming and Operating Systems”,Tata McGraw-Hill, 199724

LESSON – 4: SEMAPHORESCONTENTS4.1 Aims and Objectives4.2 Introduction to process synchronization4.3 Critical section problem4.4 Semaphores4.5 Classical problems of synchronization4.6 Let us Sum Up4.7 Points for discussion4.8 Model answers to Check your progress4.9 Lesson end Activities4.10 References4.1 Aims and ObjectivesThe aim of this lesson is to learn the concept of process synchronization, criticalsection problem, semaphores and classical problems of synchronizationThe objectives of this lesson are to make the student aware of the following conceptsa) process synchronizationb) critical section problemc) semaphoresd) and classical problems of synchronization4.2 Introduction to process synchronizationProcess synchronization will be clear with the following example. Consider the code forconsumer and producer as followsProducerwhile(1){while(counter = = buffersize);buffer[in]=nextproduced;in = (in+1) % buffersize;counter ++;}Consumerwhile(1){while(counter = = 0);25nextconsumed = buffer[out];out = (out+1) % buffersize;counter --;}Both the codes are correct separately but will not function correctly when executedconcurrently. This is because the counter++ may be executed in machine language as threeseparate statements asregister1 = counterregister1 = register1 + 1counter = register1and the counter- - asregister2 = counterregister2 = register2 - 1counter = register2The execution of these statements for the two processes may lead to the followingcondition for example.a) producer execute register1 = counter (register1 =5)b) producer execute register1 = register1 + 1 (register1 =6)c) consumer execute register2= counter (register2 =5)d) consumer execute register2 = register2 - 1 (register2 =4)e) producer execute counter = register1 (counter = 6)f) consumer execute counter = register2 (counter = 4)You can see that the answer counter =4 is wrong as there are 5 full buffers.A situation like this, where several processes access and manipulate the same dataconcurrently and the outcome of the execution depends on the particular order in which theaccess take place, is called a race condition. For this we have to make sure that only oneprocess at a time should manipulate the counter. Such situation occurs frequently in OS andwe require some form of synchronization of processes.4.3 Critical section problem

Each process will be having a segment of code called a critical section, in which theprocess may be changing a common variable, updating a table, writing a file, and so on.do{entry sectioncritical sectionexit section26reminder section}while(1);A solution should satisfy the following three requirementsa) Mutual exclusion: If a process is executing in its critical section, then no otherprocesses can be executing in their critical sectionsb) Progress: If no process is executing in its critical section and some processes wish toenter their critical sections, then only those processes that are not executing in theirremainder section can participate in the decision on which will enter its criticalsection next, and this selection cannot be postponed indefinitely.c) Bounded waiting: There exists a bound on the number of times that other processesare allowed to enter their critical sections after a process has made a request to enterits critical section and before that request is granted.There are many solutions available and can be done using various implementation methods.One of the multiprocess solutions for the critical section is given belowData structureboolean choosing[n];int number[n];do{chossing[i]=true;number[i]=max(number[0], number[1], …, number[n-1])+1;choosing[i] = false;for(j=0;j<n;j++){while(choosing[j]);while((number[j]!=0)&&(number[j, j]<number[i, i]));}CRITICAL SECTIONnumber[i]=0;REMINDER SECTION}while(1);4.4 SemaphoresThe solution described in the above section cannot be generalized most of the times. Toover come this, we have a synchronization tool called a semaphore proposed by Dijkstra.Semaphores are a pair composed of an integer variable that apart from initialization isaccessed only through two standard atomic operations: wait and signal.27• wait: decrease the counter by one; if it gets negative, block the process and enter its idin the queue.• signal: increase the semaphore by one; if it's still negative, unblock the first process ofthe queue, removing its id from the queue itself.wait(S){while(S<=0);S - -;}signal(S){S++;}The atomicity of the above operations is an essential requirement: mutual exclusion whileaccessing a semaphore must be strictly enforced by the operating system. This is often doneby implementing the operations themselves as uninterruptible system calls.It's easy to see how a semaphore can be used to enforce mutual exclusion on a sharedresource: a semaphore is assigned to the resource, it's shared among all processes that need toaccess the resource, and its counter is initialized to 1. A process then waits on the semaphoreupon entering the critical section for the resource, and signals on leaving it. The first process

will get access. If another process arrives while the former is still in the critical section, it'llbe blocked, and so will further processes. In this situation the absolute value of the counter isequal to the number of processes waiting to enter the critical section. Every process leavingthe critical section will let a waiting process use the resource by signaling the semaphore.In order to fully understand semaphores, we'll discuss them briefly before engaging anysystem calls and operational theory.The name semaphore is actually an old railroad term, referring to the crossroad ``arms''that prevent cars from crossing the tracks at intersections. The same can be said about asimple semaphore set. If the semaphore is on (the arms are up), then a resource is available(cars may cross the tracks). However, if the semaphore is off (the arms are down), thenresources are not available (the cars must wait).While this simple example may stand to introduce the concept, it is important to realizethat semaphores are actually implemented as sets, rather than as single entities. Of course, agiven semaphore set might only have one semaphore, as in our railroad example.Perhaps another approach to the concept of semaphores is to think of them as resourcecounters. Let's apply this concept to another real world scenario. Consider a print spooler,capable of handling multiple printers, with each printer handling multiple print requests. Ahypothetical print spool manager will utilize semaphore sets to monitor access to each printer.Assume that in our corporate print room, we have 5 printers online. Our print spool managerallocates a semaphore set with 5 semaphores in it, one for each printer on the system. Sinceeach printer is only physically capable of printing one job at a time, each of our five28semaphores in our set will be initialized to a value of 1 (one), meaning that they are allonline, and accepting requests.John sends a print request to the spooler. The print manager looks at the semaphore set,and finds the first semaphore which has a value of one. Before sending John's request to thephysical device, the print manager decrements the semaphore for the corresponding printerby a value of negative one (-1). Now, that semaphore's value is zero. In the world of SystemV semaphores, a value of zero represents 100% resource utilization on that semaphore. In ourexample, no other request can be sent to that printer until it is no longer equal to zero.When John's print job has completed, the print manager increments the value of thesemaphore which corresponds to the printer. Its value is now back up to one (1), which meansit is available again. Naturally, if all 5 semaphores had a value of zero, that would indicatethat they are all busy printing requests, and that no printers are available.Although this was a simple example, please do not be confused by the initial value of one(1) which was assigned to each semaphore in the set. Semaphores, when thought of asresource counters, may be initialized to any positive integer value, and are not limited toeither being zero or one. If it were possible for each of our five printers to handle 10 printjobs at a time, we could initialize each of our semaphores to 10, decrementing by one forevery new job, and incrementing by one whenever a print job was finished. Semaphores havea close working relationship with shared memory segments, acting as a watchdog to preventmultiple writes to the same memory segment.4.5 Classic problems of synchronizationThere are several problems of synchronization. Some of them are bounded bufferproblem, reader writers problem, and dining philosophers problem. In this section we explainonly the solution of bounded-buffer problem, namely the producer consumer problem.The solution for producer-consumer problem can be achieved using semaphore as shownin the following code, where mutex, empty and full are semaphores initialized to 1, n 0respectivily.CODE FOR PRODUCERdo{…produce an item in nextp…wait(empty);wait(mutex);…add nextp to buffer… signal(mutex);signal(full);}while(1);CODE FOR CONSUMER29do

{wait(full);wait(mutex);…remove an item from buffer to nextc… signal(mutex);signal(empty);…consume the item in nextc…}while(1);4.6 Let us Sum UpIn this lesson we have learnt abouta) the process synchronizationb) the critical section problemc) the Semaphoresd) and the classical problems of synchronization4.7 Points for DiscussionAfter learning this chapter, try to discuss among your friends and answer thesequestions to check your progress.a) What is synchronizationb) Define critical section problemc) Define Semaphored) Discuss about the need of Semaphorese) Discuss about synchronization based on an example4.8 Model answers to “Check your Progress”A semaphore, in computer science, is a protected variable (or abstract data type)which constitutes the classic method for restricting access to shared resources, such as sharedmemory, in a multiprogramming environment. A semaphore is a counter for a set of availableresources, rather than a locked/unlocked flag of a single resource. It was invented by EdsgerDijkstra and first used in the THE operating system. The value of the semaphore isinitialized to the number of equivalent shared resources being controlled. In the special casewhere there is a single equivalent shared resource, the semaphore is called a binarysemaphore. The general-case semaphore is often called a counting semaphore.Semaphores are the classic solution to the dining philosophers problem, although they do notprevent all resource deadlocks.4.9 Lesson end Activities30Try to implement semaphore in C under Unix4.10 Referencesa) Charles Crowley, Chapter 8 of “Operating Systems – A Design-OrientedApproach”, Tata McGraw-Hill, 2001b) H.M. Deitel, Chapter 4, 5 of “Operating Systems”, Second Edition, PearsonEducation, 2001c) Andrew S. Tanenbaum, Chapter 11 of “Modern Operating Systems”, PHI, 1996d) D.M. Dhamdhere, Chapter 13 of “Systems Programming and Operating Systems”,Tata McGraw-Hill, 199731LESSON – 5: DEADLOCK AND INDEFINITE POSTPONEMENTCONTENTS5.1 Aims and Objectives5.2 Introduction5.3 Characteristics to Deadlock5.4 Deadlock prevention and avoidance5.5 Deadlock detection and recovery5.6 Let us Sum Up5.7 Points for discussion5.8 Model answers to Check your Progress5.9 Lesson - end Activities5.10 References5.1 Aims and ObjectivesThe aim of this lesson is to learn the concept of Deadlock and indefinite postponement.The objectives of this lesson are to make the student aware of the following conceptsa) Deadlock preventionb) Deadlock Avoidance

c) Deadlock detectiond) Deadlock recovery5.2 IntroductionOne problem that arises in multiprogrammed systems is deadlock. A process or threadis in a state of deadlock (or is deadlocked) if the process or thread is waiting for a particularevent that will not occur. In a system deadlock, one or more processes are deadlocked. Mostdeadlocks develop because of the normal contention for dedicated resources (i.e., resourcesthat may be used by only one user at a time). Circular wait is characteristic of deadlockedsystems.One example of a system that is prone to deadlock is a spooling system. A commonsolution is to restrain the input spoolers so that, when the spooling files begin to reach somesaturation threshold, they do not read in more print jobs. Today's systems allow printing tobegin before the job is completed so that a full, or nearly full, spooling file can be emptied orpartially cleared even while a job is still executing. This concept has been applied tostreaming audio and video clips, where the audio and video begin to play before the clips arefully downloaded.In any system that keeps processes waiting while it makes resource-allocation andprocess scheduling decisions, it is possible to delay indefinitely the scheduling of a processwhile other processes receive the system's attention. This situation, variously called indefinite32postponement, indefinite blocking, or starvation, can be as devastating as deadlock. Indefinitepostponement may occur because of biases in a system's resource scheduling policies. Somesystems prevent indefinite postponement by increasing a process's priority as it waits for aresource—this technique is called aging.Resources can be preemptable (e.g., processors and main memory), meaning that theycan be removed from a process without loss of work, or nonpreemptible meaning that they(e.g., tape drives and optical scanners), cannot be removed from the processes to which theyare assigned. Data and programs certainly are resources that the operating system mustcontrol and allocate. Code that cannot be changed while in use is said to be reentrant. Codethat may be changed but is reinitialized each time it is used is said to be serially reusable.Reentrant code may be shared by several processes simultaneously, whereas serially reusablecode may be used by only one process at a time. When we call particular resources shared,we must be careful to state whether they may be used by several processes simultaneously orby only one of several processes at a time. The latter kind—serially reusable resources—arethe ones that tend to become involved in deadlocks.5.3 Characteristics of DeadlockThe four necessary conditions for deadlock are:a) A resource may be acquired exclusively by only one process at a time (mutualexclusion condition);b) A process that has acquired an exclusive resource may hold it while waiting toobtain other resources (wait-for condition, also called the hold-and-wait condition);c) Once a process has obtained a resource, the system cannot remove the resourcefrom the process's control until the process has finished using the resource (nopreemptioncondition);d) And two or more processes are locked in a "circular chain" in which each processin the chain is waiting for one or more resources that the next process in the chain isholding (circular-wait condition).Because these are necessary conditions for a deadlock to exist, the existence of a deadlockimplies that each of them must be in effect. Taken together, all four conditions are necessaryand sufficient for deadlock to exist (i.e., if all these conditions are in place, the system isdeadlocked).The four major areas of interest in deadlock research are deadlock prevention, deadlockavoidance, deadlock detection, and deadlock recovery.335.4 Deadlock prevention and avoidance5.4.1 Deadlock preventionIn deadlock prevention our concern is to condition a system to remove any possibilityof deadlocks occurring. Havender observed that a deadlock cannot occur if a system deniesany of the four necessary conditions.The first necessary condition, namely that processes claim exclusive use of theresources they require, is not one that we want to break, because we specifically want toallow dedicated (i.e., serially reusable) resources.Denying the "wait-for" condition requires that all of the resources a process needs tocomplete its task be requested at once, which can result in substantial resourceunderutilization and raises concerns over how to charge for resources.

Denying the "no-preemption" condition can be costly, because processes lose workwhen their resources are preempted.Denying the "circular-wait" condition uses a linear ordering of resources to preventdeadlock. This strategy can increase efficiency over the other strategies, but not withoutdifficulties.5.4.2 Deadlock avoidanceIn deadlock avoidance the goal is to impose less stringent conditions than in deadlockprevention in an attempt to get better resource utilization. Avoidance methods allow thepossibility of deadlock to loom, but whenever a deadlock is approached, it is carefullysidestepped. Dijkstra's Banker's Algorithm is an example of a deadlock avoidance algorithm.In the Banker's Algorithm, the system ensures that a process's maximum resource need doesnot exceed the number of available resources. The system is said to be in a safe state if theoperating system can guarantee that all current processes can complete their work within afinite time. If not, then the system is said to be in an unsafe state. Dijkstra's Banker'sAlgorithm requires that resources be allocated to processes only when the allocations result insafe states. It has a number of weaknesses (such as requiring a fixed number of processes andresources) that prevent it from being implemented in real systems.A deadlock avoidance algorithm dynamically examines the resource allocation stateto ensure that there can never be a circular wait condition. The resource allocation state isdefined by the number of available and allocated resources, and the maximum demands ofthe processes. A state is safe if the system can allocate resources to each process (up to itsmaximum) in some order and still avoid a deadlock.345.4.3 Banker’s algorithmLet ‘Available’ be a vector of length m indicating the number of available resourcesof each type, ‘Max’ be an ‘n x n’ matrix defining the maximum demand of each process,‘Allocation’ be an ‘n x m’ matrix defining the number of resources of each type currentlyallocated to each process, and let ‘need’ be an ‘n x m’ matrix indicating the remainingresource need of each process.Let Requesti be the request vector for process pi. If requesti[j]=k, then process piwants k instances of resource type rj. When a request for resources is made by process pi, thefollowing actions are taken:a) If Requesti< = Needi then proceed to step b. Else the process has exceeded itsmaximum claimb) If Requesti< = Available the proceed to step c. Else the resources are notavailable and pi must waitc) The system pretends to have allocated the requested resources to process pi bymodifying the state as follows.Available = Available - RequestiAllocationi = Allocationi + RequestiNeedi = Needi - RequestiIf the resulting resource allocation state is safe, the transaction is completed and process pi isallocated its resources. If the new state is unsafe, the pi must wait for Requesti and the oldresource allocation state is restored.5.4.4 Safety AlgorithmThe algorithm for finding out whether a system is in a safe state or not can bedescribed as follows.a) Let Work and Finish be vectors of length m and n. Initialize Work =Available and Finish[i] = falseb) Find an i such thata. Finish[i] = false andb. Needi < = WorkIf no such i exists, go to step dc) Work = Work + AllocationiFinish[i] = trueGo to step bc) If Finish[i] = true for all i, then the system is in a safe state.355.5 Deadlock detection and recoveryDeadlock detection methods are used in systems in which deadlocks can occur. Thegoal is to determine if a deadlock has occurred, and to identify those processes and resourcesinvolved in the deadlock. Deadlock detection algorithms can incur significant runtimeoverhead. To facilitate the detection of deadlocks, a directed graph indicates resourceallocations and requests. Deadlock can be detected using graph reductions. If a process'sresource requests may be granted, then we say that a graph may be reduced by that process. If

a graph can be reduced by all its processes, then there is no deadlock. If a graph cannot bereduced by all its processes, then the irreducible processes constitute the set of deadlockedprocesses in the graph.a) Let Work and Finish be vectors of length m and n. Initialize Work =Available. For i = 1, …, n, if Allocationi ≠ 0 then Finish[i] = false, elseFinish[i] = Trueb) Find an i such thata. Finish[i] = false andb. Requesti < = WorkIf no such i exists, go to step dc) Work = Work + AllocationiFinish[i] = trueGo to step bc) If Finish[i] = false for some i, then the system is in deadlock state.Deadlock recovery methods are used to clear deadlocks from a system so that it mayoperate free of the deadlocks, and so that the deadlocked processes may complete theirexecution and free their resources. Recovery typically requires that one or more of thedeadlocked processes be flushed from the system. The suspend/resume mechanism allows thesystem to put a temporary hold on a process (temporarily preempting its resources), and,when it is safe to do so, resume the held process without loss of work. Checkpoint/rollbackfacilitates suspend/resume capabilities by limiting the loss of work to the time at which thelast checkpoint (i.e., saved state of the system) was taken. When a process in a systemterminates (by accident or intentionally as the result of a deadlock recovery algorithm), thesystem performs a rollback by undoing every operation related to the terminated process thatoccurred since the last checkpoint. To ensure that data in the database remains in a consistentstate when deadlocked processes are terminated, database systems typically perform resourceallocations using transactions.In personal computer systems and workstations, deadlock has generally been viewedas a limited annoyance. Some systems implement the basic deadlock prevention methodssuggested by Havened, while others ignore the problem—these methods seem to besatisfactory. While ignoring deadlocks may seem dangerous, this approach can actually berather efficient. If deadlock is rare, then the processor time devoted to checking for deadlockssignificantly reduces system performance. However, given current trends, deadlock willcontinue to be an important area of research as the number of concurrent operations andnumber of resources becomes large, increasing the likelihood of deadlock in multiprocessorand distributed systems. Also, many real-time systems, which are becoming increasinglyprevalent, require deadlock-free resource allocation.365.6 Let us Sum UpIn this lesson we have learned about the characteristics of deadlock, deadlockprevention mechanism, deadlock avoidance using bankers and safety algorithms, deadlockdetection and recovery.5.7 Points for discussionAfter learning this chapter, try to discuss among your friends and answer thesequestions to check your progress.a) What are the characteristics of deadlock?b) Discuss about the deadlock prevention mechanismc) Discuss about bankers and safety algorithmd) How deadlock can be detected.e) What are the steps needed for deadlock recovery5.8 Model answers to Check your ProgressThe characteristic of deadlock can be explained by means of the four necessaryconditions for deadlock namely, a) A resource may be acquired exclusively by only oneprocess at a time (mutual exclusion condition); b) A process that has acquired an exclusiveresource may hold it while waiting to obtain other resources (wait-for condition, also calledthe hold-and-wait condition); c) Once a process has obtained a resource, the system cannotremove the resource from the process's control until the process has finished using theresource (no-preemption condition); d) And two or more processes are locked in a "circularchain" in which each process in the chain is waiting for one or more resources that the nextprocess in the chain is holding (circular-wait condition).Because these are necessary conditions for a deadlock to exist, the existence of adeadlock implies that each of them must be in effect. Taken together, all four conditions arenecessary and sufficient for deadlock to exist (i.e., if all these conditions are in place, thesystem is deadlocked).5.9 Lesson - end Activities

Try to write a program in C/C++ to implement bankers and safety algorithms5.10 Referencesi) Charles Crowley, Chapter 8 of “Operating Systems – A Design-OrientedApproach”, Tata McGraw-Hill, 2001j) H.M. Deitel, Chapter 6 of “Operating Systems”, Second Edition, PearsonEducation, 2001k) Andrew S. Tanenbaum, Chapter 6 of “Modern Operating Systems”, PHI, 1996l) D.M. Dhamdhere, Chapter 12 of “Systems Programming and Operating Systems”,Tata McGraw-Hill, 1997.37UNIT – IILESSON – 6: STORAGE MANAGEMENTCONTENTS6.1 Aims and Objectives6.2 Introduction6.3 Contiguous storage allocation6.4 Non-Contiguous Storage Allocation6.5 Fixed PartitionsMultiprogramming6.6 Variable Partitions Multiprogramming6.7 Multiprogramming with Storage Swapping6.8 Let us Sum Up6.9 Points for discussion6.10 Model answer to Check your Progress6.11 Lesson - end Activities6.12 References6.1 Aims and ObjectivesThe aim of this lesson is to learn the concept of Real storage management strategies.The objectives of this lesson are to make the student aware of the following conceptsa) Contiguous storage allocationb) Non Contiguous storage allocationc) Fixed partition multiprogrammingd) Variable partition multiprogramminge) and multiprogramming with storage swapping6.2 IntroductionThe organization and management of the main memory or primary memory or realmemory of a computer system has been one of the most important factors influencingoperating systems design. Regardless of what storage organization scheme we adopt for aparticular system, we must decide what strategies to use to obtain optimal performance.Storage Management Strategies are of four types as described belowa) Fetch strategies – concerned with when to obtain the next piece of program ordata for transfer to main storage from secondary storagea. Demand fetch – in which the next piece of program or data is brought intothe main storage when it is referenced by a running program38b. Anticipatory fetch strategies – where we make guesses about the futureprogram control which will yield improved system performanceb) Placement strategies – concerned with determining where in main storage toplace and incoming program. Examples are first fit, best fit and worst fitc) Replacement strategies – concerned with determining which piece of programor data to replace to make place for incoming programs6.3 Contiguous Storage AllocationIn contiguous storage allocation each program has to occupy a single contiguousblock of storage locations. The simplest memory management scheme is the bare machineconcept, where the user is provided with the complete control over the entire memory space.The next simplest scheme is to divide memory into two sections, one for the user andone for the resident monitor of the operating system. A protection hardware can be providedin terms of fence register to protect the monitor code and data from changes by the userprogram.The resident monitor memory management scheme may seem of little use since itappears to be inherently single user. When they switched to the next user, the currentcontents of user memory were written out to a backing storage and the memory of the nextuser is read in called as swapping6.4 Non-Contiguous Storage AllocationMemory is divided into a number of regions or partitions. Each region may haveone program to be executed. Thus the degree of multiprogramming is bounded by the

number of regions. When a region is free, a program is selected from the job queue andloaded into the free regions. Two major schemes are multiple contiguous fixed partitionallocation and multiple contiguous variable partition allocation.6.5 Fixed Partitions MultiprogrammingFixed partitions multiprogramming also called as multiprogramming with fixednumber of task (MFT) or multiple contiguous fixed partition allocation. MFT has thefollowing properties.• Several users simultaneously compete for system resources• switch between I/O jobs and calculation jobs for instance• Allowing Relocation and Transfers between partitions• Protection implemented by the use of several boundary registers : low and highboundary registers, or base register with length• Fragmentation occurs if user programs cannot completely fill a partition - wasteful.All the jobs that enters the system are put into queues. Each partition has its own jobqueue as shown in the following figure. The memory requirements of each job and theavailable regions in determining which jobs are allocated memory are taken care by the jobscheduler. When a job is allocated space, it is loaded into a region and then compete for theCPU. When a job terminates, it releases its memory region, which the job scheduler maythen fill with another job from the job queue. Another way is to allow a single unified queue39and the decisions of choosing a job reflect the choice between a best-fit-only or a bestavailable-fit job memory allocation policy.Figure :Multiprogramming - fixed partitions6.6 Variable Partitions MultiprogrammingVariable Partitions Multiprogramming also called as multiprogramming with variablenumber of task (MVT) or multiple contiguous variable partition allocation. In this scheme,there are no fixed partitions. The memory is divided into regions and allocated to programsas and when it is required.Figure: Multiprogramming - variable partitionsMVT has the following properties• Variable partitions - allowing jobs to use as much space as they needed (limit beingthe complete memory space)• No need to divide jobs into types - reduce waste if jobs could not fill partitionHowever, complete wastage is still not reduced. The OS keeps a table indicatingwhich parts of memory are available and which are occupied. Initially all memory isavailable for user programs, and is considered as one large block of available memory, a hole.When a job arrives and needs memory, we search for a hole large enough for this job. If wefind one, we allocate only as much as is needed, keeping the rest available to satisfy futurerequests. The most common algorithms for allocating memory are first-fit and best-fit.40Figure: Allocating MemoryOnce a block of memory has been allocated to a job, its program can be loaded intothat space and executed. The minimal hardware support needed is the same as with MFT:two registers containing the upper and lower bounds of the region of memory allocated to thisjob. When the CPU scheduler selects this process, the dispatcher loads these bounds registerswith the correct values. Since every address generated by the cpu is checked against theseregisters, we can protect other users programs and data from being modified by this runningprocess.6.7 Multiprogramming with Storage SwappingMultiprogramming with storage swapping has the following features. It is different from theprevious schemes where user programs remain in memory until completion.• swap job out of main storage (to secondary storage) if it requires service from someexternal routine (so another process can use the CPU)• A job may typically be need to swapping in and out many time before completion• Now, main storage is large enough to have many user programs active at the sametime (swapping in and out of main memory)• Allocate a bit more than necessary for process to grow during execution Area on disk where this happens : swap space (usually the /tmp area in unix) Sometimes, swap space is automatically allocated on process creation (hence, a fixedswap space per process) An average process in the middle of memory (after system has reached equilibrium)will encounter half allocations and half deallocations (above and below) The Fifty percent rule = if mean number of processes in memory is n, the meannumber of holes is n/2 Thats because adjacent holes are merged, adjacent processes are not (hence a process /

hole asymmetry) - just a heuristic6.8 Let us sum up41In this lesson we have learnt about the concept of Real storage management strategieslike a) Contiguous storage allocation, b) Non Contiguous storage allocation, c) Fixedpartition multiprogramming, d) Variable partition multiprogramming, e) andmultiprogramming with storage swapping6.9 Points for discussiona) What happens if no job in queue can meet the size of the slot left by the departing job?b) How do we keep track of memory in implementation6.10 Model answers to Check your ProgressThe answers for the questions given in 6.8 area) This leads to the creation of holes in main storage that must be filled.b) We can keep track of memory in implementation by use ofi) linked lists,ii) buddy system (dividing memory according to a power of 2) - leads to bigwastes (checkerboading or external fragmentation)iii) or bitmaps (use a structure, where a 0 indicates if block is free, and 1otherwise)6.11 Lesson - end activitiesAfter learning this chapter, try to discuss among your friends and answer thesequestions to check your progress.o Differentiate between MFT and MVTo Advantages of swapping426.12 Referencesm) Charles Crowley, Chapter 10, 11, 12 of “Operating Systems – A Design-OrientedApproach”, Tata McGraw-Hill, 2001n) H.M. Deitel, Chapter 7, 8, 9 of “Operating Systems”, Second Edition, PearsonEducation, 2001o) Andrew S. Tanenbaum, Chapter 3 of “Modern Operating Systems”, PHI, 1996p) D.M. Dhamdhere, Chapter 15 of “Systems Programming and Operating Systems”,Tata McGraw-Hill, 199743LESSON 7 – VIRTUAL STORAGECONTENTS7.1 Aims and Objectives7.2 Introduction7.2.1 Overlays7.2.2 Dynamic Loading7.3 Contiguous storage allocation7.4 Steps in handling page fault7.5 Page replacement algorithms7.5.1 FIFO7.5.2 Optimal replacement7.5.3 Least recently used7.6 Working sets7.7 Demand paging7.8 Page size7.9 Let us Sum Up7.10 Points for discussion7.11 Model answers to check your progress7.12 Lesson end Activities7.13 References7.1 Aims and ObjectivesThe aim of this lesson is to learn the concept of virtual storage management strategies.The objectives of this lesson are to make the student aware of the following conceptsf) virtual storage management strategiesg) page replacement strategiesh) working setsi) demand pagingj) and page size7.2 IntroductionVirtual Memory is technique which allows the execution of processes that may not becompletely in memory. The main advantage of this scheme is that user programs can be larger than

physical memory. The ability to execute a program which is only partially in memory would havemany benefits which includes (a) users can write programs for a very large virtual address space, (b)more users can be run at the same time, with a corresponding increase in cpu utilization andthroughput, but no increase in response time or turnaround time, (c) less I/O would be needed to loador swap each user into memory, so each user would run faster.447.2.1 OverlaysOverlay is a technique which keeps in memory only those instructions and data that arecurrently needed and when other instructions are needed they are loaded into space that waspreviously occupied by instructions that are no longer needed.7.2.2 Dynamic LoadingHere a routine is not called until it is called. The advantage of dynamic loading is that anunused routine is never loaded. This scheme is particularly useful when large amounts of code areneeded to handle infrequently.7.3 Virtual storage management strategiesThere are three main strategies namelyFetch strategies – concerned with when a page or segment should be brought from secondaryto primary storagePlacement strategies – concerned with where in primary storage to place an incoming page orsegmentReplacement strategies – concerned with deciding which page or segment to displace to makeroom for an incoming page or segment when primary storage is already fully committed7.4 Steps in handling page faultWhen a page is not available in the main memory a page fault occurs. When a page faultoccurs, the OS has to do some necessary steps to bring the required page from secondary storagedevice to main memory. The steps in handling page fault are as followsa) First check whether the reference is valid or not from the internal table of process controlblock (PCB)b) Bring the page if it is not already loaded and the reference is validc) Find a free framed) Read the desired page into the newly allocated framee) Then modify the internal table in the PCB to indicate that the page is now availablef) Restart the instruction that was interrupted.7.5 Page replacement algorithmsThere are many page replacement algorithms and the most important three are FIFO, optimalreplacement and least recently used. This subsection explains the above three algorithms.457.5.1 FIFOThe simplest page replacement algorithm is first in first out. In this scheme, when a pagemust be replaced, the oldest page is chosen. For example consider the page reference string1, 5, 6, 1, 7, 1, 5, 7, 6, 1, 5, 1, 7For a three frame case, the FIFO will work as follows. Let all our 3 frames are initially empty.1 1 1 7 7 7 6 65 5 5 1 1 1 76 6 6 5 5 5You can see, FIFO creates eight page faults.7.5.2 Optimal replacementIn optimal page replacement algorithm, we replace that page which will not be used for thelongest period of time. For example for the reference string1, 5, 6, 1, 7, 1, 5, 7, 6, 1, 5, 1, 7with 3 frames, the page faults will be as follows1 1 1 1 1 15 5 5 5 56 7 6 7You can see that Optimal replacement, creates six page faults7.5.3 Least recently usedMost of the case, predicting the future page references is difficult and hence implementingoptimal replacement is difficult. Hence there is a need of other scheme which approximates theoptimal replacement. Least recently used (LRU) schemes approximate the future uses by the pastused pages. In LRU scheme, we replace those pages which have not been used for the longest periodof time.For example for the reference string1, 5, 6, 1, 7, 1, 5, 7, 6, 1, 5, 1, 7with 3 frames, the page faults will be as follows1 1 1 1 1 6 6 6 7

5 5 7 7 7 7 5 56 6 5 5 1 1 1You can see that LRU creates nine page faults7.6 Working setsIf the number of frames allocated to a low-priority process falls below the minimum numberrequired, we must suspend its execution. We should then page out it remaining pages, freeing all ofits allocated frames. A process is thrashing if it is spending more time paging than executing.46Thrashing can cause severe performance problems. To prevent thrashing, we must provide a processwith as many frames as it needs. There are several techniques available to know how many frame aprocess needs. Working sets is a strategy which starts by looking at what a program is actually using.The set of most recent page references is the working set denoted by δ. The accuracy of theworking set depends upon the selection of δ. If it is too small, page fault will increase an if it is toolarge, then it is very difficult to allocate the required frames.For example,You can see that the working set (ws) at two different times for the window size δ.=11. [The workingset refers to the pages the process has used during that time for the window size δ]. So at themaximum the above given example needed atleast 5 frames, otherwise page fault will occur. In mostof the cases we will allocate the number of frames to a process depending on the average working setsize.Let Si be the average working set size for each process in the system. Then=∑ D Siis the total demand for frames. Thus process i needs Si frames. If the total demand is greater than thetotal number of available frames, thrashing will occur.7.7 Demand pagingDemand paging is the most common virtual memory system. Demand paging is similar to apaging system with swapping. When we need a program, it is swapped from the backing storage.There are also lazy swappers, which never swaps a page into memory unless it is needed. The lazyswapper decreases the swap time and the amount of physical memory needed, allowing an increaseddegree of multiprogramming.7.8 Page sizeThere is no single best page size. The designers of the Operating system will decide the pagesize for an existing machine. Page sizes are usually be in powers of two, ranging from 28 to 212 bytesor words. The size of the pages will affect in the following way.a) Decreasing the page size increases the number of pages and hence the size of the pagetable.b) Memory is utilized better with smaller pages.c) For reducing the I/O time we need to have smaller page size.d) To minimize the number of page faults, we need to have a large page size.7.9 Let us Sum UpIn this lesson we have learnt about the concept of virtual storage managementstrategies like page replacement strategies, working sets, demand paging, and page size1 2 3 1 4 1 5 2 5 2 5 3 6 2 1 3 1 4 6 1 4 5 1 1 3 1 2 1 1 3 1 2 1 3 4 5 6 6 6 3ws={2,3,1,4,5}δ = 11ws={1,3,2}δ = 11477.10 Points for discussionAfter learning this lesson, try to discuss among your friends and answer thesequestions to check your progress.a) What is the use of demand pagingb) Differentiate between optimal replacement and LRU7.11 Model answers to check your progressDemand paging is the most common virtual memory system. Demand paging is similar to apaging system with swapping. When we need a program, it is swapped from the backing storage.There are also lazy swappers, which never swaps a page into memory unless it is needed. The lazyswapper decreases the swap time and the amount of physical memory needed, allowing an increaseddegree of multiprogramming.7.12 Lesson end ActivitiesFor the reference string1, 5, 6, 1, 7, 1, 5, 7, 6, 1, 5, 1, 7find the page fault rate for various page replacement algorithms with number of frame as 5.7.13 References♦ Charles Crowley, Chapter 11, 12 of “Operating Systems – A Design-Oriented

Approach”, Tata McGraw-Hill, 2001♦ H.M. Deitel, Chapter 7, 8, 9 of “Operating Systems”, Second Edition, PearsonEducation, 2001♦ Andrew S. Tanenbaum, Chapter 3 of “Modern Operating Systems”, PHI, 1996♦ D.M. Dhamdhere, Chapter 15 of “Systems Programming and Operating Systems”,Tata McGraw-Hill, 199748UNIT – IIILESSON – 8: PROCESSOR MANAGEMENTCONTENTS8.1 Aims and Objectives8.2 Introduction8.3 Preemptive Vs Non-Preemptive scheduling8.4 Priorities8.5 Deadline scheduling8.6 Let us Sum Up8.7 Points for discussion8.8 Model answer to Check your Progress8.9 Lesson end Activities8.10 References8.1 Aims and ObjectivesA multiprogramming operating system allows more than one process to be loaded intothe executabel memory at a time and for the loaded process to share the CPU using timemultiplexing.Part of the reason for using multiprogramming is that the operating system itselfis implemented as one or more processes, so there must be a way for the operating systemand application processes to share the CPU. Another main reason is the need for processes toperf I/O operations in the normal course of computation. Since I/O operations ordinarilyrequire orders of magnitude more time to complete than do CPU instructions,multiprograming systems allocate the CPU to another process whenever a process invokes anI/O operationMake sure your scheduling strategy is good enough with the following criteria:• Utilization/Efficiency: keep the CPU busy 100% of the time with useful work• Throughput: maximize the number of jobs processed per hour.• Turnaround time: from the time of submission to the time of completion, minimizethe time batch users must wait for output• Waiting time: Sum of times spent in ready queue - Minimize this• Response Time: time from submission till the first response is produced, minimizeresponse time for interactive users• Fairness: make sure each process gets a fair share of the CPUThe aim of this lesson is to learn the concept of processor management and related issues.The objectives of this lesson are to make the student aware of the following conceptsk) preemptive scheduling49l) Non preemptive schedulingm) Prioritiesn) and deadline scheduling8.2 IntroductionWhen one or more process is runnable, the operating system must decide which oneto run first. The part of the operating system that makes decision is called the Scheduler; thealgorithm it uses is called the Scheduling Algorithm.An operating system has three main CPU schedulers namely the long term scheduler,short term scheduler and medium term schedulers. The long term scheduler determineswhich jobs are admitted to the system for processing. It selects jobs from the job pool andloads them into memory for execution. The short term scheduler selects from among the jobsin memory which are ready to execute and allocated the cpu to one of them. The mediumterm scheduler helps to remove processes from main memory and from the active contentionfor the cpu and thus reduce the degree of multiprogramming.The cpu scheduler has another component called as dispatcher. It is the module thatactually gives control of the cpu to the process selected by the short term scheduler whichinvolves loading of registers of the process, switching to user mode and jumping to the properlocation.Before looking at specific scheduling algorithms, we should think about what thescheduler is trying to achieve. After all the scheduler is concerned with deciding on policy,not providing a mechanism. Various criteria come to mind as to what constitutes a goodscheduling algorithm. Some of the possibilities include:

1. Fairness – make sure each process gets its fair share of the CPU.2. Efficiency (CPU utilization) – keep the CPU busy 100 percent of the time.3. Response Time [Time from the submission of a request until the first response isproduced] – minimize response time for interactive users.4. Turnaround time [The interval from the time of submission to the time of completion]– minimize the time batch users must wait for output.5. Throughput [Number of jobs that are completed per unit time] – maximize thenumber of jobs processed per hour.6. Waiting time – minimize the waiting time of jobs8.3 Preemptive Vs Non-PreemptiveThe Strategy of allowing processes that are logically runnable to be temporarilysuspended is called Preemptive Scheduling. ie., a scheduling discipline is preemptive if theCPU can be taken away. Preemptive algorithms are driven by the notion of prioritizedcomputation. The process with the highest priority should always be the one currently using50the processor. If a process is currently using the processor and a new process with a higherpriority enters, the ready list, the process on the processor should be removed and returned tothe ready list until it is once again the highest-priority process in the system.Run to completion is also called Nonpreemptive Scheduling. ie., a schedulingdiscipline is nonpreemptive if, once a process has been given the CPU, the CPU cannot betaken away from that process. In short, Non-preemptive algorithms are designed so that oncea process enters the running state(is allowed a process), it is not removed from the processoruntil it has completed its service time ( or it explicitly yields the processor). This leads to racecondition and necessitates of semaphores, monitors, messages or some other sophisticatedmethod for preventing them. On the other hand, a policy of letting a process run as long as itis wanted would mean that some process computing π to a billion places could deny serviceto all other processes indefinitely.8.4 PrioritiesA priority is associated with each job, and the cpu is allocated to the job with thehighest priority. Priorities are generally some fixed numbers such as 0 to 7 or 0 to 4095.However there is no general agreement on whether 0 is the highest or lowest priority.Priority can be defined either internally or externally. Examples of internal prioritiesare time limits, memory requirements, number of open files, average I/O burst time, CPUburst time, etc. External priorities are given by the user.A major problem with priority scheduling algorithms is indefinite blocking orstarvation. A solution to this problem is aging. Aging is a technique of gradually increasingthe priority of jobs that wait in the system for a long time.8.5 Deadline schedulingCertain jobs have to be completed in specified time and hence to be scheduled basedon deadline. If delivered in time, the jobs will be having high value and otherwise the jobswill be having nil value. The deadline scheduling is complex for the following reasonsa) Giving resource requirements of the job in advance is difficultb) A deadline job should be run without degrading other deadline jobsc) In the event of arriving new jobs, it is very difficult to carefully plan resourcerequirementsd) Resource management for deadline scheduling is really an overhead518.6 Let us Sum UpIn this lesson we have learnt abouta) preemptive schedulingb) Nonpreemptive schedulingc) Prioritiesd) and deadline scheduling8.7 Points for discussiona) Why CPU scheduling is important?b) How to evaluate scheduling algorithm?8.8 Model answer to Check your ProgressThe answers for the question in 8.8 are discussed below.a) Because it can can have a big effect on resource utilization and the overallperformance of the system.b) There are many possible criteria:a. CPU Utilization: Keep CPU utilization as high as possible. (What isutilization, by the way?).b. Throughput: number of processes completed per unit time.c. Turnaround Time: mean time from submission to completion of process.

d. Waiting Time: Amount of time spent ready to run but not running.e. Response Time: Time between submission of requests and first response tothe request.f. Scheduler Efficiency: The scheduler doesn't perform any useful work, soany time it takes is pure overhead. So, need to make the scheduler veryefficient.8.9 Lesson end ActivitiesAfter learning this chapter, try to discuss among your friends and answer thesequestions to check your progress.a) What is CPU scheduling?b) Discuss about deadline scheduling. How to evaluate scheduling algorithm?528.10 References♦ Charles Crowley, Chapter 8 of “Operating Systems – A Design-Oriented Approach”,Tata McGraw-Hill, 2001♦ H.M. Deitel, Chapter 10 of “Operating Systems”, Second Edition, Pearson Education,2001♦ Andrew S. Tanenbaum, Chapter 11 of “Modern Operating Systems”, PHI, 1996♦ D.M. Dhamdhere, Chapter 9 of “Systems Programming and Operating Systems”, TataMcGraw-Hill, 199753LESSON – 9: PROCESSOR SCHEDULINGCONTENTS9.1 Aims and Objectives9.2 Introduction9.3 First In First out (FIFO)9.4 Round Robin Scheduling9.5 Quantum size9.6 Shortest Job First (SJF)9.7 Shortest remaining time first (SRF)9.8 Highest response ratio next (HRN)9.9 Let us Sum Up9.10 Points for discussion9.11 Model answers to Check your Progress9.12 Lesson - end Activities9.13 References9.1 Aims and ObjectivesThe aim of this lesson is to learn the concept of processor scheduling and schedulingalgorithms.The objectives of this lesson are to make the student aware of the following conceptso) FIFOp) Round Robinq) Shortest Job firstr) Shortest remaining time firsts) Highest response ratio next (HRN)9.2 IntroductionDifferent algorithms have different properties and may favor one class of processesover another. In choosing best algorithm, the characteristic explained in the previous lessonmust be considered which includes CPU utilization, throughput, turnaround time, waitingtime, and response time.The five most important algorithms used in the CPU scheduling are FIFO, RoundRobin, Shortest Job first, Shortest remaining time first and Highest response ratio next(HRN). The following sections describe each one of it.9.3 First In First out (FIFO)54A low overhead paging algorithm is the FIFO (First in, First Out) algorithm. Toillustrate how this works, consider a supermarket that has enough shelves to display exactly kdifferent products. One day, some company introduces a new convenience food-instant,freeze-dried, organic yogurt that can be reconstituted in a microwave oven. It is an immediatesuccess, so our finite supermarket has to get rid of one old product in order to stock it.One possibility is to find the product that the supermarket has been stocking thelongest and get rid of it on the grounds that no one is interested anymore. In effect, thesupermarket maintains the linked list of all the products it currently sells in the order theywere introduced. The new one goes on the back of the list; the one at the front of the list isdropped.

As a page replacement algorithm, the same idea is applicable. The operating systemmaintains a list of all pages currently in memory, with the page at the head of the list theoldest one and the page at the tail the most recent arrival. On a page fault, the page at thehead is removed and the new page added to the tail of the list. When applied to stores, FIFOmight remove mustache wax, but it might also remove flour, salt or butter. When applied tocomputers the same problems arise. For this reason, FIFO in its pure form is rarely used.Consider for example, the following scenario of four jobs and the corresponding CPUburst time arrived in the order of job number.Job Burst time1 202 103 54 15FCFS algorithm allocates the job to the cpu in the order of there arrival and thefollowing Gantt chart shows the result of execution.The waiting times of jobs areJob 1 0Job 2 20Job 3 30Job 4 35--------------------------------------------Total waiting time = 85Hence the average waiting time is 21.25.The turnaround times of jobs areJob 1 20Job 2 30Job 1 Job 2 Job 3 Job40 20 30 35 5055Job 3 35Job 4 50--------------------------------------------Total turnaround time = 135Hence the average turnaround time is 33.25.9.4 Round Robin SchedulingOne of the oldest, simplest, fairest and most widely used algorithms is Round Robin.Each process is assigned a time interval, called the Quantum, which it is allowed to run. If theprocess is still running at the end of the quantum, the CPU is preempted and given to anotherprocess. If the process has blocked or finished before the quantum has elapsed, the CPUswitching is done. Round robin is easy to implement. All scheduler has to maintain a list ofrun able processes.The only interesting issue with the round robin is the length of the quantum.Switching from one process to another requires a certain amount of time for doing theadministration – saving and loading registers and memory maps, updating various tables andlists, etc. suppose that this process switch or context switch takes 5 msecs. Also suppose thequantum is set say 20 msecs. With these parameters, after doing 20 msecs of useful work, theCPU will have to spend 5 msecs on process switching. Twenty percent of the CPU time willbe wasted on administrative overhead.Consider for example, the following scenario of four jobs and the corresponding CPU bursttime arrived in the order of job number.Job Burst time1 202 103 54 15RR algorithm allocates a quantum of time to each job in a rotation and the followingGantt chart shows the result of execution. Let the time quantum be 5The waiting times of jobs areJob 1 0 + 15 + 10 + 5 = 30Job 2 5 + 15 = 20Job 3 10 = 10Job 4 15 + 10 + 5 = 30--------------------------------------------------------------Job 10 5 10 15 20 25 30 35 40 45 50

50Job 2 Job 3 Job 4 Job 1 Job 2 Job 4 Job 1 Job 4 Job 1B F D G A F D G A B56Total waiting time = 90Hence the average waiting time is 22.5The turnaround times of jobs areJob 1 50Job 2 30Job 3 15Job 4 45--------------------------------------------Total turnaround time = 140Hence the average turnaround time is 359.5 Quantum sizeIn round robin scheduling algorithm, no process is allocated the cpu for more than onetime quantum in a row. If its cpu burst exceeds a time quantum, it is preempted and put backin the ready queue.If the time quantum is too large, round robin becomes equivalent to FCFS. If the timequantum is too small in terms of microseconds, round robin is called as processor sharing andappears as if each of n processes has its own processors running at 1/n the speed of the realprocessor.The time quantum size must be large with respect to the context switch time. If thecontext switch time is approximately 5 percent of the time quantum, then cpu will spent 5percent of the time for context switching.9.6 Shortest Job First (SJF)Most of the above algorithms were designed for interactive systems. Now let us lookat one that is especially appropriate for batch jobs where the run times are known in advance.In an insurance company, for example, people can predict quite accurately how long it willtake to run a batch of 1000 claims, since similar work is done every day. When severalequally important jobs are sitting in the input queue waiting to be started, the schedulershould use shortest job first.8 4 6 5A B C DHere we find four jobs A, B, C, and D with run times of 8, 4, 6 and 3 minutesrespectively. By running them in that order, the turn around time for A is 8 minutes, for B is12 Minutes, for C is 18 Minutes and for D is 23 minutes for an average of 15.25 minutes.Now if we do the SJF first as follows57The turnaround times are now 4, 9, 15 and 23 minutes for an average of 12.75minutes.Shortest job first is provably optimal. Consider the case of four jobs, with runtimes of a, b, cand d respectively. The first job finishes at time a, the second finishes at time a+b and so on.The mean turnaround time is (4a+3b+2c+d)/4. It is clear that ‘a’ contributes more to theaverage than the other, so it should be the shortest job, with b next, then c and finally d as thelongest, as it affects only its own turnaround time. The same argument applies equally well toany number of jobs.Consider for example, the following scenario of four jobs and the corresponding CPU bursttime arrived in the order of job number.Job Burst time1 202 103 54 15The algorithm allocates the shortest job first.The waiting times of jobs areJob 1 30Job 2 5Job 3 0Job 4 15--------------------------------------Total waiting time = 50Hence the average waiting time is 12.5The turnaround times of jobs areJob 1 50

Job 2 15Job 3 5Job 4 30--------------------------------------------Total turnaround time = 100Hence the average turnaround time is 25B D C AJob 3 Job 2 Job4 Job 10 5 15 30 5050589.7 Shortest remaining time first (SRF)Shortest job first may be either preemptive or non-preemptive. When a new jobarrives, at the ready queue with a shortest cpu burst time, while a previous job is executing,then a preemptive shortest job first algorithm will preempt the currently executing job, whilea non-preemptive shortest-job-first algorithm will allow the currently running job to finish itscpu burst. Preemptive-shortest-job-first algorithm is also called as shortest remaining timefirst.Consider for example, the following scenario of four jobs and the corresponding CPU bursttime and arrival time.Job Burst time Arrival time1 20 02 10 23 5 44 15 19The algorithm allocates the jobs as shown in Gantt chartThe waiting times of jobs areJob 1 0 + 15 + 15 = 30Job 2 2 + 5 = 7Job 3 4 = 4Job 4 19 = 19--------------------------------------Total waiting time = 60Hence the average waiting time is 15The turnaround times of jobs areJob 1 50Job 2 17Job 3 9Job 4 34--------------------------------------------Total turnaround time = 110Hence the average turnaround time is 27.59.8 Highest response ratio next (HRN)Job 10 2 4 9 17 19 34 50Job 2 Job 3 Job 2 Job 1 Job 4 Job 159HRN is a nonpreemptive scheduling algorithm which considers both the CPU bursttime and waiting time. The priority of the job in HRN can be calculated aspriority = time waiting + service timeservice timewhere service time is the next cpu burst time. Here shortest jobs will get highest prioritysince it appears in the denominator. Since waiting time appears in the numerator, longerwaiting jobs will also get priority.9.9 Let us Sum UpIn this chapter we have learned about various scheduling algorithms like, FIFO, RR,SJF, SRT and HRN.9.10 Points for discussiona) Consider performance of FCFS algorithm for three compute-bound processes. What ifhave 4 processes P1 (takes 24 seconds), P2 (takes 3 seconds) and P3 (takes 3 seconds). Ifarrive in order P1, P2, P3, what is Waiting Time?, Turnaround Time? and Throughput?b) What about if processes come in order P2, P3, P1?c) What happens with really a really small quantum?d) What about having a really small quantum supported in hardware?e) What about a really big quantum?

9.11 Model answers to Check your progressThe answers for the question given in 9.11 area) Waiting Time? (24 + 27) / 3 = 17, Turnaround Time? (24 + 27 + 30) = 27,Throughput? 30 / 3 = 10.b) Waiting Time? (3 + 3) / 2 = 6, Turnaround Time? (3 + 6 + 30) = 13,Throughput? 30 / 3 = 10.c) It looks like you've got a CPU that is 1/n as powerful as the real CPU, where n isthe number of processes. Problem with a small quantum - context switch overhead.d) Then, you have something called multithreading. Give the CPU a bunch ofregisters and heavily pipeline the execution. Feed the processes into the pipe oneby one. Treat memory access like IO - suspend the thread until the data comesback from the memory. In the meantime, execute other threads. Use computationto hide the latency of accessing memory.e) It turns into FCFS. Rule of thumb - want 80 percent of CPU bursts to be shorterthan time quantum9.12 Lesson end ActivitiesTry to write C/C++ programs to implement FIFO, RR, SJF, SRT and HRN609.13 References♦ Charles Crowley, Chapter 8 of “Operating Systems – A Design-Oriented Approach”,Tata McGraw-Hill, 2001♦ H.M. Deitel, Chapter 10 of “Operating Systems”, Second Edition, Pearson Education,2001♦ Andrew S. Tanenbaum, Chapter 11 of “Modern Operating Systems”, PHI, 1996♦ D.M. Dhamdhere, Chapter 9 of “Systems Programming and Operating Systems”, TataMcGraw-Hill, 1997♦ Operating Systems Lecture Notes, Copyright 1997 Martin C. Rinard.61LESSON – 10: DISTRIBUTED COMPUTINGCONTENTS10.1 Aims and Objectives10.2 Introduction10.3 Classification of sequential and parallel processing10.4 Array Processors10.4.1 History10.5 Dataflow computer10.6 Let us Sum Up10.7 Points for discussion10.8 Model Answer to Check your Progress10.9 Lesson - end Activities10.10 References10.1 Aims and ObjectivesThe aim of this lesson is to learn the concept of distribute computing and parallel computing.The objectives of this lesson are to make the student aware of the following conceptst) sequential processingu) parallel processingv) Array Processorsw) and Dataflow computer10.2 IntroductionParallel computing is the simultaneous execution of some combination of multipleinstances of programmed instructions and data on multiple processors in order to obtainresults faster. The idea is based on the fact that the process of solving a problem usually canbe divided into smaller tasks, which may be carried out simultaneously with somecoordination. The technique was first put to practical use by ILLIAC IV in 1976, fully adecade after it was conceived.A parallel computing system is a computer with more than one processor for parallelprocessing. In the past, each processor of a multiprocessing system always came in its ownprocessor packaging, but recently-introduced multicore processors contain multiple logicalprocessors in a single package. There are many different kinds of parallel computers. Theyare distinguished by the kind of interconnection between processors (known as "processingelements" or PEs) and memory.Parallel computers can be modelled as Parallel Random Access Machines (PRAMs).The PRAM model ignores the cost of interconnection between the constituent computing62units, but is nevertheless very useful in providing upper bounds on the parallel solvability of

many problems. In reality the interconnection plays a significant role. The processors maycommunicate and cooperate in solving a problem or they may run independently, often underthe control of another processor which distributes work to and collects results from them (a"processor farm").Processors in a parallel computer may communicate with each other in a number ofways, including shared (either multiported or multiplexed, parallel supercomputers, NUMAvs. SMP vs. massively parallel computer systems, distributed computing (esp. computerclusters and grid computing). According to Amdahl's law, parallel processing is less efficientthan one x-times-faster processor from a computational perspective. However, since powerconsumption is a super-linear function of the clock frequency on modern processors, we arereaching the point where from an energy cost perspective it can be cheaper to run many lowspeed processors in parallel than a single highly clocked processor.10.3 Classification of sequential and parallel processingFlynn's taxonomy, one of the most accepted taxonomies of parallel architectures,classifies parallel (and serial) computers according to: whether all processors execute thesame instructions at the same time (single instruction/multiple data -- SIMD) or whether eachprocessor executes different instructions (multiple instruction/multiple data --MIMD).SISD (single instruction stream, single data stream) machines are uni-processorcomputers that process one instruction at a time and are the most common architectureavailable today. Array processors essentially performs operations simultaneously on everyelement of an array namely SIMD (Single instruction stream, multiple data stream). Anothercategory is multiple instruction stream and single data stream (MISD) which is not in use.Parallel processors or multiprocessors can handle multiple instruction stream and multipledata streams.Another major way to classify parallel computers is based on their memoryarchitectures. Shared memory parallel computers have multiple processors accessing allavailable memory as global address space. They can be further divided into two main classesbased on memory access times: Uniform Memory Access (UMA), in which access times toall parts of memory are equal, or Non-Uniform Memory Access (NUMA), in which they arenot. Distributed memory parallel computers also have multiple processors, but each of theprocessors can only access its own local memory; no global memory address space existsacross them. Parallel computing systems can also be categorized by the numbers ofprocessors in them. Systems with thousands of such processors are known as massivelyparallel. Subsequently there are what are referred to as "large scale" vs. "small scale" parallelprocessors. This depends on the size of the processor, eg. a PC based parallel system wouldgenerally be considered a small scale system. Parallel processor machines are also dividedinto symmetric and asymmetric multiprocessors, depending on whether all the processors arethe same or not (for instance if only one is capable of running the operating system code andothers are less privileged).A variety of architectures have been developed for parallel processing. For example aRing architecture has processors linked by a ring structure. Other architectures includeHypercubes, Fat trees, systolic arrays, and so on.6310.4 Array ProcessorsA vector processor, or array processor, is a CPU design that is able to runmathematical operations on multiple data elements simultaneously. This is in contrast to ascalar processor which handles one element at a time. The vast majority of CPUs are scalar(or close to it). Vector processors were common in the scientific computing area, where theyformed the basis of most supercomputers through the 1980s and into the 1990s, but generalincreases in performance and processor design saw the near disappearance of the vectorprocessor as a general-purpose CPU.Today most commodity CPU designs include some vector processing instructions,typically known as SIMD (Single Instruction, Multiple Data), common examples includeSSE and AltiVec. Modern video game consoles and consumer computer-graphics hardwarerely heavily on vector processing in their architecture. In 2000, IBM, Toshiba and Sonycollaborated to create a Cell processor, consisting of one scalar processor and eight vectorprocessors, for the Sony PlayStation 3.10.4.1 HISTORYVector processing was first worked on in the early 1960s at Westinghouse in theirSolomon project. Solomon's goal was to dramatically increase math performance by using alarge number of simple math co-processors (or ALUs) under the control of a single masterCPU. The CPU fed a single common instruction to all of the ALUs, one per "cycle", but witha different data point for each one to work on. This allowed the Solomon machine to apply asingle algorithm to a large data set, fed in the form of an array. In 1962 Westinghousecancelled the project, but the effort was re-started at the University of Illinois as the ILLIAC

IV. Their version of the design originally called for a 1 GFLOPS machine with 256 ALUs,but when it was finally delivered in 1972 it had only 64 ALUs and could reach only 100 to150 MFLOPS. Nevertheless it showed that the basic concept was sound, and when used ondata-intensive applications, such as computational fluid dynamics, the "failed" ILLIAC wasthe fastest machine in the world. It should be noted that the ILLIAC approach of usingseparate ALUs for each data element is not common to later designs, and is often referred tounder a separate category, massively parallel computing.The first successful implementation of vector processing appears to be the CDCSTAR-100 and the Texas Instruments Advanced Scientific Computer (ASC). The basic ASC(i.e., "one pipe") ALU used a pipeline architecture which supported both scalar and vectorcomputations, with peak performance reaching approximately 20 MFLOPS, readily achievedwhen processing long vectors. Expanded ALU configurations supported "two pipes" or "fourpipes" with a corresponding 2X or 4X performance gain. Memory bandwidth was sufficientto support these expanded modes. The STAR was otherwise slower than CDC's ownsupercomputers like the CDC 7600, but at data related tasks they could keep up while beingmuch smaller and less expensive. However the machine also took considerable time decodingthe vector instructions and getting ready to run the process, so it required very specific datasets to work on before it actually sped anything up.The vector technique was first fully exploited in the famous Cray-1. Instead ofleaving the data in memory like the STAR and ASC, the Cray design had eight "vectorregisters" which held sixty-four 64-bit words each. The vector instructions were applied64between registers, which is much faster than talking to main memory. In addition the designhad completely separate pipelines for different instructions, for example, addition/subtractionwas implemented in different hardware than multiplication. This allowed a batch of vectorinstructions themselves to be pipelined, a technique they called vector chaining. The Cray-1normally had a performance of about 80 MFLOPS, but with up to three chains running itcould peak at 240 MFLOPS – a respectable number even today.Other examples followed. CDC tried to re-enter the high-end market again with itsETA-10 machine, but it sold poorly and they took that as an opportunity to leave thesupercomputing field entirely. Various Japanese companies (Fujitsu, Hitachi and NEC)introduced register-based vector machines similar to the Cray-1, typically being slightlyfaster and much smaller. Oregon-based Floating Point Systems (FPS) built add-on arrayprocessors for minicomputers, later building their own minisupercomputers. However Craycontinued to be the performance leader, continually beating the competition with a series ofmachines that led to the Cray-2, Cray X-MP and Cray Y-MP. Since then the supercomputermarket has focused much more on massively parallel processing rather than betterimplementations of vector processors. However, recognizing the benefits of vector processingIBM developed Virtual Vector Architecture for use in supercomputers coupling several scalarprocessors to act as a vector processor.Today the average computer at home crunches as much data watching a shortQuickTime video as did all of the supercomputers in the 1970s. Vector processor elementshave since been added to almost all modern CPU designs, although they are typically referredto as SIMD. In these implementations the vector processor runs beside the main scalar CPU,and is fed data from programs that know it is there.10.5 Dataflow computerDataflow computing is a subject of considerable interest, since this class of computerarchitectures exposes high level of parallelism.A dataflow computer system VSPD-1 presented here is a research prototype of astatic dataflow architecture according to the Veen’s classification1. Static partitioning,linking, and distribution of actors among processing modules must be done at the stage ofdevelopment of a dataflow application for VSPD-1.VSPD-1 is designed to scale up to 16 processing nodes. The research prototype ofVSPD-1 implemented at the Computer Engineering Department, LETI (Leningrad Instituteof Electrical Engineering), consists of five processing modules (PM). Each VSPD-1processing module consists of- a microcomputer MS11200.5 with a 1MIPS microprocessor and 32-KB RAM;- PROM for a loader and a kernel of a dataflow run-time system (called heremonitor);- communication Qbus-Qbus adapters (CA);- a I/O processor (IOP), that supports up to eight DMA channels for internodecommunication- an auxiliary control unit that implements most of operations on dataflow control2.65Restrictions of physical realization motivated the following configuration of the 5-

modules system: four processing modules are configured in a ring, and the fifth module isconnected with all other PMs. The fifth PM uses a communication adapter that connects theprocessing module to a host computer DVK-4. The structure of the VSPD processing moduleis presented in Fig.1.Fig.1.Dataflow computation is realized at the level of labeled code fragments called actors.An actor is ready and can be performed, when it receives all data (operands) required for itsexecution. An application for VSPD-1 must be written in a specific dataflow style whereactors are programmed in Pascal with macros in Macro-11 assembler. Each labeled codesegment that specifies an actor ends with an assembler macro which passes a pointer to adestination list to the kernel and invokes it. The kernel sends data produced by the actor todestination actors specified in the list.A label of the actor serves as its starting address (activation point). Addresses ofdestination actors that will receive the result tokens are stored in a destination table thatactually holds a dataflow graph of the application. A dataflow program includes also a tableof starting addresses of actors, and a table of ready bit vectors. A ready-bit vector indicatesfor each of actor inputs whether a token has arrived at that input. Before computation starts,the ready bit table contains initial mapping of tokens to actor inputs. A special mask storedtogether with a ready-bit vector indicates the inputs which are always ready (constant or notin use).The system supports explicit and implicit transferring data from source-actors todestination actors. With explicit transfer, data that are sent to a destination actor, are followedby a result token, whose role is to indicate that data has been sent to the destination. A tokenis 16-bits wide and consists of the following fields: the data type tag (T) that indicateswhether the data item sent is a vector or a scalar; the number of the destination processormodule (P); the number of the destination actor (A); the number of the actor input to whichthe token is directed (I). In this way, parameters for explicit data transfer consist of twowords: a 16-bits token and a pointer to a data structure to be sent to the destination. Theformat of a destination list entry is shows in Fig.2.66Implicit data transfer is used for local data flow between actors within a processingmodule, and it is implemented through shared variables. Tokens are used for synchronizationof parallel processes in processing modules. Tokens that follow data, form a control flowsince each token indicates which destination actor can be performed after a source actor. Theactor is ready when it has collected tokens on all its inputs. The amount of token traffic canbe less than that of data flow. For example, when a source actor sends more than one dataitem to a destination actor, it is enough to send only one token after all data are sent in thevector form. If data is directed to remote actors located on the same remote node, data and alltokens are sent in one message.Token flow and data flow are controlled by a distributed software run-time systemtogether with the auxiliary hardware Control Unit (CU) in each processing node. Theauxiliary Control Unit consists of a register file (RF), a table memory (TM), a operationalunit (OU), a ready-actor queue (RAQ), microprogrammed controller (MPC), address selector(AS) and bus adapters (BA). The table memory is split into three regions that are used tostore three tables: the table of starting addresses; the table of ready-bit vectors, and the tableof destination addresses. A structure of CU is shown in Fig.3.67Fig.3.The auxiliary Control Unit receives a token and sets an appropriate ready-bit in theread-bit table to indicate that the token has arrived. If the destination actor is ready, itsnumber is inserted to the ready actor queue. On a request from the software kernel, CUfetches a ready actor (if any) from the head of the queue, and looks up the starting addresstable. A starting address of the ready actor is returned to the kernel that invokes the actor.After the actor completes, the kernel resets actor’s ready-bit vector masked by a always-readymask, and sends results and tokens to destination actors according to a destination list storedin the destination address table of CU. Sending of data and tokens to remote processingmodules is performed by a I/O co-processor1 and communication adapters. A message thatincludes data (vector or scalar), is passed to the destination processing unit via DMA. Whendata transmission completes, a corresponding token is passed to the auxiliary Control Unit ofthe destination processing module. A DMA engine is controlled by the IO co-processor. TheVSPD-1 system can be used, in particular, as a functional accelerator for a minicomputerwith a PDP-11 architecture for applications that require high performance computing, such assimulation of complex dynamic objects in real-time.10.6 Let us Sum UpIn this lesson we have learnt about

a) Sequential and Parallel processingb) Array processorsc) and data flow computers10.7 Points for discussionDiscuss abouta) array or vector proceesorb) Discuss the working of a data flow computers10.8 Model Answer to check your progressThe answer for the question a in section 10.8 can be written asA vector processor, or array processor, is a CPU design that is able to runmathematical operations on multiple data elements simultaneously. This is in contrast to ascalar processor which handles one element at a time. The vast majority of CPUs are scalar(or close to it). Vector processors were common in the scientific computing area, where theyformed the basis of most supercomputers through the 1980s and into the 1990s, but generalincreases in performance and processor design saw the near disappearance of the vectorprocessor as a general-purpose CPU. Today most commodity CPU designs include somevector processing instructions, typically known as SIMD (Single Instruction, Multiple Data),common examples include SSE and AltiVec. Modern video game consoles and consumercomputer-graphics hardware rely heavily on vector processing in their architecture. In 2000,IBM, Toshiba and Sony collaborated to create a Cell processor, consisting of one scalarprocessor and eight vector processors, for the Sony PlayStation 3.10.9 Lesson - end Activities68After learning this chapter, try to discuss among your friends and answer thesequestions to check your progress.b) Discuss about sequential processingc) Discuss about Parallel processing6910.10 References♦ Charles Crowley, Chapter 6 of “Operating Systems – A Design-Oriented Approach”,Tata McGraw-Hill, 2001♦ H.M. Deitel, Chapter 11 of “Operating Systems”, Second Edition, Pearson Education,2001♦ Andrew S. Tanenbaum, Chapter 9, 10, 11, 12, 13 of “Modern Operating Systems”,PHI, 1996♦ D.M. Dhamdhere, Chapter 19 of “Systems Programming and Operating Systems”,Tata McGraw-Hill, 199770LESSON – 11: MULTIPROCESSING AND FAULT TOLERANCECONTENTS11.1 Aims and Objectives11.2 Introduction to process11.3 Multiprocessing techniques11.3.1 Instruction and data streams11.3.2 Processor coupling11.3.4 SISD multiprocessing11.3.5 SIMD multiprocessing11.3.6 MISD multiprocessing11.3.7 MIMD multiprocessing11.4 Fault-tolerant11.4.1 Fault Tolerance Requirements11.4.2 Fault-tolerance by replication, Redundancy and Diversity11.5 Let us sum up11.6 Points for discussion11.7 Model answers to Check your Progress11.8 Lesson - end activities11.9 References11.1 Aims and ObjectivesIn this lesson we will learn about the introduction to multiprocessing and faulttolerance.The objectives of this lesson is make the students aware of the following:a) Multiprocessing techniquesb) Instruction and data streamsc) Processor couplingd) SISD, SIMD, MISD, MIMD

e) Fault-tolerant, its requirements andf) Fault-tolerance by replication, Redundancy and Diversity7111.2 IntroductionMultiprocessing is the use of two or more central processing units (CPUs) within asingle computer system. The term also refers to the ability of a system to support more thanone processor and/or the ability to allocate tasks between them. There are many variations onthis basic theme, and the definition of multiprocessing can vary with context, mostly as afunction of how CPUs are defined (multiple cores on one die, multiple chips in one package,multiple packages in one system unit, etc.).Multiprocessing sometimes refers to the execution of multiple concurrent softwareprocesses in a system as opposed to a single process at any one instant. However, the termmultiprogramming is more appropriate to describe this concept, which is implemented mostlyin software, whereas multiprocessing is more appropriate to describe the use of multiplehardware CPUs. A system can be both multiprocessing and multiprogramming, only one ofthe two, or neither of the two.In a multiprocessing system, all CPUs may be equal, or some may be reserved forspecial purposes. A combination of hardware and operating-system software designconsiderations determine the symmetry (or lack thereof) in a given system. For example,hardware or software considerations may require that only one CPU respond to all hardwareinterrupts, whereas all other work in the system may be distributed equally among CPUs; orexecution of kernel-mode code may be restricted to only one processor (either a specificprocessor, or only one processor at a time), whereas user-mode code may be executed in anycombination of processors. Multiprocessing systems are often easier to design if suchrestrictions are imposed, but they tend to be less efficient than systems in which all CPUs areutilized equally.11.3 Multiprocessing techniquesSystems that treat all CPUs equally are called symmetric multiprocessing (SMP)systems. In systems where all CPUs are not equal, system resources may be divided in anumber of ways, including asymmetric multiprocessing (ASMP), non-uniform memoryaccess (NUMA) multiprocessing, and clustered multiprocessing.11.3.1 Instruction and data streamsIn multiprocessing, the processors can be used to execute a single sequence ofinstructions in multiple contexts (single-instruction, multiple-data or SIMD, often used invector processing), multiple sequences of instructions in a single context (multipleinstruction,single-data or MISD, used for redundancy in fail-safe systems and sometimesapplied to describe pipelined processors or hyperthreading), or multiple sequences ofinstructions in multiple contexts (multiple-instruction, multiple-data or MIMD).7211.3.2 Processor couplingTightly-coupled multiprocessor systems contain multiple CPUs that are connected atthe bus level. These CPUs may have access to a central shared memory (SMP or UMA), ormay participate in a memory hierarchy with both local and shared memory (NUMA). TheIBM p690 Regatta is an example of a high end SMP system. Intel Xeon processorsdominated the multiprocessor market for business PCs and were the only x86 option till therelease of AMD's Opteron range of processors in 2004. Both ranges of processors had theirown onboard cache but provided access to shared memory; the Xeon processors via acommon pipe and the Opteron processors via independent pathways to the system RAM.Chip multiprocessors, also known as multi-core computing, involves more than oneprocessor placed on a single chip and can be thought of the most extreme form of tightlycoupledmultiprocessing. Mainframe systems with multiple processors are often tightlycoupled.Loosely-coupled multiprocessor systems (often referred to as clusters) are based onmultiple standalone single or dual processor commodity-computers interconnected via a highspeed communication system (Gigabit Ethernet is common). A Linux Beowulf cluster is anexample of a loosely-coupled system.Tightly-coupled systems perform better and are physically smaller than looselycoupledsystems, but have historically required greater initial investments and may depreciaterapidly; nodes in a loosely-coupled system are usually inexpensive commodity computersand can be recycled as independent machines upon retirement from the cluster.Power consumption is also a consideration. Tightly-coupled systems tend to be muchmore energy efficient than clusters. This is because considerable economies can be realizedby designing components to work together from the beginning in tightly-coupled systems,whereas loosely-coupled systems use components that were not necessarily intendedspecifically for use in such systems.11.3.3 SISD multiprocessing

All processors of 8-bit and 16-bit instruction set are SISD mutliprocessors.7311.3.4 SIMD multiprocessingSIMD multiprocessing is well suited to parallel or vector processing, in which a verylarge set of data can be divided into parts that are individually subjected to identical butindependent operations. A single instruction stream directs the operation of multipleprocessing units to perform the same manipulations simultaneously on potentially largeamounts of data.For certain types of computing applications, this type of architecture can produceenormous increases in performance, in terms of the elapsed time required to complete a giventask. However, a drawback to this architecture is that a large part of the system falls idlewhen applications or system tasks are executed that cannot be divided into units that can beprocessed in parallel.Additionally, applications must be carefully and specially written to take maximumadvantage of the architecture, and often special optimizing compilers designed to producecode specifically for this environment must be used. Some compilers in this category providespecial constructs or extensions to allow programmers to directly specify operations to beperformed in parallel (e.g., DO FOR ALL statements in the version of FORTRAN used onthe ILLIAC IV, which was a SIMD multiprocessing supercomputer).SIMD multiprocessing finds wide use in certain domains such as computer simulation,but is of little use in general-purpose desktop and business computing environments.11.3.5 MISD multiprocessingMISD multiprocessing offers mainly the advantage of redundancy, since multipleprocessing units perform the same tasks on the same data, reducing the chances of incorrectresults if one of the units fails. MISD architectures may involve comparisons betweenprocessing units to detect failures. Apart from the redundant and fail-safe character of thistype of multiprocessing, it has few advantages, and it is very expensive. It does not improveperformance. It can be implemented in a way that is transparent to software.7411.3.6 MIMD multiprocessingMIMD multiprocessing architecture is suitable for a wide variety of tasks in whichcompletely independent and parallel execution of instructions touching different sets of datacan be put to productive use. For this reason, and because it is easy to implement, MIMDpredominates in multiprocessing.Processing is divided into multiple threads, each with its own hardware processor state,within a single software-defined process or within multiple processes. Insofar as a system hasmultiple threads awaiting dispatch (either system or user threads), this architecture makesgood use of hardware resources.MIMD does raise issues of deadlock and resource contention, however, since threadsmay collide in their access to resources in an unpredictable way that is difficult to manageefficiently. MIMD requires special coding in the operating system of a computer but does notrequire application changes unless the applications themselves use multiple threads (MIMDis transparent to single-threaded applications under most operating systems, if theapplications do not voluntarily relinquish control to the OS). Both system and user softwaremay need to use software constructs such as semaphores (also called locks or gates) toprevent one thread from interfering with another if they should happen to cross paths inreferencing the same data. This gating or locking process increases code complexity, lowersperformance, and greatly increases the amount of testing required, although not usuallyenough to negate the advantages of multiprocessing.75Similar conflicts can arise at the hardware level between CPUs (cache contention andcorruption, for example), and must usually be resolved in hardware, or with a combination ofsoftware and hardware (e.g., cache-clear instructions).11.4 Fault-tolerantFault-tolerant describes a computer system or component designed so that, in theevent that a component fails, a backup component or procedure can immediately take itsplace with no loss of service. Fault tolerance can be provided with software, or embedded inhardware, or provided by some combination.In the software implementation, the operating system provides an interface that allowsa programmer to "checkpoint" critical data at pre-determined points within a transaction. Inthe hardware implementation (for example, with Stratus and its VOS operating system), theprogrammer does not need to be aware of the fault-tolerant capablilities of the machine.At a hardware level, fault tolerance is achieved by duplexing each hardwarecomponent. Disks are mirrored. Multiple processors are "lock-stepped" together and theiroutputs are compared for correctness. When an anomaly occurs, the faulty component is

determined and taken out of service, but the machine continues to function as usual.Fault-tolerance or graceful degradation is the property that enables a system (oftencomputer-based) to continue operating properly in the event of the failure of (or one or morefaults within) some of its components. If its operating quality decreases at all, the decrease isproportional to the severity of the failure, as compared to a naively-designed system in whicheven a small failure can cause total breakdown. Fault-tolerance is particularly sought-after inhigh-availability or life-critical systems.Fault-tolerance is not just a property of individual machines; it may also characterisethe rules by which they interact. For example, the Transmission Control Protocol (TCP) isdesigned to allow reliable two-way communication in a packet-switched network, even in thepresence of communications links which are imperfect or overloaded. It does this byrequiring the endpoints of the communication to expect packet loss, duplication, reorderingand corruption, so that these conditions do not damage data integrity, and only reducethroughput by a proportional amount.Data formats may also be designed to degrade gracefully. HTML for example, isdesigned to be forward compatible, allowing new HTML entities to be ignored by Webbrowsers which do not understand them without causing the document to be unusable.Recovery from errors in fault-tolerant systems can be characterised as either rollforwardor roll-back. When the system detects that it has made an error, roll-forward recoverytakes the system state at that time and corrects it, to be able to move forward. Roll-backrecovery reverts the system state back to some earlier, correct version, for example usingcheckpointing, and moves forward from there. Roll-back recovery requires that theoperations between the checkpoint and the detected erroneous state can be made idempotent.Some systems make use of both roll-forward and roll-back recovery for different errors ordifferent parts of one error.76Within the scope of an individual system, fault-tolerance can be achieved byanticipating exceptional conditions and building the system to cope with them, and, ingeneral, aiming for self-stabilization so that the system converges towards an error-free state.However, if the consequences of a system failure are catastrophic, or the cost of making itsufficiently reliable is very high, a better solution may be to use some form of duplication. Inany case, if the consequence of a system failure is catastrophic, the system must be able touse reversion to fall back to a safe mode. This is similar to roll-back recovery but can be ahuman action if humans are present in the loop.11.4.1 Fault Tolerance RequirementsThe basic characteristics of fault tolerance require:1. No single point of failure2. No single point of repair3. Fault isolation to the failing component4. Fault containment to prevent propagation of the failure5. Availability of reversion modesIn addition, fault tolerant systems are characterized in terms of both planned serviceoutages and unplanned service outages. These are usually measured at the application leveland not just at a hardware level. The figure of merit is called availability and is expressed as apercentage. A five nines system would therefore statistically provide 99.999% availability.11.4.2 Fault-tolerance by replication, Redundancy and DiversitySpare components addresses the first fundamental characteristic of fault-tolerance inthree ways:Replication: Providing multiple identical instances of the same system or subsystem,directing tasks or requests to all of them in parallel, and choosing the correct result on thebasis of a quorum;Redundancy: Providing multiple identical instances of the same system and switchingto one of the remaining instances in case of a failure (failover);Diversity: Providing multiple different implementations of the same specification, andusing them like replicated systems to cope with errors in a specific implementation.A redundant array of independent disks (RAID) is an example of a fault-tolerantstorage device that uses data redundancy.A lockstep fault-tolerant machine uses replicated elements operating in parallel. Atany time, all the replications of each element should be in the same state. The same inputs areprovided to each replication, and the same outputs are expected. The outputs of thereplications are compared using a voting circuit. A machine with two replications of eachelement is termed Dual Modular Redundant (DMR). The voting circuit can then only detect a77mismatch and recovery relies on other methods. A machine with three replications of eachelement is termed Triple Modular Redundancy (TMR). The voting circuit can determine

which replication is in error when a two-to-one vote is observed. In this case, the votingcircuit can output the correct result, and discard the erroneous version. After this, the internalstate of the erroneous replication is assumed to be different from that of the other two, and thevoting circuit can switch to a DMR mode. This model can be applied to any larger number ofreplications.Lockstep fault tolerant machines are most easily made fully synchronous, with eachgate of each replication making the same state transition on the same edge of the clock, andthe clocks to the replications being exactly in phase. However, it is possible to build lockstepsystems without this requirement.Bringing the replications into synchrony requires making their internal stored statesthe same. They can be started from a fixed initial state, such as the reset state. Alternatively,the internal state of one replica can be copied to another replica.One variant of DMR is pair-and-spare. Two replicated elements operate in lockstep asa pair, with a voting circuit that detects any mismatch between their operations and outputs asignal indicating that there is an error. Another pair operates exactly the same way. A finalcircuit selects the output of the pair that does not proclaim that it is in error. Pair-and-sparerequires four replicas rather than the three of TMR, but has been used commercially.11.5 Let us sum upIn this lesson we have learnt abouta) the multiprocessing techniquesb) and fault tolerance11.6 Points for discussionDiscuss the followinga) SISDb) SIMDc) MISDd) MIMD11.7 Model answers to check your progressThe questions given in section 11.7 are explained here.In computing, SISD (Single Instruction, Single Data) is a term referring to anarchitecture in which a single processor executes a single instruction stream, to operate ondata stored in a single memory. Corresponds to the von Neumann architecture.In computing, SIMD (Single Instruction, Multiple Data) is a technique employed toachieve data level parallelism, like with vector processor. First made popular in large-scale78supercomputers (contrary to MIMD parallelization), smaller-scale SIMD operations havenow become widespread in personal computer hardware.In computing, MISD (Multiple Instruction, Single Data) is a type of parallelcomputing architecture where many functional units perform different operations on the samedata. Pipeline architectures belong to this type, though a purist might say that the data isdifferent after processing by each stage in the pipeline. Fault-tolerant computers executingthe same instructions redundantly in order to detect and mask errors, in a manner known astask replication, may] be considered to belong to this type. Not many instantiations of thisarchitecture exist, as MIMD and SIMD are often more appropriate for common data paralleltechniques. Specifically, they allow better scaling and use of computational resources thanMISD does.In computing, MIMD (Multiple Instruction stream, Multiple Data stream) is atechnique employed to achieve parallelism. Machines using MIMD have a number ofprocessors that function asynchronously and independently. At any time, different processorsmay be executing different instructions on different pieces of data. MIMD architectures maybe used in a number of application areas such as computer-aided design/computer-aidedmanufacturing, simulation, modeling, and as communication switches. MIMD machines canbe of either shared memory or distributed memory categories. These classifications are basedon how MIMD processors access memory. Shared memory machines may be of the busbased,extended, or hierarchical type. Distributed memory machines may have hypercube ormesh interconnection schemes.11.8 Lesson - end activitiesAfter learning this chapter, try to discuss among your friends and answer thesequestions to check your progress.b) Discuss about various multiprocessing techniquesc) Discuss about fault tolerance11.9. Referencesa) Charles Crowley, Chapter 5, 6 of “Operating Systems – A Design-Oriented Approach”,Tata McGraw-Hill, 2001b) H.M. Deitel, Chapter 11 of “Operating Systems”, Second Edition, Pearson Education,

2001c) Andrew S. Tanenbaum, Chapter 9, 10, 11, 12, 13 of “Modern Operating Systems”, PHI,1996d) D.M. Dhamdhere, Chapter 18, 19 of “Systems Programming and Operating Systems”,Tata McGraw-Hill, 199779UNIT – IVLESSON – 12: DEVICE AND DISK MANAGEMENTCONTENTS12.1 Aims and Objectives12.2 Introduction12.3 Need for Disk Scheduling12.4 Disk Scheduling Strategies12.4.1 First Come First Served (FCFS)12.4.2 Shortest Seek Time First (SSTF)12.4.3 SCAN12.4.4 Circular SCAN (C-SCAN)12.5 RAM(Random access memory)12.5.1 Overview12.5.2 Recent developments12.5.3 Memory wall12.5.4 DRAM packaging12.6 Optical Disks12.7 Let us sum up12.8 Points for discussion12.9 Model answers to Check your Progress12.10 Lesson - end activities12.11 References2.2 Aims and ObjectivesIn this lesson we will learn about the introduction to Disk Scheduling Strategies,RAM and optical disks.The objective of this lesson is to make sure that the student understands the following.a) Disk Scheduling,b) First Come First Served (FCFS),c) Shortest Seek Time First (SSTF)d) SCANe) Circular SCAN (C-SCAN)80f) RAM andg) Optical Disks12.2 IntroductionIn multiprogramming systems several different processes may want to use thesystem's resources simultaneously. For example, processes will contend to access an auxiliarystorage device such as a disk. The disk drive needs some mechanism to resolve thiscontention, sharing the resource between the processes fairly and efficiently.A magnetic disk consists of a collection of platters which rotate on about a centralspindle. These platters are metal disks covered with magnetic recording material on bothsides. Each disk surface is divided into concentric circles called tracks. Disk divides eachtrack into sectors, each typically contains 512 bytes. While reading and writing the headmoves over the surface of the platters until it finds the track and sector it requires. This is likefinding someone's home by first finding the street (track) and then the particular housenumber (sector). There is one head for each surface on which information is stored each onits own arm. In most systems the arms are connected together so that the heads move inunison, so that each head is over the same track on each surface. The term cylinder refers tothe collection of all tracks which are under the heads at any time.In order to satisfy an I/O request the disk controller must first move the head to thecorrect track and sector. Moving the head between cylinders takes a relatively long time so inorder to maximize the number of I/O requests which can be satisfied the scheduling policyshould try to minimize the movement of the head. On the other hand, minimizing headmovement by always satisfying the request of the closest location may mean that somerequests have to wait a long time. Thus, there is a trade-off between throughput (the averagenumber of requests satisfied in unit time) and response time (the average time between arequest arriving and it being satisfied).12.3 Need for Disk SchedulingAccess time has two major components namely seek time and rotational latency. Seek

time is the time for the disk are to move the heads to the cylinder containing the desiredsector. Rotational latency is the additional time waiting for the disk to rotate the desired81sector to the disk head. In order to have fast access time we have to minimize the seek timewhich is approximately equal to the seek distance.Disk bandwidth is the total number of bytes transferred, divided by the total timebetween the first request for service and the completion of the last transfer. The operatingsystem is responsible for using hardware efficiently for the disk drives, to have a fast accesstime and disk bandwidth. This in turn needs a good disk scheduling.12.4 Disk Scheduling StrategiesThree criteria to measure strategies are throughput, mean response time, and varianceof response times. Throughput is the number of requests serviced per unit of time. Meanresponse time is the average time spent waiting for request to be serviced. Variance ofresponse times is the measure of the predictability of response times. Hence the overall goalsof the disk scheduling strategies are to maximize the throughput and minimize both responsetime and variance of response time.12.4.1 First Come First Served (FCFS)The disk controller processes the I/O requests in the order in which they arrive, thusmoving backwards and forwards across the surface of the disk to get to the next requestedlocation each time. Since no reordering of request takes place the head may move almostrandomly across the surface of the disk. This policy aims to minimize response time withlittle regard for throughput.Figure illustrates this method. The figure depicts requests for 63, 33, 72, 47, 8, 99, 74,52, 75. If the requests have arrived in the sequence, they are also serviced in that sequence,causing the head movement as shown in the figure.FCFS is a 'just' algorithm, because, the process to make a request first is served first,but it may not be the best in terms of reducing the head movement, as is clear from the figure.82Seek pattern under the FCFS strategy.Disk request pattern.First-come-first-served (FCFS) scheduling has major drawbacks namely, (i) Seekingto randomly distributed locations results in long waiting times, (ii) Under heavy loads, systemcan become overwhelmed, (iii) Requests must be serviced in logical order to minimizedelays, (iv) and Service requests with least mechanical motion12.4.2 Shortest Seek Time First (SSTF)Each time an I/O request has been completed the disk controller selects the waitingrequest whose sector location is closest to the current position of the head. The movementacross the surface of the disk is still apparently random but the time spent in movement isminimized. This policy will have better throughput than FCFS but a request may be delayedfor a long period if many closely located requests arrive just after it.83Seek pattern under the SSTF strategyAdvantages of SSTF are higher throughput and lower response times than FCFS andit is a reasonable solution for batch processing systems. The disadvantages of SSTF are (i) itdoes does not ensure fairness, (ii) there are possibility of indefinite postponement, (iii) therewill be high variance of response times and (iv) the response time generally will beunacceptable for interactive systems.12.4.3 SCANThe drive head sweeps across the entire surface of the disk, visiting the outermostcylinders before changing direction and sweeping back to the innermost cylinders. It selectsthe next waiting requests whose location it will reach on its path backwards and forwardsacross the disk. Thus, the movement time should be less than FCFS but the policy is clearlyfairer than SSTF.12.4.4 Circular SCAN (C-SCAN)C-SCAN is similar to SCAN but I/O requests are only satisfied when the drive head istraveling in one direction across the surface of the disk. The head sweeps from the innermostcylinder to the outermost cylinder satisfying the waiting requests in order of their locations.When it reaches the outermost cylinder it sweeps back to the innermost cylinder withoutsatisfying any requests and then starts again.12.5 RAM(Random access memory)Random access memory (usually known by its acronym, RAM) is a type of datastorage used in computers. It takes the form of integrated circuits that access the stored datato be accessed in any order — that is, at random and without the physical movement of thestorage medium or a physical reading head. RAM is a volatile memory as the information orinstructions stored in it will be lost if the power is switched off.

The word "random" refers to the fact that any piece of data can be returned in aconstant time, regardless of its physical location and whether or not it is related to theprevious piece of data. This contrasts with storage mechanisms such as tapes, magnetic discsand optical discs, which rely on the physical movement of the recording medium or a reading84head. In these devices, the movement takes longer than the data transfer, and the retrievaltime varies depending on the physical location of the next item.A 1GB DDR RAM memoryRAM is usually writable as well as readable, so "RAM" is often used interchangeablywith "read-write memory". The alternative to this is "ROM", or Read Only Memory. Mosttypes of RAM lose their data when the computer powers down. "Flash memory" is aROM/RAM hybrid that can be written to, but which does not require power to maintain itscontents. RAM is not strictly the opposite of ROM, however. The word random indicates acontrast with serial access or sequential access memory."Random access" is also the name of an indexing method: hence, disk storage is oftencalled "random access" because the reading head can move relatively quickly from one pieceof data to another, and does not have to read all the data in between. However the final "M" iscrucial: "RAM" (provided there is no additional term as in "DVD-RAM") always refers to asolid-state device.Many CPU-based designs actually have a memory hierarchy consisting of registers,on-die SRAM caches, DRAM, paging systems, and virtual memory or swap space on a harddrive.This entire pool of memory may be referred to as "RAM" by many developers, eventhough the various subsystems can have very different access times, violating the originalconcept behind the "random access" term in RAM. Even within a hierarchy level such asDRAM, the specific row/column/bank/rank/channel/interleave organization of thecomponents make the access time variable, although not to the extent that rotating storagemedia or a tape is variable.12.5.1 OverviewThe key benefit of RAM over types of storage which require physical movement isthat retrieval times are short and consistent. Short because no physical movement isnecessary, and consistent because the time taken to retrieve a piece of data does not dependon its current distance from a physical head; it requires practically the same amount of time toaccess any piece of data stored in a RAM chip. Most other technologies have inherent delaysfor reading a particular bit or byte. The disadvantage of RAM over physically moving mediais cost, and the loss of data when power is turned off.Because of this speed and consistency, RAM is used as 'main memory' or primarystorage: the working area used for loading, displaying and manipulating applications anddata. In most personal computers, the RAM is not an integral part of the motherboard or85CPU—it comes in the easily upgraded form of modules called memory sticks or RAM sticksabout the size of a few sticks of chewing gum. These can quickly be removed and replaced. Asmaller amount of random-access memory is also integrated with the CPU, but this is usuallyreferred to as "cache" memory, rather than RAM.Modern RAM generally stores a bit of data as either a charge in a capacitor, as indynamic RAM, or the state of a flip-flop, as in static RAM. Some types of RAM can detect orcorrect random faults called memory errors in the stored data, using RAM parity and errorcorrection codes.Many types of RAM are volatile, which means that unlike some other forms ofcomputer storage such as disk storage and tape storage, they lose all data when the computeris powered down. For these reasons, nearly all PCs use disks as "secondary storage".Software can "partition" a portion of a computer's RAM, allowing it to act as a muchfaster hard drive that is called a RAM disk. Unless the memory used is non-volatile, a RAMdisk loses the stored data when the computer is shut down. However, volatile memory canretain its data when the computer is shut down if it has a separate power source, usually abattery.If a computer becomes low on RAM during intensive application cycles, the computercan resort to so-called virtual memory. In this case, the computer temporarily uses hard drivespace as additional memory. Constantly relying on this type of backup memory it is calledthrashing, which is generally undesirable, as virtual memory lacks the advantages of RAM. Inorder to reduce the dependency on virtual memory, more RAM can be installed.12.5.2 Recent developmentsCurrently, several types of non-volatile RAM are under development, which willpreserve data while powered down. The technologies used include carbon nanotubes and themagnetic tunnel effect.In summer 2003, a 128 KB magnetic RAM chip manufactured with 0.18 μm

technology was introduced. The core technology of MRAM is based on the magnetic tunneleffect. In June 2004, Infineon Technologies unveiled a 16 MB prototype again based on 0.18μm technology.Nantero built a functioning carbon nanotube memory prototype 10 GB array in 2004.In 2006, Solid state memory came of age, especially when implemented as "Solidstate disks", with capacities exceeding 150 gigabytes and speeds far exceeding traditionaldisks. This development has started to blur the definition between traditional random accessmemory and disks, dramatically reducing the difference in performance.12.5.3 Memory wallThe "memory wall" is the growing disparity of speed between CPU and memoryoutside the CPU chip. An important reason of this disparity is the limited communicationbandwidth beyond chip boundaries. From 1986 to 2000, CPU speed improved at an annualrate of 55% while memory speed only improved at 10%. Given these trends, it was expected86that memory latency would become an overwhelming bottleneck in computer performance.Currently, CPU speed improvements have slowed significantly partly due to major physicalbarriers and partly because current CPU designs have already hit the memory wall in somesense. Intel summarized these causes in their Platform 2015 documentation“First of all, as chip geometries shrink and clock frequencies rise, the transistorleakage current increases, leading to excess power consumption and heat (more on powerconsumption below). Secondly, the advantages of higher clock speeds are in part negated bymemory latency, since memory access times have not been able to keep pace with increasingclock frequencies. Third, for certain applications, traditional serial architectures are becomingless efficient as processors get faster (due to the so-called Von Neumann bottleneck), furtherundercutting any gains that frequency increases might otherwise buy. In addition, resistancecapacitance(RC) delays in signal transmission are growing as feature sizes shrink, imposingan additional bottleneck that frequency increases don't address.”The RC delays in signal transmission were also noted in Clock Rate versus IPC: TheEnd of the Road for Conventional Microarchitectures which projects a maximum of 12.5%average annual CPU performance improvement between 2000 and 2014. The data on IntelProcessors clearly shows a slowdown in performance improvements in recent processors.However, Intel's new processors, Core 2 Duo (codenamed Conroe) show a significantimprovement over previous Pentium 4 processors; due to a more efficient architecture,performance increased while clock rate actually decreased.12.5.4 DRAM packagingFor economic reasons, the large (main) memories found in personal computers,workstations, and non-handheld game-consoles (such as Playstation and Xbox) normallyconsists of dynamic RAM (DRAM). Other parts of the computer, such as cache memoriesand data buffers in hard disks, normally use static RAM (SRAM).12.6 Optical DisksAn optical disc is an electronic data storage medium that can be written to and readusing a low-powered laser beam. Originally developed in the late 1960s, the first optical disc,created by James T. Russell, stored data as micron-wide dots of light and dark. A laser readthe dots, and the data was converted to an electrical signal, and finally to audio or visualoutput. However, the technology didn't appear in the marketplace until Philips and Sonycame out with the compact disc (CD) in 1982. Since then, there has been a constantsuccession of optical disc formats, first in CD formats, followed by a number of DVDformats.Optical disc offers a number of advantages over magnetic storage media. An opticaldisc holds much more data. The greater control and focus possible with laser beams (incomparison to tiny magnetic heads) means that more data can be written into a smaller space.Storage capacity increases with each new generation of optical media. Emerging standards,such as Blu-ray, offer up to 27 gigabytes (GB) on a single-sided 12-centimeter disc. Incomparison, a diskette, for example, can hold 1.44 megabytes (MB). Optical discs areinexpensive to manufacture and data stored on them is relatively impervious to mostenvironmental threats, such as power surges, or magnetic disturbances.87Optical disc offers a number of advantages over magnetic storage media. An opticaldisc holds much more data. The greater control and focus possible with laser beams (incomparison to tiny magnetic heads) means that more data can be written into a smaller space.Storage capacity increases with each new generation.12.7 Let us sum upIn this lesson we have learnt abouta) the various disk scheduling strategiesb) the Random access memory

c) and the optical disk12.8 Points for discussiona) Compare disk scheduling algorithms12.9 Model answers to Check your ProgressThe answer for the question given in section 12.9 is discussed here for helping thestudents to check their progress. FCFS (First Come, First Served)o perform operations in order requestedo no reordering of work queueo no starvation: every request is servicedo poor performance SSTF (Shortest Seek Time First)o after a request, go to the closest request in the work queue, regardless ofdirectiono reduces total seek time compared to FCFSo Disadvantages starvation is possible; stay in one area of the disk if very busy switching directions slows things down SCANo go from the outside to the inside servicing requests and then back from theoutside to the inside servicing requests.o repeats this over and over.o reduces variance compared to SSTF. LOOKo like SCAN but stops moving inwards (or outwards) when no more requests inthat direction exist. C-SCAN (circular scan)o moves inwards servicing requests until it reaches the innermost cylinder; thenjumps to the outside cylinder of the disk without servicing any requests.o repeats this over and over.o variant: service requests from inside to outside, and then skip back to theinnermost cylinder. C-LOOKo moves inwards servicing requests until there are no more requests in thatdirection, then it jumps to the outermost outstanding requests.88o repeast this over and over.o variant: service requests from inside to outside, then skip back to theinnermost request.12.10 Lesson - end activitiesAfter learning this chapter, try to discuss among your friends and answer these questions tocheck your progress.a) Discuss about various disk scheduling strategiesb) Explain about FCFSc) Explain about SSTFd) Explain about SCANe) Discuss about RAMf) Discuss about Optical storages12.11 References♦ Charles Crowley, Chapter 15 of “Operating Systems – A Design-Oriented Approach”,Tata McGraw-Hill, 2001♦ H.M. Deitel, Chapter 12 of “Operating Systems”, Second Edition, Pearson Education,2001♦ Andrew S. Tanenbaum, Chapter 5 of “Modern Operating Systems”, PHI, 1996♦ D.M. Dhamdhere, Chapter 16, 17 of “Systems Programming and Operating Systems”,Tata McGraw-Hill, 199789LESSON – 13: FILE SYSTEMS AND ORGANIZATIONCONTENTS13.1 Aims and Objectives13.2 Introduction13.2.1 Aspects of file systems13.3 Types of file systems13.3.1 Disk file systems

13.3.2 Flash file systems13.3.3 Database file systems13.3.4 Transactional file systems13.3.5 Network file systems13.3.6 Special purpose file systems13.3.7 Flat file systems13.4 File systems and operating systems13.5 Multiple Filesystem Support13.6 Organization13.7 Examples of file systems13.7.1 File systems under Unix and Linux systems13.7.2 File systems under Mac OS X13.7.3 File systems under Plan 9 from Bell Labs13.7.4 File systems under MicrosoftWindows13.8 Let us sum up13.9 Points for Discussion13.10 Model Answers to Check your Progress13.11 Lesson - end activities13.12 References13.1 Aims and ObjectivesIn this lesson we will learn the basics, history, versions and environment of DOS. Thislesson also covers the basic definitions and history of UNIX.The objectives of this lesson is to make the student aware of the basic concepts andhistory of both DOS and UNIX9013.1 IntroductionIn computing, a file system (often also written as filesystem) is a method for storingand organizing computer files and the data they contain to make it easy to find and accessthem. File systems may use a data storage device such as a hard disk or CD-ROM andinvolve maintaining the physical location of the files, they might provide access to data on afile server by acting as clients for a network protocol (e.g., NFS, SMB, or 9P clients), or theymay be virtual and exist only as an access method for virtual data.More formally, a file system is a set of abstract data types that are implemented forthe storage, hierarchical organization, manipulation, navigation, access, and retrieval of data.File systems share much in common with database technology, but it is debatable whether afile system can be classified as a special-purpose database (DBMS).1.3.2 Aspects of file systemsThe most familiar file systems make use of an underlying data storage device thatoffers access to an array of fixed-size blocks, sometimes called sectors, generally 512 byteseach. The file system software is responsible for organizing these sectors into files anddirectories, and keeping track of which sectors belong to which file and which are not beingused.However, file systems need not make use of a storage device at all. A file system canbe used to organize and represent access to any data, whether it be stored or dynamicallygenerated (eg, from a network connection).Whether the file system has an underlying storage device or not, file systems typicallyhave directories which associate file names with files, usually by connecting the file name toan index into a file allocation table of some sort, such as the FAT in an MS-DOS file system,or an inode in a Unix-like file system. Directory structures may be flat, or allow hierarchieswhere directories may contain subdirectories. In some file systems, file names are structured,with special syntax for filename extensions and version numbers. In others, file names aresimple strings, and per-file metadata is stored elsewhere.Other bookkeeping information is typically associated with each file within a filesystem. The length of the data contained in a file may be stored as the number of blocksallocated for the file or as an exact byte count. The time that the file was last modified may bestored as the file's timestamp. Some file systems also store the file creation time, the time itwas last accessed, and the time that the file's meta-data was changed. (Note that many earlyPC operating systems did not keep track of file times.) Other information can include thefile's device type (e.g., block, character, socket, subdirectory, etc.), its owner user-ID andgroup-ID, and its access permission settings (e.g., whether the file is read-only, executable,etc.).The hierarchical file system was an early research interest of Dennis Ritchie of Unixfame; previous implementations were restricted to only a few levels, notably the IBMimplementations, even of their early databases like IMS. After the success of Unix, Ritchieextended the file system concept to every object in his later operating system developments,

such as Plan 9 and Inferno.91Traditional file systems offer facilities to create, move and delete both files anddirectories. They lack facilities to create additional links to a directory (hard links in Unix),rename parent links (".." in Unix-like OS), and create bidirectional links to files.Traditional file systems also offer facilities to truncate, append to, create, move, deleteand in-place modify files. They do not offer facilities to prepend to or truncate from thebeginning of a file, let alone arbitrary insertion into or deletion from a file. The operationsprovided are highly asymmetric and lack the generality to be useful in unexpected contexts.For example, interprocess pipes in Unix have to be implemented outside of the file systembecause the pipes concept does not offer truncation from the beginning of files.Secure access to basic file system operations can be based on a scheme of accesscontrol lists or capabilities. Research has shown access control lists to be difficult to secureproperly, which is why research operating systems tend to use capabilities. Commercial filesystems still use access control lists.Arbitrary attributes can be associated on advanced file systems, such as XFS,ext2/ext3, some versions of UFS, and HFS+, using extended file attributes. This feature isimplemented in the kernels of Linux, FreeBSD and Mac OS X operating systems, and allowsmetadata to be associated with the file at the file system level. This, for example, could be theauthor of a document, the character encoding of a plain-text document, or a checksum.1.4 Types of file systemsFile system types can be classified into disk file systems, network file systems andspecial purpose file systems.1.4.1 Disk file systemsA disk file system is a file system designed for the storage of files on a data storagedevice, most commonly a disk drive, which might be directly or indirectly connected to thecomputer. Examples of disk file systems include FAT, FAT32, NTFS, HFS and HFS+, ext2,ext3, ISO 9660, ODS-5, and UDF. Some disk file systems are journaling file systems orversioning file systems.1.4.2 Flash file systemsA flash file system is a file system designed for storing files on flash memory devices.These are becoming more prevalent as the number of mobile devices is increasing, and thecapacity of flash memories catches up with hard drives.While a block device layer can run emulate hard drive behavior and store regular filesystems on a flash device, this is suboptimal for several reasons:Erasing blocks: Flash memory blocks have to be explicitly erased before they can bewritten to. The time taken to erase blocks can be significant, thus it is beneficial to eraseunused blocks while the device is idle.92Random access: Disk file systems are optimized to avoid disk seeks wheneverpossible, due to the high cost of seeking. Flash memory devices impose no seek latency.Wear levelling: Flash memory devices tend to "wear out" when a single block isrepeatedly overwritten; flash file systems try to spread out writes as evenly as possible.It turns out that log-structured file systems have all the desirable properties for a flashfile system. Such file systems include JFFS2 and YAFFS.1.4.3 Database file systemsA new concept for file management is the concept of a database-based file system.Instead of, or in addition to, hierarchical structured management, files are identified by theircharacteristics, like type of file, topic, author, or similar metadata.1.4.4 Transactional file systemsThis is a special kind of file system in that it logs events or transactions to files. Eachoperation that you do may involve changes to a number of different files and disk structures.In many cases, these changes are related, meaning that it is important that they all beexecuted at the same time. Take for example a bank sending another bank some moneyelectronically. The bank's computer will "send" the transfer instruction to the other bank andalso update its own records to indicate the transfer has occurred. If for some reason thecomputer crashes before it has had a chance to update its own records, then on reset, therewill be no record of the transfer but the bank will be missing some money. A transactionalsystem can rebuild the actions by resynchronizing the "transactions" on both ends to correctthe failure. All transactions can be saved as well, providing a complete record of what wasdone and where. This type of file system is designed and intended to be fault tolerant, andnecessarily incurs a high degree of overhead.931.4.5 Network file systemsA network file system is a file system that acts as a client for a remote file access

protocol, providing access to files on a server. Examples of network file systems includeclients for the NFS, SMB protocols, and file-system-like clients for FTP and WebDAV.1.4.6 Special purpose file systemsA special purpose file system is basically any file system that is not a disk file system ornetwork file system. This includes systems where the files are arranged dynamically bysoftware, intended for such purposes as communication between computer processes ortemporary file space.Special purpose file systems are most commonly used by file-centric operating systemssuch as Unix. Examples include the procfs (/proc) file system used by some Unix variants,which grants access to information about processes and other operating system features.Deep space science exploration craft, like Voyager I & II used digital tape based specialfile systems. Most modern space exploration craft like Cassini-Huygens used Real-timeoperating system file systems or RTOS influenced file systems. The Mars Rovers are onesuch example of an RTOS file system, important in this case because they are implemented inflash memory.1.4.7 Flat file systemsIn a flat file system, there are no subdirectories—everything is stored at the same (root)level on the media, be it a hard disk, floppy disk, etc. While simple, this system rapidlybecomes inefficient as the number of files grows, and makes it difficult for users to organisedata into related groups.Like many small systems before it, the original Apple Macintosh featured a flat filesystem, called Macintosh File System. Its version of Mac OS was unusual in that the filemanagement software (Macintosh Finder) created the illusion of a partially hierarchical filingsystem on top of MFS. This structure meant that every file on a disk had to have a uniquename, even if it appeared to be in a separate folder. MFS was quickly replaced withHierarchical File System, which supported real directories.1.5 File systems and operating systemsMost operating systems provide a file system, and is an integral part of any modernoperating system. Early microcomputer operating systems' only real task was filemanagement — a fact reflected in their names (see DOS). Some early operating systems hada separate component for handling file systems which was called a disk operating system. Onsome microcomputers, the disk operating system was loaded separately from the rest of theoperating system. On early operating systems, there was usually support for only one, native,unnamed file system; for example, CP/M supports only its own file system, which might becalled "CP/M file system" if needed, but which didn't bear any official name at all.Because of this, there needs to be an interface provided by the operating system softwarebetween the user and the file system. This interface can be textual (such as provided by a94command line interface, such as the Unix shell, or OpenVMS DCL) or graphical (such asprovided by a graphical user interface, such as file browsers). If graphical, the metaphor ofthe folder, containing documents, other files, and nested folders is often used (see also:directory and folder).1.6 Multiple Filesystem SupportWith the expansion of network computing, it became desirable to support both localand remote filesystems. To simplify the support of multiple filesystems, the developers addeda new virtual node or vnode interface to the kernel. The set of operations exported from thevnode interface appear much like the filesystem operations previously supported by the localfilesystem. However, they may be supported by a wide range of filesystem types:• Local disk-based filesystems• Files imported using a variety of remote filesystem protocols• Read-only CD-ROM filesystems• Filesystems providing special-purpose interfaces - for example, the /proc filesystemA few variants of 4.4BSD, such as FreeBSD, allow filesystems to be loadeddynamically when the filesystems are first referenced by the mount system call.1.7 Organization1. A file is organized logically as a sequence of records.2. Records are mapped onto disk blocks.3. Files are provided as a basic construct in operating systems, so we assume theexistence of an underlying file system.4. Blocks are of a fixed size determined by the operating system.5. Record sizes vary.6. In relational database, tuples of distinct relations may be of different sizes.7. One approach to mapping database to files is to store records of one length in agiven file.8. An alternative is to structure files to accommodate variable-length records. (Fixedlength

is easier to implement.)951.8 Examples of file systems1.8.1 File systems under Unix and Linux systemsUnix operating systems create a virtual file system, which makes all the files on allthe devices appear to exist in a single hierarchy. This means, in Unix, there is one rootdirectory, and every file existing on the system is located under it somewhere. Furthermore,the Unix root directory does not have to be in any physical place. It might not be on your firsthard drive - it might not even be on your computer. Unix can use a network shared resourceas its root directory.Unix assigns a device name to each device, but this is not how the files on that deviceare accessed. Instead, to gain access to files on another device, you must first inform theoperating system where in the directory tree you would like those files to appear. Thisprocess is called mounting a file system. For example, to access the files on a CD-ROM, onemust tell the operating system "Take the file system from this CD-ROM and make it appearunder such-and-such directory". The directory given to the operating system is called themount point - it might, for example, be /media. The /media directory exists on many Unixsystems (as specified in the Filesystem Hierarchy Standard) and is intended specifically foruse as a mount point for removable media such as CDs, DVDs and like floppy disks. It maybe empty, or it may contain subdirectories for mounting individual devices. Generally, onlythe administrator (i.e. root user) may authorize the mounting of file systems.Unix-like operating systems often include software and tools that assist in themounting process and provide it new functionality. Some of these strategies have been coined"auto-mounting" as a reflection of their purpose.1. In many situations, file systems other than the root need to be available as soon as theoperating system has booted. All Unix-like systems therefore provide a facility formounting file systems at boot time. System administrators define these file systems inthe configuration file fstab, which also indicates options and mount points.2. In some situations, there is no need to mount certain file systems at boot time,although their use may be desired thereafter. There are some utilities for Unix-likesystems that allow the mounting of predefined file systems upon demand.3. Removable media have become very common with microcomputer platforms. Theyallow programs and data to be transferred between machines without a physicalconnection. Common examples include USB flash drives, CD-ROMs and DVDs.Utilities have therefore been developed to detect the presence and availability of amedium and then mount that medium without any user intervention.4. Progressive Unix-like systems have also introduced a concept called supermounting.For example, a floppy disk that has been supermounted can be physically removedfrom the system. Under normal circumstances, the disk should have beensynchronised and then unmounted before its removal. Provided synchronisation hasoccurred, a different disk can be inserted into the drive. The system automaticallynotices that the disk has changed and updates the mount point contents to reflect thenew medium. Similar functionality is found on standard Windows machines.965. A similar innovation preferred by some users is the use of autofs, a system that, likesupermounting, eliminates the need for manual mounting commands. The differencefrom supermount, other than compatibility in an apparent greater range of applicationssuch as access to file systems on network servers, is that devices are mountedtransparently when requests to their file systems are made, as would be appropriatefor file systems on network servers, rather than relying on events such as the insertionof media, as would be appropriate for removable media.1.8.2 File systems under Mac OS XMac OS X uses a file system that it inherited from Mac OS called HFS Plus. HFSPlus is a metadata-rich and case preserving file system. Due to the Unix roots of Mac OS X,Unix permissions were added to HFS Plus. Later versions of HFS Plus added journaling toprevent corruption of the file system structure and introduced a number of optimizations tothe allocation algorithms in an attempt to defragment files automatically without requiring anexternal defragmenter.Filenames can be up to 255 characters. HFS Plus uses Unicode to store filenames. OnMac OS X, the filetype can come from the type code stored in file's metadata or the filename.HFS Plus has three kinds of links: Unix-style hard links, Unix-style symbolic linksand aliases. Aliases are designed to maintain a link to their original file even if they aremoved or renamed; they are not interpreted by the file system itself, but by the File Managercode in userland.Mac OS X also supports the UFS file system, derived from the BSD Unix Fast File

System via NeXTSTEP.1.8.3 File systems under Plan 9 from Bell LabsPlan 9 from Bell Labs was originally designed to extend some of Unix's good points,and to introduce some new ideas of its own while fixing the shortcomings of Unix.With respect to file systems, the Unix system of treating things as files was continued,but in Plan 9, everything is treated as a file, and accessed as a file would be (i.e., no ioctl ormmap). Perhaps surprisingly, while the file interface is made universal it is also simplifiedconsiderably, for example symlinks, hard links and suid are made obsolete, and an atomiccreate/open operation is introduced. More importantly the set of file operations becomes welldefined and subversions of this like ioctl are eliminated.Secondly, the underlying 9P protocol was used to remove the difference betweenlocal and remote files (except for a possible difference in latency). This has the advantagethat a device or devices, represented by files, on a remote computer could be used as thoughit were the local computer's own device(s). This means that under Plan 9, multiple file serversprovide access to devices, classing them as file systems. Servers for "synthetic" file systemscan also run in user space bringing many of the advantages of micro kernel systems whilemaintaining the simplicity of the system.97Everything on a Plan 9 system has an abstraction as a file; networking, graphics,debugging, authentication, capabilities, encryption, and other services are accessed via I-Ooperations on file descriptors. For example, this allows the use of the IP stack of a gatewaymachine without need of NAT, or provides a network-transparent window system without theneed of any extra code.Another example: a Plan-9 application receives FTP service by opening an FTP site.The ftpfs server handles the open by essentially mounting the remote FTP site as part of thelocal file system. With ftpfs as an intermediary, the application can now use the usual filesystemoperations to access the FTP site as if it were part of the local file system. A furtherexample is the mail system which uses file servers that synthesize virtual files and directoriesto represent a user mailbox as /mail/fs/mbox. The wikifs provides a file system interface toa wiki.These file systems are organized with the help of private, per-process namespaces,allowing each process to have a different view of the many file systems that provideresources in a distributed system.1.8.4 File systems under MicrosoftWindowsWindows makes use of the FAT and NTFS (New Technology File System) filesystems. The FAT (File Allocation Table) filing system, supported by all versions ofMicrosoft Windows, was an evolution of that used in Microsoft's earlier operating system(MS-DOS which in turn was based on 86-DOS). FAT ultimately traces its roots back to theshortlived M-DOS project and Standalone disk BASIC before it. Over the years variousfeatures have been added to it, inspired by similar features found on file systems used byoperating systems such as UNIX.Older versions of the FAT file system (FAT12 and FAT16) had file name lengthlimits, a limit on the number of entries in the root directory of the file system and hadrestrictions on the maximum size of FAT-formatted disks or partitions. Specifically, FAT12and FAT16 had a limit of 8 characters for the file name, and 3 characters for the extension.This is commonly referred to as the 8.3 filename limit. VFAT, which was an extension toFAT12 and FAT16 introduced in Windows NT 3.5 and subsequently included in Windows95, allowed long file names (LFN). FAT32 also addressed many of the limits in FAT12 andFAT16, but remains limited compared to NTFS.NTFS, introduced with the Windows NT operating system, allowed ACL-basedpermission control. Hard links, multiple file streams, attribute indexing, quota tracking,compression and mount-points for other file systems (called "junctions") are also supported,though not all these features are well-documented.Unlike many other operating systems, Windows uses a drive letter abstraction at theuser level to distinguish one disk or partition from another. For example, the pathC:\WINDOWS represents a directory WINDOWS on the partition represented by the letter C. TheC drive is most commonly used for the primary hard disk partition, on which Windows isinstalled and from which it boots. This "tradition" has become so firmly ingrained that bugscame about in older versions of Windows which made assumptions that the drive that theoperating system was installed on was C. The tradition of using "C" for the drive letter can betraced to MS-DOS, where the letters A and B were reserved for up to two floppy disk drives;98in a common configuration, A would be the 3½-inch floppy drive, and B the 5¼-inch one.Network drives may also be mapped to drive letters.13.8 Let us sum up

In this lesson we have learnt abouta) the various types of file systemsb) the multiple file system supportc) and examples of file systems13.9 Points for Discussiona) Define file systemb) Define flat file system13.10 Model Answers to Check your ProgressIn order to check your progress, answer for the first question is given hereIn computing, a file system (often also written as filesystem) is a method for storing andorganizing computer files and the data they contain to make it easy to find and access them.File systems may use a data storage device such as a hard disk or CD-ROM and involvemaintaining the physical location of the files, they might provide access to data on a fileserver by acting as clients for a network protocol (e.g., NFS, SMB, or 9P clients), or theymay be virtual and exist only as an access method for virtual data (e.g., procfs).More formally, a file system is a set of abstract data types that are implemented for thestorage, hierarchical organization, manipulation, navigation, access, and retrieval of data. Filesystems share much in common with database technology, but it is debatable whether a filesystem can be classified as a special-purpose database (DBMS)13.11 Lesson - end activitiesAfter learning this chapter, try to discuss among your friends and answer these questionsto check your progress.b) Discuss about various file systemsc) Discuss about UNIX and Linux file systemsd) Discuss about Microsoft Windows file systems13.12 References♦ Charles Crowley, Chapter 16, 17 of “Operating Systems – A Design-OrientedApproach”, Tata McGraw-Hill, 2001♦ H.M. Deitel, Chapter 13 of “Operating Systems”, Second Edition, Pearson Education,2001♦ Andrew S. Tanenbaum, Chapter 4 of “Modern Operating Systems”, PHI, 1996♦ D.M. Dhamdhere, Chapter 17 of “Systems Programming and Operating Systems”,Tata McGraw-Hill, 199799LESSON – 14: FILE ALLOCATIONCONTENTS14.1 Aims and Objectives14.2 Introduction14.3 Free Space Management14.4 Contiguous allocation14.5 Linked allocation14.6 Indexed allocation14.7 Implementation issues14.7.1 Memory allocation14.7.2 Fixed Sized Allocation14.7.3 Variable Size Allocation14.7.4 Memory Allocation with PAGING14.7.5 MemoryMapped files14.7.6 Hardware support14.7.7 Copy-on-Write14.8 Let us sum up14.9 Points for discussion14.10 Model answer to check your progress14.11 Lesson - end activities14.12 References14.1 Aims and ObjectivesIn this lesson we will learn about the basics of file allocation.The objective of this lesson is to make the student aware of the basic concepts of thefollowinga) Free Space Management,b) Contiguous allocation, Linked allocation, and Indexed allocationc) Implementation issues10014.2 IntroductionThe main discussion here is how to allocate space to files so that disk space is effectively

utilized and files can be quickly accessed. Three major methods of allocating disk space arein wide use and they are contiguous allocation, linked allocation and indexed allocation, eachone having its own advantages and disadvantages.14.3 Free Space ManagementTo keep track of the free space, the file system maintains a free space list which recordsall disk blocks which are free. We search the free space list to create a file for the requiredamount of space and allocate it to the new file. This space is then removed from the freespace list. When a file is deleted, its disk space is added to the free space list.Bit-VectorFrequently, the free-space list is implemented as a bit map or bit vector. Each block isrepresented by a 1 bit. If the block is free, the bit is 0; if the block is allocated, the bit is 1.For example, consider a disk where blocks 2, 3, 4, 5, 8, 9, 10, 11, 12, 13, 17, 18, 25, 26,and 27 are free, and the rest of the blocks are allocated. The free-space bit map would be:11000011000000111001111110001111…The main advantage of this approach is that it is relatively simple and efficient to find nconsecutive free blocks on the disk. Unfortunately, bit vectors are inefficient unless the entirevector is kept in memory for most accesses. Keeping it main memory is possible for smallerdisks such as on microcomputers, but not for larger ones.Linked ListAnother approach is to link all the free disk blocks together, keeping a pointer to thefirst free block. This block contains a pointer to the next free disk block, and so on. In theprevious example, a pointer could be kept to block 2, as the first free block. Block 2 wouldcontain a pointer to block 3, which would point to block 4, which would point to block 5,which would point to block 8, and so on. This scheme is not efficient; to traverse the list,each block must be read, which requires substantial I/O timeGroupingA modification of the free-list approach is to store the addresses of n free blocks in thefirst free block. The first n-1 of these are actually free. The last one is the disk address ofanother block containing addresses of another n free blocks. The importance of thisimplementation is that addresses of a large number of free blocks can be found quickly.101CountingAnother approach is to take advantage of the fact that, generally, several contiguousblocks may be allocated or freed simultaneously, particularly when contiguous allocation isused. Thus, rather than keeping a list of free disk addresses, the address of the first free blockis kept and the number n of free contiguous blocks that follow the first block. Each entry inthe free-space list then consists of a disk address and a count. Although each entry requiresmore space than would a simple disk address, the overall list will be shorter, as long as thecount is generally greater than 1.14.4 Contiguous allocationThe contiguous allocation method requires each file to occupy a set of contiguousaddress on the disk. Disk addresses define a linear ordering on the disk. Notice that, with thisordering, accessing block b+1 after block b normally requires no head movement. When headmovement is needed (from the last sector of one cylinder to the first sector of the nextcylinder), it is only one track. Thus, the number of disk seeks required for accessingcontiguous allocated files in minimal, as is seek time when a seek is finally needed.Contiguous allocation of a file is defined by the disk address and the length of the first block.If the file is n blocks long, and starts at location b, then it occupies blocks b, b+1, b+2, …,b+n-1. The directory entry for each file indicates the address of the starting block and thelength of the area allocated for this file.The difficulty with contiguous allocation is finding space for a new file. If the file to becreated is n blocks long, then the OS must search for n free contiguous blocks. First-fit, bestfit,and worst-fit strategies (as discussed in Chapter 4 on multiple partition allocation) are themost common strategies used to select a free hole from the set of available holes. Simulationshave shown that both first-fit and best-fit are better than worst-fit in terms of both timestorage utilization. Neither first-fit nor best-fit is clearly best in terms of storage utilization,but first-fit is generally faster.These algorithms also suffer from external fragmentation. As files are allocated anddeleted, the free disk space is broken into little pieces. External fragmentation exists whenenough total disk space exists to satisfy a request, but this space not contiguous; storage isfragmented into a large number of small holes.Another problem with contiguous allocation is determining how much disk space isneeded for a file. When the file is created, the total amount of space it will need must beknown and allocated. How does the creator (program or person) know the size of the file tobe created. In some cases, this determination may be fairly simple (e.g. copying an existing

file), but in general the size of an output file may be difficult to estimate.14.5 Linked allocationThe problems in contiguous allocation can be traced directly to the requirement that thespaces be allocated contiguously and that the files that need these spaces are of differentsizes. These requirements can be avoided by using linked allocation.102In linked allocation, each file is a linked list of disk blocks. The directory contains apointer to the first and (optionally the last) block of the file. For example, a file of 5 blockswhich starts at block 4, might continue at block 7, then block 16, block 10, and finally block27. Each block contains a pointer to the next block and the last block contains a NIL pointer.The value -1 may be used for NIL to differentiate it from block 0.With linked allocation, each directory entry has a pointer to the first disk block of thefile. This pointer is initialized to nil (the end-of-list pointer value) to signify an empty file. Awrite to a file removes the first free block and writes to that block. This new block is thenlinked to the end of the file. To read a file, the pointers are just followed from block to block.There is no external fragmentation with linked allocation. Any free block can be used tosatisfy a request. Notice also that there is no need to declare the size of a file when that file iscreated. A file can continue to grow as long as there are free blocks. Linked allocation, doeshave disadvantages, however. The major problem is that it is inefficient to support directaccess;it is effective only for sequential-access files. To find the ith block of a file, it muststart at the beginning of that file and follow the pointers until the ith block is reached. Notethat each access to a pointer requires a disk read.Another severe problem is reliability. A bug in OS or disk hardware failure might resultin pointers being lost and damaged. The effect of which could be picking up a wrong pointerand linking it to a free block or into another file.14.6 Indexed allocationThe indexed allocation method is the solution to the problem of both contiguous andlinked allocation. This is done by bringing all the pointers together into one location calledthe index block. Of course, the index block will occupy some space and thus could beconsidered as an overhead of the method. In indexed allocation, each file has its own indexblock, which is an array of disk sector of addresses. The ith entry in the index block points tothe ith sector of the file. The directory contains the address of the index block of a file. Toread the ith sector of the file, the pointer in the ith index block entry is read to find the desiredsector. Indexed allocation supports direct access, without suffering from externalfragmentation. Any free block anywhere on the disk may satisfy a request for more space.14.7 Implementation issues14.7.1 Memory allocationDynamic memory allocation involves 2 central commands: malloc allocates a portionof unused memory for use by a process, while free frees a previously-allocated portion ofmemory, allowing it to be reused. The operating system must allocate memory among allrunning processes, but processes themselves must also allocate memory at a finer granularity.Many of the issues are the same in each case.So how can we implement these commands efficiently? First, let's consider anextremely simple algorithm: fixed size allocation.10314.7.2 Fixed Sized AllocationAssume N = 32 bytes: everything allocated is exactly 32 bytes long.If we are given a 32 MB Heap like this:Fig. 1: 32MB Heap1 0 0 1 … 0 0 0Fig 2: 1MB Free bitmapThis 32 MB Heap is divided into 32 byte chunks. In order to determine which chunksare free we need to do a little bookkeeping. Since there 1 MB of chunks, you will need 1 MBfree bitmap to do the bookkeeping. Each byte would represent a chunk in the heap. If thatbyte is 1, then the chunk is being used, if the byte is 0, then the corresponding chunk is freefor use.With this bitmap, the algorithm for allocating a chunk would be:Allocate Search through bitmapLook for free locationTurn bit to 1Return ChunkKeep in mind this whole sequence should be synchronized. There lies the problem oftwo processes attempting to allocate the same chunk at the same time. This would cause avery dangerous race condition.To free a memory location the algorithm would be:

Free(chunk #i)bitmap[i/8] &= (1 << (i % 8));Also note that without synchronization, if two threads free two chunks in the samebyte at the same time, one chunk might not look free.There are both positives and negatives to this design. The positive is that it uses asingle local data structure. However, this positive is more useful for disks than memory.Memory is much faster than disk in regards to changing to different chunks. The negative ismuch more glaring in this design. It is an O(n) algorithm for allocation! This is far tooinefficient to work as memory allocation. As a result, we should look for other solutions.One proposed solution would be to use 2 pointers: one pointer at the end of the heapand one free pointer that points to the most recently freed chunk. It would look like this:104Fig 3: Free and End pointer implementationThe allocation of memory would go like this:AllocateIf free < endReturn free++;This would have an allocation algorithm of O(1). However there is a serious problemwith this design: How would we ever free a chunk that was not the most recently allocated?So what's a better solution? Make a free list: a linked list of free chunks. The head ofthe free list points to the first free chunk, and each free chunk points to the next free chunk.The design would be:AllocateIf(free != NULL)chunk = freefree = free-> next;return chunkFreep->next = freefree = pThis uses the concept of using free space to maintain bookkeeping state. The morefree space we have, the more free pointers we need. Conversely, the less free space we have,the less of a need for free pointers. As a result, this design also results in optimal use ofspace. We can allocate all but one of the chunks (namely, the chunk containing the freepointer itself). And, since all allocations return exactly one chunk, and chunks are 100%independent of one another, the heap is nearly 100% utilizable. But a malloc() that onlyworks for N=32 is not much of a malloc at all! What happens when we do not have fixedsized allocation?Let us look at cases when N is an arbitrary number and the heap is still 32 MB. Thisis called Variable Size Allocation.14.7.3 Variable Size AllocationIn variable size allocation, unlike fixed-size allocation, we need to keep track of somebookkeeping information for allocated chunks as well as for free chunks. In particular, thefree function must know the size of the chunk it's freeing! One common way to do this is to105store bookkeeping information immediately before the pointer returned by malloc. Forexample:typedef struct malloc_chunk {int sentinel;struct malloc_bookkeeping *prev;struct malloc_bookkeeping *next;char actual_data[]; /* pointer returned from malloc() points here */} malloc_chunk_t;Then, free can just use simple pointer arithmetic to find the bookkeeping information.A simple heap using this bookkeeping structure might look like this.Fig 4: Implementation of variable sized allocation.The first component of each chunk is called the "sentinel". It shows whether thechunk is allocated (A) or not (F). The second component has a pointer to the previous chunk,and the third has a pointer to the next chunk. (Notice that those pointers point at the chunkdata, not at the initial sentinel!) An "X" the second or third column indicates that there is noprevious/next chunk. The fourth component is the chunk data; we write the size of the datathere. The first three columns are 4 bytes and the fourth column is the size of allocation.The list of chunks is kept in sorted order by address; the size of any chunk can becalculated through pointer arithmetic using the next field. As before, free chunks areadditionally kept on their own free list, using the data areas; we don't show this list in the

diagrams.What happens when alloc(3000) is called to an empty heap?Fig 5: The heap after a call to alloc(3000).What happens when we call alloc(1MB)? We will just look at the latter half of theheap from now on.106Fig 6: The heap after a call to alloc(1MB).What happens when we call alloc(10MB)?Fig 7: The heap after a call to alloc(10MB).What happens when we call free(the 1MB chunk)?Fig 8: The heap after a call to free(the 1MB chunk).Now, what happens when we call alloc(22MB)?The call will fail. Although 22MB are available, the free space is split into twochunks, a 21MB chunk and a 1MB chunk. The chunks are noncontiguous and there's no wayto move them together. The heap is fragmented, since it is unable to allocate 22MB even if ithas 22MB of space. This type of fragmentation is called external fragmentation.(There can also be internal fragmentation if malloc() returns chunks that arebigger than the user requested. For example, malloc(3) might commonly return a chunk with8 bytes of data area, rather than 3; the 5 remaining bytes are lost to internal fragmentation.)Free space is divided into noncontiguous chunks. Allocation can fail even though there isenough free space.How can we solve external fragmentation? We could use compaction to solve this problem.The heap is copied, but the free space is compacted to the right side. This allowscalls like alloc(22MB) now! But compaction is extremely expensive. It requires copying a107ton of data, and it also requires help from the programming language so that pointer variablescan be updated correctly. Compaction is not used on OSes for modern architectures.Compared to memory allocation with constraints N = 32 bytes, variable sizeallocation has more overheads, can run into external fragmentation, and internalfragmentation. To avoid these issues, we turn to memory allocation with paging. This lets theoperating system use fixed-size allocation, with all its benefits, to provide applications withvariable-size allocation!14.7.4 Memory Allocation with PAGINGFig 9: Virtual Address Space ImplementationVirtual Address Spaces:1. Reduces fragmentation by separating contiguous virtual pages from contiguity inphysical address space.2. Provides for isolation between processes. Each process has its own space.Fig 10: PagingPaging allows the OS to avoid external fragmentation. Variable sized allocationbuilt from fixed size allocation + hardware supported address indirection.Page Faults occur when you try to access an invalid address. This can happen onexecute, read, and write commands. The CPU traps on invalid accesses.14.7.5 MemoryMapped filesWhen attempting a sequential read on a file on disk, we need to use system callssuch as open(), read(), and write(). This can be quite costly if the file is rather large. Also,sharing becomes quite the pain. One alternative is memory mapped files. Memory mappingis when a process marks a portion of its memory as corresponding to some file. For example,suppose you wanted to read and write to /tmp/foo. This can be accomplished by memorymapping:Page DirectoryPage Tables%cr3Physical Addresses0 1MB 2MB4KB0 1GB-14GB (232)Physical AddressesVirtual Addresses108/tmp/foo -> memory address 0x10000So when someone accesses memory address 0x10000, they are now accessing thestart of the file foo. Because of this, the function that invokes memory mapping I/O wouldneed to return a pointer to the start of the file. This function is called:

void *mmap(void *addr, size_t len, int prot, int flags, int fildes, off_t off);The mmap function will return a pointer to the starting address, also known as thestart of the file. In the *addr parameter, it allows the user to determine what address it wouldlike to begin at. It is often best to place NULL and allow the operating system to decide. Ifthe user were to place an address that the operating system does not like, an error will occur.The len parameter is merely the length in which the user wants to map the file. The protparameter involves protection mechanism the user would like to enforce. The user can addsuch flags as PROT_READ, PROT_WRITE, and PROT_EXEC as permissions for the newmemory mapped file. The parameter flags are any miscellaneous flags that the user wishes toset. For example, to allow sharing, the user can set the MAP_SHARED flag. The parameterfildes is the file descriptor that the opened file is on. Finally, the off parameter is the offset inthe file that you want to start mapping from. An example of an invocation of this functionwould be:addr = mmap(NULL, length, PROT_READ, 0, fd, offset);Suppose that the we invoke the mmap function with a length of 1 MB and filedescriptor 2. It would cause the following effect:Fig 11: Fault on 0x308002, loads from physical address space.What occurs now is that the files are read in the background into physical memory space.When the process faults:OS checks if the addr is in mmapped spaceIf it is, then it will load that page of data from disk (unless already cached)Add virtual memory mapping for that pageSome advantages/positives to memory mapping include no need for copying whennot needed, and the ability to share a file amongst processes. If processes read the same file,they can share the same physical address. However, there are also some disadvantages tomemory mapped I/O. A simple problem is that data must be page-aligned; user programsmust be careful to ensure this. A more complex problem is the need to synchronize writeswith the disk. Once a file is memory mapped, will the OS have to write out the entire fileback to disk, even if only one byte was written? Another complex problem is unwantedsharing. Say a process P1 is reading a file, and another process P2 is writing to the same file.fd fd fd3MB 4MBPhysical Address Space:Virtual Address fd109Before memory-mapped I/O, P1 could get a stable version of the file by reading its data intomemory; no one could change P1's memory copy of the file data. But with memory-mappedI/O, it's possible that P1 would see all of P2's changes to the file, as P2 made those changes!14.7.6 Hardware supportAfter talks between software and hardware manufacturers, the hardware manufacturers wereable to improve the hardware to help support memory management. The result was a PageTable Entry like the following below:Fig 12: Page Entry TableThe new accessed bit and dirty bit are set by the hardware. The CPU sets theaccess bit to 1 when a portion of memory has been read or written, and sets the dirty bit to 1when the portion of memory has been written to. The hardware never clears these bits; theoperating system is expected to clear them as necessary.This allows for more efficient writes to mmapped files. The operating system onlyneeds to write pages of mmapped files when they have the dirty bit set to 1. All other pageshave not been altered and consequently do not need to be written back to disk. This reducesbus traffic as well as overhead. Once a read has occurred, the file is changed and the OSclears the dirty bits on the pages of the mmapped file.Fig 13:Writing to a mmapped file.Trick:When synchronizing files with the disk, the operating system clears the dirty bit on allpages of the memory mapped file when they are first read in.Note: Only write pages with D = 1! (Dirty bit set to 1). This reduces disk traffic for writes.14.7.7 Copy-on-Write3MB 4MBVA1 x fdPA fd fd12 bits of flagsA D U W P20 bitsUser/ Supervisor

Write/Read PermissionPresentDirty bitAccess BitPhysical Address110Fig 14: Copy-on-WriteTo fix unwanted sharing, a copy-on-write algorithm can be applied. This algorithmallows sharing of pages between two processes until one process decides to write to it. Whenone process tries to write on a page, the page is copied so that each process has its ownversion of the page, as depicted above.To copy-on-write, the operating system marks the pages in bookkeeping structure ascopy-on-write and the virtual address mapping is set to NON-WRITABLE. When a processtries to write to the page, it faults!When the operating system catches a fault, it copies the page, and changes thevirtual address mapping to the new copied page. It sets the virtual address mapping toWRITABLE. After this event, each process has its own copy of the page and is able to writeto its own copies.14.8 Let us sum upIn this lesson we have learnt abouta) the memory allocationb) and the allocation and freeing space14.9 Points for discussiona) Discuss about linked allocationb) Discuss about indexed allocation14.10 Model answer to Check your ProgressIn order to check the progress of the student, answer for second question in section 14.10is given here.The indexed allocation method is the solution to the problem of both contiguous andlinked allocation. This is done by bringing all the pointers together into one location calledthe index block. Of course, the index block will occupy some space and thus could beconsidered as an overhead of the method. In indexed allocation, each file has its own indexblock, which is an array of disk sector of addresses. The ith entry in the index block points tothe ith sector of the file. The directory contains the address of the index block of a file. Toread the ith sector of the file, the pointer in the ith index block entry is read to find the desiredsector. Indexed allocation supports direct access, without suffering from externalfragmentation. Any free block anywhere on the disk may satisfy a request for more space.VA1 fd VA2 fdPA fd fd11114.11 Lesson - end activitiesAfter learning this lesson, try to discuss among your friends and answer these questions tocheck your progress.b) Discuss about Memory allocationc) Discuss about free space management14.12 References♦ Charles Crowley, Chapter 16, 17 of “Operating Systems – A Design-OrientedApproach”, Tata McGraw-Hill, 2001♦ H.M. Deitel, Chapter 13 of “Operating Systems”, Second Edition, PearsonEducation, 2001♦ Andrew S. Tanenbaum, Chapter 4 of “Modern Operating Systems”, PHI, 1996♦ D.M. Dhamdhere, Chapter 17 of “Systems Programming and Operating Systems”,Tata McGraw-Hill, 1997112LESSON – 15: FILE DESCRIPTORS AND ACCESS CONTROLCONTENTS15.1 Aims and Objectives15.2 Introduction15.3 File descriptor in programming15.4 Operations on file descriptors15.4.1 Creating file descriptors15.4.2 Deriving file descriptors15.4.3 Operations on a single file descriptor15.4.4 Operations on multiple file descriptors

15.4.5 Operations on the file descriptor table15.4.6 Operations that modify process state15.4.7 File locking15.4.8 Sockets15.4.9 Miscellaneous15.4.10 Upcoming Operations15.5 Access Control Matrix15.6 Let us sum up15.7 Points for Discussion15.8 Model Answers to Check your Progress15.9 Lesson - end activities15.10 References15.1 Aims and ObjectivesIn this lesson we will learn about the file descriptors and access control.The objectives of this lesson is to make the candidate aware of the followinga) file descriptorsb) operations on file descriptora. creatingb. derivingc. modifying, etc.c) access control matrix15.2 Introduction113A file descriptor or file control block is a control block containing information thesystem needs to manage a file. The file descriptor is controlled by the operating system andis brought to the primary storage when a file is opened. A file descriptor containsinformation regarding (i) symbolic file name, (ii) location of file, (iii) file organization(sequential, indexed, etc.), (iv) device type, (v) access control data, (vi) type (data file, objectprogram, C source program, etc.), (vii) disposition (temporary or permanent), (viii) date andtime of creation, (ix) destroy date, (x) last modified date and time, (xi) access activity counts(number of reads, etc.).15.3 File descriptor in programmingIn computer programming, a file descriptor is an abstract key for accessing a file.The term is generally used in POSIX operating systems. In Microsoft Windows terminologyand in the context of the C standard I/O library, "file handle" is preferred, though the lattercase is technically a different object (see below).In POSIX, a file descriptor is an integer, specifically of the C type int. There are 3standard POSIX file descriptors which presumably every process (save perhaps a daemon)should expect to have:Integer value Name0 Standard Input (stdin)1 Standard Output (stdout)2 Standard Error (stderr)Generally, a file descriptor is an index for an entry in a kernel-resident data structurecontaining the details of all open files. In POSIX this data structure is called a file descriptortable, and each process has its own file descriptor table. The user application passes theabstract key to the kernel through a system call, and the kernel will access the file on behalfof the application, based on the key. The application itself cannot read or write the filedescriptor table directly.In Unix-like systems, file descriptors can refer to files, directories, block or characterdevices (also called "special files"), sockets, FIFOs (also called named pipes), or unnamedpipes.The FILE * file handle in the C standard I/O library routines is technically a pointer toa data structure managed by those library routines; one of those structures usually includes anactual low level file descriptor for the object in question on Unix-like systems. Since filehandle refers to this additional layer, it is not interchangeable with file descriptor.To further complicate terminology, Microsoft Windows also uses the term file handleto refer to the more low-level construct, akin to POSIX's file descriptors. Microsoft's Clibraries also provide compatibility functions which "wrap" these native handles to supportthe POSIX-like convention of integer file descriptors as detailed above.A program is passed a set of ``open file descriptors'', that is, pre-opened files. Asetuid/setgid program must deal with the fact that the user gets to select what files are open114and to what (within their permission limits). A setuid/setgid program must not assume thatopening a new file will always open into a fixed file descriptor id, or that the open will

succeed at all. It must also not assume that standard input (stdin), standard output (stdout),and standard error (stderr) refer to a terminal or are even open.The rationale behind this is easy; since an attacker can open or close a file descriptorbefore starting the program, the attacker could create an unexpected situation. If the attackercloses the standard output, when the program opens the next file it will be opened as though itwere standard output, and then it will send all standard output to that file as well. Some Clibraries will automatically open stdin, stdout, and stderr if they aren't already open (to/dev/null), but this isn't true on all Unix-like systems. Also, these libraries can't be completelydepended on; for example, on some systems it's possible to create a race condition that causesthis automatic opening to fail (and still run the program).When using ILE C/400 stream I/O functions as defined by the American NationalStandards Institute (ANSI) to perform operations on a file, you identify the file through use ofpointers. When using the integrated file system C functions, you identify the file byspecifying a file descriptor. A file descriptor is a positive integer that must be unique in eachjob. The job uses a file descriptor to identify an open file when performing operations on thefile. The file descriptor is represented by the variable fildes in C functions that operate on theintegrated file system and by the variable descriptor in C functions that operate on sockets.Each file descriptor refers to an open file description, which contains informationsuch as a file offset, status of the file, and access modes for the file. The same open filedescription can be referred to by more than one file descriptor, but a file descriptor can referto only one open file description.Figure 1. File descriptor and open file descriptionIf an ILE C/400 stream I/O function is used with the integrated file system, the ILEC/400 run-time support converts the file pointer to a file descriptor.When using the "root" (/), QOpenSys, or user-defined file systems, you can passaccess to an open file description from one job to another, thus allowing the job to access thefile. You do this by using the givedescriptor() or takedescriptor() function to pass the filedescriptor between jobs.11515.4 Operations on file descriptorsA modern Unix typically provides the following operations on file descriptors.15.4.1 Creating file descriptorsopen(), open64(), creat(), creat64()socket()socketpair()pipe()15.4.2 Deriving file descriptorsfileno()dirfd()15.4.3 Operations on a single file descriptorread(), write()recv(), send()recvmsg(), sendmsg() (inc. allowing sending FDs)sendfile()lseek(), lseek64()fstat(), fstat64()fchmod()fchown()fdopen()gzdopen()ftruncate()15.4.4 Operations on multiple file descriptorsselect(), pselect()poll(), epoll()15.4.5 Operations on the file descriptor tableclose()dup()dup2()fcntl (F_DUPFD)fcntl (F_GETFD and F_SETFD)15.4.6 Operations that modify process statefchdir(): sets the process's current working directory based on a directory filedescriptormmap(): maps ranges of a file into the process's address space116

15.4.7 File lockingflock()fcntl (F_GETLK, F_SETLK and F_SETLKW)lockf()15.4.8 Socketsconnect()bind()listen()accept(): creates a new file descriptor for an incoming connectiongetsockname()getpeername()getsockopt(), setsockopt()shutdown(): shuts down one or both halves of a full duplex connection11715.4.9 Miscellaneousioctl(): a large collection of miscellaneous operations on a single file descriptor, oftenassociated with a device15.4.10 Upcoming OperationsA series of new operations on file descriptors has been added to Solaris and Linux, aswell as numerous C libraries, to be standardized in a future version of POSIX. The at suffixsignifies that the function takes an additional first argument supplying a file descriptor fromwhich relative paths are resolved, the forms lacking the at suffix thus becoming equivalent topassing a file descriptor corresponding to the current working directory.openat()faccessat()fchmodat()fchownat()fstatat()futimesat()linkat()mkdirat()mknodat()readlinkat()renameat()symlinkat()unlinkat()mkfifoat()15.5 Access Control MatrixAccess Control Matrix or Access Matrix is an abstract, formal computer protectionand security model used in computer systems, that characterizes the rights of each subjectwith respect to every object in the system. It was first introduced by Butler W. Lampson in1971. It is the most general description of operating system protection mechanism.According to the model a computer system consists of a set of objects O, that is theset of entities that needs to be protected (e.g. processes, files, memory pages) and a set ofsubjects S, that consists of all active entities (e.g. users, processes). Further there exists a setof rights R of the form r(s,o). A right thereby specifies the kind of access a subject is allowedto process with regard to an object.118ExampleIn this matrix example there exists two processes, a file and a device. The first processhas the ability to execute the second, read the file and write some information to the device,while the second process can only send information to the first.Asset 1 Asset 2 file deviceRole 1 read, write, execute, own execute read writeRole 2 Read read, write, execute, ownUtilityBecause it does not define the granularity of protection mechanisms, the AccessControl Matrix can be used as a model of the static access permissions in any type of accesscontrol system. It does not model the rules by which permissions can change in any particularsystem, and therefore only gives an incomplete description of the system's access controlsecurity policy.An Access Control Matrix should be thought of only as an abstract model ofpermissions at a given point in time; a literal implementation of it as a two-dimensional arraywould have excessive memory requirements. Capability-based security and access controllists are categories of concrete access control mechanisms whose static permissions can be

modeled using Access Control Matrices. Although these two mechanisms have sometimesbeen presented (for example in Butler Lampson's Protection paper) as simply row-based andcolumn-based implementations of the Access Control Matrix, this view has been criticized asdrawing a misleading equivalence between systems that does not take into account dynamicbehaviour.15.6 Let us sum upIn this lesson we have learnt abouta) the file descriptorsb) and the access control matrix15.7 Points for Discussiona) Define file descriptorb) Explain about the implementation issues of file descriptor11915.8 Model Answers to Check your ProgressIn order to check the progress of the candidate the answer for the first question in the section15.8 is given below.A file descriptor or file control block is a control block containing information thesystem needs to manage a file. The file descriptor is controlled by the operating system andis brought to the primary storage when a file is opened. A file descriptor containsinformation regarding (i) symbolic file name, (ii) location of file, (iii) file organization(sequential, indexed, etc.), (iv) device type, (v) access control data, (vi) type (data file, objectprogram, C source program, etc.), (vii) disposition (temporary or permanent), (viii) date andtime of creation, (ix) destroy date, (x) last modified date and time, (xi) access activity counts(number of reads, etc.).15.9 Lesson - end activitiesAfter learning this lesson, try to discuss among your friends and answer these questions tocheck your progress.b) Discuss about file operationsc) Discuss about access control matrix15.10 Referencesa) Charles Crowley, Chapter 16, 17 of “Operating Systems – A Design-OrientedApproach”, Tata McGraw-Hill, 2001b) H.M. Deitel, Chapter 13 of “Operating Systems”, Second Edition, PearsonEducation, 2001c) Andrew S. Tanenbaum, Chapter 4 of “Modern Operating Systems”, PHI, 1996d) D.M. Dhamdhere, Chapter 17 of “Systems Programming and Operating Systems”,Tata McGraw-Hill, 1997120UNIT – VLESSON – 16: MS-DOSCONTENTS16.1 Aims and Objectives16.2 Introduction16.3 Accessing hardware under DOS16.4 Early History of MS-DOS16.5 User’s view of MS-DOS16.5.1 File Management16.5.2 Change Directory16.5.3 Make Directory16.5.4 Copy16.5.5 Delete and Undelete16.5.6 Rename16.5.7 Parent directory16.5.8 F316.5.9 Breaking16.5.10 Wildcards (*) and (?)16.6 Executing, Viewing, Editing, Printing16.7 Backup Files16.8 Other commands16.8.1 Change the Default Drive16.8.2 Change Directory Command16.8.3 DIR (Directory) Command16.8.4 ERASE Command16.8.5 FORMAT Command16.8.6 Rebooting the computer (Ctrl-Alt-Del)

16.8.7 RENAME (REN) Command16.8.8 RMDIR (RD) Remove Directory Command16.8.9 Stop Execution (Ctrl-Break)16.9 The system view of MS-DOS16.10 The future of MS-DOS16.11 Let us sum up16.12 Points for Discussion16.13 Model Answers to Check your Progress16.14 Lesson - end activities16.15 References12116.1 Aims and ObjectivesIn this lesson we will learn about the introduction to MS-DOS and its commands.The objectives of this lesson is to make the candidate aware of the followingd) History of MS-DOSe) MS-DOS Commandsf) Users view of MS-DOSg) System View of MS-DOSh) And the future of MS-DOS16.2 IntroductionA full time operating system was needed for IBM's 8086 line of computers, butnegotiations for the use of CP/M on these broke down. IBM approached Microsoft's CEO,Bill Gates, who purchased QDOS from SCP allegedly for $50,000. This became MicrosoftDisk Operating System, MS-DOS. Microsoft also licensed their system to multiple computercompanies, who used their own names. Eventually, Microsoft would require the use of theMS -DOS name, with the exception of the IBM variant, which would continue to bedeveloped concurrently and sold as PC-DOS (this was for IBM's new 'PC' using the 8088CPU (internally the same as the 8086)).Early versions of Microsoft Windows were little more than a graphical shell for DOS,and later versions of Windows were tightly integrated with MS-DOS. It is also possible to runDOS programs under OS/2 and Linux using virtual-machine emulators. Because of the longexistence and ubiquity of DOS in the world of the PC-compatible platform, DOS was oftenconsidered to be the native operating system of the PC compatible platform.There are alternative versions of DOS, such as FreeDOS and OpenDOS. FreeDOSappeared in 1994 due to Microsoft Windows 95, which differed from Windows 3.11 by beingnot a shell and dispensing with MS-DOS.MS-DOS (and the IBM PC-DOS which was licensed therefrom), and its predecessor,86-DOS, was inspired by CP/M (Control Program / (for) Microcomputers) - which was thedominant disk operating system for 8-bit Intel 8080 and Zilog Z80 based microcomputers.Tim Paterson at Seattle Computer Products developed a variant of CP/M-80, intended as aninternal product for testing the SCP's new 8086 CPU card for the S-100 bus. It did not run onthe 8080 CPU needed for CP/M-80. The system was named 86-DOS (it had initially beencalled QDOS, which stood for Quick and Dirty Operating System).Digital Research would attempt to regain the market with DR-DOS, an MS-DOS andCP/M hybrid. Digital Research would later be bought out by Novell, and DR DOS becameNovell DOS 7. DR DOS would later be part of Caldera (as OpenDOS), Lineo (as DR DOS),and DeviceLogics.Early versions of Microsoft Windows were shell programs that ran in DOS. Windows3.11 extended the shell by going into protected mode and added 32-bit support. These were16-bit/32-bit hybrids. Microsoft Windows 95 further reduced DOS to the role of the bootloader. Windows 98 and Windows Me were the last Microsoft OS to run on DOS. The DOS122based branch was eventually abandoned in favor of Windows NT, the first true 32-bit systemthat was the foundation for Windows XP and Windows Vista.Windows NT, initially NT OS/2 3.0, was the result of collaboration betweenMicrosoft and IBM to develop a 32-bit operating system that had high hardware and softwareportability. Because of the success of Windows 3.0, Microsoft changed the applicationprogramming interface to the extended Windows API, which caused a split between the twocompanies and a branch in the operating system. IBM would continue to work on OS/2 andOS/2 API, while Microsoft renamed its operating system Windows NT.16.3 Accessing hardware under DOSThe operating system offers a hardware abstraction layer that allows development ofcharacter-based applications, but not for accessing most of the hardware, such as graphicscards, printers, or mice. This required programmers to access the hardware directly, resultingin each application having its own set of device drivers for each hardware peripheral.Hardware manufacturers would release specifications to ensure device drivers for popular

applications were available.16.4 Early History of MS-DOSMicrosoft bought non-exclusive rights for marketing 86-DOS in October 1980. InJuly 1981, Microsoft bought exclusive rights for 86-DOS (by now up to version 1.14) andrenamed the operating system MS-DOS.The first IBM branded version, PC-DOS 1.0, was released in August, 1981. Itsupported up to 640 kB of RAM[3] and four 160 kB 5.25" single sided floppy disks.Various versions of DOS:Version - 1.1In May 1982, PC-DOS 1.1 added support for 320 kB double-sided floppy disks.Version – 2.0PC-DOS 2.0 and MS-DOS 2.0, released in March 1983, were the first versions tosupport the PC/XT and fixed disk drives (commonly referred to as hard disk drives). Floppydisk capacity was increased to 180 kB (single sided) and 360 kB (double sided) by using ninesectors per track instead of eight.At the same time, Microsoft announced its intention to create a GUI for DOS. Its firstversion, Windows 1.0, was announced on November 1983, but was unfinished and did notinterest IBM. By November 1985, the first finished version, Microsoft Windows 1.01, wasreleased.Version – 3.0MS-DOS 3.0, released in September 1984, first supported 1.2Mb floppy disks and32Mb hard disks.123Version - 3.1MS-DOS 3.1 released November that year, introduced network support.Version - 3.2MS-DOS 3.2, released in April 1986, was the first retail release of MS-DOS. It addedsupport of 720 kB 3.5" floppy disks. Previous versions had been sold only to computermanufacturers who pre-loaded them on their computers, because operating systems wereconsidered part of a computer, not an independent product.Version - 3.3MS-DOS 3.3, released in April 1987, featured logical disks. A physical disk could bedivided into several partitions, considered as independent disks by the operating system.Support was also added for 1.44 MB 3.5" floppy disks.The first version of DR DOS was released in May of 1988, and was compatible withMS/PC-DOS 3.3. Later versions of DR DOS would continue to identify as "DOS 3.31." toapplications, despite using newer version numbers.Version - 4.0MS-DOS 4.0, released in July 1988, supported disks up to 2 GB (disk sizes weretypically 40-60 MB in 1988), and added a full-screen shell called DOSSHELL. Other shells,like Norton Commander and PCShell, already existed in the market. In November of 1988,Microsoft addressed many bugs in a service release, MS-DOS 4.01.DR DOS skipped version 4 due to perceived unpopularity of MS-DOS 4.x. Wishingto get a jump on Microsoft, Digital Research released DR DOS 5 in May 1990, whichincluded much more powerful utilities that previous DOS versions.Version - 5.0MS-DOS 5.0 was released in April 1991, mainly as a follow-up to DR DOS 5. Itincluded the full-screen BASIC interpreter QBasic, which also provided a full-screen texteditor (previously, MS-DOS had only a line-based text editor, edlin). A disk cache utilitySmartDrive, undelete capabilities, and other improvements were also included. It had severeproblems with some disk utilities, fixed later in MS-DOS 5.01, released later in the sameyear.Version - 6.0MS-DOS 6.0 was released in March 1993. Following competition from DigitalResearch's SuperStor, Microsoft added a disk compression utility called DoubleSpace. At thetime, typical hard disk sizes were about 200-400 MB, and many users badly needed moredisk space. It turned out that DoubleSpace contained stolen code from another compressionutility, Stacker, which led to later legal problems. MS-DOS 6.0 also featured the disk124defragmenter DEFRAG, backup program MSBACKUP, memory optimization withMEMMAKER, and rudimentary virus protection via MSAV.Version - 6.2As with versions 4.0 and 5.0, MS-DOS 6.0 turned out to be buggy. Due to complaintsabout loss of data, Microsoft released an updated version, MS-DOS 6.2, with an improvedDoubleSpace utility, a new disk check utility, SCANDISK (similar to fsck from Unix), and

other improvements.December 1993 saw the release of Novell DOS 7, which was DR DOS under a newname. Its multiple bugs, as well as DR DOS' already declining market share and Windows 95looming on the horizon, led to low sales. By this time, PC DOS was at version 6.1, and IBMsplit its development from Microsoft. From this point, the two developed independently.Version - 6.21The next version of MS-DOS, 6.21 (released March 1994), appeared due to legalproblems. Stac Electronics sued Microsoft due to stolen source code from their utility,Stacker, and forced them to remove DoubleSpace from their operating system.Version - 6.22In May 1994, Microsoft released MS-DOS 6.22, with another disk compressionpackage, DriveSpace, licensed from VertiSoft Systems. MS-DOS 6.22 was the last standaloneversion of MS-DOS available to the general public. MS-DOS was removed frommarketing byMicrosoft on November 30, 2001.Version - 6.23Microsoft also released versions 6.23 to 6.25 for banks and American militaryorganizations. These versions introduced FAT32 support.Version - 7.0Microsoft Windows 95 incorporated MS-DOS version 7.0, but only as the kernel (asWindows became the full operating system). Windows 98 also used MS-DOS 7. At thispoint, Microsoft announced abandonment of the DOS kernel and released Windows 2000 onthe NT kernel, but following its commercial failure, released one more DOS kernelWindows,Windows ME. The next system, Windows XP, was based on the NT kernel. Windows MEused MS-DOS 8; Windows XP and Vista continue to use MS-DOS 8 on emergency startupdisks.IBM released PC-DOS 7.0 in early 1995. It incorporated many new utilities such asanti-virus software, comprehensive backup programs, PCMCIA support, and DOS Penextensions. Also added were new features to enhance available memory and disk space. Thelast version of PC DOS was PC DOS 2000, released in 1998. Its major feature was Y2Kcompatibility.12516.5 User’s view of MS-DOSThe user’s view of MS-DOS deals with ‘what the user can accomplish with the MSDOScommands. This section deals with some of the important MS-DOS commands andother commands can be obtained from MS-DOS manual.In MS-DOS, the Command processor interprets user commands. The Commandprocessor is stored in the file COMMAND.COM, which is loaded automatically when MSDOSis started. Internal commands are part of this file. External commands are brought intomemory from disk as and when needed. The command processor also executes programfiles. Executable files have one of the extensions .COM, .EXE, and .BAT. MS-DOSprompts the user to enter commands. The standard prompt is the letter of the current drivefollowed by the greater than sign.16.5.1 File ManagementBefore we get started with perhaps the most fundamental aspect of the operatingsystem, file management, let's make sure we are at the root directory, a sort of home base.Many commands in the DOS environment follow a certain format. In order to tellDOS to perform a function, at the command prompt, you would type the command followedby arguments which specify what you want DOS to do. For example:C:\>copy practice.txt a:"COPY" is the command that you want DOS to perform."PRACTICE.TXT A: " is an example of an argument which specifies what will beaffected by the command.In this case, DOS will copy the file practice.txt from the C: drive to the A: drive.Commands such as edit, del, rename, and many other commands require arguments similar tothe example listed above.16.5.2 Change DirectoryTo move to a different directory, use the command "cd" (for "change directory")followed by the directory you wish to move to. The backslash by itself always represents theroot directory.C:\>cd \Now let's create a file. At this point it is important to mention that DOS requires thatfilenames be no longer than eight characters with the option of a three character extension fordescriptive purposes. Also, no spaces or slashes are acceptable. Again, some of this harkensback to the limitations of earlier days. Since it is going to be a simple text file, let's call ourfile "practice.txt".

• C:\> edit practice.txt• Type your name or some other message.126• Then press the Alt key, followed by the f key to display the File menu.(You can alsouse the mouse.) Press [ALT] + [F]• Then press the s key to save the file. Press [S]• To exit, press the Alt key, followed by the f key followed by the x key. Press [ALT]+ [F] + [X]If we take a look at the root directory, the PRACTICE.TXT file will be included. DOS seesthe new file as C:\PRACTICE.TXT.16.5.3 Make DirectoryNow, let's make a couple of directories so we can put our file away in a place thatmakes sense to us. The command for making a directory is "md" (for "make directory") andis followed by the name of the directory you wish to make.• C:\> md dosclass(to make the directory DOSCLASS)• C:\> dir(to view the directory with the new DOSCLASS subdirectory)• C:\> cd dosclass(to change to the DOSCLASS directory)• C:\DOSCLASS> md samples(to make the directory "samples" inside the DOSCLASS directory)• C:\DOSCLASS> dir(to view the directory with the new SAMPLES subdirectory)Note that as soon as we changed our directory, (cd dosclass) the prompt changed torepresent the new directory. Remember, if you want to get your bearings, you can take a lookat the command prompt or display a directory list of the current directory (dir).16..5.4 CopyNow that we have created the directories, we can put the practice file we created intothe new directory, SAMPLES (C:\DOSCLASS\SAMPLES). To keep things simple, let's "cd"back to the root directory where the practice file is.C:\>cd \And now let's copy that file to the SAMPLES directory which is inside theDOSCLASS directory. In order to copy something, you must first issue the command (copy),then identify the file to be copied (source), and then the directory to which you wish to copythe file (destination).C:\>copy practice.txt dosclass\samplesA somewhat unfriendly yet useful diagram of the command format would looksomething like this (where things in brackets are optional).copy [volume+pathname+]filename [volume+pathname+]directory127What this means is that you don't have to include the volume and pathname of thesource file and the destination directory (we didn't in the first example). This is because DOSassumes that any filename or directory included in the copy command is in the currentdirectory. Because we had moved to the root directory, both PRACTICE.TXT andDOSCLASS were in the current directory.But the nice thing about command-line interfaces is that you don't have to be in thedirectory of the files you wish to act on. The command-line interface makes it possible for theuser to use one command to copy any file anywhere to any other location. From any directory(within volume C:), we could have used the following command to copy the same file to thesame directory:C:\DOSCLASS>copy \practice.txt \dosclass\samplesThis command would perform the same function as the first command. We just toldthe computer where to find the source file since it wasn't in the current directory by placing abackslash in front of the filenames (which told the computer that the file was in the rootdirectory). This applies to copies between volumes as well. All you have to do is specify thevolume:Z:\ANYWHERE>copy c:\practice.txt c:\dosclass\samplesor, a very common and useful example:C:\>copy practice.txt a:\backup\dosclassThis command copied the file to a floppy disk in the PC's floppy drive (which already had thedirectories, BACKUP\DOSCLASS).16.5.5 Delete and UndeleteThere is a slight problem now. The PRACTICE.TXT file was copied into theSAMPLES directory, but there is no reason to have two copies of PRACTICE.TXT. To

delete a file, use the DEL command (from "delete") followed by the name of the file.C:\>del practice.txtAgain, you can delete the file from any directory by including the full pathname.Z:\>del c:\practice.txtIf you accidentally delete something, there is a possibility of retrieving it using the"undelete" command. This, however, will only work for files just deleted.C:\undeleteA list of any files that can be undeleted will appear. You will need to replace the firstcharacter of the file because DOS removes the first letter of a deleted file (that's how it keepstrack of what can be written over).16.5.6 Rename128Like the above commands, you needn't be in the same directory as the file you wish torename provided you include the pathname of the file you wish to change. But the secondargument of this command, the new filename, will not accept a pathname designation. Inother words, the second argument should just be a filename (with no pathname):C:\>ren \dosclass\samples\practice.txt prac.txtNote: Being able to designate a new path would in effect allow the user to move a filefrom one place to another without having to copy and delete. A common complaint aboutDOS is that there is no "move" command. Therefore, the only way to move files from onelocation to another is first to copy them and then to delete the originals.16.5.7 Parent directoryIf you wish to move up one level in the hierarchy of the file structure (change to theparent directory), there is a shortcut--two consecutive periods: ".."C:\DOSCLASS\SAMPLES>cd ..This works in a pathname as well:C:\DOSCLASS\SAMPLES>ren ..\practice.txt prac.txtIf the file PRACTICE.TXT were in the directory above SAMPLES (the DOSCLASSdirectory), the above command would change it to PRAC.TXT.16.5.8 F3The F3 function key can be a time saver if you're prone to typos. Pressing the F3 button willretype the last command for you.16.5.9 BreakingSometimes, you may start a procedure such as a long directory listing and wish to stopbefore it is completed. The Break command often works when running batch files and otherexecutable files. To stop a procedure, press [CTRL] + [BREAK]. [CTRL] is located in thelower right-hand corner of the keyboard. [BREAK] is located in the upper right hand cornerof the keyboard and is also labeled [PAUSE].16.5.10 Wildcards (*) and (?)Another benefit of the command-prompt interface is the ability to use wildcards. Ifyou want, for example, to copy only files with the .txt extension, you employ a wildcard:C:\>copy *.txt a:\backup\txtfilesThis command would copy all of the files with the .txt extension onto a floppy diskinside the TXTFILES directory which is inside the BACKUP directory. To copy all filesfrom the C drive to the A drive, you would type:129C:\>copy *.* a:The wildcard is often used when retrieving a directory of similarly named files such as:C:\>dir *.txtThis command would display all files ending with the .txt extension. To list all files thatbegin with the letter g, you would type the following:C:\>dir g*.*Additionally the ? can be used to substitute for individual letters. If there are many similarlynamed files that end with the letters GOP but begin with letters A through G, then you cantype the following to list those files:C:\>dir ?gop.*The following command would list all similar files beginning with the letters REP and endingwith A.C:\>dir rep?a.*The ? wildcard can be used to replace any letter in any part of the filename. Wildcards suchas * and ? can be useful when you do not know the full name of a file or files and wish to listthem separately from the main directory listing.16.6 Executing, Viewing, Editing, PrintingExecutingBinary files ending in .exe are usually "executed" by typing the filename as if it were

a command. The following command would execute the WordPerfect application whichappears on the disk directory as WP.EXE:C:\APPS\WP51>wpBinary files ending in .com often contain one or more commands for execution eitherthrough the command prompt or through some program.ViewingText files, on the other hand can be viewed quickly with the type command:C:\>cd dosclass\samplesC:\DOSCLASS\SAMPLES>type practice.txt | moreEditing130Or you can view the file in the provided text editor (just as we did when we first createdpractice.txt):C:\DOSCLASS\SAMPLES>edit practice.txtPrintingIf you want to send a text file to the printer, there is the print command. But there aretwo steps:C:\DOSCLASSES\SAMPLES>print practice.txtName of list device [PRN]: lpt2If you wish to print to a networked printer, usually lpt2 is the correct response. Forlocal printers, the correct response is usually lpt1.16.7 Backup FilesIt is possible to lose files by mistake, although the more you practice the less likely itbecomes. For your own peace of mind, it is good practice to make backup copies of yourmost valuable files on a separate diskette. Store your backup disk in a safe place and don'tcarry it through a metal detector. Use the COPY command to create the backup.There is no need to backup every file you create, only the ones in which you'veinvested much work. Also, prune your backup diskette every week or two using the ERASEcommand. Backup files which have been made redundant by subsequent additions willsimply create clutter on your backup diskette. An effective file naming convention is essentialto keeping track of your backups.16.8 Other commands16.8.1 Change the Default DriveTo change the default drive, simply type the letter of the your choice. The new defaultwill be listed in subsequent DOS prompts.Example:• C> A: [enter]• Changes the default drive from C to A.• A> C: [enter]• Changes the default drive from A to C.[enter] means that you must press the Enter Key before the format command will execute.[Enter] is required after any DOS command, it is assumed in all commands found below.16.8.2 Change Directory CommandOnce you have located the directory you want, you may move from directory todirectory using the CD command (change directory)131Example:• C> cd furnitureMoves you to the directory called 'FURNITURE'• C> cd \furniture\chairsMoves you to the directory called 'CHAIRS' under the directory called'FURNITURE'.• C> cd ..Moves you up one level in the path.• C> cd \Takes you back to the root directory (c: in this case).16.8.3 DIR (Directory) CommandThe DIRECTORY command lists the names and sizes of all files located on aparticular disk.Example:• C> dir a:Shows directory of drive A• C> dir b:Shows directory of drive B• C> dir \agis

Shows files in a subdirectory on drive C (default)• C> dirShows directory of drive C• C> dir /wShows directory in wide format, as opposed to a vertical listing.All the files are listed at the screen, you can stop the display by typing CTRL-BREAK. Ifyou ask for a directory on the A or B drives, be sure there is a diskette in the drive and thatthe diskette has been formatted. If the drive is empty, or if the diskette is unformatted, theDOS will respond with an error message.16.8.4 ERASE CommandThe ERASE command deletes specified files.Example:C> erase a:myfile.txtErases the file MYFILE.TXT from the diskette in the A drive. If no drive specification isentered, the system looks to delete the specified file form drive C (in this case).IMPORTANT WARNING: This command is easy to use, but it is the most dangerousone you will encounter in DOS (apart form FORMAT). If you aren't careful, you may deletea file which you--or someone else--needs. And, unless you have saved a backup of that file,the erased file is gone for good. For this reason it is good practice to use only complete filespecifications with the ERASE command (and to keep backups of your most valuable files).As a safety precaution, never use the wild-card characters '*' and '?' in ERASE commands.13216.8.5 FORMAT CommandYou must format new disks before using them on the IBM computers. The formatcommand checks a diskette for flaws and creates a directory where all the names of thediskette's files will be stored.Example:C> format a:Formats the diskette in the A drive.After entering this command, follow the instructions on the screen. When the FORMAToperation is complete, the system will ask if you wish to FORMAT more diskettes. If you areworking with only one diskette, answer N (No) and carry on with you work. If you wish toFORMAT several diskettes, answer Y (Yes) until you have finished formatting all yourdiskettes.BEWARE: Executing the format command with a diskette which already contains fileswill result in the deletion of all the contents of the entire disk. It is best to execute the formatcommand only on new diskettes. If you format an old diskette make sure it contains nothingyou wish to save.16.8.6 Rebooting the computer (Ctrl-Alt-Del)In some cases, when all attempts to recover from a barrage of error messages fails, asa last resort you can reboot the computer. To do this, you press, all at once, the control,alternate and delete.BEWARE: If you re-boot, you may loose some of your work--any data active inRAM which has not yet been saved to disk.16.8.7 RENAME (REN) CommandThe RENAME command permits users to change the name of a file without making acopy of it.Example:C> ren a:goofy.txt pluto.txtChanges the name of 'GOOFY.TXT' on the A drive to 'PLUTO.TXT'.This command is very simple to use, just remember two points: the file name andextension must be complete for the source file and no drive specification is given for thetarget. Renaming can only occur on a single disk drive (otherwise COPY must be used).16.8.8 RMDIR (RD) Remove Directory CommandThis command removes a directory. It is only possible to execute this command if thedirectory you wish to remove is empty.Example:133C> rd mineRemoves directory called 'MINE'.16.8.9 Stop Execution (Ctrl-Break)If you wish to stop the computer in the midst of executing the current command, youmay use the key sequence Ctrl-Break. Ctrl-Break does not always work with non-DOScommands. Some software packages block its action in certain situations, but it is worthtrying before you re-boot.

16.9 The system view of MS-DOSThe systems view of MS-DOS deals with the internal organization of the operatingsystem. On booting the boot sector gets transferred to memory which then loads theinitialization routine (the DOS first portion) to organize the memory and then loads the nextportion of MS-DOS. Then the resident code and the transient code are loaded into the lowand high memory respectively. Interrupt handlers and system calls belong to resident code,while command processor and the internal commands belong to transient code.16.10 The future of MS-DOSEarly versions of Microsoft Windows were shell programs that ran in DOS. Windows3.11 extended the shell by going into protected mode and added 32-bit support. These were16-bit/32-bit hybrids. Microsoft Windows 95 further reduced DOS to the role of the bootloader. Windows 98 and Windows Me were the last Microsoft OS to run on DOS. The DOSbasedbranch was eventually abandoned in favor of Windows NT, the first true 32-bit systemthat was the foundation for Windows XP and Windows Vista.Windows NT, initially NT OS/2 3.0, was the result of collaboration betweenMicrosoft and IBM to develop a 32-bit operating system that had high hardware and softwareportability. Because of the success of Windows 3.0, Microsoft changed the applicationprogramming interface to the extended Windows API, which caused a split between the twocompanies and a branch in the operating system. IBM would continue to work on OS/2 andOS/2 API, while Microsoft renamed its operating system Windows NT.16.11 Let us sum upIn this lesson we have learnt abouta) the History of MS-DOSb) and the various versions of MS-DOS16.12 Points for DiscussionTry to answer the following questionsa) 5 MS-DOS commandsb) Users view of MS-DOS16.13 Model Answers to Check your Progress134In order to check your progress try to answer the following questionsa) Systems view of MS-DOSb) How to delete a filec) How to delete a directory16.14 Lesson - end activitiesAfter learning this chapter, try to discuss among your friends and answer these questions tocheck your progress.b) Discuss about the future of MS-DOSc) Discuss about the various versions of MS-DOS16.15 References♦ H.M. Deitel, Chapter 19 of “Operating Systems”, Second Edition, PearsonEducation, 2001♦ Andrew S. Tanenbaum, Chapter 8 of “Modern Operating Systems”, PHI, 1996135LESSON – 17: UNIXCONTENTS17.1 Aims and Objectives17.2 Introduction17.3 History of UNIX17.4 Hierarchical File System17.5 The UNIX File System Organization17.5.1 The bin Directory17.5.2 The dev Directory17.5.3 The etc Directory17.5.4 The lib Directory17.5.5 The lost+found Directory17.5.6 The mnt and sys Directories17.5.7 The tmp Directory17.5.8 The usr Directory17.6 Other Miscellaneous Stuff at the Top Level17.7 Let us sum up17.8 Points for Discussion17.9 Model Answers to Check your Progress17.10 Lesson - end activities17.11 References

17.1 Aims and ObjectivesIn this lesson we will learn about the introduction to UNIX Operating system.The objectives of this lesson is to make the candidate aware of the followingi) History of UNIXj) Various versions of UNIXk) UNIX file system organization17.2 IntroductionThe Unix operating system was created more than 30 years ago by a group ofresearchers at AT&T’s Bell Laboratories. During the three decades of constant developmentthat have followed, Unix has found a home in many places, from the ubiquitous mainframe tohome computers to the smallest of embedded devices. This lesson provides a brief overview136of the history of Unix, discusses some of the differences among the many Unix systems inuse today, and covers the fundamental concepts of the basic Unix operating system.UNIX is a computer operating system, a control program that works with users to runprograms, manage resources, and communicate with other computer systems. Several peoplecan use a UNIX computer at the same time; hence UNIX is called a multiuser system. Any ofthese users can also run multiple programs at the same time; hence UNIX is calledmultitasking. Because UNIX is such a pastiche—a patchwork of development—it’s a lotmore than just an operating system. UNIX has more than 250 individual commands. Theserange from simple commands—for copying a file, for example—to the quite complex: thoseused in high-speed networking, file revision management, and software development. Mostnotably, UNIX is a multichoice system. As an example, UNIX has three different primarycommand-line-based user interfaces (in UNIX, the command-line user interface is called ashell ): The three choices are the Bourne shell, C shell, and Korn shell. Often, soon after youlearn to accomplish a task with a particular command, you discover there’s a second or thirdway to do that task. This is simultaneously the greatest strength of UNIX and a source offrustration for both new and current users.Why is having all this choice such a big deal? Think about why Microsoft MS-DOSand the Apple Macintosh interfaces are considered so easy to use. Both are designed to givethe user less power. Both have dramatically fewer commands and precious little overlap incommands: You can’t use copy to list your files in DOS, and you can’t drag a Mac file iconaround to duplicate it in its own directory. The advantage to these interfaces is that, in eithersystem, you can learn the one-and-only way to do a task and be confident that you’re assophisticated in doing that task as is the next person. It’s easy. It’s quick to learn. It’s exactlyhow the experts do it, too.UNIX, by contrast, is much more like a spoken language, with commands acting asverbs, command options (which you learn about later in this lesson) acting as adjectives, andthe more complex commands acting akin to sentences. How you do a specific task can,therefore, be completely different from how your UNIX-expert friend does the same task.Worse, some specific commands in UNIX have many different versions, partly because ofthe variations from different UNIX vendors. (You’ve heard of these variations and vendors,I’ll bet: UNIXWare from Novell, Solaris from Sun, SCO from Santa Cruz, System V Release4 (pronounce that “system five release four” or, to sound like an ace, “ess-vee-are-four”), andBSD UNIX (pronounced “bee-ess-dee”) from University of California at Berkeley are theprimary players. Each is a little different from the other.) Another contributor to the sprawl ofmodern UNIX is the energy of the UNIX programming community; plenty of UNIX usersdecide to write a new version of a command in order to solve slightly different problems, thusspawning many versions of a command.In terms of computers, Unix has a long history. Unix was developed at AT&T’s BellLaboratories after Bell Labs withdrew from a long-term collaboration with General Electric(G.E.) and MIT to create an operating system called MULTICS (Multiplexed Operating andComputing System) for G.E.’s mainframe. In 1969, Bell Labs researchers created the firstversion of Unix (then called UNICS, or Uniplexed Operating and Computing System), whichhas evolved into the common Unix systems of today.137Unix was gradually ported to different machine architectures from the original PDP-7minicomputer and was used by universities. The source code was made available at a smallfee to encourage its further adoption. As Unix gained acceptance by universities, studentswho used it began graduating and moving into positions where they were responsible forpurchasing systems and software. When those people began purchasing systems for theircompanies, they considered Unix because they were familiar with it, spreading adoptionfurther. Since the first days of Unix, the operating system has grown significantly, so that itnow forms the backbone of many major corporations’ computer systems.Unix no longer is an acronym for anything, but it is derived from the UNICS

acronym. Unix developers and users use a lot of acronyms to identify things in the system andfor commands.Unlike DOS, Windows, OS/2, the Macintosh, VMS, MVS, and just about any otheroperating system, UNIX was designed by a couple of programmers as a fun project, and itevolved through the efforts of hundreds of programmers, each of whom was exploring his orher own ideas of particular aspects of OS design and user interaction. In this regard, UNIX isnot like other operating systems, needless to say! It all started back in the late 1960s in a darkand stormy laboratory deep in the recesses of the American Telephone and Telegraph(AT&T) corporate facility in New Jersey. Working with the Massachusetts Institute ofTechnology, AT&T Bell Labs was codeveloping a massive, monolithic operating systemcalled Multics. On the Bell Labs team were Ken Thompson, Dennis Ritchie, BrianKernighan, and other people in the Computer Science Research Group who would prove tobe key contributors to the new UNIX operating system.When 1969 rolled around, Bell Labs was becoming increasingly disillusioned withMultics, an overly slow and expensive system that ran on General Electric mainframecomputers that themselves were expensive to run and rapidly becoming obsolete. Theproblem was that Thompson and the group really liked the capabilities Multics offered,particularly the individual-user environment and multiple-user aspects.17.3 History of UNIXIn the 1960s, the Massachusetts Institute of Technology, AT&T Bell Labs, andGeneral Electric worked on an experimental operating system called Multics (MultiplexedInformation and Computing Service), which was designed to run on the GE-645 mainframecomputer. The aim was the creation of a commercial product, although this was never a greatsuccess. Multics was an interactive operating system with many novel capabilities, includingenhanced security. The project did develop production releases, but initially these releasesperformed poorly.AT&T Bell Labs pulled out and deployed its resources elsewhere. One of thedevelopers on the Bell Labs team, Ken Thompson, continued to develop for the GE-645mainframe, and wrote a game for that computer called Space Travel. However, he found thatthe game was too slow on the GE machine and was expensive, costing $75 per execution inscarce computing time.138Thompson thus re-wrote the game in assembly language for Digital EquipmentCorporation's PDP-7 with help from Dennis Ritchie. This experience, combined with hiswork on the Multics project, led Thompson to start a new operating system for the PDP-7.Thompson and Ritchie led a team of developers, including Rudd Canady, at Bell Labsdeveloping a file system as well as the new multi-tasking operating system itself. Theyincluded a command line interpreter and some small utility programs.Editing a shell script using the ed editor. The dollar-sign at the top of the screen is theprompt printed by the shell. 'ed' is typed to start the editor, which takes over from that pointon the screen downwards.1970sIn 1970 the project was named Unics, and could - eventually - support twosimultaneous users. Brian Kernighan invented this name as a contrast to Multics; the spellingwas later changed to UNIX.Up until this point there had been no financial support from Bell Labs. When theComputer Science Research Group wanted to use UNIX on a much larger machine than thePDP-7, Thompson and Ritchie managed to trade the promise of adding text processingcapabilities to UNIX for a PDP-11/20 machine. This led to some financial support from Bell.For the first time in 1970, the UNIX operating system was officially named and ran on thePDP-11/20. It added a text formatting program called roff and a text editor. All three werewritten in PDP-11/20 assembly language. Bell Labs used this initial "text processing system",made up of UNIX, roff, and the editor, for text processing of patent applications. Roff soonevolved into troff, the first electronic publishing program with a full typesetting capability.1973sIn 1973, the decision was made to re-write UNIX in the C programming language.The change meant that it was easier to modify UNIX to work on other machines (thusbecoming portable), and other developers could create variations. The code was now moreconcise and compact, leading to accelerated development of UNIX. AT&T made UNIXavailable to universities and commercial firms, as well as the United States government underlicenses. The licenses included all source code including the machine-dependent parts of thekernel, which were written in PDP-11 assembly code.1975sVersions of the UNIX system were determined by editions of its user manuals, so that(for example) "Fifth Edition UNIX" and "UNIX Version 5" have both been used to designate

the same thing. Development expanded, with Versions 4, 5, and 6 being released by 1975.These versions added the concept of pipes, leading to the development of a more modularcode-base, increasing development speed still further. Version 5 and especially Version 6 ledto a plethora of different Unix versions both inside and outside Bell Labs, includingPWB/UNIX, IS/1 (the first commercial Unix), and the University of Wollongong's port to theInterdata 7/32 (the first non-PDP Unix).1391978sIn 1978, UNIX/32V, for the VAX system, was released. By this time, over 600machines were running UNIX in some form.1979sVersion 7 UNIX, the last version of Research Unix to be released widely, wasreleased in 1979. Versions 8, 9 and 10 were developed through the 1980s but were onlyreleased to a few universities, though they did generate papers describing the new work. Thisresearch led to the development of Plan 9 from Bell Labs, a new portable distributed system.1980sA late-80s style UNIX desktop running the X Window System graphical user interface.Shown are a number of client applications common to the MIT X Consortium's distribution,including Tom's Window Manager, an X Terminal, Xbiff, xload, and a graphical manualpage browser.AT&T now licensed UNIX System III, based largely on Version 7, for commercialuse, the first version launching in 1982. This also included support for the VAX. AT&Tcontinued to issue licenses for older UNIX versions. To end the confusion between all itsdiffering internal versions, AT&T combined them into UNIX System V Release 1. Thisintroduced a few features such as the vi editor and curses from the Berkeley SoftwareDistribution of Unix developed at the University of California, Berkeley. This also includedsupport for the Western Electric 3B series of machines.Since the newer commercial UNIX licensing terms were not as favorable foracademic use as the older versions of Unix, the Berkeley researchers continued to developBSD Unix as an alternative to UNIX System III and V, originally on the PDP-11 architecture(the 2.xBSD releases, ending with 2.11BSD) and later for the VAX-11 (the 4.x BSDreleases). Many contributions to UNIX first appeared on BSD systems, notably the C shellwith job control (modeled on ITS). Perhaps the most important aspect of the BSDdevelopment effort was the addition of TCP/IP network code to the mainstream UNIX kernel.The BSD effort produced several significant releases that contained network code: 4.1cBSD,4.2BSD, 4.3BSD, 4.3BSD-Tahoe ("Tahoe" being the nickname of the CCI Power 6/32architecture that was the first non-DEC release of the BSD kernel), Net/1, 4.3BSD-Reno (tomatch the "Tahoe" naming, and that the release was something of a gamble), Net/2, 4.4BSD,and 4.4BSD-lite. The network code found in these releases is the ancestor of much TCP/IPnetwork code in use today, including code that was later released in AT&T System V UNIXand early versions of Microsoft Windows. The accompanying Berkeley Sockets API is a defacto standard for networking APIs and has been copied on many platforms.1982sOther companies began to offer commercial versions of the UNIX System for theirown mini-computers and workstations. Most of these new UNIX flavors were developedfrom the System V base under a license from AT&T; however, others were based on BSDinstead. One of the leading developers of BSD, Bill Joy, went on to co-found Sun140Microsystems in 1982 and create SunOS (now Solaris) for their workstation computers. In1980, Microsoft announced its first Unix for 16-bit microcomputers called Xenix, which theSanta Cruz Operation (SCO) ported to the Intel 8086 processor in 1983, and eventuallybranched Xenix into SCO UNIX in 1989.For a few years during this period (before PC compatible computers with MS-DOSbecame dominant), industry observers expected that UNIX, with its portability and richcapabilities, was likely to become the industry standard operating system formicrocomputers.1984sIn 1984 several companies established the X/Open consortium with the goal ofcreating an open system specification based on UNIX. Despite early progress, thestandardization effort collapsed into the "Unix wars," with various companies forming rivalstandardization groups. The most successful Unix-related standard turned out to be theIEEE's POSIX specification, designed as a compromise API readily implemented on bothBSD and System V platforms, published in 1988 and soon mandated by the United Statesgovernment for many of its own systems.Between 1987 - 1989

AT&T added various features into UNIX System V, such as file locking, systemadministration, streams, new forms of IPC, the Remote File System and TLI. AT&Tcooperated with Sun Microsystems and between 1987 and 1989 merged features from Xenix,BSD, SunOS, and System V into System V Release 4 (SVR4), independently of X/Open.This new release consolidated all the previous features into one package, and heralded theend of competing versions. It also increased licensing fees.The Common Desktop Environment or CDE, a graphical desktop for UNIX codevelopedin the 1990s by HP, IBM, and Sun as part of the COSE initiative.1990sIn 1990, the Open Software Foundation released OSF/1, their standard UNIXimplementation, based on Mach and BSD. The Foundation was started in 1988 and wasfunded by several Unix-related companies that wished to counteract the collaboration ofAT&T and Sun on SVR4. Subsequently, AT&T and another group of licensees formed thegroup "UNIX International" in order to counteract OSF. This escalation of conflict betweencompeting vendors gave rise again to the phrase "Unix wars".1991sIn 1991, a group of BSD developers (Donn Seeley, Mike Karels, Bill Jolitz, and TrentHein) left the University of California to found Berkeley Software Design, Inc (BSDI). BSDIproduced a fully functional commercial version of BSD Unix for the inexpensive andubiquitous Intel platform, which started a wave of interest in the use of inexpensive hardwarefor production computing. Shortly after it was founded, Bill Jolitz left BSDI to pursuedistribution of 386BSD, the free software ancestor of FreeBSD, OpenBSD, and NetBSD.1411993sIn 1993 most commercial vendors had changed their variants of UNIX to be based onSystem V with many BSD features added on top. The creation of the COSE initiative thatyear by the major players in UNIX marked the end of the most notorious phase of the UNIXwars, and was followed by the merger of UI and OSF in 1994. The new combined entity,which retained the OSF name, stopped work on OSF/1 that year. By that time the only vendorusing it was Digital, which continued its own development, rebranding their product DigitalUNIX in early 1995.Shortly after UNIX System V Release 4 was produced, AT&T sold all its rights toUNIX® to Novell. (Dennis Ritchie likened this to the Biblical story of Esau selling hisbirthright for the proverbial "mess of pottage".) Novell developed its own version, UnixWare,merging its NetWare with UNIX System V Release 4. Novell tried to use this to battleagainst Windows NT, but their core markets suffered considerably.In 1993, Novell decided to transfer the UNIX® trademark and certification rights tothe X/Open Consortium. In 1996, X/Open merged with OSF, creating the Open Group.Various standards by the Open Group now define what is and what is not a "UNIX" operatingsystem, notably the post-1998 Single UNIX Specification.1995sIn 1995, the business of administering and supporting the existing UNIX licenses,plus rights to further develop the System V code base, were sold by Novell to the Santa CruzOperation. Whether Novell also sold the copyrights is currently the subject of litigation (seebelow).1997sIn 1997, Apple Computer sought out a new foundation for its Macintosh operatingsystem and chose NEXTSTEP, an operating system developed by NeXT. The core operatingsystem was renamed Darwin after Apple acquired it. It was based on the BSD family and theMach kernel. The deployment of Darwin BSD Unix in Mac OS X makes it, according to astatement made by an Apple employee at a USENIX conference; the most widely used Unixbasedsystem in the desktop computer market.2000 to presentIn 2000, SCO sold its entire UNIX business and assets to Caldera Systems, whichlater on changed its name to The SCO Group. This new player then started legal actionagainst various users and vendors of Linux. SCO have alleged that Linux containscopyrighted UNIX code now owned by The SCO Group. Other allegations include tradesecretviolations by IBM, or contract violations by former Santa Cruz customers who havesince converted to Linux. However, Novell disputed the SCO group's claim to hold copyrighton the UNIX source base. According to Novell, SCO (and hence the SCO Group) areeffectively franchise operators for Novell, which also retained the core copyrights, veto rightsover future licensing activities of SCO, and 95% of the licensing revenue. The SCO Groupdisagreed with this, and the dispute had resulted in the SCO v. Novell lawsuit.142In 2005, Sun Microsystems released the bulk of its Solaris system code (based on

UNIX System V Release 4) into an open source project called OpenSolaris. New Sun OStechnologies such as the ZFS file system are now first released as open source code via theOpenSolaris project; as of 2006 it has spawned several non-Sun distributions such asSchilliX,Belenix, Nexenta and MarTux.The Dot-com crash has led to significant consolidation of UNIX users as well. Of themany commercial flavors of UNIX that were born in the 1980s, only Solaris, HP-UX, andAIX are still doing relatively well in the market, though SGI's IRIX persisted for quite sometime. Of these, Solaris has the most market share, and may be gaining popularity due to itsfeature set and also since it now has an Open Source version.17.4 Hierarchical File SystemIn a nutshell, a hierarchy is a system organized by graded categorization. A familiarexample is the organizational structure of a company, where workers report to supervisorsand supervisors report to middle managers. Middle managers, in turn, report to seniormanagers, and senior managers report to vice-presidents, who report to the president of thecompany. Graphically, this hierarchy looks like Figure 1.1.Figure 1.1. A typical organizational hierarchyYou’ve doubtless seen this type of illustration before, and you know that a higherposition indicates more control. Each position is controlled by the next highest position orrow. The president is top dog of the organization, but each subsequent manager is also incontrol of his or her own small fiefdom.To understand how a file system can have a similar organization, simply imagine eachof the managers in the illustration as a “file folder” and each of the employees as a piece ofpaper, filed in a particular folder. Open any file cabinet, and you probably see thingsorganized this143Moving About in the File System way: filed papers are placed in labeled folders, andoften these folders are filed in groups under specific topics. The drawer might then have aspecific label to distinguish it from other drawers in the cabinet, and so on.That’s exactly what a hierarchical file system is all about. You want to have your fileslocated in the most appropriate place in the file system, whether at the very top, in a folder, orin a nested series of folders. With careful usage, a hierarchical file system can containhundreds or thousands of files and still allow users to find any individual file quickly. On mycomputer, the chapters of this book are organized in a hierarchical fashion, as shown inFigure 1.2.Figure 1.2. File organization for the chapters of Teach17.5 The UNIX File System OrganizationA key concept enabling the UNIX hierarchical file system to be so effective is thatany thing that is not a folder is a file. Programs are files in UNIX, device drivers are files,documents and spreadsheets are files, your keyboard is represented as a file, your display is afile, and even your try line and mouse are files.What this means is that as UNIX has developed, it has avoided becoming an ungainlymess. UNIX does not have hundreds of cryptic files stuck at the top (this is still a problem inDOS)or tucked away in confusing folders within the System Folder (as with theMacintosh).The top level of the UNIX file structure (/) is known as the root directory or slashdirectory, and it always has a certain set of subdirectories, including bin, dev, etc, lib, mnt,tmp, and usr. There can be a lot more, however. Listing 1.1 shows files found at the top levelof the mentor file system (the system I work on). Typical UNIX directories are shownfollowed by a slash in the listing.AA boot flags/ rf/ userb/ var/OLD/ core gendynix stand/ userc/archive/ dev/ lib/ sys/ users/ats/ diag/ lost+found/ tftpboot/ usere/backup/ dynix mnt/ tmp/ users/bin/ etc/ net/ usera/ usr/144You can obtain a listing of the files and directories in your own top-level directory byusing the ls –C -F / command. (You’ll learn all about the ls command in the next hour. Fornow, just be sure that you enter exactly what’s shown in the example.) On a differentcomputer system, here’s what I see when I enter that command:% ls –C -F /Mail/ export/ public/News/ home/ reviews/add_swap/ kadb* sbin/apps/ layout sys@archives/ lib@ tftpboot/

bin@ lost+found/ tmp/boot mnt/ usr/cdrom/ net/ utilities/chess/ news/ var/dev/ nntpserver vmunix*etc/ pcfs/In this example, any filename that ends with a slash (/) is a folder (UNIX calls thesedirectories). Any filename that ends with an asterisk (*) is a program. Anything ending withan at sign (@) is a symbolic link, and everything else is a normal, plain file. As you can seefrom these two examples, and as you’ll immediately find when you try the commandyourself, there is much variation in how different UNIX systems organize the top leveldirectory. There are some directories and files in common, and once you start examining thecontents of specific directories, you’ll find that hundreds of programs and files always showup in the same place from UNIX to UNIX. It’s as if you were working as a file clerk at a newlaw firm. Although this firm might have a specific approach to filing information, theapproach may be similar to the filing system of other firms where you have worked in thepast. If you know the underlying organization, you can quickly pick up the specifics of aparticular organization. Try the command ls –C -F / on your computer system, and identify,as previously explained, each of the directories in your resultant listing. The output of the lscommand shows the files and directories in the top level of your system. Next, you learn whatthey are.17.5.1 The bin DirectoryIn UNIX parlance, programs are considered executables because users can executethem. (In this case, execute is a synonym for run, not an indication that you get to wanderabout murdering innocent applications!) When the program has been compiled (usually froma C listing), it is translated into what’s called a binary format. Add the two together, and youhave a common UNIX description for an application—an executable binary.3It’s no surprise that the original UNIX developers decided to have a directory labeled“binaries” to store all the executable programs on the system. Remember the primitiveteletypewriter discussed in the last hour? Having a slow system to talk with the computer hadmany ramifications that you might not expect. The single most obvious one was thateverything became quite concise. There were no lengthy words like binaries or listfiles, butrather succinct abbreviations: bin and ls are, respectively, the UNIX equivalents. The bindirectory is where all the executable binaries were kept in early UNIX. Over time, as more145and more executables were added to UNIX, having all the executables in one place provedunmanageable, and the bin directory split into multiple parts (/bin, /sbin, /usr/bin).17.5.2 The dev DirectoryAmong the most important portions of any computer are its device drivers. Withoutthem, you wouldn’t have any information on your screen (the information arrives courtesy ofthe display device driver). You wouldn’t be able to enter information (the information is readand given to the system by the keyboard device driver), and you wouldn’t be able to use yourfloppy disk drive (managed by the floppy device driver).Earlier, you learned how almost anything in UNIX is considered a file in the filesystem, and the dev directory is an example. All device drivers—often numbering into thehundreds— are stored as separate files in the standard UNIX dev (devices) directory.Pronounce this directory name “dev,” not “dee-ee-vee.”17.5.3 The etc DirectoryUNIX administration can be quite complex, involving management of user accounts,the file system, security, device drivers, hardware configurations, and more. To help, UNIXdesignates the etc directory as the storage place for all administrative files and information.Pronounce the directory name either “ee-tea-sea”, “et-sea,” or “etcetera.” All threepronunciations are common.17.5.4 The lib DirectoryLike your neighborhood community, UNIX has a central storage place for functionand procedural libraries. These specific executables are included with specific programs,allowing programs to offer features and capabilities otherwise unavailable. The idea is that ifprograms want to include certain features, they can reference just the shared copy of thatutility in the UNIX library rather than having a new, unique copy.Many of the more recent UNIX systems also support what’s called dynamic linking,where the library of functions is included on-the-fly as you start up the program. The wrinkleis that instead of the library reference being resolved when the program is created, it’sresolved only when you actually run the program itself. Pronounce the directory name “libe”or “lib” (to rhyme with the word bib).

17.5.5 The lost+found DirectoryWith multiple users running many different programs simultaneously, it’s been achallenge over the years to develop a file system that can remain synchronized with theactivity of the computer. Various parts of the UNIX kernel—the brains of the system—helpwith this problem. When files are recovered after any sort of problem or failure, they areplaced here, in the lost+found directory, if the kernel cannot ascertain the proper location inthe filesystem. This directory should be empty almost all the time. This directory iscommonly pronounced “lost and found” rather than “lost plus found.”17.5.6 The mnt and sys Directories146The mnt (pronounced “em-en-tea”) and sys (pronounced “sis”) directories also aresafely ignored by UNIX users. The mnt directory is intended to be a common place to mountexternal media—hard disks, removable cartridge drives, and so on—in UNIX. On manysystems, though not all, sys contains files indicating the system configuration.17.5.7 The tmp DirectoryA directory that you can’t ignore, the tmp directory—say “temp”—is used by many ofthe programs in UNIX as a temporary file-storage space. If you’re editing a file, for example,the program makes a copy of the file and saves it in tmp, and you work directly with that,saving the new file back to your original file only when you’ve completed your work. Onmost systems, tmp ends up littered with various files and executables left by programs thatdon’t remove their own temporary files. On one system I use, it’s not uncommon to find 10–30 megabytes of files wasting space here. Even so, if you’re manipulating files or workingwith copies of files, tmp is the best place to keep the temporary copies of files. Indeed, onsome UNIX workstations, tmp actually can be the fastest device on the computer, allowingfor dramatic performance improvements over working with files directly in your homedirectory.17.5.8 The usr DirectoryFinally, the last of the standard directories at the top level of the UNIX file systemhierarchy is the usr—pronounced “user”—directory. Originally, this directory was intendedto be the central storage place for all user-related commands. Today, however, manycompanies have their own interpretation, and there’s no telling what you’ll find in thisdirectory.317.6 Other Miscellaneous Stuff at the Top LevelBesides all the directories previously listed, a number of other directories and filescommonly occur in UNIX systems. Some files might have slight variations in name on yourcomputer, so when you compare your listing to the following files and directories, be alert forpossible alternative spellings.A file you must have to bring up UNIX at all is one usually called unix or vmunix, ornamed after the specific version of UNIX on the computer. The file contains the actual UNIXoperating system. The file must have a specific name and must be found at the top level ofthe file system. Hand-in-hand with the operating system is another file called boot, whichhelps during initial startup of the hardware.Notice on one of the previous listings that the files boot and dynix appear. (DYNIX isthe name of the particular variant of UNIX used on Sequent computers.) By comparison, thelisting from the Sun Microsystems workstation shows boot and vmunix as the two files.Another directory that you might find in your own top-level listing is diag—pronounced“dye-ag”—which acts as a storehouse for diagnostic and maintenance programs. If you haveany programs within this directory, it’s best not to try them out without proper training! Thehome directory, also sometimes called users, is a central place for organizing all files uniqueto a specific user. Listing this directory is usually an easy way to find out what accounts areon the system, too, because by convention each individual account directory is named after147the user’s account name. On one system I use, my account is taylor, and my individualaccount directory is also called taylor. Home directories are always created by the systemadministrator. The net directory, if set up correctly, is a handy shortcut for accessing othercomputers on your network. The tftpboot directory is a relatively new feature of UNIX. Theletters stand for “trivial file Transfer protocol boot.” Don’t let the name confuse you, though;this directory contains Versions of the kernel suitable for X Window System-based terminalsand diskless workstations to run UNIX. Some UNIX systems have directories named forspecific types of peripherals that can be attached. On the Sun workstation, you can seeexamples with the directories cd rom and pcfs.The former is for a CD-ROM drive and thelatter for DOS-format floppy disks. There are many more directories in UNIX, but this willgive you an idea of how things are organized.17.7 Let us sum up

In this lesson we have learnt abouta) The History of UNIXb) And UNIX file system organization17.8 Points for DiscussionTry to discuss about the followinga) Evolution of UNIXb) Future of UNIX17.9 Model Answers to Check your ProgressIn order to check your progress, try to answer the uses of the following directoriesa) bin directoryb) dev directoryc) lib directory17.10 Lesson - end activitiesAfter learning this chapter, try to discuss among your friends and answer these questionsto check your progress.b) Discuss about UNIX file system organizationc) Discuss about various version of UNIX17.11 References♦ H.M. Deitel, Chapter 18 of “Operating Systems”, Second Edition, PearsonEducation, 2001♦ Andrew S. Tanenbaum, Chapter 7 of “Modern Operating Systems”, PHI, 1996148LESSON – 18: KERNEL AND SHELLCONTENTS18.1 Aims and Objectives18.2 Introduction18.2.1 The kernel18.2.2 The shell18.3 Simple commands18.4 Background commands18.5 Input output redirection18.6 Pipelines and filters18.7 File name generation18.8 Quoting18.9 Prompting18.10 Shell procedures18.11 Control Flow18.12 Shell variables18.13 The test command18.14 Other Control flows18.15 Command grouping18.16 Debugging shell procedures18.17 Other important commands18.18 Let us sum up18.19 Points for Discussion18.20 Model Answers to Check your Progress18.21 Lesson - end activities18.22 References18.1 Aims and ObjectivesIn this lesson we will learn about the introduction to Kernel and Shell.The objectives of this lesson is to make the candidate aware of the followingl) Kernelm) Shelln) And some important shell commands14918.2 IntroductionIn this section we discuss about kernel and shell.18.2.1 The kernelThe kernel of UNIX is the hub of the operating system: it allocates time and memoryto programs and handles the filestore and communications in response to system calls. Thekernel is responsible for carrying out all fundamental low-level system operations, likescheduling processes, opening and closing files, and sending instructions to the actualhardware CPU chips that process your data. There is only one kernel and it is the heart of themachine.As an illustration of the way that the shell and the kernel work together, suppose a

user types rm myfile (which has the effect of removing the file myfile). The shell searchesthe filestore for the file containing the program rm, and then requests the kernel, throughsystem calls, to execute the program rm on myfile. When the process rm myfile has finishedrunning, the shell then returns the UNIX prompt % to the user, indicating that it is waiting forfurther commands.18.2.2 The shellThe shell acts as an interface between the user and the kernel. When a user logs in, thelogin program checks the username and password, and then starts another program called theshell. The shell is a command line interpreter (CLI). It interprets the commands the user typesin and arranges for them to be carried out. The commands are themselves programs: whenthey terminate, the shell gives the user another prompt (% on our systems).The adept user can customise his/her own shell, and users can use different shells onthe same machine. Staff and students in the school have the tcsh shell by default.The tcsh shell has certain features to help the user inputting commands.Filename Completion - By typing part of the name of a command, filename or directory andpressing the [Tab] key, the tcsh shell will complete the rest of the name automatically. If theshell finds more than one name beginning with those letters you have typed, it will beep,prompting you to type a few more letters before pressing the tab key again.History - The shell keeps a list of the commands you have typed in. If you need to repeat acommand, use the cursor keys to scroll up and down the list or type history for a list ofprevious commands.The shell is both a command language and a programming language that provides aninterface to the UNIX operating system.18.3 Simple commands150Simple commands consist of one or more words separated by blanks. The first word isthe name of the command to be executed; any remaining words are passed as arguments tothe command. For example,whois a command that prints the names of users logged in. The commandls –lprints a list of files in the current directory. The argument -l tells ls to print statusinformation, size and the creation date for each file.18.4 Background commandsTo execute a command the shell normally creates a new process and waits for it tofinish. A command may be run without waiting for it to finish. For example,cc pgm.c &calls the C compiler to compile the file pgm.c. The trailing & is an operator that instructs theshell not to wait for the command to finish. To help keep track of such a process the shellreports its process number following its creation. A list of currently active processes may beobtained using the ps command.18.5 Input output redirectionMost commands produce output on the standard output that is initially connected tothe terminal. This output may be sent to a file by writing, for example,ls -l >fileThe notation >file is interpreted by the shell and is not passed as an argument to ls. Iffile does not exist then the shell creates it; otherwise the original contents of file are replacedwith the output from ls. Output may be appended to a file using the notationls -l >>fileIn this case file is also created if it does not already exist.The standard input of a command may be taken from a file instead of the terminal bywriting, for example,wc <fileThe command wc reads its standard input (in this case redirected from file) and printsthe number of characters, words and lines found. If only the number of lines is required thenwc -l <filecould be used.15118.6 Pipelines and filtersThe standard output of one command may be connected to the standard input ofanother by writing the `pipe' operator, indicated by |, as in,ls -l | wcTwo commands connected in this way constitute a pipeline and the overall effect isthe same asls -l >file; wc <file

except that no file is used. Instead the two processes are connected by a pipe and are run inparallel.Pipes are unidirectional and synchronization is achieved by halting wc when there isnothing to read and halting ls when the pipe is full.A filter is a command that reads its standard input, transforms it in some way, andprints the result as output. One such filter, grep, selects from its input those lines that containsome specified string. For example,ls | grep oldprints those lines, if any, of the output from ls that contain the string old. Another useful filteris sort. For example,who | sortwill print an alphabetically sorted list of logged in users.A pipeline may consist of more than two commands, for example,ls | grep old | wc -lprints the number of file names in the current directory containing the string old.18.7 File name generationMany commands accept arguments which are file names. For example,ls -l main.cprints information relating to the file main.c.The shell provides a mechanism for generating a list of file names that match apattern. For example,ls -l *.c152generates, as arguments to ls, all file names in the current directory that end in .c. Thecharacter * is a pattern that will match any string including the null string. In general patternsare specified as follows.*Matches any string of characters including the null string.?Matches any single character.[...]Matches any one of the characters enclosed. A pair of characters separated by a minuswill match any character lexically between the pair.For example,[a-z]*matches all names in the current directory beginning with one of the letters a through z./usr/fred/test/?matches all names in the directory /usr/fred/test that consist of a single character. If no filename is found that matches the pattern then the pattern is passed, unchanged, as an argument.This mechanism is useful both to save typing and to select names according to somepattern. It may also be used to find files. For example,echo /usr/fred/*/corefinds and prints the names of all core files in sub-directories of /usr/fred. (echo is a standardUNIX command that prints its arguments, separated by blanks.) This last feature can beexpensive, requiring a scan of all sub-directories of /usr/fred.There is one exception to the general rules given for patterns. The character `.' at thestart of a file name must be explicitly matched.echo *will therefore echo all file names in the current directory not beginning with `.'.echo .*will echo all those file names that begin with `.'. This avoids inadvertent matching of thenames `.' and `..' which mean `the current directory' and `the parent directory' respectively.(Notice that ls suppresses information for the files `.' and `..'.)18.8 QuotingCharacters that have a special meaning to the shell, such as < > * ? | &, are calledmetacharacters. Any character preceded by a \ is quoted and loses its special meaning, if any.The \ is elided so that153echo \?will echo a single ?, andecho \\will echo a single \. To allow long strings to be continued over more than one line thesequence \newline is ignored.\ is convenient for quoting single characters. When more than one character needs quoting theabove mechanism is clumsy and error prone. A string of characters may be quoted by

enclosing the string between single quotes. For example,echo xx'****'xxwill echoxx****xxThe quoted string may not contain a single quote but may contain newlines, which arepreserved. This quoting mechanism is the most simple and is recommended for casual use. Athird quoting mechanism using double quotes is also available that prevents interpretation ofsome but not all metacharacters.15418.9 PromptingWhen the shell is used from a terminal it will issue a prompt before reading acommand. By default this prompt is `$ '. It may be changed by saying, for example,PS1=yesdearthat sets the prompt to be the string yesdear. If a newline is typed and further input is neededthen the shell will issue the prompt `> '. Sometimes this can be caused by mistyping a quotemark. If it is unexpected then an interrupt (DEL) will return the shell to read anothercommand. This prompt may be changed by saying, for example,PS2=moreThe shell and loginFollowing login (1) the shell is called to read and execute commands typed at theterminal. If the user's login directory contains the file .profile then it is assumed to containcommands and is read by the shell before reading any commands from the terminal.18.10 Shell proceduresThe shell may be used to read and execute commands contained in a file. Forexample,sh file [ args ... ]calls the shell to read commands from file. Such a file is called a command procedure or shellprocedure. Arguments may be supplied with the call and are referred to in file using thepositional parameters $1, $2, .... For example, if the file wg containswho | grep $1thensh wg fredis equivalent towho | grep fredUNIX files have three independent attributes, read, write and execute. The UNIXcommand chmod (1) may be used to make a file executable. For example,chmod +x wgwill ensure that the file wg has execute status. Following this, the commandwg fredis equivalent tosh wg fred155This allows shell procedures and programs to be used interchangeably. In either case anew process is created to run the command.As well as providing names for the positional parameters, the number of positionalparameters in the call is available as $#. The name of the file being executed is available as$0.A special shell parameter $* is used to substitute for all positional parameters except $0. Atypical use of this is to provide some default arguments, as in,nroff -T450 -ms $*which simply prepends some arguments to those already given.18.11 Control FlowControl flow - forA frequent use of shell procedures is to loop through the arguments ($1, $2, ...)executing commands once for each argument. An example of such a procedure is tel thatsearches the file /usr/lib/telnos that contains lines of the form...fred mh0123bert mh0789...The text of tel isfor ido grep $i /usr/lib/telnos; doneThe commandtel fred

prints those lines in /usr/lib/telnos that contain the string fred.tel fred bertprints those lines containing fred followed by those for bert.The for loop notation is recognized by the shell and has the general formfor name in w1 w2 ...do command-listdoneA command-list is a sequence of one or more simple commands separated orterminated by a newline or semicolon. Furthermore, reserved words like do and done areonly recognized following a newline or semicolon. name is a shell variable that is set to thewords w1 w2 ... in turn each time the command-list following do is executed. If in w1 w2 ... is156omitted then the loop is executed once for each positional parameter; that is, in $* isassumed.Another example of the use of the for loop is the create command whose text isfor i do >$i; doneThe commandcreate alpha betaensures that two empty files alpha and beta exist and are empty. The notation >file may beused on its own to create or clear the contents of a file. Notice also that a semicolon (ornewline) is required before done.Control flow - caseA multiple way branch is provided for by the case notation. For example,case $# in1) cat >>$1 ;;2) cat >>$2 <$1 ;;*) echo \'usage: append [ from ] to\' ;;esacis an append command. When called with one argument asappend file$# is the string 1 and the standard input is copied onto the end of file using the cat command.append file1 file2appends the contents of file1 onto file2. If the number of arguments supplied to append isother than 1 or 2 then a message is printed indicating proper usage.The general form of the case command iscase word inpattern) command-list;;...esacThe shell attempts to match word with each pattern, in the order in which the patternsappear. If a match is found the associated command-list is executed and execution of the caseis complete. Since * is the pattern that matches any string it can be used for the default case.A word of caution: no check is made to ensure that only one pattern matches the caseargument. The first match found defines the set of commands to be executed. In the examplebelow the commands following the second * will never be executed.157case $# in*) ... ;;*) ... ;;esacAnother example of the use of the case construction is to distinguish betweendifferent forms of an argument. The following example is a fragment of a cc command.for ido case $i in-[ocs]) ... ;;-*) echo \'unknown flag $i\' ;;*.c) /lib/c0 $i ... ;;*) echo \'unexpected argument $i\' ;;esacdoneTo allow the same commands to be associated with more than one pattern the casecommand provides for alternative patterns separated by a |. For example,case $i in-x|-y) ...esac

is equivalent tocase $i in-[xy]) ...EsacThe usual quoting conventions apply so thatcase $i in\?) ...will match the character ?.18.12 Shell variablesThe shell provides string-valued variables. Variable names begin with a letter andconsist of letters, digits and underscores. Variables may be given values by writing, forexample,user=fred box=m000 acct=mh0000which assigns values to the variables user, box and acct. A variable may be set to the nullstring by saying, for example,null=The value of a variable is substituted by preceding its name with $; for example,echo $userwill echo fred.Variables may be used interactively to provide abbreviations for frequently usedstrings. For example,158b=/usr/fred/binmv pgm $bwill move the file pgm from the current directory to the directory /usr/fred/bin. A moregeneral notation is available for parameter (or variable) substitution, as in,echo ${user}which is equivalent toecho $userand is used when the parameter name is followed by a letter or digit. For example,tmp=/tmp/psps a >${tmp}awill direct the output of ps to the file /tmp/psa, whereas,ps a >$tmpawould cause the value of the variable tmpa to be substituted.Except for $? the following are set initially by the shell. $? is set after executing eachcommand.$?The exit status (return code) of the last command executed as a decimal string. Mostcommands return a zero exit status if they complete successfully, otherwise a nonzeroexit status is returned. Testing the value of return codes is dealt with later underif and while commands.$#The number of positional parameters (in decimal). Used, for example, in the appendcommand to check the number of parameters.$$The process number of this shell (in decimal). Since process numbers are uniqueamong all existing processes, this string is frequently used to generate uniquetemporary file names. For example,ps a >/tmp/ps$$...rm /tmp/ps$$$!The process number of the last process run in the background (in decimal).$-The current shell flags, such as -x and -v.Some variables have a special meaning to the shell and should be avoided for general use.$MAILWhen used interactively the shell looks at the file specified by this variable before itissues a prompt. If the specified file has been modified since it was last looked at theshell prints the message you have mail before prompting for the next command. Thisvariable is typically set in the file .profile, in the user's login directory. For example,MAIL=/usr/mail/fred$HOMEThe default argument for the cd command. The current directory is used to resolve

file name references that do not begin with a /, and is changed using the cd command.For example,cd /usr/fred/binmakes the current directory /usr/fred/bin.cat wn159will print on the terminal the file wn in this directory. The command cd with noargument is equivalent tocd $HOMEThis variable is also typically set in the the user's login profile.$PATHA list of directories that contain commands (the search path). Each time a commandis executed by the shell a list of directories is searched for an executable file. If$PATH is not set then the current directory, /bin, and /usr/bin are searched bydefault. Otherwise $PATH consists of directory names separated by :. For example,PATH=:/usr/fred/bin:/bin:/usr/binspecifies that the current directory (the null string before the first :), /usr/fred/bin,/bin and /usr/bin are to be searched in that order. In this way individual users canhave their own `private' commands that are accessible independently of the currentdirectory. If the command name contains a / then this directory search is not used; asingle attempt is made to execute the command.$PS1The primary shell prompt string, by default, `$ '.$PS2The shell prompt when further input is needed, by default, `> '.$IFSThe set of characters used by blank interpretation (see section 3.4).18.13 The test commandThe test command, although not part of the shell, is intended for use by shellprograms. For example,test -f filereturns zero exit status if file exists and non-zero exit status otherwise. In general testevaluates a predicate and returns the result as its exit status. Some of the more frequentlyused test arguments are given here, see test (1) for a complete specification.test strue if the argument s is not the null stringtest -f filetrue if file existstest -r filetrue if file is readabletest -w filetrue if file is writabletest -d filetrue if file is a directory18.14 Other Control flowsControl flow - whileThe actions of the for loop and the case branch are determined by data available tothe shell. A while or until loop and an if then else branch are also provided whose actionsare determined by the exit status returned by commands. A while loop has the general formwhile command-list1do command-list2160doneThe value tested by the while command is the exit status of the last simple commandfollowing while. Each time round the loop command-list1 is executed; if a zero exit status isreturned then command-list2 is executed; otherwise, the loop terminates. For example,while test $1do ...shiftdoneis equivalent tofor ido ...doneshift is a shell command that renames the positional parameters $2, $3, ... as $1, $2, ... and

loses $1.Another kind of use for the while/until loop is to wait until some external eventoccurs and then run some commands. In an until loop the termination condition is reversed.For example,until test -f filedo sleep 300; donecommandswill loop until file exists. Each time round the loop it waits for 5 minutes before trying again.(Presumably another process will eventually create the file.)Control flow - ifAlso available is a general conditional branch of the form,if command-listthen command-listelse command-listfithat tests the value returned by the last simple command following if.The if command may be used in conjunction with the test command to test for theexistence of a file as inif test -f filethen process fileelse do something elsefiAn example of the use of if, case and for constructions is given in section 2.10.A multiple test if command of the formif ...then ...else if ...then ...161else if ......fififimay be written using an extension of the if notation as,if ...then ...elif ...then ...elif ......fiThe following example is the touch command which changes the `last modified' timefor a list of files. The command may be used in conjunction with make (1) to forcerecompilation of a list of files.flag=for ido case $i in-c) flag=N ;;*) if test -f $ithen ln $i junk$$; rm junk$$elif test $flagthen echo file \'$i\' does not existelse >$ifiesacdoneThe -c flag is used in this command to force subsequent files to be created if they donot already exist. Otherwise, if the file does not exist, an error message is printed. The shellvariable flag is set to some non-null string if the -c argument is encountered. The commandsln ...; rm ...make a link to the file and then remove it thus causing the last modified date to be updated.The sequenceif command1then command2

fimay be writtencommand1 && command2Conversely,command1 || command2executes command2 only if command1 fails. In each case the value returned is that of the lastsimple command executed.18.15 Command groupingCommands may be grouped in two ways,162{ command-list ; }and( command-list )In the first command-list is simply executed. The second form executes command-listas a separate process. For example,(cd x; rm junk )executes rm junk in the directory x without changing the current directory of the invokingshell.The commandscd x; rm junkhave the same effect but leave the invoking shell in the directory x.18.16 Debugging shell proceduresThe shell provides two tracing mechanisms to help when debugging shell procedures.The first is invoked within the procedure asset -v(v for verbose) and causes lines of the procedure to be printed as they are read. It is useful tohelp isolate syntax errors. It may be invoked without modifying the procedure by sayingsh -v proc ...where proc is the name of the shell procedure. This flag may be used in conjunction with the-n flag which prevents execution of subsequent commands. (Note that saying set -n at aterminal ill render the terminal useless until an end- of-file is typed.)The commandset -xwill produce an execution trace. Following parameter substitution each command is printedas it is executed. (Try these at the terminal to see what effect they have.) Both flags may beturned off by sayingset -and the current setting of the shell flags is available as $-.18.17 Other important commands18.17.1 The man commandThe following is the man command which is used to print sections of the UNIXmanual. It is called, for example, as$ man sh$ man -t ed$ man 2 forkIn the first the manual section for sh is printed. Since no section is specified, section 1 isused. The second example will typeset (-t option) the manual section for ed.cd /usr/man: 'colon is the comment command'163: 'default is nroff ($N), section 1 ($s)'N=n s=1for ido case $i in[1-9]*) s=$i ;;-t) N=t ;;-n) N=n ;;-*) echo unknown flag \'$i\' ;;*) if test -f man$s/$i.$sthen ${N}roff man0/${N}aa man$s/$i.$selse : 'look through all manual sections'found=nofor j in 1 2 3 4 5 6 7 8 9do if test -f man$j/$i.$jthen man $j $i

found=yesfidonecase $found inno) echo \'$i: manual page not found\'esacfiesacdone18.17.2 Keyword parametersShell variables may be given values by assignment or when a shell procedure isinvoked. An argument to a shell procedure of the form name=value that precedes thecommand name causes value to be assigned to name before execution of the procedurebegins. The value of name in the invoking shell is not affected. For example,user=fred commandwill execute command with user set to fred. The -k flag causes arguments of the formname=value to be interpreted in this way anywhere in the argument list. Such names aresometimes called keyword parameters. If any arguments remain they are available aspositional parameters $1, $2, ....The set command may also be used to set positional parameters from within aprocedure. For example,set - *will set $1 to the first file name in the current directory, $2 to the next, and so on. Note thatthe first argument, -, ensures correct treatment when the first file name begins with a -.16418.17.3 Parameter transmissionWhen a shell procedure is invoked both positional and keyword parameters may besupplied with the call. Keyword parameters are also made available implicitly to a shellprocedure by ecifying in advance that such parameters are to be exported. For example,export user boxmarks the variables user and box for export. When a shell procedure is invoked copies aremade of all exportable variables for use within the invoked procedure. Modification of suchvariables within the procedure does not affect the values in the invoking shell. It is generallytrue of a shell procedure that it may not modify the state of its caller without explicit requeston the part of the caller. (Shared file descriptors are an exception to this rule.)Names whose value is intended to remain constant may be declared readonly. Theform of this command is the same as that of the export command,readonly name ...Subsequent attempts to set readonly variables are illegal.18.17.4 Parameter substitutionIf a shell parameter is not set then the null string is substituted for it. For example, ifthe variable d is not setecho $dorecho ${d}will echo nothing. A default string may be given as inecho ${d-.}which will echo the value of the variable d if it is set and `.' otherwise. The default string isevaluated using the usual quoting conventions so thatecho ${d-'*'}will echo * if the variable d is not set. Similarlyecho ${d-$1}will echo the value of d if it is set and the value (if any) of $1 otherwise. A variable may beassigned a default value using the notationecho ${d=.}which substitutes the same string asecho ${d-.}165and if d were not previously set then it will be set to the string `.'. (The notation ${...=...} isnot available for positional parameters.)If there is no sensible default then the notationecho ${d?message}will echo the value of the variable d if it has one, otherwise message is printed by the shelland execution of the shell procedure is abandoned. If message is absent then a standardmessage is printed. A shell procedure that requires some parameters to be set might start as

follows.: ${user?} ${acct?} ${bin?}...Colon (:) is a command that is built in to the shell and does nothing once its arguments havebeen evaluated. If any of the variables user, acct or bin are not set then the shell willabandon execution of the procedure.18.17.5 Command substitutionThe standard output from a command can be substituted in a similar way toparameters. The command pwd prints on its standard output the name of the current directory.For example, if the current directory is /usr/fred/bin then the commandd=`pwd`is equivalent tod=/usr/fred/binThe entire string between grave accents (`...`) is taken as the command to be executed and isreplaced with the output from the command. The command is written using the usual quotingconventions except that a ` must be escaped using a \. For example,ls `echo "$1"`is equivalent tols $1Command substitution occurs in all contexts where parameter substitution occurs includinghere documents) and the treatment of the resulting text is the same in both cases. Thismechanism allows string processing commands to be used within shell procedures. Anexample of such a command is basename which removes a specified suffix from a string. Forexample,basename main.c .cwill print the string main. Its use is illustrated by the following fragment from a cc command.166case $A in...*.c) B=`basename $A .c`...Esacthat sets B to the part of $A with the suffix .c stripped.Here are some composite examples.· for i in `ls -t`; do ...The variable i is set to the names of files in time order, most recent first.· set `date`; echo $6 $2 $3, $4will print, e.g., 1977 Nov 1, 23:59:5918.17.6 Evaluation and quotingThe shell is a macro processor that provides parameter substitution, commandsubstitution and file name generation for the arguments to commands. This section discussesthe order in which these evaluations occur and the effects of the various quoting mechanisms.Commands are parsed initially according to the grammar given in appendix A. Beforea command is executed the following substitutions occur.parameter substitution, e.g. $usercommand substitution, e.g. `pwd`Only one evaluation occurs so that if, for example, the value of the variable X is thestring $y thenecho $Xwill echo $y.18.17.7 Error handlingThe treatment of errors detected by the shell depends on the type of error and onwhether the shell is being used interactively. An interactive shell is one whose input andoutput are connected to a terminal (as determined by gtty (2)). A shell invoked with the -i flagis also interactive.Execution of a command (see also 3.7) may fail for any of the following reasons.• Input - output redirection may fail. For example, if a file does not exist or cannot becreated.• The command itself does not exist or cannot be executed.167• The command terminates abnormally, for example, with a "bus error" or "memoryfault". See Figure 2 below for a complete list of UNIX signals.• The command terminates normally but returns a non-zero exit status.In all of these cases the shell will go on to execute the next command. Except for the lastcase an error message will be printed by the shell. All remaining errors cause the shell to exit

from a command procedure. An interactive shell will return to read another command fromthe terminal. Such errors include the following.• Syntax errors. e.g., if ... then ... done• A signal such as interrupt. The shell waits for the current command, if any, to finishexecution and then either exits or returns to the terminal.• Failure of any of the built-in commands such as cd.The shell flag -e causes the shell to terminate if any error is detected. The following are someof the values of the unix signals1hangup2interrupt3*quit4*illegal instruction5*trace trap6*IOT instruction7*EMT instruction8*floating point exception9kill (cannot be caught or ignored)10*bus error11*segmentation violation12*bad argument to system call13write on a pipe with no one to read it14alarm clock15software termination (from kill (1))168Those signals marked with an asterisk produce a core dump if not caught. However,the shell itself ignores quit which is the only external signal that can cause a dump. Thesignals in this list of potential interest to shell programs are 1, 2, 3, 14 and 15.18.17.8 Fault handlingShell procedures normally terminate when an interrupt is received from the terminal.The trap command is used if some cleaning up is required, such as removing temporary files.For example,trap 'rm /tmp/ps$$; exit' 2sets a trap for signal 2 (terminal interrupt), and if this signal is received will execute thecommandsrm /tmp/ps$$; exitexit is another built-in command that terminates execution of a shell procedure. The exit isrequired; otherwise, after the trap has been taken, the shell will resume executing theprocedure at the place where it was interrupted.UNIX signals can be handled in one of three ways. They can be ignored, in whichcase the signal is never sent to the process. They can be caught, in which case the processmust decide what action to take when the signal is received. Lastly, they can be left to causetermination of the process without it having to take any further action. If a signal is beingignored on entry to the shell procedure, for example, by invoking it in the background (see3.7) then trap commands (and the signal) are ignored.The use of trap is illustrated by this modified version of the touch command. Thecleanup action is to remove the file junk$$. The following is the touch command.flag=trap 'rm -f junk$$; exit' 1 2 3 15for ido case $i in

-c) flag=N ;;*) if test -f $ithen ln $i junk$$; rm junk$$elif test $flagthen echo file \'$i\' does not existelse >$ifiesacdoneThe trap command appears before the creation of the temporary file; otherwise itwould be possible for the process to die without removing the file.169Since there is no signal 0 in UNIX it is used by the shell to indicate the commands to beexecuted on exit from the shell procedure.A procedure may, itself, elect to ignore signals by specifying the null string as the argumentto trap. The following fragment is taken from the nohup command.trap '' 1 2 3 15which causes hangup, interrupt, quit and kill to be ignored both by the procedure and byinvoked commands.Traps may be reset by sayingtrap 2 3which resets the traps for signals 2 and 3 to their default values. A list of the current values oftraps may be obtained by writingtrapThe procedure scan (given below) is an example of the use of trap where there is noexit in the trap command. scan takes each directory in the current directory, prompts with itsname, and then executes commands typed at the terminal until an end of file or an interrupt isreceived. Interrupts are ignored while executing the requested commands but causetermination when scan is waiting for input.d=`pwd`for i in *do if test -d $d/$ithen cd $d/$iwhile echo "$i:"trap exit 2read xdo trap : 2; eval $x; donefidoneread x is a built-in command that reads one line from the standard input and places theresult in the variable x. It returns a non-zero exit status if either an end-of-file is read or aninterrupt is received.18.17.9 Command executionTo run a command (other than a built-in) the shell first creates a new process usingthe system call fork. The execution environment for the command includes input, output andthe states of signals, and is established in the child process before the command is executed.The built-in command exec is used in the rare cases when no fork is required and simplyreplaces the shell with a new command. For example, a simple version of the nohupcommand looks liketrap \'\' 1 2 3 15170exec $*The trap turns off the signals specified so that they are ignored by subsequentlycreated commands and exec replaces the shell by the command specified.Most forms of input output redirection have already been described. In the followingword is only subject to parameter and command substitution. No file name generation orblank interpretation takes place so that, for example,echo ... >*.cwill write its output into a file whose name is *.c. Input output specifications are evaluatedleft to right as they appear in the command.> wordThe standard output (file descriptor 1) is sent to the file word which is created if itdoes not already exist.>> wordThe standard output is sent to file word. If the file exists then output is appended (by

seeking to the end); otherwise the file is created.< wordThe standard input (file descriptor 0) is taken from the file word.<< wordThe standard input is taken from the lines of shell input that follow up to but notincluding a line consisting only of word. If word is quoted then no interpretation ofthe document occurs. If word is not quoted then parameter and command substitutionoccur and \ is used to quote the characters \ $ ` and the first character of word. In thelatter case \newline is ignored (c.f. quoted strings).>& digitThe file descriptor digit is duplicated using the system call dup (2) and the result isused as the standard output.<& digitThe standard input is duplicated from file descriptor digit.<&-The standard input is closed.>&-The standard output is closed.171Any of the above may be preceded by a digit in which case the file descriptor createdis that specified by the digit instead of the default 0 or 1. For example,... 2>fileruns a command with message output (file descriptor 2) directed to file.... 2<&1runs a command with its standard output and message output merged. (Strictly speaking filedescriptor 2 is created by duplicating file descriptor 1 but the effect is usually to merge thetwo streams.)The environment for a command run in the background such aslist *.c | lpr &is modified in two ways. Firstly, the default standard input for such a command is the emptyfile /dev/null. This prevents two processes (the shell and the command), which are running inparallel, from trying to read the same input. Chaos would ensue if this were not the case. Forexample,ed file &would allow both the editor and the shell to read from the same input at the same time.The other modification to the environment of a background command is to turn offthe QUIT and INTERRUPT signals so that they are ignored by the command. This allowsthese signals to be used at the terminal without causing background commands to terminate.For this reason the UNIX convention for a signal is that if it is set to 1 (ignored) then it isnever changed even for a short time. Note that the shell command trap has no effect for anignored signal.18.17.10 Invoking the shellThe following flags are interpreted by the shell when it is invoked. If the firstcharacter of argument zero is a minus, then commands are read from the file .profile.-c stringIf the -c flag is present then commands are read from string.-sIf the -s flag is present or if no arguments remain then commands are read from thestandard input. Shell output is written to file descriptor 2.-iIf the -i flag is present or if the shell input and output are attached to a terminal (astold by gtty) then this shell is interactive. In this case TERMINATE is ignored (so that172kill 0 does not kill an interactive shell) and INTERRUPT is caught and ignored (sothat wait is interruptable). In all cases QUIT is ignored by the shell.18.18 Let us sum upIn this lesson we have learnt aboutc) the Kerneld) and shell commands18.19 Points for Discussiona) Discuss about the importance of Unixb) Differentiate between kernel and shell18.20 Model Answers to Check your ProgressIn order to check your progress, try to answer the followingo What is a shell variable

o What is the use of MAN commando Describe about various control flows in UNIX shell18.21 Lesson - end activitiesAfter learning this chapter, try to discuss among your friends and answer thesequestions to check your progress.d) Discuss about Kernele) Discuss about any 10 shell commands18.22 References♦ H.M. Deitel, Chapter 18 of “Operating Systems”, Second Edition, PearsonEducation, 2001♦ Andrew S. Tanenbaum, Chapter 7 of “Modern Operating Systems”, PHI, 1996173LESSON – 19: PROCESS, MEMORY AND I/O OF UNIXCONTENTS19.1 Aims and Objectives19.2 Introduction to process management19.2.1 ps19.2.2 Runaway Processes19.2.3 Killing Processes19.2.4 Top19.2.5 nice and renice19.2.6 Job Control19.3 Memory Management19.3.1 Kinds of Memory19.3.2 OS Memory Uses19.3.3 Process Memory Uses19.4 The Input/output system19.4.1 Descriptors and I/O19.4.2 Descriptor Management19.4.3 Devices19.4.4 Socket IPC19.4.5 Scatter/Gather I/O19.5 Let us sum up19.6 Lesson - end activities19.7 Points for Discussion19.8 Model Answers to Check your Progress19.9 References19.1 Aims and ObjectivesIn this lesson we will learn about the introduction to process, memory and i/o of UNIX.The objectives of this lesson is to make the candidate aware of the followingo) Process conceptp) Memory conceptq) And Input/output of Unix17419.2 Introduction to Process ManagementAny time you use a command on a Unix system, that command runs as a process onthe system. Once the command runs to completion, or is otherwise interrupted, the processshould terminate. Most commands do this flawlessly, however sometimes odd things willhappen. This document will attempt to help you understand how processes work and how tomanage your own processes.19.2.1 psThe first topic to cover is how to get information on processes. There several ways todo this and this is not meant to be an exhaustive listing of methods to use. To find out whichprocesses are running under your username at this moment, use the command ps -uusername replacing the word "username" with your own username.The output from ps shows four columns by default: PID, TTY, TIME, and CMD. PIDis short for process ID, a number which uniquely identifies the process on the system.Whenever a process is started, it is given a PID. You can use this PID to interact with theprocess at a later time. The TTY field shows the terminal associated with the process. You donot generally need to worry about the value of this field. The TIME field displays the totalamount of CPU time the process has used, not the time since the process was started. Thiswill typically be quite small, unless your process is very CPU intensive. The final field isCMD, which shows the name of the command associated with that process.If you are using Borg or Locutus, you will notice that the bash process will berunning any time you are logged in. This is the shell that you are currently using to run

commands. So long as bash is set as your default shell, it will be there when you are loggedin. If you get a friend to run this command from their account with your username while youare not logged in, nothing should show up at all.Please note that this is not the only way to use ps and for more information youshould type man ps on the command line for a detailed listing of the different options youcan use.19.2.2 Runaway ProcessesSo what happens if you are not logged in and there are still processes running? Thiscan mean one of several things, but more than likely you have set a process to run in thebackground by adding a & at the end of the command (more on this later) and did not closethe program or kill the process. Another possibility is that you have a program that that isrunaway. This means that the process should have completed properly and you have taken allthe correct steps, but for any number of reasons, it did not terminate properly.17519.2.3 Killing ProcessesIn order to kill a process that has run away or that you have otherwise lost control of,you should use the kill command. There are two popular ways of using kill: kill pid where"pid" is the process ID number listed under the PID column when you use ps. This is agraceful way to kill your process by having the operating system send the process a signalthat tells it to stop what it's doing and terminate. Sometimes this doesn't work and you have touse a more aggressive version of kill. kill -9 pid will get the operating system to kill theprocess immediately. This can be dangerous if the process has control of a lock mechanisms,but your process will stop. If you find yourself completely stuck, terminals have frozen uponyou and you just don't know what to do, you can log in again and use kill -9 -1. Thiscommand will kill all of your processes including your current log in. The next time you login, everything that was running or was locked up will have terminated. Please note that usingthis command as root will bring down a system quick, fast and in a hurry so use thiscommand only as a user, never as root. This may also result in lost work, so use it only as alast resort.19.2.4 TopAnother very useful command is the top command. This will allow you to see theprocesses that are using the most CPU time, memory, etc... If you find that one of yourprocesses is running near the top of the listing offered by top, it means that you are taking upmore CPU time than most people on the system. You will see a column that has%CPU as itsheading. If your process is taking more than 5% of the CPU on Borg or Locutus for anyextended period of time, you should investigate to see what it's doing. It is quite possible thatyou are using a program that does require this much CPU time. This is perfectly alright andwill not pose a problem. If, however, you find that the program isn't doing anything or shouldalready have completed you should consider killing it.19.2.5 nice and reniceIf you are about to run some program that you know will consume a significantamount of resources on the servers, please send an e-mail to cshelp first to warn our technicalsupport team. If possible, you should first consider running such a process on workstationsrather than on the server. You can log in and run any process on our Sun workstations just aswell as on our servers. If you must run a job that requires a lot of CPU time, you should nicethe process. The nice and renice commands change the priority at which a process runs. Thenice is used to start a process with a different priority, and renice changes the priority of arunning process. To start a process with nice, run nice -n nicevalue command argumentswhere nicevalue is a number between 1 and 19, and command and arguments are thecommand you want to run and its arguments. The larger the number given to nice, the lowerthe priority command is given. Note that only the root user can increase priority with nice.If you have already begun to run a process which is using a lot of CPU time, you canalso re-nice that process. renice acts the same way as nice; except it is used on runningprocesses. To re-nice a process, simply type renice -n nicevalue -p pid. Here, nicevalue isthe new priority, and pid is the process ID of the process you want to re-nice. You can alsorenice all your processes, useful if your feeling guilty about the amount of CPU time yourprocesses are using. To do this, simply use renice -n nicevalue -u username. Note that you176can never increase your priority with renice, nor can you use it on processes owned by otherusers.19.2.6 Job ControlMany shells, including the default bash shell, support what is known as job control.This allows you to run several processes, at the same time, from the same terminal. You canrun processes that may take a long time to complete in the background, freeing the terminalfor other use. You can suspend running processes, move them to the background, put

background processes back into the foreground, and list all your currently running jobs.Running processes in the background is important for processes that take a long timeto complete, but don't require any user interaction from the shell. If you start a process suchas netscape from the command line, it is a good idea to place it in the background so the shellis not tied up. All the output from the process is still visible, unless it is specifically disabled.To run a process in the background, simply add a & character to the end of the commandline.The command line prompt has returned immediately, and it has provided you withsome important information regarding the background process. The number inside the squarebrackets (here it is 1) indicates the job number of the background process. This number isused by other job control commands to identify this process. The second number (here it is5044) is the process ID of the process. The difference between the two numbers is the jobnumber is unique to the shell, while the process ID is unique to the operating system.If you run a program normally, but then decide you would like to run it in thebackground, you must first suspend the process by hitting Ctrl-Z. This suspends execution ofthe process, and gives you a command line. To move the suspended process to thebackground, type bg. The process then resumes execution, exactly as if you had typed &when starting the process. To later bring a background process back into the foreground, usethe fg command. This command takes an argument specifying which background process toplace in the foreground. This is the job number displayed when the process was placed in thebackground. If you cannot remember the job number of the process you wish to bring back tothe foreground, the jobs command will list all the jobs you are currently running and theirnumbers. This is similar to a small version of ps, but is not quite the same: the jobs commandonly lists processes started from that particular shell.19.3 Memory ManagementUnlike traditional PC operating systems, Unix related systems use very sophisticatedmemory management algorithms to make efficient use of memory resources. This makes thequestions "How much memory do I have?" and "How much memory is being used?" rathercomplicated to answer. First you must realize that there are three different kinds of memory,three different ways they can be used by the operating system, and three different ways theycan be used by processes.19.3.1 Kinds of Memory177Main - The physical Random Access Memory located on the CPU motherboard that mostpeople think of when they talk about RAM. Also called Real Memory. This does notinclude processor caches, video memory, or other peripheral memory.File System - Disk memory accessible via pathnames. This does not include raw devices,tape drives, swap space, or other storage not addressable via normal pathnames. It doesinclude all network file systems.Swap Space - Disk memory used to hold data that is not in Real or File System memory.Swap space is most efficient when it is on a separate disk or partition, but sometimes it is justa large file in the File System.19.3.2 OS Memory UsesKernel - The Operating System's own (semi-)private memory space. This is always in Mainmemory.Cache - Main memory that is used to hold elements of the File System and other I/Ooperations. Not to be confused with the CPU cache or or disk drive cache, which are not partof main memory.Virtual - The total addressable memory space of all processes running on the given machine.The physical location of such data may be spread among any of the three kinds of memory.19.3.3 Process Memory UsesData - Memory allocated and used by the program (usually via malloc, new, or similarruntime calls).Stack - The program's execution stack (managed by the OS).Mapped - File contents addressable within the process memory space.The amount of memory available for processes is at least the size of Swap, minusKernel. On more modern systems (since around 1994) it is at least Main plus Swap minusKernel and may also include any files via mapping.19.4 The Input/output systemThe basic model of the UNIX I/O system is a sequence of bytes that can be accessedeither randomly or sequentially. There are no access methods and no control blocks in atypical UNIX user process.Different programs expect various levels of structure, but the kernel does not imposestructure on I/O. For instance, the convention for text files is lines of ASCII charactersseparated by a single newline character (the ASCII line-feed character), but the kernel knows

nothing about this convention. For the purposes of most programs, the model is furthersimplified to being a stream of data bytes, or an I/O stream. It is this single common dataform that makes the characteristic UNIX tool-based approach work Kernighan & Pike, 1984.An I/O stream from one program can be fed as input to almost any other program. (This kindof traditional UNIX I/O stream should not be confused with the Eighth Edition stream I/Osystem or with the System V, Release 3 STREAMS, both of which can be accessed astraditional I/O streams.)17819.4.1 Descriptors and I/OUNIX processes use descriptors to reference I/O streams. Descriptors are smallunsigned integers obtained from the open and socket system calls. The open system call takesas arguments the name of a file and a permission mode to specify whether the file should beopen for reading or for writing, or for both. This system call also can be used to create a new,empty file. A read or write system call can be applied to a descriptor to transfer data. Theclose system call can be used to deallocate any descriptor.Descriptors represent underlying objects supported by the kernel, and are created bysystem calls specific to the type of object. In 4.4BSD, three kinds of objects can berepresented by descriptors: files, pipes, and sockets.• A file is a linear array of bytes with at least one name. A file exists until all its namesare deleted explicitly and no process holds a descriptor for it. A process acquires adescriptor for a file by opening that file's name with the open system call. I/O devicesare accessed as files.• A pipe is a linear array of bytes, as is a file, but it is used solely as an I/O stream, andit is unidirectional. It also has no name, and thus cannot be opened with open. Instead,it is created by the pipe system call, which returns two descriptors, one of whichaccepts input that is sent to the other descriptor reliably, without duplication, and inorder. The system also supports a named pipe or FIFO. A FIFO has propertiesidentical to a pipe, except that it appears in the filesystem; thus, it can be opened usingthe open system call. Two processes that wish to communicate each open the FIFO:One opens it for reading, the other for writing.• A socket is a transient object that is used for interprocess communication; it existsonly as long as some process holds a descriptor referring to it. A socket is created bythe socket system call, which returns a descriptor for it. There are different kinds ofsockets that support various communication semantics, such as reliable delivery ofdata, preservation of message ordering, and preservation of message boundaries.In systems before 4.2BSD, pipes were implemented using the filesystem; whensockets were introduced in 4.2BSD, pipes were reimplemented as sockets.The kernel keeps for each process a descriptor table, which is a table that the kerneluses to translate the external representation of a descriptor into an internal representation.(The descriptor is merely an index into this table.) The descriptor table of a process isinherited from that process's parent, and thus access to the objects to which the descriptorsrefer also is inherited. The main ways that a process can obtain a descriptor are by opening orcreation of an object, and by inheritance from the parent process. In addition, socket IPCallows passing of descriptors in messages between unrelated processes on the same machine.Every valid descriptor has an associated file offset in bytes from the beginning of theobject. Read and write operations start at this offset, which is updated after each data transfer.For objects that permit random access, the file offset also may be set with the lseek systemcall. Ordinary files permit random access, and some devices do, as well. Pipes and sockets donot.When a process terminates, the kernel reclaims all the descriptors that were in use bythat process. If the process was holding the final reference to an object, the object's manager179is notified so that it can do any necessary cleanup actions, such as final deletion of a file ordeallocation of a socket.19.4.2 Descriptor ManagementMost processes expect three descriptors to be open already when they start running.These descriptors are 0, 1, 2, more commonly known as standard input, standard output, andstandard error, respectively. Usually, all three are associated with the user's terminal by thelogin process (see Section 14.6) and are inherited through fork and exec by processes run bythe user. Thus, a program can read what the user types by reading standard input, and theprogram can send output to the user's screen by writing to standard output. The standard errordescriptor also is open for writing and is used for error output, whereas standard output isused for ordinary output.These (and other) descriptors can be mapped to objects other than the terminal; suchmapping is called I/O redirection, and all the standard shells permit users to do it. The shell

can direct the output of a program to a file by closing descriptor 1 (standard output) andopening the desired output file to produce a new descriptor 1. It can similarly redirectstandard input to come from a file by closing descriptor 0 and opening the file.Pipes allow the output of one program to be input to another program withoutrewriting or even relinking of either program. Instead of descriptor 1 (standard output) of thesource program being set up to write to the terminal, it is set up to be the input descriptor of apipe. Similarly, descriptor 0 (standard input) of the sink program is set up to reference theoutput of the pipe, instead of the terminal keyboard. The resulting set of two processes andthe connecting pipe is known as a pipeline. Pipelines can be arbitrarily long series ofprocesses connected by pipes.The open, pipe, and socket system calls produce new descriptors with the lowestunused number usable for a descriptor. For pipelines to work, some mechanism must beprovided to map such descriptors into 0 and 1. The dup system call creates a copy of adescriptor that points to the same file-table entry. The new descriptor is also the lowestunused one, but if the desired descriptor is closed first, dup can be used to do the desiredmapping. Care is required, however: If descriptor 1 is desired, and descriptor 0 happens alsoto have been closed, descriptor 0 will be the result. To avoid this problem, the systemprovides the dup2 system call; it is like dup, but it takes an additional argument specifyingthe number of the desired descriptor (if the desired descriptor was already open, dup2 closesit before reusing it).19.4.3 DevicesHardware devices have filenames, and may be accessed by the user via the samesystem calls used for regular files. The kernel can distinguish a device special file or specialfile, and can determine to what device it refers, but most processes do not need to make thisdetermination. Terminals, printers, and tape drives are all accessed as though they werestreams of bytes, like 4.4BSD disk files. Thus, device dependencies and peculiarities are keptin the kernel as much as possible, and even in the kernel most of them are segregated in thedevice drivers.180Hardware devices can be categorized as either structured or unstructured; they areknown as block or character devices, respectively. Processes typically access devices throughspecial files in the filesystem. I/O operations to these files are handled by kernel-residentsoftware modules termed device drivers. Most network-communication hardware devices areaccessible through only the interprocess-communication facilities, and do not have specialfiles in the filesystem name space, because the raw-socket interface provides a more naturalinterface than does a special file.Structured or block devices are typified by disks and magnetic tapes, and include mostrandom-access devices. The kernel supports read-modify-write-type buffering actions onblock-oriented structured devices to allow the latter to be read and written in a totally randombyte-addressed fashion, like regular files. Filesystems are created on block devices.Unstructured devices are those devices that do not support a block structure. Familiarunstructured devices are communication lines, raster plotters, and unbuffered magnetic tapesand disks. Unstructured devices typically support large block I/O transfers.Unstructured files are called character devices because the first of these to beimplemented were terminal device drivers. The kernel interface to the driver for these devicesproved convenient for other devices that were not block structured.Device special files are created by the mknod system call. There is an additionalsystem call, ioctl, for manipulating the underlying device parameters of special files. Theoperations that can be done differ for each device. This system call allows the specialcharacteristics of devices to be accessed, rather than overloading the semantics of othersystem calls. For example, there is an ioctl on a tape drive to write an end-of-tape mark,instead of there being a special or modified version of write.19.4.4 Socket IPCThe 4.2BSD kernel introduced an IPC mechanism more flexible than pipes, based onsockets. A socket is an endpoint of communication referred to by a descriptor, just like a fileor a pipe. Two processes can each create a socket, and then connect those two endpoints toproduce a reliable byte stream. Once connected, the descriptors for the sockets can be read orwritten by processes, just as the latter would do with a pipe. The transparency of socketsallows the kernel to redirect the output of one process to the input of another process residingon another machine. A major difference between pipes and sockets is that pipes require acommon parent process to set up the communications channel. A connection between socketscan be set up by two unrelated processes, possibly residing on different machines.System V provides local interprocess communication through FIFOs (also known asnamed pipes). FIFOs appear as an object in the filesystem that unrelated processes can openand send data through in the same way as they would communicate through a pipe. Thus,

FIFOs do not require a common parent to set them up; they can be connected after a pair ofprocesses are up and running. Unlike sockets, FIFOs can be used on only a local machine;they cannot be used to communicate between processes on different machines. FIFOs areimplemented in 4.4BSD only because they are required by the POSIX.1 standard. Theirfunctionality is a subset of the socket interface.The socket mechanism requires extensions to the traditional UNIX I/O system calls toprovide the associated naming and connection semantics. Rather than overloading the181existing interface, the developers used the existing interfaces to the extent that the latterworked without being changed, and designed new interfaces to handle the added semantics.The read and write system calls were used for byte-stream type connections, but six newsystem calls were added to allow sending and receiving addressed messages such as networkdatagrams. The system calls for writing messages include send, sendto, and sendmsg. Thesystem calls for reading messages include recv, recvfrom, and recvmsg. In retrospect, the firsttwo in each class are special cases of the others; recvfrom and sendto probably should havebeen added as library interfaces to recvmsg and sendmsg, respectively.19.4.5 Scatter/Gather I/OIn addition to the traditional read and write system calls, 4.2BSD introduced theability to do scatter/gather I/O. Scatter input uses the readv system call to allow a single readto be placed in several different buffers. Conversely, the writev system call allows severaldifferent buffers to be written in a single atomic write. Instead of passing a single buffer andlength parameter, as is done with read and write, the process passes in a pointer to an array ofbuffers and lengths, along with a count describing the size of the array.This facility allows buffers in different parts of a process address space to be writtenatomically, without the need to copy them to a single contiguous buffer. Atomic writes arenecessary in the case where the underlying abstraction is record based, such as tape drivesthat output a tape block on each write request. It is also convenient to be able to read a singlerequest into several different buffers (such as a record header into one place and the data intoanother). Although an application can simulate the ability to scatter data by reading the datainto a large buffer and then copying the pieces to their intended destinations, the cost ofmemory-to-memory copying in such cases often would more than double the running time ofthe affected application.Just as send and recv could have been implemented as library interfaces to sendto andrecvfrom, it also would have been possible to simulate read with readv and write with writev.However, read and write are used so much more frequently that the added cost of simulatingthem would not have been worthwhile.18219.5 Let us sum upIn this lesson we have learnt abouta) the UNIX process managementb) the MemoryManagement in UNIXc) and input/output in UNIX19.6 Points for DiscussionDiscuss about Input/output in UNIX19.7 Model Answers to Check your ProgressTo check your progress try to answer the following questionsa) Socket IPCb) Descriptor management19.8 Lesson - end activitiesAfter learning this chapter, try to discuss among your friends and answer these questions tocheck your progress.b) Discuss about process and memory management in UNIXc) Discuss about MemoryManagement in UNIX19.9 References♦ H.M. Deitel, Chapter 18 of “Operating Systems”, Second Edition, Pearson Education,2001♦ Andrew S. Tanenbaum, Chapter 7 of “Modern Operating Systems”, PHI, 1996