B. Prabhakaran 1

Advanced Operating Systems

Course Reading Material: Lecture notes, class discussions will be the material

for examination purposes. Recommend reading appropriate reference papers for

the different algorithms. Other books discussing the algorithms will help too.

B. Prabhakaran 2

Advanced Operating Systems

Project Reference Text: “Unix Network programming”, Richard Stevens,

Prentice Hall.

B. Prabhakaran 3

Contact InformationB. PrabhakaranDepartment of Computer ScienceUniversity of Texas at DallasMail Station EC 31, PO Box 830688Richardson, TX 75083Email: praba@utdallas.eduFax: 972 883 2349URL: http://www.utdallas.edu/~praba/cs6378.htmlPhone: 972 883 4680Office: ES 3.706Office Hours: 12.45-1.15pm Saturdays

10.30-11am WednesdaysOther times by appointments through email

Announcements: Made in class and on course web page.TA: TBA.

B. Prabhakaran 4

Course Outline

Introduction to operating systems, inter-process communication.

Distributed Operating Systems Architecture Clock Synchronization, Ordering Distributed Mutual Exclusion Distributed Deadlock Detection Agreement Protocols

Distributed Resource Management Distributed File Systems Distributed Shared Memory Distributed Scheduling

Proposed Outline. Might be modified based on time availability:

B. Prabhakaran 5

Recovery & Fault Tolerance Concurrency Control/ Security

Depending on time availability

Course Outline ...

Additional/modified topics might be introduced from other texts and/or papers. References to those materials will begiven at appropriate time.

B. Prabhakaran 6

Evaluation 1 Mid-terms: in class. 75 minutes. Mix of MCQs

(Multiple Choice Questions) & Short Questions. 1 Final Exam: 75 minutes or 2 hours (Mix of MCQs and

Short Questions). 2 Quiz: in class. 5-6 MCQs or very short questions. 20 –

25 minutes. Homeworks/assignments: at most 2-3 Programming Projects:

1 preparatory (to warm up on thread and socket programming) – NOT Graded

2 projects based on algorithms discussed in class.

B. Prabhakaran 7

Grading 2 Home works: 5% 2 Quizzes: 15% 1 Mid-term + Final: 50%

Equally distributed 25% each

Intermediate & Final Projects (combined): 30% Possible Letter Grades: A, A-, B+, B, B-, C+, C Likely: B will be around class average Letter grade trend will be informed after 2nd Quiz.

B. Prabhakaran 8

Schedule Quiz 1: September 25, 2010 Quiz 2: November 20, 2010 Mid-term: October 9, 2010 Final Exam: As per UTD schedule (Dec. 11) or Last day of class Subject to minor changes Homework schedules will be announced in class and course web

page, giving sufficient time for submission. Tutorials may be given, if sufficient time available.

Likely Homework Due Dates: September 18, 2010 & November 13, 2010.

Likely project deadlines: Preparatory project: September 5, 2010 (NOT Graded) Project 1: October 17, 2010 Project 2: December 5, 2010

B. Prabhakaran 9

Programming Projects No copying/sharing of code/results will be tolerated. Any

instance of cheating in projects/homeworks/exams will be reported to the University.

No copying code from the Internet. 2 individual students copying code from Internet

independently: still considered copying in the project !! Individual projects. Deadlines will be strictly followed for projects and

homeworks submissions. Projects submissions through Web CT. Demo will be needed

B. Prabhakaran 10

Two-track Course Programming Project Discussions:

Announcements in class Minimal Discussions in class Design discussions during office hours based

on individual needs Theory (Algorithms) Discussions:

Full discussion in class Office hours for clarification if needed

B. Prabhakaran 11

eLearning Go to:

https://elearning.utdallas.edu/webct/entryPageIns.dowebct

It has a discussion group that can be used for project and other course discussions.

B. Prabhakaran 12

Cheating Academic dishonesty will be taken seriously. Cheating students will be handed over to

Head/Dean for further action. Remember: home works/projects (exams too !)

are to be done individually. Any kind of cheating in home works/ projects/

exams will be dealt with as per UTD guidelines. Cheating in any stage of projects will result in 0

for the entire set of projects.

B. Prabhakaran 13

Projects Involves exercises such as ordering, deadlock detection, load

balancing, message passing, and implementing distributed algorithms (e.g., for scheduling, etc.).

Platform: Linux/Windows, C/C++/Java. Network programming will be needed. Multiple systems will be used.

Specific details and deadlines will be announced in class and course webpage.

Suggestion: Learn network socket programming and threads, if you do not know already. Try simple programs for file transfer, talk, etc.

Sample programs and tutorials available at: http://www.utdallas.edu/~praba/projects.html

B. Prabhakaran 14

Homeworks At most 2-3 home works, announced in class and course

web page. Homeworks Submission:

Submit on eLearning (or) paper to TA/Instructor.

B. Prabhakaran 15

Basic Computer Organization Input Unit Output Unit CPU Memory ALU (Arithmetic &

Logic Unit) Secondary Storage

CPU

ALUMemory Disk

I/O

Keyboard

Display

B. Prabhakaran 16

Simplified View of OS

Memory Space

PhysicalMemory

VirtualMemory

OS Kernel

OS Tools

User Processes

Tools ++

User Processes ..

Data

Data

Data

Data

CodeCode Code

Code

User iProcess j

B. Prabhakaran 17

Distributed View of the System

hardware hardware

hardwarehardware

hardware

Process

B. Prabhakaran 18

Inter-Process Communication Need for exchanging data/messages among processes belonging

to the same or different group. IPC Mechanisms:

Shared Memory: Designate and use some data/memory as shared. Use the shared memory to exchange data. Requires facilities to control access to shared data.

Message Passing: Use “higher” level primitives to “send” and “receive” data. Requires system support for sending and receiving messages.

Operation oriented language constructs Request-response action Similar to message passing with mandatory response Can be implemented using shared memory too.

B. Prabhakaran 19

IPC Examples Parallel/distributed computation such as sorting: shared

memory is more apt. Using message passing/RPC might need an array/data manager

of some sort. Client-server type: message passing or RPC may suit

better. Shared memory may be useful, but the program is more clear

with the other types of IPCs. RPC vs. Message Passing: if response is not a must,

atleast immediately, simple message passing should suffice.

B. Prabhakaran 20

Shared Memory

Shared Memory

Only one process can write at anypoint in time. Noaccess to readers.

Multiple readers canaccess. No access toWriters.

Writers/Producers

Readers/Consumers

B. Prabhakaran 21

Shared Memory: Possibilities Locks (unlocks) Semaphores Monitors Serializers Path expressions

B. Prabhakaran 22

Message Passing Blocked Send/Receive: Both sending and receiving

process get blocked till the message is completely received. Synchronous.

Unblocked Send/Receive: Both sender and receiver are not blocked. Asynchronous.

Unblocked Send/Blocked Receive: Sender is not blocked. Receiver waits till message is received.

Blocked Send/Unblocked Receive: Useful ? Can be implemented using shared memory. Message

passing: a language paradigm for human ease.

B. Prabhakaran 23

Un/blocked Blocked message exchange

Easy to: understand, implement, verify correctness Less powerful, may be inefficient as sender/receiver might

waste time waiting

Unblocked message exchange More efficient, no waste on waiting Needs queues, i.e., memory to store messages Difficult to verify correctness of programs

B. Prabhakaran 24

Message Passing: Possibilities

sender

receiver i

receiver j

receiver k

sender

receiver i

receiver j

receiver k

B. Prabhakaran 25

Message Passing: Possibilities...

receiver

sender i

sender j

sender k

receiver

sender i

sender j

sender k

B. Prabhakaran 26

Naming Direct Naming

Specify explicitly the receiver process-id. Simple but less powerful as it needs the sender/receiver to know the actual

process-id to/from which a message is to be sent/received. Not suitable for generic client-server models

Port Naming receiver uses a single port for getting all messages, good for client-server. more complex in terms of language structure, verification

Global Naming (mailbox) suitable for client-server, difficult to implement on a distributed network. complex for language structure and verification

Indirect Naming Use a naming server, e.g., as in RPCs.

B. Prabhakaran 27

Communicating Sequential Processes (CSP)

process reader-writer OKtoread, OKtowrite: integer (initially = value); busy: boolean (initially = 0);

*[ busy = 0; writer?request() ->busy := 1; writer!OKtowrite;

busy = 0; reader?request() ->

busy := 1; reader!OKtoread;busy = 1; reader?readfin() -> busy := 0;busy = 1; writer?writefn() -> busy := 0;

]

B. Prabhakaran 28

CSP: Drawbacks Requires explicit naming of processes in I/O commands. No message buffering; input/output command gets

blocked (or the guards become false) -> Can introduce delay and inefficiency.

B. Prabhakaran 29

Operation oriented constructs

Service declarations

Task A Task B

……RPC: abc(xyz, ijk);

…….

Service declarationssend xyz; wait for result

return ijk

Remote Procedure Call (RPC):

• Service declaration: describes in and out parameters• Can be implemented using message passing• Caller: gets blocked when RPC is invoked.• Callee implementation possibilities:

• Can loop “accepting” calls• Can get “interrupted” on getting a call• Can fork a process/thread for calls

B. Prabhakaran 30

RPC: Issues Pointer passing, global variables passing can be difficult. If processes on different machines, data size (number of

bits for a data type) variations need to be addressed. Abstract Data Types (ADTs) are generally used to take care of

these variations. ADTs are language like structures that specify how many bits

are being used for integer, etc… What does this imply?

Multiple processes can provide the same service? Naming needs to be solved.

Synchronous/blocked message passing is equivalent to RPC.

B. Prabhakaran 31

Ada

Task body <name> isDeclaration of local variablesbegin list of statements ….. accept <entry id> (<formal parameters> do body of the accept statement end<entry id> exceptions Exception handlersend;

task [type] <name> isentry specificationsend

task proc-buffer is entry store(x:buffer); remove(y:buffer);end;

task body proc-buffer is temp: buffer;begin loop when flag accept store(x: buffer);

temp := x; flag :=0; end store; When !flag accept remove(y:buffer); y := temp; flag :=1; end remove; end loopend proc-buffer.

B. Prabhakaran 32

Ada Message Passing

….accept Store(.)…...

….Store(xyz);…..

entry store(...)xyz is sent fromTask B to A

Task A Task B

Somewhat similar to executing procedure call. Parameter valuefor the entry procedure is supplied by the calling task. Value of Result, if any, is returned to the caller.

result

B. Prabhakaran 33

RPC Design Structure

Caller: local call + stub Callee: stub + actual procedure

Binding Where to execute? Name/address of the server that offers a

service Name server with inputs from service specifications of a task.

Parameter & results Packing: Convert to remote machine format Unpacking: Convert to local machine format

B. Prabhakaran 34

RPC Execution

Local call

Return

Querybindingserver

Paramspacking

Wait

Unpackresult

Unpack

Localcall

Packresults

Executeprocedure

Return

ReceiveQuery

ReturnServer Address

RegisterServices

BindingServer

Caller Callee

Stub StubLocal Proc. Remote Proc.

B. Prabhakaran 35

RPC Semantics At least once

A RPC results in zero or more invocation. Partial call, i.e., unsuccessful call: zero, partial, one or more

executions.

Exactly once Only one call maximum Unsuccessful? : zero, partial, or one execution

At most once Zero or one. No partial executions.

B. Prabhakaran 36

RPC Implementation Sending/receiving parameters:

Use reliable communication? : Use datagrams/unreliable? Implies the choice of semantics: how many times a RPC may be

invoked.

B. Prabhakaran 37

RPC Disadvantage

Incremental results communication not possible: (e.g.,) response from a database cannot return first few matches immediately. Got to wait till all responses are decided.

B. Prabhakaran 38

Distributed Operating Systems

Global Knowledge Naming Scalability Compatibility Process Synchronization Resource Management Security Structuring Client-Server Model

Issues :

B. Prabhakaran 39

DOS: Issues .. Global Knowledge

Lack of global shared memory, global clock, unpredictable message delays

Lead to unpredictable global state, difficult to order events (A sends to B, C sends to D: may be related)

Naming Need for a name service: to identify objects (files, databases),

users, services (RPCs). Replicated directories? : Updates may be a problem. Need for name to (IP) address resolution. Distributed directory: algorithms for update, search, ...

B. Prabhakaran 40

DOS: Issues .. Scalability

System requirements should (ideally) increase linearly with the number of computer systems

Includes: overheads for message exchange in algorithms used for file system updates, directory management...

Compatibility Binary level: Processor instruction level compatibility Execution level: same source code can be compiled and

executed Protocol level: Mechanisms for exchanging messages,

information (e.g., directories) understandable.

B. Prabhakaran 41

DOS: Issues .. Process Synchronization

Distributed shared memory: difficult.

Resource Management Data/object management: Handling migration of files, memory

values. To achieve a transparent view of the distributed system. Main issues: consistency, minimization of delays, ..

Security Authentication and authorization

B. Prabhakaran 42

DOS: Issues .. Structuring

Monolithic Kernel: Not needed (e.g.,) file management not needed fully on diskless workstations.

Collective kernel: distributed functionality on all systems. Micro kernel + set of OS processes Micro kernel: functionality for task, memory, processor

management. Runs on all systems. OS processes: set of tools. Executed as needed.

Object-oriented system: services as objects. Object types: process, directory, file, … Operations on the objects: encapsulated data can be manipulated.

B. Prabhakaran 43

DOS: CommunicationComputer

Switch

B. Prabhakaran 44

ISO-OSI Reference Model

Network

Application

Presentation

Session

Transport

Datalink

Physical

Network

Application

Presentation

Session

Transport

Datalink

Physical

Network

Datalink

Physical

Network

Datalink

Physical

Communication Network

B. Prabhakaran 45

Un/reliable Communication Reliable Communication

Virtual circuit: one path between sender & receiver. All packets sent through the path.

Data received in the same order as it is sent. TCP (Transmission Control Protocol) provides reliable

communication.

Unreliable communication Datagrams: Different packets are sent through different paths. Data might be lost or out of sequence. UDP (User datagram Protocol) provides unreliable

communication.