Chapter 18 Distributed Control Algorithms Copyright © 2008

Chapter 18

Distributed Control AlgorithmsCopyright © 2008

Operating Systems, by Dhananjay Dhamdhere Copyright © 2008 18.2Operating Systems, by Dhananjay Dhamdhere 2

Introduction

• Operation of Distributed Control Algorithms• Correctness of Distributed Control Algorithms• Distributed Mutual Exclusion• Distributed Deadlock Handling• Distributed Scheduling Algorithms• Distributed Termination Detection• Election Algorithms• Practical Issues in Using Distributed Control Algorithms

Operating Systems, by Dhananjay Dhamdhere Copyright © 2008 18.3

OS Control Functions in a Distributed Environment

• Special features of distributed OS control functions– Mutual exclusion

• Involves synchronization of processes in different computers

– Deadlock handling• Deadlocks may involve use of resources in different hosts

– Scheduling• Perform load balancing for comparable loading of computers

– Termination detection• Check whether all processes of a computation, which may

operate in different computers, have completed

– Election• Elect coordinator for a privileged function

Operating Systems, by Dhananjay Dhamdhere 3


Operation of Distributed Control Algorithms (continued)

• A distributed control algorithm operates in parallel with its clients, so that it can respond readily to events related to its service– Each process has a control part and a basic part


Correctness of Distributed Control Algorithms

• Algorithm correctness has two facets:– Liveness: eventually performs correct actions

• i.e., without indefinite delays

– Safety: does not perform wrong actions

• Proving correctness of a distributed algorithm is a complex task


Correctness of Distributed Control Algorithms (continued)


Distributed Mutual Exclusion

• A Permission-Based Algorithm• Token-Based Algorithms


• Ricart and Agrawala algorithm: CS entry in FCFS order– Requires 2 x (n – 1 ) messages per CS entry


A Permission-Based Algorithm



Maekawa Algorithm

• Each process has a request set of processes; it seeks the permission of only processes in the request set (Ri represents the request set of process Pi)

– Correctness is ensured through the following rules:• For all Pi : Pi is included in Ri

• For all Pi, Pj: Ri ∩ Rj is non-null

– The algorithm requires 2 x √n messages per CS entry



Token-Based Algorithms for Mutual Exclusion

• A token represents the privilege to use a CS– Only a process possessing token can enter CS

• Safety of algorithm follows from this rule• Liveness: must ensure token eventually reaches a

(requesting) process

• These algorithms use abstract system models– Edges represent the paths used to pass control

messages

– For example:• Abstract ring and tree topologies


An Algorithm Employing the Ring Topology


Raymond’s Token-Based Algorithm

• Features of the algorithm– The algorithm uses an abstract inverted tree to reduce

the number of messages. It has three invariants• Pholder, the process holding the token, is at root of the tree

• Each process in the system belongs to the tree

• Each process other than the Pholder has only one edge, which points to its parent in the tree

Thus, each process has a path that ends on Pholder

– Each process has a local request queue • When it receives a request, it puts the requestor’s id in the

queue• When it makes a request, it puts its own id in the queue



• Uses an abstract inverted tree as the system model


Raymond’s Algorithm

Token is transferred to P3



Distributed Deadlock Handling

• Deadlock detection, prevention, and avoidance approaches studied earlier use state information– Problems arise in extending these approaches to a

distributed system

• Approaches in distributed systems:– Detection: centralized and distributed

– Prevention

• No special techniques for distributed deadlock avoidance have been discussed in OS literature

• Model used in this section:– SISR model of resource allocation


Problems in Centralized Deadlock Detection

• Steps in centralized deadlock detection:– Collect WFGs of all nodes at a central node

– Superimpose them to form a merged WFG

– Use a conventional deadlock detection algorithm

• Problem: can lead to phantom deadlocks– Violates safety property in deadlock detection


Distributed Deadlock Detection

• Key issues in deadlock detection:– A cycle indicates a deadlock in an SISR system

– A knot indicates a deadlock in an MISR system

• Distributed deadlock detection approach:– Cycles and knots detected through joint actions of nodes

in system

– Every node can detect and declare a deadlock

• Two such algorithms:– Diffusion computation-based algorithm

– Edge chasing algorithm


Diffusion Computation-Based Deadlock Detection

• Diffusion computation: used to collect info about nodes– Diffusion phase

• Computation that has originated in one node, spreads to other nodes

– A control message called a query is sent along each edge

– The first query received by a node is called an engaging query. On receiving it, the node sends queries along all its out-edges

– Information collection phase• Each node sends information in response to each query

– It sends a dummy reply for a non-engaging query

– It collects information from all replies it received, adds its own information, and sends the result as the reply to the engaging query. We call it an engaging reply



Diffusion Computation-Based Algorithm

• Algorithm 18.4 was proposed by Chandy, Misra, and Haas (1983)– Works for SISR and MISR systems

• P2, P3 are blocked. P1 becomes blocked and sends a query but does not receive a reply because P4 is not blocked• P4 requests a resource held by P1, becomes blocked and sends a query. It would receive a reply and declare a deadlock


Diffusion Computation-Based Algorithm (continued)


Mitchell–Merritt Algorithmfor Distributed Deadlock Detection

• It is an edge chasing algorithm—control messages are sent over WFG edges to detect cycles– A provision is made to ensure that the cycle has not been

broken before it was detected• Each process is assigned a public label and a private label

– The labels are identical when a process is created

– The public label of a process changes when it gets blocked on a resource request

– It also changes when it waits for a process having a larger public label

• A wait-for edge with a specific relation between public and private labels of its source and destination processes indicates presence of a deadlock



• Rules of the algorithm:


Mitchell-Merritt Algorithm

public labelprivate label


Distributed Deadlock Prevention

• Cycles are prevented as follows:– A pair (local time, node id) is used to time-stamp creation of

a process

– When process Pi requests a resource allocated to Pj, time-stamps of Pi, Pj are used to decide whether Pi can wait for Pj

• Two approaches– Wait-or-die

• Pi is allowed to wait if older than Pj; otherwise, it is killed

– Would-or-wait• Pi is allowed to wait if younger than Pj; otherwise Pj is killed

• A killed process retains original timestamp if restartedOperating Systems, by Dhananjay Dhamdhere 24


Distributed Scheduling Algorithms

• A distributed scheduling algorithm balances computational loads in the nodes– Uses process migration

• Apply a threshold to decide if a node is heavily or lightly loaded


Distributed Scheduling Algorithms (continued)

• Migration may be preemptive or nonpreemptive• Stability is an important issue

– An unstable algorithm may lead to a situation similar to thrashing

• A process is transferred very often; does not make progress

• Algorithms can be sender- or receiver-initiated– A heavily loaded node is a sender

– A lightly loaded node is a receiver

– A symmetrically initiated algorithm contains features of both






Distributed Termination Detection

• When a process terminates, OS frees its resources– This approach is not adequate for distributed systems

• Processes of a distributed computation execute in different nodes of a distributed system. They should be terminated when all of them have completed their tasks– These processes perform work assigned to them

• A process is active when it is performing work, and passive when it has no work

• Work is assigned to a process through a message– Passive process becomes active on receiving a message

– Distributed termination condition (DTC):• All processes of a distributed computation are passive• No basic messages are in transit


Distributed Termination Detection (continued)

• Credit-distribution-based termination detection– Every process is assigned a numerical credit

• It sends some of its credit in each message

– When a process becomes passive, it sends its credit to collector process

– The distributed computation is known to have terminated when credit accumulated by collector is C

• Diffusion computation-based termination detection– Each process that becomes passive initiates a diffusion

computation to determine if DTC holds


Distributed Termination Detection (continued)


Election Algorithms

• Critical system functions are assigned to a coordinator (for the function)– E.g., replacing lost token in a token-based algorithm

• Coordinator is typically the highest-priority process in set

• A process that finds that coordinator is not responding assumes it has failed– Initiates election algorithm

• Algorithm chooses highest-priority nonfailed process as new coordinator

– Then, announces its id to all nonfailed processes


Election Algorithms for Unidirectional Ring Topologies

• Links in ring are assumed to be FIFO channels• Assumption: control part of failed process continues to

function and forwards received messages along out-edge

• Election performed by obtaining ids of all nonfailed processes and electing highest-priority process

• Two types of messages:– “elect me” and “new coordinator”

• O(n2) messages or 3n−1 (worst case) in refined version


Election Algorithms

• Algorithm for the ring topology• Process Pi initiates by sending (“elect me”, Pi) message

• Process Pj receiving an (“elect me”, Pi) message sends an (“elect me”, Pj) message and then forwards Pi’s message

• Pi receives back its own message after receiving message of every other process; it elects the highest priority process as leader– It sends a “new coordinator” message to inform others

– Algorithm 2: Refinement of algorithm 1• In Step 2, Pj sends its own message if its priority is higher

than Pi’s; otherwise, it sends Pi’s message

• Only highest priority process would get back its own message



Election Algorithm

• Bully algorithm1. Initiator Pi sends (“elect me”, Pi) messages to all higher

priority processes and starts a time-out interval T1

a. If a time-out occurs, it sends a “new coordinator” message to lower priority processes

b. If it receives a “don’t you dare” message from a higher priority process Pj, it starts another time-out interval T2

i. If a time-out occurs, it starts a new election

2. If a process Pj receives an “elect me” message from a lower priority processa. It sends a “don’t you dare” message to its sender

b. Starts a new election by sending (“elect me”, Pj) messages

• Requires O(n2) messages per electionOperating Systems, by Dhananjay Dhamdhere 35


Practical Issues in Using Distributed Control Algorithms

• Resource Management • Process Migration


Resource Management

• Name server in each node must be updated when resources are added– Solution: use an arrangement of name servers as in the

domain name service (DNS)• Only the name server of a domain needs to be updated

when a resource is added


Process Migration

• Reasons for process migration:– To achieve load balancing

– To reduce network traffic involved in utilizing a remote resource

– To provide availability of services when a node has to be shut down for maintenance


Process Migration

• Difficulties– Process state is distributed in various data structures of

the OS

– Process id’s may change due to migration• Process id’s are used in interprocess communication• Solution: Use global process ids as in Sun cluster

– Delivery of messages requires a special provision• A node receiving a message would redirect it if the

destination process has migrated out of it– This residual state causes poor reliability

• Alternatively, all processes may be informed when a process is migrated

– Requires a complex protocol



Summary

• Actions of a distributed control algorithms are performed in many nodes of the system– Two aspects of correctness are liveness and safety

• Distributed system models: physical and logical• Examples of distributed control algorithms:

– Distributed mutual exclusion: e.g., token based

– Distributed deadlock detection: diffusion computation

– Distributed scheduling (to balance load)

– Distributed termination: e.g., credit-based

– Election (highest priority process wins)

Documents

Chapter 18 Distributed Control Algorithms Copyright © 2008