Chapter 8Chapter 8-2 : -2 : MultiMulticomputerscomputers

Multiprocessors vs multicomputersMultiprocessors vs multicomputers Interconnection topologiesInterconnection topologies Switching schemesSwitching schemes Communication with messagesCommunication with messages Remote procedure callsRemote procedure calls Distributed shared memoryDistributed shared memory Multicomputer schedulingMulticomputer scheduling VirtualizationVirtualization

1

MulticomputersMulticomputers

MultiprocessorsMultiprocessors share the common share the common memorymemory

MulticomputersMulticomputers are tightly coupled CPUs are tightly coupled CPUs with memories of their own and do not with memories of their own and do not share themshare them

These systems are also known as cluster These systems are also known as cluster computers and COWS (Clusters of computers and COWS (Clusters of Workstations)Workstations)

The secret of high performance is the The secret of high performance is the interconnection networkinterconnection network

2

3

Multiprocessor vs Multiprocessor vs MulticomputerMulticomputer

Figure 8-29. Comparison of three Figure 8-29. Comparison of three kinds of multiple CPU systems.kinds of multiple CPU systems.

4

Interconnection Interconnection TTopologiesopologies

Figure 8-16. Various interconnect topologies. Figure 8-16. Various interconnect topologies. (a) A single switch (a) A single switch (b) A ring (b) A ring (c) A grid(c) A grid or mesh or mesh (d) (d) A double torus A double torus (e) A cube (f) A 4D hypercube(e) A cube (f) A 4D hypercube

Note : A Note : A diameterdiameter is defined as the longest path is defined as the longest path between any two nodesbetween any two nodes

Switching SchemesSwitching Schemes Store-and-forward packet switching Store-and-forward packet switching : :

messages are broken into packets. Packets messages are broken into packets. Packets move from one switch to anothermove from one switch to another This scheme has increasing latency This scheme has increasing latency

(delay) problem because of store-and-(delay) problem because of store-and-forwarding in intermediate switchesforwarding in intermediate switches

Circuit switching Circuit switching : a path is established : a path is established from the source to the destination. Once from the source to the destination. Once this path is setup bits are pumped from this path is setup bits are pumped from source to destination with no buffering in source to destination with no buffering in the intermediate switchesthe intermediate switches Circuit setup may need some time but Circuit setup may need some time but

then the transfer is fastthen the transfer is fast5

User Level User Level Communication SoftwareCommunication Software Multicomputers communicate through Multicomputers communicate through

messagesmessages Send(dest, &mptr) Send(dest, &mptr) : send a message pointed : send a message pointed

by by mptr mptr to a process identified by to a process identified by destdest Receive(addr, &mptr) Receive(addr, &mptr) : : addraddr is usually CPU is usually CPU

number and a process or port numbernumber and a process or port number Send calls can be Send calls can be blocked blocked or or nonblockednonblocked Receive calls are always blocked. That is, Receive calls are always blocked. That is,

the receiving process has to wait if the the receiving process has to wait if the message has not been sent beforemessage has not been sent before

6

7

Blocking Blocking CallsCalls

Figure 8-19. (a) A blocking send Figure 8-19. (a) A blocking send call.call.

8

Nonblocking CallsNonblocking Calls

Figure 8-19. (b) A nonblocking Figure 8-19. (b) A nonblocking send call.send call.

9

Remote Procedure CallRemote Procedure Call

Figure 8-20. Steps in making a remote Figure 8-20. Steps in making a remote procedure call. procedure call.

The The stubsstubs are are local procedures implementing local procedures implementing the interface between the client and the serverthe interface between the client and the server

Parameter and result passing are done by Parameter and result passing are done by messagesmessages

10

Distributed Shared Distributed Shared Memory (1)Memory (1)

Figure 8-21. Various layers where Figure 8-21. Various layers where shared memory can be implementedshared memory can be implemented (a) The hardware(a) The hardware (b) The operating (b) The operating system (c) User-level softwaresystem (c) User-level software

11

Distributed Shared Distributed Shared Memory (Memory (22))

Figure 8-22. (a) Pages of the address space Figure 8-22. (a) Pages of the address space distributed among four machines. distributed among four machines.

When a CPU references an address that is not local, a When a CPU references an address that is not local, a trap occurs, and the DSM software fetches the page trap occurs, and the DSM software fetches the page containing the address and restarts the faulting containing the address and restarts the faulting instructioninstruction

12

Distributed Shared Distributed Shared Memory (Memory (33))

Figure 8-22. (b) Situation after CPU Figure 8-22. (b) Situation after CPU 00 references references page 10 and the page is moved there. page 10 and the page is moved there.

13

Distributed Shared Distributed Shared Memory (Memory (44))

Figure 8-22. (c) Situation if page Figure 8-22. (c) Situation if page 10 is read10 is read only and replication is only and replication is used.used.

Multicomputer Multicomputer SchedulingScheduling On a multiprocessor, all processes On a multiprocessor, all processes

reside on the shared memoryreside on the shared memory On a multicomputer each node has its On a multicomputer each node has its

own memory and its own set of own memory and its own set of processesprocesses

Scheduling within a node is easier but Scheduling within a node is easier but allocation of processes to nodes is much allocation of processes to nodes is much more important (more important (load balancingload balancing))

The algorithms and heuristics for doing The algorithms and heuristics for doing this assignment are known as this assignment are known as processor processor allocation algorithmsallocation algorithms

14

Scheduling AlgorithmsScheduling Algorithms

Graph-Theoretic Deterministic Graph-Theoretic Deterministic AlgorithmAlgorithm

Sender-Initiated Distributed Sender-Initiated Distributed Heuristic AlgorithmHeuristic Algorithm

Receiver-Initiated Distributed Receiver-Initiated Distributed Heuristic AlgorithmHeuristic Algorithm

Please read from the bookPlease read from the book

15

VirtualizationVirtualization Virtualization is the virtual machine concept Virtualization is the virtual machine concept

introduced in Chapter 1 – Introductionintroduced in Chapter 1 – Introduction VM technology allows a single computer to host VM technology allows a single computer to host

multiple virtual machines, each potentially multiple virtual machines, each potentially running a different operating systemrunning a different operating system

Advantages :Advantages : Failure in one VM does not automatically bring down Failure in one VM does not automatically bring down

any others since VMs are isolatedany others since VMs are isolated Fewer machines to provide an environment for different Fewer machines to provide an environment for different

applications (instead of having one server for each of the applications (instead of having one server for each of the different apllications, a server with VMs can do the job)different apllications, a server with VMs can do the job)

Ability to run different and probably older versions of Ability to run different and probably older versions of Oss needed by some appicationsOss needed by some appications

16

Virtualization MethodsVirtualization Methods

Type 1 hypervisor (virtual machine Type 1 hypervisor (virtual machine monitor)monitor)

Type 2 hypervisorType 2 hypervisor

17