Distributed System Examples

7/28/2019 Distributed System Examples

1/33

NJIT

7/16/20131

Distributed SystemExamples

Introduction to key topics in

Distributed Computing


2/33

7/16/20132

Warning

This is a difficult course, because it requires youto learn how to configure heterogeneouslanguages and environments.

You will work in teams to solve problems. Theinstructor cannot provide individual technicalsupport. All technical support will be on Moodle,

with students answering each others questions. You should be familiar with Java, and have

some knowledge of C or C++.


3/33


4/33

7/16/20134

The Fourth Wave

Starting about 2000, there has been a movetoward parallel computing using inexpensive

blade servers, often using Linux. The next slide give an example from a

company named Cassat formed byspecialists from the Cray supercomputercompany. Note that the example gives 7times the performance for one tenth the cost!


5/33

7/16/20135

The Fourth Wave

Cassatts program, Collage sits atop a pool of

servers and shifts processing power on the fly

Pharmaceutical giant Pfizer became a Cassat

customer after a demo in which Collage enabled a$300,000 cluster of IBM blade servers, running 24ordinary Intel microprocessors, to crunch through anOracle data warehousing job in 45 minutes.Previously Pfizer had run the job on a $3 million Sunmachine. It took five and a half hours. 11Forbes magazine, September 18, 2006, page 106.


6/33

7/16/20136

Subsequent Developments

Key developments since 1998 include theFurther development of the Internet,

including portable Internet technologiessuch as WiFi and Bluetooth, as well as thegrowth of Peer-to-Peer systems starting

with the great popularity of Napster.


7/337/16/20137

Not about technology

The Distributed Systems marketplace wasdriven by technology until 2000. That is

no longer true. Now it is driven bybusiness value.

Knowing the latest and greatest

technology will no longer get you a job.Now you have to deliver increased profitand/or lower cost to a company.


8/337/16/20138

Distributed Computing -Strategic Factors

End-User

Organization

Flexibility

Real-Time Access

to information

ScalabilityFaster Development

And deployment of

Business Solutions

Code

Reusability

Interoperability

Portability

Reduced

Complexity

Increased

Complexity

Lack of

ManagementTools

Changing

Technology

Personalized Environment

Predictable Response

Location IndependencePlatform Independence

Applic

ationDeveloper

SystemAdministr

ator


9/33


10/33


11/337/16/201311

Distributed ComputingReference Model

Network

Managem

ent

Mana

gement

AndS

upport

Enterprise Systems:

Perform enterprise activities

Application Systems:

Support enterprise systems

Distributed Computing platform

Application Support Services (OS,

DB support, Directories, RPC)

Communication Network Services

(Network protocols,Physical devices)

Hardware

Interoper

ability

P

ortability

Integration


12/337/16/201312

Expanded Model

Common Network Services

Network protocols & interconnectivityOSI

ProtocolsTCP/IP SNA

Distributed Computing Platform

Application Support Services

C/S SupportDist. Data

Trans. Mgmt.Distributed

OS

Application Systems:

User

Interfaces

Processing

Programs

Data files &

Databases

Enterprise Systems:

.Engineering Systems .Manufacturing

.Business Systems .Office Systems

Network

Manag

ement&

Manageme

nt

Support

Interoperability

Portability

Integration


13/337/16/201313

The Internet

The Internet is a vast interconnectedcollection of computer networks of many

different types. [Coulouriset al, p.3] It is the dominant distributed system at the

current time, although intranets, the public

switched telephone network (PSTN) andother networks are still important.


14/337/16/201314

Effect of the Web

Some industries have already beenchanged beyond all recognition by the

web. Nearly all new car buyers haveshopped the web first. That hasdramatically changed the balance of

power between dealer and customer. Thestock brokerage and travel businesseshave also been impacted dramatically.


15/33


16/33

7/16/201316

Mobile and UbiquitousComputing

Small portable devices have beenintegrated into information systems with

wireless networking. These devices include laptop computers,

handheld devices such as personal digital

assistants, cell phones, wearable deviceslike smart watches, and devicesembedded in appliances and automobiles.


17/33

7/16/201317

Resource Sharing and theWeb

Distributed Systems began with thesharing of printers and files. Today, we

take for granted the ready access to vastquantities of information. While theInternet may be little more than the ability

to share information from a vast collectionof files, the speed, search capabilities, andtools have changed much of our society.


18/33

7/16/201318

Web Portable Protocols

KISS

Platform Independent

Read-only; first generation Internet as backbone

URL Global naming

HTTP RPC-Like protocol HTML documents

Web browsers as universal clients


19/33

7/16/201319

A Web Client/Server Interaction

Simple Web Client/Server Interaction

1. Select target URL

2. Browser send HTTP request to server3. Server processes request

a. Well-known port 80 for HTTP

b. Send back requested HTML file; close

connection4. Browser interprets HTML commands

If HTML type, displays page.

Otherwise, launches helper


20/33


21/33

7/16/201321

URL (Uniform Resource Locator)

Naming scheme + how to get to resource

Protocol scheme

HTTP, Gopher, News, FTP, Mailto, nntp, Telnet

HTTP is native Web protocol

Servername

Internet host domain name or raw IP address

Port number (or default)

HTTP = 80; Gopher = 70; FTP = 21

Path to resource


22/33


23/33

7/16/201323

HTTP:

Web's RPC on top of TCP/IP

Stateless protocol

Separate TCP connection to download each BLOBdocument with 5 inline images = 6 connections

Typed data using RPC

Negotiate representation for each connection

MIME-like content minus Internet Mail

MIME's 7 types: Plain text, audio, video, stillimages, message, multipart message, application-specific data


24/33

7/16/201324

CGI: Common Gateway Interface

Supported by most Web servers Appropriate for small amounts of dynamic content

E.g. mostly static web pages plus phone number

lookup Unacceptable for highly interactive web sites

Each CGI request starts a processstartup delayProcess-per-request requirement: overloads theserver if many requests

Now mostly obsolete, but important as anillustration of emerging need for efficiency.


25/33


26/33

7/16/201326

XML: Structured Data in aText File

l Spreadsheets, address books,configuration parameters, financialtransactions, product catalogs

l XML defines a set of rules andconventions for designing text formats forsuch data

l Easy to generate and read by computer

l Extensible


27/33

7/16/201327

Role of XML

Applications built on differenttechnologies can communicate via XML.

New integration tools and integrationservers capitalize on emergence of XMLas an integration technology.


28/33

7/16/201328

Challenges

Heterogeneity

Openness

Security Scalability

Failure Handling

Concurrency

Transparency

The concernsintroduced at the endof chapter one in

Coulouris et alwill bemajor topics fordiscussion in futurelectures in thiscourse.


29/33


30/33

7/16/201330

Distributed System Challengesfrom a Systems Perspective

Communication

Processes

Naming Synchronization

Data Storage/Access

Consistency/Replication Fault Tolerance

Security

API and Transparency

Scalability/Modularity


31/33

7/16/201331

Distributed System Challengesfrom an Algorithmic Perspective

Execution models andFrameworks

Distributed Graph andRouting Algorithms

Time and Global States

Synchronization and

Coordination

Group communicationand multicasting

Monitoring distributedevents

Design and verificationtools

Distributed debugging

Replication and

consistency

Web design cache,search, scheduling


32/33

7/16/201332

More Challenges from anAlgorithmic Perspective

Distributed sharedmemory

Reliable and faulttolerant systems

Load balancing

Real time scheduling

Performance

These are broad topics,and each of them mayhave several sub-topics. For example,Distributed sharedmemory is concerned

with wait-freealgorithms, mutualexclusion, andconsistency models.


33/33

Documents

Distributed System Examples