Upload
youngsu-son
View
1.394
Download
4
Embed Size (px)
DESCRIPTION
POSA2에 의해 소개된 Web Server로 다양한 패턴들을 엮어 구성되어 있다. 웹서버의 내부와 함께 패턴이 어떻게 상호작용하는지 좋은 가이드라인을 제시하는 패턴이다.
Citation preview
JAWS(A FRAMEWORK FOR HIGH-PERFORMANCE WEB SERVERS)
TP VERSION 1.1
JAWS(A FRAMEWORK FOR HIGH-PERFORMANCE WEB SERVERS)
TP VERSION 1.1
YoungSu, [email protected]
Microsoft MVPDevpia Architecture Sysop
EvaCast Study Leader (http://www.EvaCast.net)
YoungSu, [email protected]
Microsoft MVPDevpia Architecture Sysop
EvaCast Study Leader (http://www.EvaCast.net)
What’s the What’s the
Good Web Good Web Server?Server?
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 22
ArchitectArchitect
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 33
Concurrency
Thread-per-Session
I/O
Synchronous
Cache
LRU
Protocol
&
Filter
Read Request
Parse Request
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 44
Thread-per Request
Thread Pool
Reactvie
Asynchronous
LFU
Structured Cache
Hinted Cache
Parse Request
Parse Header
Perform Request
Log Request
Seed Paper
� James C. Hu, Douglas Schmidt,
JAWS: A Framework for High-Performance Web Servers,
Adobe Acrobat 7.0 Document
Domain-Specific Application Frameworks: Frameworks Experience By Industry,
John Wiley & Sons, October, 1999.
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 55
Agenda
� Motivation
� Applying Patterns & Frameworks to Web Server� Common Pitfalls of Developing Web Server
� Overcoming Web Server Pitfalls
� Relationship Between Frameworks, Patterns & Other Reuse Technique
� The JAWS Web Server� The JAWS Web Server� Overview of the JAWS Framework
� Overview of the Design Pattern in JAWS
� Concurrency Strategy
� I/O Strategy
� Protocol Pipeline Strategy
� File Caching Strategy
� Web Server Benchmarking
� Concluding Remarks9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 66
� Common Pitfalls of Developing Web Server Software
� Overcoming Web Server Pitfalls � Overcoming Web Server Pitfalls
with Patterns and Frameworks
� Relationship between
Frameworks, Patterns, and Other Reuse Techniques
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 77
2.1 Common Pitfalls
of Developing Web Server SW
� Excessive low-level detail
� Continuous rediscovery & reinvention of
incompatible higher-level programming abstraction.
� High potential for errors� High potential for errors
� Lack of portability
� Steep learning curve
� Inability to handle increasing complexity
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 88
2.2 Overcoming Web Server Pitfalls
with Patterns & Frameworks
� Limitations of Class Library
� do not capture
the control flow and collaboration
among families of related SW components.
� Class Library reuse often
re-invent and re-implement
the overall Software architecture
for each new Application.
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 99
2.2 Overcoming Web Server Pitfalls
with Patterns & Frameworks
� The Benefits of Patterns
� Expert’s Design Experiences
� Good Methods to use Framework
� The infrastructure of Framework
� The Benefits of Frameworks
� Expert’s intellectual products
� Help developers avoid costly re-invention
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 1010
2.3 Relationship Between Frameworks, Patterns,
and Other Reuse Techniques
Class Libraries vs Framework
App SpecificApp SpecificLogicLogic
DATABASEDATABASE ADTsADTs
MATHMATH NETWORKINGNETWORKINGInvocationsInvocations
Class LibraryClass Library
ReactorReactor
NETWORKINGNETWORKING
Active ObjectActive Object
GUIGUI
StateState
MATHMATH
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 1111
OO DesignOO Design
EventEventLoopLoop
GRAPHICSGRAPHICS GUIGUI
SingletonSingleton StrategyStrategy
ReactorReactor AdapterAdapter
StateState Active ObjectActive Object
Class Library Class Library Component ArchitectureComponent Architecture
SelectionsSelections
Design PatternDesign Pattern
DATABASEDATABASE
SingletonSingleton
GRAPHICSGRAPHICS
AdapterAdapter
EventEventLoopLoop
App SpecificApp SpecificLogicLogic
ADTsADTs
CallbacksCallbacks
InvocationsInvocations
Application FrameworkApplication FrameworkComponent ArchitectureComponent Architecture
2.3 Relationship Between Frameworks, Patterns,
and Other Reuse Techniques
� Frameworks define “semi-complete” application
that embody domain-specific object structures and functionality.� Class Libraries don’t offer explicit guidance to system design
� Components in a Framework manage
the canonical flow of control within App
� Frameworks are active and exhibit “IoC (Inversion of Control)” at runtime.� Template Method Pattern (Hollywood Principle)
� In Practice, Frameworks, Class Libraries,
and Components are complementary technologies.
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 1212
� Overview of the JAWS Framework,
� Overview Design Pattern in JAWS.
� 4 Core Components in JAWS� Concurrency.� Concurrency.
� I/O.
� Protocol/Pipeline.
� File Caching.
� JAWS Framework Revisited
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 1313
3.1 Overview of the JAWS Framework
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 1414
3.2 Overview Design Pattern in JAWS
� Strategic Pattern� Message Composition/Decomposition
� Pipe & Filter , Layer, Composite Message
� Dispatcher� Acceptor –Connector / Reactor / Proactor
� Active Object
� Service Configurator
� Tactical Pattern� Strategy
� Adapter
� State
� Singleton
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 1515
Strategic Pattern Overview
DataData
LayerLayer
DecompositeDecomposite
MessageMessage
DispatchingDispatching
LayerLayer
Composite MessageComposite Message
MessagesMessages
UnMarshalingUnMarshaling
MarshalingMarshaling
MessageMessage
3.2.1 Strategic Pattern
� Marshaling / UnMarshaling
� Composite Message / Layer / Pipe & Filter
� Dispatcher � Dispatcher
� Acceptor-Connector / Reactor / Proactor
� Active Object
� Service Configurator
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 1717
Composite Message
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 1818
Acceptor/Connector Pattern
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 1919
Acceptor Pattern
� Acceptor is Factory.
� Creates, Accepts and Activates a new Protocol Handler
Whenever the Event Dispatcher notifies Acceptor
that a connection has arrived from a Client.
Protocol Handler
Protocol Handler
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 2020
Event Dispatcher Concurrency
TaskAcceptor
peer_acceptor_accept()
Protocol Handler
peer_stream_open()
Reactor Pattern
Reactor
handle_events()register_handler(in handle)remove_handler(in handle)
Event Handler
handle_event(in type)get_handle()
handlehandlesetset
ownsowns
dispatchdispatch
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 2121
SynchronousEvent
Demultiplexer
select()
Concrete Event Handler
handle_event(in type)get_handle()
Concrete Event Handler
handle_event(in type)get_handle()
Handle<<uses>><<uses>>
notifiesnotifies
ownsowns
Reactor Pattern� decouples the synch demultiplexing and dispatching logic.
� the Single-Thread Web Server Concurrency model
InputOutput Handler Protocol PipelineInputOutput
Reactive IO Filecache Handle
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 2222
Synchronous IO Reactor
send_file()recv_file()send_data()recv_data()
Asynchronous IO
Initiation Dispatcher
handle_event()register_handler(h)remove_handler(h)
Timer_Queue
schdule_timer(h)cancel_timer(h)expire_timers(h)
Reactive IO Handler
handle_input()handle_output()
IO Handle
Proactor Pattern
Handle
<<uses>><<uses>>
is associated withis associated with
AsynchronousOperation Processor
AsynchronousOperation
Initiator
Completion Handler
handle_event()
<<uses>><<uses>> <<invokes>><<invokes>>
<<uses>><<uses>>
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 2323
Proactor
handle_events()
AsynchronousEvent Demultiplexer
get_completion_event()
Concrete Completion Handler
**
<<enqueues>><<enqueues>>
CompletionEvent Queue
execute_async_operation() async_operation()
handle_event()
<<executes>><<executes>>
<<dequeues>><<dequeues>>
<<demultiplexes & <<demultiplexes & dispatches>>dispatches>>
Proactor Pattern (IOCP)
Socket ASocket A
Socket BSocket B Completion PortCompletion Port
2.1 Create Completion Port2.1 Create Completion Port
5. 1 Send 5. 1 Send Completion Socket InfoCompletion Socket Info
3. Create Thread3. Create Thread
7. Thread Returns Thread Pool7. Thread Returns Thread Pool
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 2424
Socket BSocket B
Socket CSocket C
Completion PortCompletion Port
2.2 Create 2.2 Create Completion QueueCompletion Queue
ThreadThread
ThreadThread
ThreadThread
1. Create Socket1. Create Socket
4. Link 4. Link Socket and PortSocket and Port 5. 2 Send 5. 2 Send
Completion PacketCompletion Packet
Thread PoolThread Pool
6. Process Completion Events6. Process Completion Events
Proactor Pattern� decouples the Asynch demultiplexing and dispatching logic.
� The Asynch variant of the Thread Pool
� Use IOCP (IO Completion Port)
InputOutput Handler Protocol PipelineInputOutput
Asynchronous IO Filecache Handle
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 2525
Synchronous IO Proactor
send_file()recv_file()send_data()recv_data()
Reactive IO
Asynchronous IO
Completion Dispatcher
handle_event()register_handler(h)remove_handler(h)
Timer_Queue
schdule_timer(h)cancel_timer(h)expire_timers(h)
Filecache Handle
Proactive IO Handler
handle_read_file()handle_write_file()
IO Handle
Asynch Op
open()cancel()
Active Object Pattern
Scheduler
Dispatch() Insert()
Proxy
method_1()method_n()
ActivationQueue
Insert()Remove()
createscreates createscreates maintainsmaintains
invokeinvoke
11 1111 11
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 2626
Future MethodRequest
Can_run()Call()
**
Servant
method_1()method_n()
createscreates createscreates maintainsmaintains
ConcreteMethodRequest 1
ConcreteMethodRequest N
Client
write towrite to
obtain result fromobtain result from
Active Object Pattern� decouples method invocation from method execution.
� Thread-per-Request, Thread Pool, Thread-per-Session Model
use Active Object pattern.
Protocol Handler
http_request()ftp_request()
delegates
delegates
for(;;){
m = aq->remove();
dispatch(m); }aq->insert(http)
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 2727
Scheduler
http_request()ftp_request()snmp_request()dispatch()
Activation Queue
insert()remove()
Method Object
ftp_request()snmp_request()
Resource Representation
Protocol Pipeline
delegates
delegates
Service Configurator Pattern
Component
+Init()+Fini()+Suspend()+Resume()
ComponentRepository
Service
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 2828
+Resume()+Info()
CoreComponent
CoreComponent
ServiceConfigurator
Service Configurator Pattern
� Provides System Configuration at Runtime.
� Dynamically optimize, control and reconfigure
the behavior of Web server strategies a installation-time or during run-time.
ServiceFilecache
Protocol Handler
Service Repository
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 2929
+init()+fini()+suspend()+resume()+info()
Protocol Handler
Filter Repository
Cache Strategy Repository
Protocol Pipeline DLL Cache Strategy
Read Request Filter LRU Strategy LFU Strategy
Parse Request Log Request
3.2.2 Tactical Patterns
� Strategy� Configure Cache replacement strategies
without afffecting the core SW Architecture.
� Adapter� Uniformly encapsulate the operation
of synch, asynch and reactive I/O.
� State� Event Dispatcher use State
to seamlessly support different concurrency strategies
and both synch and asynch.
� Singleton� To provide a global point to access object.
� Exist one copy of Cached Virtual File System.
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 3030
Review
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 3131
3.3 4 Components in JAWS
Concurrency
Thread-per-Session
I/O
Synchronous
Cache
LRU
Protocol
&
Filter
Read Request
Parse Request
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 3232
Thread-per Request
Thread Pool
Reactvie
Asynchronous
LFU
Structured Cache
Hinted Cache
Parse Request
Parse Header
Perform Request
Log Request
3.3 Concurrency Strategy
� Design Challenge
� A large portion of non-I/O related Web server overhead is due to
the Web server’s concurrency strategy.
� Key Overhead� Key Overhead
� Synchronization
� Thread/Process Creation
� Context Switching
� Choosing an efficient concurrency strategy
is crucial to achieve high-performance
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 3333
3.3 Concurrency Strategy
� Alternative Solution Strategies
� Thread-per-Request
� Thread-per-Session
� Thread Pool� Thread Pool
� Single-Thread
� Experiments
� JAWS Concurrency Strategy Framework
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 3434
3.3.1 Alternative Solution Strategies
� Concurreny Strategy of Web Servers
� Single Thread
� Roxen
� Process-BasedDynamic FactorsDynamic Factors
Machine LoadMachine Load
# of simultaneous requests# of simultaneous requests Memory UseMemory Use
Memory WorkloadMemory WorkloadProcess-Based
� Apach
� Zeus
� MultiThread
� PHTTD
� JAWS
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 3535
Static FactorsStatic Factors
H/W ConfigurationH/W Configuration OS PlatformOS Platform
Web Server Use CaseWeb Server Use Case
Machine LoadMachine Load Memory WorkloadMemory Workload
Thread-per-Request
� Allows each client request to run concurrently in a separate thread.
� Each Request arrives,
a new thread is crated to process request.
� This design appropriate synch I/O mechanismsThis design appropriate synch I/O mechanisms
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 3636
Event Dispatcher Concurrency
Task
Protocol Handler
Protocol Handler
Acceptor
Protocol Handler
Protocol Handler
Thread-per-Request (cont’d)
� Advantage� Simplicity
� Exploit parallelism on multi-processor platform
� LimitationThe # of running threads may grow without bound, exhausting� The # of running threads may grow without bound, exhaustingavailable memory and CPU resources
� Compatible Fields� Use Light loaded servers with low latency
� Don’t use time consuming task based system
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 3737
Thread-per-Session
� Allows each client connection to run concurrently
� All of these requests are submitted through a connection
between each client and a separate thread in the web server process
� Advantage� Advantage
� Does not spawn a separate thread for each request.
� Limitation
� This model still vulnerable to unbounded resource consumption as the # of clients grows.
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 3838
Thread-per-Session (cont’d)
� Suited for HTTP 1.1 but not HTTP 1.0
� HTTP 1.0� support open/operation/close
� One URL Fetch per Connection� Needs Reconnection
� Transporting Data Size Limitation� Transporting Data Size Limitation
� URL Size Limitation
� HTTP 1.1� Persistent Connection
� Pipeline Functionality � Many Requests -> Support Serial Response
� Compressed Data Transporting
� Use Proxy Server and Cache
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 3939
Thread Pool
� Allows up to N requests to execute concurrently within a
pool of threads.
� Support prespawning of Thread Creation
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 4040
Event Dispatcher Concurrency
Task
Protocol HandlerProtocol Handler
Acceptor Protocol Handler
Protocol Handler
Thread Pool (cont’d)
� Advantage
� Lower overhead that Thread-per-Request.
� Thread Prespawning
� thread creation cost is amortized
� Pool Size is Fixed
� Resource Consume is bounded
� Limitation
� Small Pool Size
� New incoming requests be dropped or wait indefinitely.
� Large Pool Size
� Resource Consumption may be no better than using Thread-Per-Request
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 4141
Single-Threaded
� All connections and requests
are handled by the same thread of control.
� Single-Threaded servers process requests iteratively.
� Compatible Filed
� Asynch or Reactive I/O Multiple Request
� Inadequate for high volume server
� Subsequent requests are blocked until they are reached
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 4242
Experiments
� Static adaptivity
� Select the concurrency strategy
that best meets
The static requirements
of the system.
� Dynamic adaptivity
� Adapt dynamically
to current server condition
� multi-threaded concurrency
� More suited to
A multiple-processor machinethan
a uni-processor machine
� In order to accommodate Unanticipated
load
� Support to increase
the # of available threads in the Pool
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 4343
JAWS Concurrency Strategy
FrameworkEvent Dispatcher
dispatch()
Concurrncy
Task
Protocol Handler
open()
Acceptor
accept()
server->dispatch()
serverserver
tasktask
acceptoracceptor
handlerhandler
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 4444
Concurrncy
dispatch()
accept()activiate()enque()deque()svc()
Thread Pool
dispatch()
Thread-per-Connection
dispatch()
task->accept()
task->activate()
task->enque()
acceptor->accept()
3.4 I/O Strategy� Design Challenge
� is to devise efficient data retrieval and delivery strategies,
collectively referred to as I/O.
� Key Overhead
� Arranging Multiple I/O operation.
� Example
� Synch
� Web Transaction involving monetary fund transfer
� ASynch
� Web access to static information (Web page – ASP..)
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 4545
3.4 I/O Strategy
� Alternative Solution Strategies
� The Synchronous I/O Strategy
� The Reactive I/O Strategy
� The Asynchronous I/O Strategy� The Asynchronous I/O Strategy
� Experiments
� JAWS I/O Strategy Framework
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 4646
The Synch I/O Strategy
� Synch I/O describes the model of I/O interaction between a Web server process and kernel.
� Kernel doesn’t return the thread of control to the server until the requested I/O operation either complete , completes partially, or fails.
� Advantage (Easy Use)
� Disadvantage (Resource Exhaust)� If combined with a single threaded concurrency strategy,
It is not possible to perform multiple synch I/O operation simultaneously.
� When using multiple thread,
It is still possible for an I/O request to block indefinitely.
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 4747
The Reactive I/O Strategy
� Reactive I/O alleviates the blocking problems
of synchronous I/O without resorting to polling.
� Web Server uses
an OS event demultiplexing system call.
� select() in UNIX
� WaitForMultipleObjects() in Win32
� Refers Reactor Pattern
� Disadvantage
� May not make effective use of multiple CPU.
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 4848
The Asynch I/O Strategy
� Asynch I/O simplifies
the de-multiplexing of multiple events
in one or more thread of control
without blocking the Web server.
� Advantage� The Web Server not block on I/O request.
� Disadvantage� Asynch I/O is not available on many OS Platform.
� more complicated than writing synch program.
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 4949
Experiments
� The Result reveal that
each I/O strategy behaves differently
under different load condition.
� JAWS I/O Strategy Framework
addresses this issue
by allowing the I/O Strategy
to adapt dynamically
to run-time server condition.
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 5050
JAWS I/O Strategy Framework
Perform Request
svc()
fhfhInputOutput Handler
receive_file()receive_data()send_file()send_data()
InputOutput
receive_file()receive_data()send_file()send_data()
Filecache Handle
iohioh
ioio
ioioio->send_file(“index.html”)
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 5151
complete()error()
send_data()
Synch IO
receive_file()receive_data()send_file()send_data()
Asynch IO
receive_file()receive_data()send_file()send_data()
Proactor
Reactive IO
receive_file()receive_data()send_file()send_data()
Reactor
write(fh)Proactor->
send_file(fh)
Reactor->
send_file(fh)
3.5 Protocol Pipeline Strategies
� Design Challenge� Early Web Server
� Simply retrieved the requested file.
� Transferred its contents to the request.
� Modern Web Server� Modern Web Server� Support Various File Format� text, image, audio, video.
� Introducing Server Side Engine (ASP, JSP..)� search engine, map generation, interfacing database,
security, financial transaction.
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 5252
3.5 Protocol Pipeline Strategy
� Alternative Solution Strategies
� Pipeline.
� Reflection + Component Configurator.
� Examples� Examples
� Advanced Search Engine.
� Image Servers.
� Adaptive Web Contents.
� JAWS Protocol Pipeline Strategy Framework
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 5353
Using PipelineHTTP/ 1.0 200 OK
Date: Thu, 09 Oct 01:26:00 GMT
Server: Eva/1.0
Last-Modified: Wed, 08 Oct 1997 ..
Content-Length : 12345
Content-Type : text/html
<HTML>
<TITLE> Homepage </TITLE>
..
</HTML>
CONNECTION
USER_AGENT
HOST
ACCEPT
ACCEPT
ACCEPT
Keep-Alive
Mothra/0.1
any.net
image/gif
image/jpeg
*/*
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 5454
PerformPerform
RequestRequest
LogLog
RequestRequest
any.net - -
[09/Oct/1997:01:26:00 -0500]
“GET /~jxh/home.html HTTP/1.0
ReadRead
RequestRequest
GET /~jxh/hoem.html HTTP/1.0
Connection: Keep-Alive
User-Agent: Mothra/0.1
Host: any.net
Accept: image/gif, image/jpeg, */*
ParseParse
RequestRequest
GET
/users/jxh/public_thml/home.html
HTTP /1.0
ParseParse
HeadersHeaders
Static vs Dynamic
� Static Configuration� HTTP 1.0 Request
� limited # of processing operation
� Marsharing & Demarsharing
� Demutlipelxing� Demutlipelxing
� Dynamic Configuration� Unlimited # of Processing operation
� Requires a lot of process arranging cost.
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 5555
Example
� Advanced Search Engine
� May construct data filters dynamically.
� “(perfomrnace AND (NOT symphonic))”
� Requesting for Web pages containing the word
“performance” but not the word “symphonic”.
� Could be implemented as a pipeline of
� a positive match component coupled with
� a negative match component
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 5656
Example (cont’d)
� Image Servers
� User may request that
� Image be cropped, scaled, rotated, and dithered.
� Adaptive Web Contents� Adaptive Web Contents
� Dynamic Delivery for End User.
� PDA User
Web Content Overviews and Smaller Image.
� Workstation & Personal Computer
Full Multimedia Enriched Pages.
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 5757
JAWS Protocol Pipeline Strategy
Protocol Handler InputOutput Handler
Protocol Pipeline
svc()
Read Request
svc()
Filter
component->svc()io->receive_data()
pipepipe ioio
componentcomponent
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 5858
svc()
Filter
svc()parse()
Parse Headers
svc()parse()
Perform Request
svc()perform()
Log Request
svc()log()
Filter::svc()
parse()
Filter::svc()
perform()
Filter::svc()
log()
3.6 File Caching Strategies
� Design Challenges� Accessing the File System is
a significant source of web server overhead.
� Cache is a storage medium
that provides more efficient retrieval.
� Alternative Solution Strategies� Least Recently Used (LRU) Caching.
� Least Frequently used (LFU) Caching.
� Hinted Caching.
� Structured Caching.
� JAWS Cached Virtual File System Framework
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 5959
Alternative Solution Strategies
� Least Recently Used Caching
� Least Frequently Used Caching
� Structured Caching
� Refers the Next Page.� Refers the Next Page.
� Hinted Caching
� Analysis of Web page retrieval pattern.
� Web pages have spatial locality.
� Using statistical information about links (hinted).
� Using Pre-fetching Method.
� Refers “Hinted Caching in the Web”9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 6060
Alternative Solution Strategies
� Structured Caching
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 6161
JAWS Cached Virtual FS Framework
InputOutput
receive_file()send_file()
filename
InputOutput
handle()mapaddr()name()size()open()
Filecache Object
handle()mapaddr()name()size()
fhfh fofo
fh->open(file)cvf->open(file,fo)
cvfcvf
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 6262
HashTable
Filecache
find()fetch()remove()create()findish()
rw_lock[]
Cache Strategy
insert()
LRU Strategy
insert()
LFU Strategy
insert()
replace least
recently used
Replace least
Frequently used
hthtcachecache
JAWS Cached Virtual FS Framework
Perform Request
svc()
fhfhInputOutput Handler
receive_file()receive_data()send_file()send_data()
InputOutput
receive_file()receive_data()send_file()send_data()
Filecache Handle
iohioh
ioio
ioioio->send_file(“index.html”)
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 6363
complete()error()
send_data()
Synch IO
receive_file()receive_data()send_file()send_data()
Asynch IO
receive_file()receive_data()send_file()send_data()
Proactor
Reactive IO
receive_file()receive_data()send_file()send_data()
Reactor
write(fh)Proactor->
send_file(fh)
Reactor->
send_file(fh)
3.7 The JAWS Framework Visited
Concurrency
Thread-per-Session
I/O
Synchronous
Cache
LRU
Protocol
&
Filter
Read Request
Parse Request
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 6464
Thread-per Request
Thread Pool
Reactvie
Asynchronous
LFU
Structured Cache
Hinted Cache
Parse Request
Parse Header
Perform Request
Log Request
� Hardware Testbed
� Software Request Generator
� Experimental Results
� A Summary of Techniques
for Optimizing Web Servers.
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 6565
4.1 Hardware Testbed
� System Specification
� Two Micron Millerinia PRO2 plus workstation
� 128M RAM
� Client – 200 MHZ
� Server – 180 MHZ� Server – 180 MHZ
� ENI-155P-MF-S ATM Card
� ATM Network running through
� a FORE Systems ASX-200BX (bandwith 622 Mbps)
� Limitation of LAN emulation mode = 120 Mbps
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 6666
4.2 Software Request Generator
� Experiment Purpose� Using WebSTONE Stress Tool
� Collect Client and Server Side Metric
� average server throughput, client latency
� Gather Loads and Statistics
� using various particular files sizes
� Refers the next page.
� Using various concurrency & dispatching strategies
� research the impacts of results.
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 6767
4.2 Software Request Generator
� File Access Patterns
Document Size
500 bytes
Frequency
35%
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 6868
500 bytes
5 Kbytes
50 Kbytes
5M Bytes
35%
50%
14%
1%
4.3 Experimental Results
� Using 3 Models
� Definition of Terms
� Throughput Comparisons
� Latency Comparisons� Latency Comparisons
� Summary of Benchmark Results
� A Summary of Techniques
for Optimizing Web Servers
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 6969
4.3 Experimental Results
� Using 3 Models
� Synchronous Thread-per-Request
� Using Synchronous I/O
Synchronous Thread Pool� Synchronous Thread Pool
� Using Synchronous I/O
� Asynchronous Thread Pool
� Using TransmitFile() for Asynchronous I/O
� TransmitFile() is a custom Win32 Function()
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 7070
4.3 Experimental Results
� Definitions of Terms� Throughput
� The average # of bits received per second by client.
� Start Time
� HTTP Request received by client.
� End Time
� Connection is closed at the client end.� Connection is closed at the client end.
� Latency
� The average amount of delay in milliseconds
� Start Time
� HTTP GET Request received by client.
� End Time
� Completely Received File.
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 7171
4.3 Experimental Results
� Throughput Comparisons.
� Connection ↑ - Throughput ↓
� As expected,
The throughput for each connection degrades
as the connections per second increase.
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 7272
4.3 Experimental Results
� Throughput Comparisons.� Refer Figure 3.
� Experiment Results from 50K Files.
� Thread-Per-Request Model� Thread-Per-Request Model
� Connection load ↑ -> Throughput ↓ rapidly
� Incurs higher thread creation overhead.
� Synchronous Thread Pool
� Connection load ↑ -> Throughput ↓ more gracefully
� Uses Pre-spawning thread.
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 7373
4.3 Experimental Results
� Throughput Comparisons.
� Refer Figure 1 through 5.
� TransmitFile() performs extremely poorly
for small files ( i.e. < 50K bytes)
� TransmitFile() is forced
� to wait while the kernel services incoming request.
� As the Size of the file grows,
� TransmitFile() rapidly outperforms the synch model.
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 7474
4.3 Experimental Results
� Throughput Comparisons.
Fig 1. 500 Byte File
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 7575
Fig 1. 500 Byte File
Fig 2. 5K Byte File
4.3 Experimental Results
� Throughput Comparisons.
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 7676
Fig 3. 50K File
Fig 4. 500K File
4.3 Experimental Results
� Throughput Comparisons.
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 7777
Fig 5. 5M File
4.4 A Summary of Techniques (for Optimizing Web Server)
� Lightweight Concurrency� Don’t use Process-based concurrency Mechanism.
� Using POSIX thread.
� Specialized OS Feature� Using IOCP� Using IOCP
� Request Lifecycle system call overhead.� Reducing synchronization.
� Caching files.� Reducing Iterative System I/O Call (open, read, write..)
� Using “gather-write”� Using writev (Unix System Call)
� Pre-computation HTTP response
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 7878
4.4 A Summary of Techniques (for Optimizing Web Server)
� Logging Overhead� File System access
� Makes a significant number of I/O Calls.
� Synchronization overhead� Thread/Process are required to log request to a common shared log file, � Thread/Process are required to log request to a common shared log file,
access to the log file needs to be synchronized.
� Reverse hostname lookups� Hostname is typically more useful information in the log file.
� Ident Lookups � For identifying User Name
� Make setting up a new TCP/IP Connection
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 7979
4.4 A Summary of Techniques (for Optimizing Web Server)
� Transport layer optimizations
� The listen backlog
� Give Queue Size Higher.
� Socket Send buffers� Socket Send buffers
� Should be set to the highest permissible limit.
� On Solaris, 64KB limit
� Nagle’s Algorithms
� Congestion Control Algorithm
� Server latency critical App (such as X-Windows)
disable this algorithm.
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 8080
5. Concluding Remarks
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 8181
References
� POSA1
� Layer , Pipe & Filter
� POSA2� POSA2
� Component Configurator , Active Object,
Dispatcher (Reactor, Con-Acc, Proactor)
� Composite Message Pattern
9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 8282