Presentation Slides

MD850: e-Service OperationsMD850: e-Service Operations

Capacity Management

OverviewOverview

Motivation Background Capacity Management

Manufacturing Operations Person-to-Person Service Operations e-Service Operations

Capacity Management Strategies Software Demonstration

MercuryInteractive’s Astra Load Test

MotivationMotivation


Many e-Service Problems Exist that Affect Service Quality and Hurt Revenues Downtime Slow times (service slowdown)

37% of site pages exhibit unacceptable performance as defined by their managers (Mercury Interactive 2001)

Unpredicted traffic volume spikes Transaction failure

1 in 20 transactions fail on the average corporate website (Mercury Interactive 2001)


52% of e-Service applications failed to scale according to original design expectations (Newport Group 1999) Automated load testing tools were not used

60% used no load testing tool prior to deployment 34% used load testing late in development or post deployment

only Budget and time overruns above average

23.5 days, $67,083 Scalability expectations were not realistic

Thought they would handle 4300 concurrent users on average Actually could only handle 2200 concurrent users on average

BackgroundBackground

BackgroundBackgroundCapacity ConceptsCapacity Concepts

Capacity rate at which output can be produced by an operating unit (e.g., a

machine, a process, a facility, a company) capacity = [number of units of output]/[time period]

Design Capacity maximum rate at which the process can operate on a continual

basis under ideal conditions

Effective Capacity rate of production that can be achieved for extended periods under

normal conditions, taking into account … [various infrastructure and environmental factors]

BackgroundBackgroundCapacity ConceptsCapacity Concepts

Capacity Utilization= [actual output]/[design capacity]

Capacity Efficiency= [actual output]/[effective capacity]

0

DesignCapacity

EffectiveCapacity

Actual OutputPeriod #5

Actual OutputPeriod #8

Managerial Focus On:Loss in Production

Capability

Capacity Management in Capacity Management in Manufacturing OperationsManufacturing Operations

Capacity ManagementCapacity ManagementManufacturing OperationsManufacturing Operations

Supply Chain Processes Supply Chain Modularity Facilities Facility Location Facility Focus Process Design

Product Design Product Variety Product Quality Production Scheduling Materials Management Maintenance Job Design and Personnel

Management


Common Approaches Forecasting demand Translate demands (d1, d2, …, dn) for products (1, …, n)

into capacity requirements Break-Even Analysis Decision Analysis


Demand-Side Management of Capacity Counter-cyclic products

Toro - lawnmowers, snowblowers

Promotion Pricing


Medium-term to Short-term approaches

Aggregate planning chase demand strategy level demand/workforce

strategy

Inventory management Materials management Operations scheduling Personnel scheduling

Typical capacity analysis & management tools

Optimization

MRP … CRP Inventory review

Queueing Simulation Optimization


Capacity Strategies Facility

Product-Organized: facility makes one product type Process-Organized: multiple products or parts made through one process Market-Organized: close physical proximity to the customer Focused Factory/Plant-Within-A-Plant

Capacity Expansion Demand-Leading: maintain excess capacity Demand-Trailing: capacity lags behind demand Demand-Matching: match capacity to demand Steady Expansion: add capacity on regular basis, based on long-term

needs, but not on demand fluctuations

Capacity Management in Person-Capacity Management in Person-to-Person Service Operationsto-Person Service Operations

Capacity ManagementCapacity ManagementPerson-to-Person ServicesPerson-to-Person Services

Capacities to Manage Facilitating Goods

When produced internally, same issues as manufacturing Often procured from supplier

Services Persons who can deliver service at a certain rate

Information Printed information … either inventoried, or printed on demand at

some rate Communicated in person at some rate


Demand Side Management of Customer Behaviors Appointment reminders Pricing

penalty for being late yield-management

Wave scheduling Capacity sharing

One printer, 20 professors Ask customer to stop consuming/conserve

Energy efficient light bulbs Stop filling demand/shut off service

California electric service


Service Facility Design Queuing model of service system queues Simulation models to study activities within system and

to estimate effective capacity, service times, and so forth

Daily/Weekly Capacity Management Service Personnel/Staff Scheduling

Nurses, police, paramedics optimization - integer linear programming

Capacity Management in Capacity Management in e-Service Operationse-Service Operations

Capacity ManagementCapacity Managemente-Service Operationse-Service Operations

e-Service Capacities to Manage Digital network capacities Goods network capacities Person-to-Person service network capacities Inter-layer digital service capacities

Service Personnel

Service Networks, Intelligent Goods Networks

Digital Networks & Digital Content

Service-Product Service-Process


Digital Networks

Networks of Physical Objects

People

Client/ Server

DistributedComponentApplications

Service-Product Service-Process

Info:Digital Content

Capacity ManagementCapacity Managemente-Service Capacitye-Service Capacity

Determinants of e-Service Capacity Number of customers and internal users

demand

“Service-Product” site content

“Service-Process” server capacity and configuration of hardware and

software

Capacity ManagementCapacity ManagementDeterminers of e-Service CapacityDeterminers of e-Service Capacity

Desired Capacity = Average

Digital ContentDemand Per User

x Number of UsersPer Time Period

= Load on HardwarePer User

/ Number of Supported

Concurrent UsersHardwareCapacity

(design)

Goods InfoServices Info

Digital ServicesContent

Consumed Service-Product

Web browsersWireless apps

Other processes (e.g., ERP system)User Mix

ServerNetwork Infrastructure

Software ModulesService-Process



Common e-Service Problems and Causes Long response time from end users’ point of view Long response time as measured by the servers Memory leaks High CPU usage Too many open connections between the application and end users Lengthy queues for end user requests Too many table scans of the database Erroneous data returned HTTP errors Pages not available


Typical e-Service Infrastructure Managed to Achieve Desired Capacity Application Server Software Web Server Software Database Software Networking Software Load Balancing Software Application Server Hardware Web Server Hardware Database Hardware Networking Hardware

Capacity ManagementCapacity ManagementStrategies for e-Service CapacityStrategies for e-Service Capacity

Strategies for Managing e-Service Capacity Manage demand Manage “Service-Product” Manage “Service-Process”

Managing e-Service Demand Managing e-Service Demand PatternsPatterns

Capacity ManagementCapacity Managemente-Service Demandse-Service Demands

6am 12pm 6pm 12am 6am 12pm 6pm 12amDay 1 Day 2

6am 12pm 6pm 12am 6am 12pm 6pm 12amDay 1 Day 2

Demand Surge

CyclicalRandom & Infrequent

Managing the e-Service ProductManaging the e-Service Product

Capacity ManagementCapacity Managemente-Service Reference Model e-Service Reference Model (Menasce and Almeida, (Menasce and Almeida, Scaling for e-BusinessScaling for e-Business, 2000), 2000)

Characteristicsof the Business

NavigationalStructure &

Function

Patterns of CustomerBehavior

Site Architectureand

Service Demands

BusinessModel

FunctionalModel

CustomerModel

ResourceModelTechnological

View

BusinessView

InternalMetrics

ExternalMetrics

Capacity PlanningCapacity PlanningTranslating the Service-Product into Loads Translating the Service-Product into Loads

Characteristicsof the Business

NavigationalStructure &

Function

BusinessModel

FunctionalModel

Service-Product

GoodsServices

Information

WWW SiteContent

A Network of Paths BetweenPages/Objects

Heim andSinha (2000),Spiller and

Lohse (1998)

Menasce andAlmeida (2000)

Managing the e-Service ProcessManaging the e-Service Process

Capacity ManagementCapacity Managemente-Service Process Strategiese-Service Process Strategies

Implications of Classic Aggregate Planning Strategies chase demand strategy

add a server when you sense it is needed take away a server when it is not needed

level demand strategy build an inventory of e-Services (IMPOSSIBLE)


Implications of Classic Capacity Expansion Strategies Demand-Leading: maintain excess capacity

expensive solution service level stays reasonable customers satisfied

Demand-Trailing: capacity lags behind demand queue of requests builds service slows as servers average capacity across all requests server computers grind to a halt when demand severely surpasses

capacity … time to purchase new servers dissatisfied customers


Implications of Classic Capacity Expansion Strategies Demand-Matching: match capacity to demand

as demand grows (shrinks), in-house (and outsourced) servers are dynamically added (removed)

capacity for static files (GIF, JPEG, .EXEs, etc.) outsourced to a third-party

as demand grows (shrinks), third-party senses growth and re-allocates files across a larger (smaller) subset of their network

you manage basic content (e.g., HOME.HTML file) caching strategy (Akamai, Inktomi, others vendors)

Steady Expansion: add capacity on regular basis, based on long-term needs, but not on demand fluctuations intermittent periods of poor service responsiveness if demand grows too fast, system may fail, customers dissatisfied

e-Service Capacity Planning, e-Service Capacity Planning, Analysis, and ManagementAnalysis, and Management


Best Practices Set conservative goals with the intent to expand them out in a

systematic fashion Have a plan for short-term incremental achievements of long-term

goals Integrate load testing into the development process of web

applications early and often Always test the limits of any e-Service application prior to going live

Leverage pre-deployment test assets in the production environment to monitor live web application performance

Set a load capacity threshold for growth which is continuously monitored

At 70-80% of the existing system’s handling capacity, start executing plans to add more capacity

(Source: Newport Group 1999, 2000)


Best Practice Objectives Minimized Latency

Keep waiting time between making a request and beginning to see a result as low as possible

Maximized Throughput Number of items potentially processed per unit time should

be as high as possible Mid-to-high Utilization

Actual capacity utilization of components should be kept around 75% … latency suffers as you go over this level

Maximized Efficiency Keep overall performance high and cost low


Best Practice Metrics End User Response Time

Measures the performance of an application from the end-user perspective

The amount of time required for an end user to receive a response or to execute a business transaction

Application Availability Page availability Transaction availability User-perceived availability

Reliability Correct Content Delivery in Multi-Step Transactions

(Sources: Newport Group 1999, Mercury Interactive 2001)


Best Practice Guidelines1. Focus on critical business processes2. Use the right monitoring solution to meet your business needs3. Get a consistent performance baseline and watch for trends4. Avoid an alerting flood …send alerts conservatively5. Think “recurring” when acting on alerts6. Correlate end-user performance with back-end issues7. When a problem is identified, prioritize IT resources8. Optimize your existing infrastructure9. Define escalation procedures to follow to address performance

issues10. Use application performance monitoring to avoid having every

department scramble when performance goes bad

(Source: Mercury Interactive, 2001)

Capacity Management Techniques Capacity Management Techniques During e-Service Design and During e-Service Design and DevelopmentDevelopment

Capacity ManagementCapacity ManagementDesign & Development StageDesign & Development Stage

Basic Question: How to design communication between e-Service application layers? Capacity Issue

How to set up appropriate connections between tiers? How to avoid too few connections between tiers? How to avoid bad effects from failure of a server in one of the

tiers? Methods

Separation of presentation logic from business logic from database management

Load balancing Outsource network storage for caching of content


Commonly Observed Design Problems General

Insufficient memory Incompatible service packs and application extensions (e.g., .DLLs) Excessive queueing requests Too many secure HTTPS connections in use

Web Server Insufficient memory Poor web server design High CPU usage

Application Server Poor cache management and high CPU usage Lack of memory Poor session management Poor database tuning


Commonly Observed Design Problems Database

Inefficient indexing Fragmented databases Out-of-date statistics Faulty application design

Network Inadequate Internet pipe Hidden bottlenecks between the customer’s Web site and the

ISP Faulty hops (misdirected traffic and lost packets) Misconfigured software and incompatible hardware

(Source: Mercury Interactive 2001)


Load Balancing Objective is to be able to allocate demand – as it is requested – to

resources that will provide best service response Many sites use N-tier systems with many servers in each tier that

perform same functions Reliability Backup in case of failure and when server maintenance must be

performed Possible techniques:

RANDOM DISTRIBUTION – Round-robin (random) allocation of a customer request to an IP address for a web server. If a server fails, customer requests will still be allocated to it until its IP address is removed from service.

INTELLIGENT DISTRIBUTION – Customer requests are allocated to servers based upon current utilization at each server. The lowest utilization server will receive the next job.


Outsource network storage for cache capacityAkamai, Inktomi, etc. provide services for

storing and serving static content Many of these companies are also setting up

caching procedures for dynamic content


Basic Question: Theoretically, how many people can my homepage be served to simultaneously? Capacity Issue:

What is my website’s design capacity? Back of the Envelope Method:

Home page (average) size: 50,000 bits = 6250 bytes Rented T3 line = 45,000,000 bits/sec [45,000,000 bits/sec] / [50,000 bits/customer] =

900 happy customers/sec … the “design capacity” 900 customers/sec * 60 * 60 * 24 =

77,760,000 happy customers/day assuming 900 customers request the page simultaneously at the beginning of each second, and

each has a fast enough modem to receive the file by the end of the second


Load Analysis and Testing Tools Brute Force Calculations

Queueing theory

Brute Force Load Testing Thousands of employees at their computers

Simulation-Based Load Testing & Monitoring Software Load-Test: “Can our site handle 25 users?” Stress-Test: “Stable? Reliable? Over long period?” Capacity-Test: “Maximum number of concurrent customers?”


Simulation Based Load Testing Employ virtual (i.e. fake) users on client computers Have virtual users use the e-Service system based on

scripted behaviors recorded for them by the load testing program

Collect data about the performance that the virtual users experience

Modify service process depending upon the typical performance observed


Major Vendors & Applications Mercury Interactive

LoadRunner + WinRunner + TestDirector Astra LoadTest + Astra Quicktest + Astra SiteManager

Segue SilkPerformer

Empirix Compuware Radware


Development Deployment

DeploymentDeployment

Software Product

Service

M.I. LoadRunner, etc.

M.I. Astra LoadTest, etc.

Segue SilkPerformer

Capacity/Load Test Monitoring

Capacity Management Techniques Capacity Management Techniques After e-Service DeploymentAfter e-Service Deployment

Capacity ManagementCapacity ManagementAfter e-Service DeploymentAfter e-Service Deployment

General Patterns As richness of content increases, capacity decreases

(for a fixed service process) As customers converge to more actively involved

behavior types (are retained, and thus consume more content), capacity decreases (for a fixed service process)

As network of site paths increases in complexity, the complexity of managing capacity will increase


General Responses Decrease content richness (when appropriate) Make customers more efficient in their

behavior Manage site complexity


Content TuningServe out content at a lower level of richness, or

remove some contentBy lowering the number of bytes for each file, less

capacity will be used by each individual userSaved capacity can then be used to serve

additional customers


Content Tuning Possible techniques:

DESIGN TIME – Create separate sets of high resolution and low resolution images for your e-Service, and program your site so that you can change which images are dynamically inserted into your pages

DESIGN TIME – Create a program that loads an image directory on the web server with high or low resolution images, depending upon present demands

RUN TIME – Use a program procedure or filter to pre-process each image to a lower resolution, as the image is requested. Once the image has first been processed, the filter could just serve out the lower resolution image. (Note: this approach lowers image bandwidth but increases page processing time.)

RUN TIME – Change heavily downloaded page from dynamic content (3.5 seconds on average to generate and then download) to static page (1.5 seconds on average to download)


Managing customers’ efficiency New customers – difficult to do

Better site design as you figure out how new customers behave

Longer-term customers/users – should experience a learning effect that will cause them to be more directed in their activities

They may consume more content per visit as site stickiness keeps them around

They may consume less content if they become more efficient at shopping and other tasks on the site


Basic Question: Is resource X running? Capacity Issue:

Has server failed? Has network router failed?

Method: Systems performance monitoring tools for each infrastructure

component Tend to monitor and provide information in a stovepipe

fashion, but useful if you need to know about a specific resource

Note – system monitors can report that each individual component is running fine, yet the system overall can exhibit poor performance


Basic Question: Is my site running? Capacity Issue:

Is capacity > 0 right now? How long has it been 0?

Method: Simple site monitor, similar to “ping” service on any computer

Example: ArsDigita.com’s free “Uptime” web site monitoring service ... Every

15 minutes, it tries to download a text file called “http://www.mysite.com/textfile.txt” from your web site

If it fails, it emails you that your site is down


Basic Question: What portions of my “service-product” are popular? Capacity Issue:

Which files are being requested frequently? Which content configurations are requested frequently? Which processes deliver that content? Am I paying too much for my ISP service contract? Can I get by on a

lower-bandwidth contract? Method:

Site log file analysis Add up all http: transactions made to your web site during some time

period


Log File AnalysisAnalyze http requests stored on the serverTranslate requests into aggregate historical

demands for certain time period bucketsVendors

Webalizer (www.mrunix.com/webalizer) NetTracker (www.sane.com/products/NetTracker) many more


Customer Profiling (via data mining)Translate customer behaviors within system

into common customer profilesLink profiles to the resources that are required

to service the respective activities

Network ofSite Paths

Subset of Paths Having Some Relationship (Business Activities) Within an E-Service

WWW Site Log(http GET, page referrals between pages )

Subset #1of paths commonly

visited together


visited together


visted together

“DataMining”

is made up of ...

has some business meaning, and is

stored in ...

Three Consumer Types … 1, 2, and 3

additional analysis

Subset #1 … “browse only”Subset #2 … “directly buy”Subset #3 … “browse, then buy”


Translating the service product

into demandloads

WWW SiteContent

Content Located Together Within an E-Service

WWW Site Log(http GET commands for pages visited)

Goods XServices XContent X

Goods YServices YContent Y

Goods ZServices ZContent Z

“DataMining”

is made up of ...

has some business meaning, and is

stored in ...

Three Consumer Types … X, Y, and Z

additional analysis


Translating the service product

into demandloads


Basic Question: How do popular areas of my site affect my capacity?Capacity Issue:

How to link common behaviors to specific capacity providing resources?

Methods Queueing models Load testing tools Site monitoring tools


Enter

Home Page

Search Page

Add to Cart PayPage 1 Page 2

Page 3 Page 4

prob. = 0.2

prob. = 0.5

prob. = 0.3

1.0 0.3

0.30.6

0.1

0.4

0.8

0.4

0.1

Exit

0.0 1.0

Business Activity = “Browse”

Identify user navigation paths within e-Service … then use queueing theory

equations to determine long-run impact of various paths


Enter Home Exit

Enter

Enter Home Browse Exit

Browse Add to Cart Pay Exit

Three Simple Consumer Behavior “Types”Observable in Navigation Model

Identify user navigation paths within e-Service … then use queueing theory

equations to determine long-run impact of various paths


Web Application Performance MonitoringWeb application monitoring tools work to send

up red flags when the application under surveillance fails to meet its performance objectives


Web Performance Monitoring Active (synthetic) monitoring

Identify several key transactions in the e-Service Create an emulated (or simulated) client for each key transaction Execute transactions against the simulated client on a regular

basis Measure transactions in detail and collect service availability data If performance violates some predetermined threshold, send an

alert to e-Service manager


Web Performance Monitoring Passive (observational) monitoring

Collect data from actual end user activity and store it in a database

Analyze data for patterns Identify several key transactions in the e-Service Measure transactions in detail Generate performance metrics Link performance to activities experienced within the e-Service

system


Based on actual end user data High degree of detail Capture client processing time Aids in root-cause analysis

Active Passive

Data collected opportunistically Reactive by nature … to real

customer problems Potential for collecting excessive

amount of data

24 x 7 monitoring Constant controlled data collection Data collection provides baseline Proactive in nature

Not based on true end-user data Limited insight into the back-end

infrastructure performance Difficult to execute “real”

transactions Creation/maintenance of scripts

Advantage

Drawback


Site Monitoring Software/Services Perform Systems Analysis

Manager must understand the current system architecture Translate architecture into load testing objectives Define input data for testing Choose a testing/monitoring strategy

Create Virtual User Scripts Record scripts that virtual users will use to interact with the e-

Service system Define User Behaviors

Specify how the virtual users will behave (random think times, browser types to use, dial-up speeds, etc.)


Site Monitoring Software/Services Create a User Scenario

A scenario is a set of user behaviors combined together to test a web site

Monitor Performance Run the user scenario Collect data (client, network components, hardware

components, server/software components) on performance for user requests

Analyze Performance Statistics Graphs



Prominent Vendors & ApplicationsSoftware Products

Mercury Interactive Topaz Segue SilkPerformer

Services Mercury Interactive ActiveWatch (Topaz-based

service) Mercury Interactive ActiveTest (LoadRunner-based

service)


“Test on Demand” ServicesDeliverables

Consulting time Set of tailored test scripts for an e-Service Application test run Final report providing details about the tests Suggestions for areas of e-Service process/

infrastructure in need of improvement


Development Deployment

DeploymentDeployment

Software Product

Service

M.I. LoadRunner, etc.

M.I. Astra LoadTest, etc.

Segue SilkPerformer

M.I. Topaz

M.I. ActiveWatch

GomezNetworks

M.I. ActiveTest

Capacity/Load Test Monitoring

Software DemonstrationSoftware Demonstration

Software DemonstrationSoftware DemonstrationMercury Interactive Astra LoadTestMercury Interactive Astra LoadTest

Some basic capabilities Record customer activities; save as “script”

Activities that have meaning Content consumption … “service-product” subset Economic activity … “checkout system” Network paths … typical customer paths through system

Site and/or service parameters … automatically iterate through subset of or all possible values

Multiple-option click boxes Drop-down options

Combine multiple customer activities into a demand “load scenario”

Software DemonstrationSoftware DemonstrationMercury Interactive Astra LoadTestMercury Interactive Astra LoadTest

Some basic capabilities Run scenario using actual web site or n-tier service process

Prior to going “live” After going “live”

Data collection Content transaction Process technology “monitors”for typical

WWW servers & hardware off-the-shelf WWW component software

Data analysis Averages, stdev, graphs Drill-down Data export

Documents

Presentation Slides