May 20, 2009Innovations in Measurement Science
1
Measurement Science for Complex Information Systems
Measurement Science for Measurement Science for Complex Information SystemsComplex Information Systems
D. Cho, C. Dabrowski, J. D. Cho, C. Dabrowski, J. FillibenFilliben, J. , J. HagedornHagedorn, C. , C. HouardHouard, F. Hunt, D. , F. Hunt, D. GeninGenin, V. , V. MarbukhMarbukh and and K. MillsK. Mills
and various studentsand various students
Innovations in Measurement ScienceInnovations in Measurement Science MayMay 20, 200920, 2009
May 20, 2009Innovations in Measurement Science
2
Measurement Science for Complex Information Systems
Plan for Presentation
• Introduce NIST project to develop Measurement Science for Complex Information Systems
• Show an application of measurement science to compare seven alternate congestion-control algorithms for the Internet
May 20, 2009Innovations in Measurement Science
3
Measurement Science for Complex Information Systems
What are complex systems?Large collections of interconnected components whose interactions lead to macroscopic behaviors
http://autoinfo.smartlink.net/quake/quake.htmhttp://autoinfo.smartlink.net/quake/quake.htm
http://www.sover.net/~kenandeb/fire/hotshot.htmlhttp://www.sover.net/~kenandeb/fire/hotshot.html
http://www.avalanche.org/http://www.avalanche.org/
– Physical systems (e.g., earthquakes, avalanches, forest fires)
© http://www.nationalgeographic.com/© http://www.nationalgeographic.com/
©http://emergent.brynmawr.edu 2003©http://emergent.brynmawr.edu 2003
– Biological systems (e.g., slime molds, ant colonies, embryos)
http://www.wtopnews.com/http://www.wtopnews.com/
http://www.english.uiuc.edu/maps/depression/photoessay.htmhttp://www.english.uiuc.edu/maps/depression/photoessay.htm
©http://www.waag.org/realtime/©http://www.waag.org/realtime/
– Social systems (e.g., transportation networks, cities, economies)
http://www.emulab.net/pix/pc3k-back.jpghttp://www.emulab.net/pix/pc3k-back.jpg
http://www.kk.org/thetechnium/images/aol-server-farm.jpghttp://www.kk.org/thetechnium/images/aol-server-farm.jpg
– Information systems (e.g., Internet, Web services, compute grids)
May 20, 2009Innovations in Measurement Science
4
Measurement Science for Complex Information Systems
What is the problem? No one understands how to measure, predict or control macroscopic behavior in complex information systems
“[Despite] society’s profound dependence on networks, fundamental knowledge about them is primitive. [G]lobal communication …networks have quite advanced technological implementations but their behavior under stress still cannot be predicted reliably.… There is no science today that offers the fundamental knowledge necessary to design large complex networks [so] that their behaviors can be predicted prior to building them.”— Network Science, NRC report released in 2006
May 20, 2009Innovations in Measurement Science
5
Measurement Science for Complex Information Systems
Technical Approach• Evaluate models and analysis methods
– Computationally tractable?– Reveal macroscopic behavior?– Establish causality?
• Evaluate distributed control techniques– Can economic mechanisms elicit desired behaviors?– Can biological mechanisms organize elements?
Leverage models and mathematics from the physical sciences to define a systematic method to measure, understand and control macroscopic behavior in the Internet and distributed software systems built on the Internet
Internet
Distributed Systems
Scheduler
TaskControl
NegotiationControl
DSI
Grid Processor
Service Negotiator
Agreement
Grid ProcessorGrid Processor
DSIService
NegotiatorAgreement
Execution Control
CLIENT
Application
Client Negotiator
Task 1
DiscoveryControl
Task 2
Grid Processor
DSF
DSIService
NegotiatorAgreement
Client Negotiator
Task 3
TaskControl
NegotiationControl
ApplicationTask 1
DiscoveryControl
Task 2
Scheduler
DRMS Front-End
DRMS Front-End
DRMS Front-End
DRMS Front-End
DSF
spawnsspawns
negotiatesnegotiates
monitors
monitors
requests reservation
spawns
Supervisory Process Supervisory Process
spawns
GIIS
GRIS
GIIS
GRIS
GIIS
ProviderSite
ClientSite
ProviderSite
Scheduler
TaskControl
NegotiationControl
DSI
Grid Processor
Service Negotiator
Agreement
Grid ProcessorGrid Processor
DSIService
NegotiatorAgreement
Execution Control
CLIENT
Application
Client Negotiator
Task 1
DiscoveryControl
Task 2
Grid Processor
DSF
DSIService
NegotiatorAgreement
Client Negotiator
Task 3
TaskControl
NegotiationControl
ApplicationTask 1
DiscoveryControl
Task 2
Scheduler
DRMS Front-End
DRMS Front-End
DRMS Front-End
DRMS Front-End
DSF
spawnsspawns
negotiatesnegotiates
monitors
monitors
requests reservation
spawns
Supervisory Process Supervisory Process
spawns
GIIS
GRIS
GIIS
GRIS
GIIS
ProviderSite
ClientSite
ProviderSite
What is the new idea?
May 20, 2009Innovations in Measurement Science
6
Measurement Science for Complex Information Systems
Previous NIST Groundwork (2000-2005)
Complex Dynamics in Communications Networks, December 2005(including Macroscopic Dynamics in Large-Scale Data Networks by Yuan and Mills)
Yuan and Mills, A Cross-Correlation-based Method forSpatial-Temporal Traffic Analysis, July 2005
Yuan and Mills, Monitoring the Macroscopic Effects of DistributedDenial of Service (DDoS) Flooding Attacks, October 2005
Yuan and Mills, Simulating Timescale Dynamics of NetworkTraffic Using Homogeneous Modeling, May-June 2006
Preliminary investigation to identify hard technical issues
May 20, 2009Innovations in Measurement Science
7
Measurement Science for Complex Information Systems
Why is this hard? Why can we succeed?
A5. Evaluate distributed control regimesH5. Controlling behavior
A4. Evaluate analysis techniquesH4. Causal analysis
A3. Cluster analysis and statistical analysesH3. Tractable analysis
A2. Sensitivity analysis & key comparisonsH2. Model validation
A1. Scale-reduction techniquesH1. Model scalePlausible ApproachesHard Issues
Project Start Date: October 2006
May 20, 2009Innovations in Measurement Science
8
Measurement Science for Complex Information Systems
Multidisciplinary Project
V. Marbukh
K. MillsF. Hunt
C. DabrowskiD. Genin
SimulationAnalytical
Modeling Methods
J. Filliben
Experiment Design Methods
D. Y. Cho
J. Filliben
Data Analysis Methods
C. Houard
J. Hagedorn
Visualization Methods
Disciplinary Expertise Problem Domains
A
B
D
C
F
G
H
I
K
J
A1
A2
A1a
A1b
A1c
A1d
A1e
E
A2b
A2gA1f
A2a
B0a
B1
B1b
B2g
B2f
B1a
B2
B2aB2b
B2c
B2d
B2e D1
D1b
D2g
F1d
D1a
D2
D2a
D2bD2c D2d
D2e
C1
C1a
C1bC1c C1d
C1e
C2
C1f
C2f
C2eC2dC2c
C2b
C2a E1
E1b
C2g
E2f
E1a
E2E2aE2bE2c
E2d E2e
G1b
G1c G1dG1e
G2e
G2c
G2b
G2a
C1G1G1a
G1f
G2
G2dG2f
F0a
F1
F1b
F1a
F1c
F2
F2aF2b F2c F2d
F2e
H2
H2a
H2b
H2c H2dH2e
H1
H1b
H2g
H1a
F2f
J1
J1e
J1dJ1cJ1b
J1aJ1f
J2
J2a
J2f
J2b
J2c
J2d
J2eI0a
I1c
I1bI1a
I1
I1d
I2
I2cI2bI2a
I2d
K1
K1b
J2g
K1a
I2f
K2
K2b
G2g
H2f
K2a
I2e
D2f
E0a
K0a
J1g
I1e I1f
G1g
I1g
I2g
F2g
C1g E2g
F1g
F1fF1e
A2d
A2c
B1c
B1d
D1dD1cH1c
E1c E1d
H1d
K1dK1c
K2dK2c
C0a
Internet
Problem Approaches
Mesoscopic Modeling
Fluid-Flow Modeling
Markov Modeling
Perturbation Analysis
Mean-Field Approximation
Orthogonal FractionalFactorial Design
Sensitivity Analysis
Clustering Analysis
Principal Components Analysis
Correlation Analysis
Multidimensional Data Visualization
Computational Grids
May 20, 2009Innovations in Measurement Science
9
Measurement Science for Complex Information Systems
Sample Challenge Problems Under Investigation• Predict effect on global behavior and user experience from
adopting proposed replacement congestion-control algorithms for the Internet* (Filliben, Cho, Houard & Mills)
• Evaluate accuracy of proposed fluid-flow models for TCP, characterize limits of applicability of such models and propose improved analytical models (Genin & Marbukh)
• Devise efficient Markov models to accurately simulate large-scale systems and apply perturbation analysis to predict system changes that could lead to undesired behaviors (Dabrowski & Hunt)
• Investigate the use of economic methods for resource allocation in large distributed systems (e.g., computational grids and networks) (Dabrowski, Marbukh & Mills)
*Today I use this challenge problem to illustrate some of our approaches
May 20, 2009Innovations in Measurement Science
10
Measurement Science for Complex Information Systems
Sample Artifacts Produced by the ProjectMesoNet – a medium scale network simulator that includes seven
congestion control algorithms: BIC, CTCP, FAST, HSTCP,H-TCP, Scalable TCP and TCP Reno
EconoGrid – a detailed simulation model of a standards-based Gridcompute economy
Flexi-Cluster – a flexible simulation model of a compute cluster thatincludes alternate, replaceable functions forpricing, admission control, scheduling and queuemanagement
Markov-Model Rewriter – software to systematically perturb a Markovmodel with bounds defined by a user
DiVisa – software for interactive exploration of multidimensional data
May 20, 2009Innovations in Measurement Science
11
Measurement Science for Complex Information Systems
Challenge Problem: Study of Proposed Replacement Congestion-Control Algorithms for the Internet
• Modeling the network– Parameter state-space reduction techniques– Response state-space reduction techniques– Orthogonal fractional-factorial experiment design– Sensitivity analysis
• Modeling congestion-control algorithms– Unified model with phase and procedure alignment– Validation against empirical measurements
• Comparing congestion-control algorithms– Cluster analysis– Detailed analysis of individual responses– Condition-response summary analysis– Causality analysis – through domain expertise
May 20, 2009Innovations in Measurement Science
12
Measurement Science for Complex Information Systems
The Modeling State-Space Problem
y1, …, ym = f(k.x1, …, k.xn)
Response State‐Space Stimulus State‐Space
n Number of inputs (i.e., stimulus factors)k Factor range (i.e., number of values each factor can assume)m Number of outputs (i.e., responses)
May 20, 2009Innovations in Measurement Science
13
Measurement Science for Complex Information Systems
Parameter State-Space Reductionkn
k(n‐r1)
k(n‐r1‐r2)
2(n‐r1‐r2)
2(n‐r1‐r2‐r3)
2‐levelexperiment
design
modelingbrilliance
factorclustering
OFF (orthogonal fractional factorial)experiment design
SCIENTIFIC DOMAIN EXPERTISE
STATISTICALEXPERIMENT
DESIGNEXPERTISE
kn
k(n‐r1)
k(n‐r1‐r2)
2(n‐r1‐r2)
2(n‐r1‐r2‐r3)
2‐levelexperiment
design
modelingbrilliance
factorclustering
OFF (orthogonal fractional factorial)experiment design
SCIENTIFIC DOMAIN EXPERTISE
STATISTICALEXPERIMENT
DESIGNEXPERTISE
STATISTICALEXPERIMENT
DESIGNEXPERTISE
May 20, 2009Innovations in Measurement Science
14
Measurement Science for Complex Information Systems
Parameter State-Space Reduction: MesoNet Example
(232)1000
220
28
k = 2
r1 = 944
O(109633) [ 1080 = atoms in visible universe]
(232)56 O(10539)
(232)20r2 = 36
O(10192)
O(106)
r3 = 12 256
Kevin
Jim
(232)1000
220
28
k = 2
r1 = 944
O(109633) [ 1080 = atoms in visible universe]
(232)56 O(10539)
(232)20r2 = 36
O(10192)
O(106)
r3 = 12 256
(232)1000
220
28
k = 2
r1 = 944
O(109633) [ 1080 = atoms in visible universe]
(232)56 O(10539)
(232)20r2 = 36
O(10192)
O(106)
r3 = 12 256
Kevin
Jim
Kevin
Jim
May 20, 2009Innovations in Measurement Science
15
Measurement Science for Complex Information Systems
220-12 OFF Design Improves Computational Feasibility of Searching Parameter State Space
Assumptions
Processing Time for One Run 8 CPU‐Hours
Number of Available CPUs 48
Assumptions
Processing Time for One Run 8 CPU‐Hours
Number of Available CPUs 48
(220 runs x 8 CPU‐Hours Per Run)/48 CPUs = 174,762.67 Hours
2‐Level Experiment Design Requires 20 Years*
OFF Experiment Design Reduces Requirement To Under 2 Days(28 runs x 8 CPU‐Hours Per Run)/48 CPUs = 42.67 Hours
*If we had 1,000 CPUs to dedicate to this problem, then we could compute the 220 runs in just under 1 year
May 20, 2009Innovations in Measurement Science
16
Measurement Science for Complex Information Systems
Response State-Space Reduction
m responses (y1, … ym)
Principal Components
Analysis
m – d1 responses
CorrelationAnalysis
m – d2 responses
DomainExpertise
m – d3 responses
SCIENTIFIC DOMAIN EXPERTISE
DATAANALYSISEXPERTISE
m responses (y1, … ym)
Principal Components
Analysis
m – d1 responses
CorrelationAnalysis
m – d2 responses
DomainExpertise
m – d3 responses
m responses (y1, … ym)
Principal Components
Analysis
m – d1 responses
CorrelationAnalysis
m – d2 responses
DomainExpertise
m – d3 responses
SCIENTIFIC DOMAIN EXPERTISE
SCIENTIFIC DOMAIN EXPERTISE
DATAANALYSISEXPERTISE
DATAANALYSISEXPERTISE
May 20, 2009Innovations in Measurement Science
17
Measurement Science for Complex Information Systems
Response State-Space Reduction: MesoNet Example
22 responses (y1, … y22)
Principal Components
Analysis
22 – 17 responses
CorrelationAnalysis
22 – 15 responses
DomainExpertise
22 – 14 responses
Kevin
Jim
Jim
22 responses (y1, … y22)
Principal Components
Analysis
22 – 17 responses
CorrelationAnalysis
22 – 15 responses
DomainExpertise
22 – 14 responses
22 responses (y1, … y22)
Principal Components
Analysis
22 – 17 responses
CorrelationAnalysis
22 – 15 responses
DomainExpertise
22 – 14 responses
Kevin
Jim
Jim
May 20, 2009Innovations in Measurement Science
18
Measurement Science for Complex Information Systems
22-Dimension Response State Space from MesoNet
Macroscopic Network Behavior
UserExperience
May 20, 2009Innovations in Measurement Science
19
Measurement Science for Complex Information Systems
Correlation Analysis Identifies 7 Dimensions
Flow ThroughputTypical Flows
Congestion
Flow ThroughputAdvantaged Flows
Delay
AggregatePacket
Throughput
Flow ThroughputVery Advantaged Flows
Aggregate Rate ofFlow Completions
Flow ThroughputTypical Flows
Congestion
Flow ThroughputAdvantaged Flows
Delay
AggregatePacket
Throughput
Flow ThroughputVery Advantaged Flows
Aggregate Rate ofFlow Completions
May 20, 2009Innovations in Measurement Science
20
Measurement Science for Complex Information Systems
Principal Components Analysis Suggests 4 Dimensions
PC1 – Congestion
PC3 – Throughput for Advantaged Flows
PC2 – Delay
PC4 – Aggregate Throughput
May 20, 2009Innovations in Measurement Science
21
Measurement Science for Complex Information Systems
Domain Expert Selects 8 Dimensions
Domain Expert Sides with the Correlation Analysis and Adds One Response (y1)
May 20, 2009Innovations in Measurement Science
22
Measurement Science for Complex Information Systems
Sensitivity Analysis as a Model Validation Step
What parameters (or combinations of parameters)determine a model’s responses?
Does the influence of parameters on responsesmake sense to a domain expert?
Can any unexpected responses be explained anddo the explanations make sense?
May 20, 2009Innovations in Measurement Science
23
Measurement Science for Complex Information Systems
11 of 20 MesoNet Parameters Selected for Sensitivity Analysis* selected parameter
other parameters fixed Network Parameters
Topology
Propagation Delay
Speed
Buffer Sizing
Network Parameters
Topology
Propagation Delay
Speed
Buffer Sizing
Simulation Control
Measurement‐Interval Size
Number of Measurement Intervals
Random‐Number Seed
Simulation Control
Measurement‐Interval Size
Number of Measurement Intervals
Random‐Number Seed
User Behavior
Think Time
File‐Size Distribution
Pattern of Long‐Lived Flows
User Behavior
Think Time
File‐Size Distribution
Pattern of Long‐Lived Flows
Protocol Parameters
Initial Congestion Window
Initial Slow‐Start Threshold
Type of Slow Start Regime
Protocol Parameters
Initial Congestion Window
Initial Slow‐Start Threshold
Type of Slow Start Regime
Characteristics of Sources & Receivers
Number of Sources
Distribution of Sources
Number of Receivers
Distribution of Receivers
Distribution of Network‐Interface Speeds
Distribution of Congestion‐Control Mechanisms
Startup Pattern for Sources
Characteristics of Sources & Receivers
Number of Sources
Distribution of Sources
Number of Receivers
Distribution of Receivers
Distribution of Network‐Interface Speeds
Distribution of Congestion‐Control Mechanisms
Startup Pattern for Sources
***
*****
*
**None
Next slide
200 ms
6000
200000
2 packets
Limited slow start
All TCP
25% at a time
May 20, 2009Innovations in Measurement Science
24
Measurement Science for Complex Information Systems
Representative Heterogeneous Four-Tier Topology SelectedFast access routers
Directly-connected access routers
Normal access routers
May 20, 2009Innovations in Measurement Science
25
Measurement Science for Complex Information Systems
211-5 Orthogonal Fraction Factorial (OFF) Experiment Design
Design Template
Full Factorial Design Requires 211 = 2048 runs
We can only afford 26 = 64 runs; thus,
we require a 211-5 OFF Experiment Design
No confounding of main effects with two-terminteractions
Main effects may be confounded with three-termand higher interactions
May 20, 2009Innovations in Measurement Science
26
Measurement Science for Complex Information Systems
Values Selected to Replace +1 and -1 in Design Template
Initial Slow-Start Threshold (packets)
Distribution of Receivers
Distribution of Sources
Multiplier on Number Sources Per Access Router
Probability of Fast Network Interface*
Probability of a Larger File*
Average Think Time (ms)
Average File Size (packets)
Buffer Sizing Algorithm
Backbone Router Speed (ppms)*
Multiplier for Propagation Delay
Parameter
431.07x109x11Protocol Factors
WEBP2Px10
WEBP2Px9
23x8
0.40.2x7
Source &
Receiver Factors
0.020.01x6
20005000x5
50100x4UserFactors
RTTxC/SQR(n)RTTxCx3
800400x2
12x1NetworkFactors
-1+1Factor
*By convention +1 is coded with larger value and -1 is coded with smaller value. Here, coding was reversed in three cases.This makes no difference in the construction of the experiment, but must be managed during data analysis
May 20, 2009Innovations in Measurement Science
27
Measurement Science for Complex Information Systems
Speed of All Routers Derived from Speed of Backbone Routers(Topology Tiers 1 to 3)
= x2/R2/R4xDC
= x2/R2/R4xFA
= x2/R2/R4
= x2/R2
= x2
Equation
DC (= 10)
FA (= 2)
R3 (= 10)
R2 (= 4)
x2
Parameter
200 ppms100 ppmsDirectly Connected Access
40 ppms20 ppmsFast Access
20 ppms10 ppmsTypical Access
200 ppms100 ppmsPOP
800 ppms400 ppmsBackbone
-1+1Router Type
May 20, 2009Innovations in Measurement Science
28
Measurement Science for Complex Information Systems
Number & Distribution of Sources & Receivers (Tier 4)
45.2548.276.4627,840P2PWEB3
45.2548.276.4618,560P2PWEB2
75.5420.144.3241,700WEBP2P3
75.5420.144.3227,800WEBP2P2
45.2548.276.4627,840WEBWEB3
45.2548.276.4618,560WEBWEB2
75.5420.144.3241,700P2PP2P3
75.5420.144.3227,800P2PP2P2
% underN Routers
% underF Routers
% underD Routers
TotalSourcesx10x9x8
Sources
75.5420.144.32166,800P2PWEB3
75.5420.144.32111,200P2PWEB2
86.0611.472.45219,600WEBP2P3
86.0611.472.45146,400WEBP2P2
86.0611.472.45219,600WEBWEB3
86.0611.472.45146,400WEBWEB2
75.5420.144.32166,800P2PP2P3
75.5420.144.32111,200P2PP2P2
% underN Routers
% underF Routers
% underD Routers
Total Receiversx10x9x8
Receivers
May 20, 2009Innovations in Measurement Science
29
Measurement Science for Complex Information Systems
Resulting Distribution of Flow Classes
Flow classes defined by relative locations of source & receiver
34.1845.589.726.833.380.279P2PWEB3
34.1845.589.726.833.380.279P2PWEB2
65.0126.002.315.570.990.106WEBP2P3
65.0126.002.315.570.990.106WEBP2P2
38.9546.745.536.671.920.159WEBWEB3
38.9546.745.536.671.920.159WEBWEB2
57.0630.434.056.521.740.186P2PP2P3
57.0630.434.056.521.740.186P2PP2P2
% NNFlows
% FNFlows
% FFFlows
% DNFlows
% DFFlows
% DDFlowsx10x9x8
May 20, 2009Innovations in Measurement Science
30
Measurement Science for Complex Information Systems
Influence of other Factors on Average Buffer Sizes
258816277651068002
12948139325538001
12948139325534002
6474070162774001
Access RouterBuffers (avg.)
POP RouterBuffers (avg.)
Backbone Router Buffers
(avg.)x2x1
RTTxC
1052707288002
531353648001
531353644002
27681824001
Access RouterBuffers (avg.)
POP RouterBuffers (avg.)
Backbone Router Buffers
(avg.)x2x1
RTTxC/SQR(n)
May 20, 2009Innovations in Measurement Science
31
Measurement Science for Complex Information Systems
Sensitivity Analysis Relies on Main Effects Plots
May 20, 2009Innovations in Measurement Science
32
Measurement Science for Complex Information Systems
Sensitivity Analysis Driven by Correlation Analysis
Active Flows Retransmission Rate
Avg. TP NN Flows SRTT
Aggregate TP Flow Completion Rate Avg. TP DD Flows Avg. TP FF Flows
Seven Responses Chosen by Correlation Analysis + 1
May 20, 2009Innovations in Measurement Science
33
Measurement Science for Complex Information Systems
Major Factors Influencing Model Behavior
(1) Network Speed
(2) File Size
(3) Think Time
(4) Number of Sources
(5) Propagation Delay
(7) Buffer Size – small buffer size reduces delay variability
& larger buffer size has greater effect under high network speed
Correlation-based Analysis
Principal Components-based Analysis(6) Distribution of Sources
8964111032715Ordinal Rank
7.678.175.424.679.508.584.53.55.832.925.25Average Rank
7926111045831y20
5810.53558210.581y17
10107610854132y15
108351010551.51.57y10
7.57.554109326111y6
6.56.554119.523819.5y4
x11x10x9x8x7x6x5x4x3x2x1
8964111032715Ordinal Rank
7.678.175.424.679.508.584.53.55.832.925.25Average Rank
7926111045831y20
5810.53558210.581y17
10107610854132y15
108351010551.51.57y10
7.57.554109326111y6
6.56.554119.523819.5y4
x11x10x9x8x7x6x5x4x3x2x1
8976111024415Ordinal Rank
7.258.636.755.509.759.253.254.004.002.505.13Average Rank
6897111023415PC4
10.594710.5861523PC3
5101148.58.537162PC2
7.57.534910.5256110.5PC1
x11x10x9x8x7x6x5x4x3x2x1
8976111024415Ordinal Rank
7.258.636.755.509.759.253.254.004.002.505.13Average Rank
6897111023415PC4
10.594710.5861523PC3
5101148.58.537162PC2
7.57.534910.5256110.5PC1
x11x10x9x8x7x6x5x4x3x2x1
May 20, 2009Innovations in Measurement Science
34
Measurement Science for Complex Information Systems
Optional mode switch between TCP & alternate procedures:BIC, CTCP, HSTCP, H-TCP, Scalable TCP
Current timetime
Timeout for current SYNsynTO
Number of SYNs that have been sentsynSENT
Maximum number of SYNs to sendsynMAX
Timeout interval for initial SYNsynINT
DefinitionSymbol
Current timetime
Timeout for current SYNsynTO
Number of SYNs that have been sentsynSENT
Maximum number of SYNs to sendsynMAX
Timeout interval for initial SYNsynINT
DefinitionSymbol
Threshold to terminate initial slow start (varies with experiment)
sstINT
Threshold to switch from exponential to logarithmic increase (varies with experiment)
sstMAX
Current slow‐start thresholdsst
Initial congestion window (we use cwndINT = 2)cwndINT
Current congestion windowcwnd
DefinitionSymbol
Threshold to terminate initial slow start (varies with experiment)
sstINT
Threshold to switch from exponential to logarithmic increase (varies with experiment)
sstMAX
Current slow‐start thresholdsst
Initial congestion window (we use cwndINT = 2)cwndINT
Current congestion windowcwnd
DefinitionSymbol
Window increase procedures (ACK)
Window decrease procedures (Loss)
Timeout procedures
Optional periodic procedures:CTCP, FAST, H-TCP
Standard or Limited
MS Windows® TCP
Unified Model for Congestion-Control Algorithms
May 20, 2009Innovations in Measurement Science
35
Measurement Science for Complex Information Systems
Sample: CTCP Congestion-Avoidance Model
Average Smoothed Round‐Trip Time experienced on CTCP flowSRTTC
Minimum round‐trip time experienced on CTCP flowminRTTC
Low‐window threshold ( LWC = 41) for applying CTCP proceduresLWC
Exponent (kC = 0.8) for CTCP window‐increase procedureskC
CTCP zeta parameter ( C = 0.1) defining reduction speed in delay windowC
Expected throughput (cwnd/minRTTC) on CTCP flowEC
CTCP delay windowdwnd
Difference between expected and actual throughput experienced on CTCP flowDC
CTCP gamma threshold ( C = 30) for detecting early congestionC
Boolean denoting whether early congestion has been detected (true) or not (false)CDC
Window decrease ( C = 0.5) weight for CTCPC
Actual throughput (cwnd/SRTTC) experienced on CTCP flowAC
Window increase ( C = 0.125) weight for CTCPC
DefinitionSymbol
Average Smoothed Round‐Trip Time experienced on CTCP flowSRTTC
Minimum round‐trip time experienced on CTCP flowminRTTC
Low‐window threshold ( LWC = 41) for applying CTCP proceduresLWC
Exponent (kC = 0.8) for CTCP window‐increase procedureskC
CTCP zeta parameter ( C = 0.1) defining reduction speed in delay windowC
Expected throughput (cwnd/minRTTC) on CTCP flowEC
CTCP delay windowdwnd
Difference between expected and actual throughput experienced on CTCP flowDC
CTCP gamma threshold ( C = 30) for detecting early congestionC
Boolean denoting whether early congestion has been detected (true) or not (false)CDC
Window decrease ( C = 0.5) weight for CTCPC
Actual throughput (cwnd/SRTTC) experienced on CTCP flowAC
Window increase ( C = 0.125) weight for CTCPC
DefinitionSymbol
Mode Switch Procedures
Periodic Procedures
IncreaseProcedures
DecreaseProcedures
TimeoutProcedures
May 20, 2009Innovations in Measurement Science
36
Measurement Science for Complex Information Systems
Flow 1 Flow 1
Flow 2 Flow 2
PATH
Flow 1 Flow 1
Flow 2 Flow 2
PATH
PARAMETERS• Number of flows sharing path• Start time of each flow• Bottleneck bandwidth• Round-trip propagation delay• Number of buffers
1. Yee-Ting, Leith and Shorten, “Experimental Evaluation of TCP Protocols for High-Speed Networks”, IEEE/ACM Transactions on Networking, (15)5, October 2007, pp. 1109 – 1122.– Covers BIC, FAST, HSTCP, H-TCP, Scalable TCP– Uses implementations within Linux kernel
2. Leith, Andrew, Quetchenbach, Shorten and Lavi, “Experimental Evaluation of Delay/Loss-based TCP Congestion Control Algorithms”, Proceedings of the 6th International Workshop on Protocols for Fast Long-Distance Networks (PFLDnet 2008), March 5-7,2008, Manchester, UK.– Covers CTCP and TCP Illinois– Uses CTCP implementation in Windows® VISTA and TCP Illinois
implementation within Linux kernel
Empirical Studies of Six Alternate Congestion-Control Algorithms
May 20, 2009Innovations in Measurement Science
37
Measurement Science for Complex Information Systems
Simulated Behavior of MesoNet CTCP Model
0 1000 2000 3000 4000 5000 6000 7000 8000 9000 1 .1040
100
200
300
400
500
600
700
800
900
1000
1100
1200
Time
Con
gest
ion
Win
dow
avg. cwnd = 798 avg. cwnd = 836
avg. red cwnd = 419avg. blue cwnd = 417
0 1000 2000 3000 4000 5000 6000 7000 8000 9000 1 .1040
100
200
300
400
500
600
700
800
900
1000
1100
1200
Time
Con
gest
ion
Win
dow
avg. cwnd = 798 avg. cwnd = 836
avg. red cwnd = 419avg. blue cwnd = 417
0 1000 2000 3000 4000 5000 6000 7000 8000 9000 1 .1040
500
1000
1500
2000
2500
3000
3500
4000
4500
Time
Con
gest
ion
Win
dow
avg. cwnd = 2745 avg. cwnd = 3406
avg. red cwnd = 1853avg. blue cwnd = 1553
0 1000 2000 3000 4000 5000 6000 7000 8000 9000 1 .1040
500
1000
1500
2000
2500
3000
3500
4000
4500
Time
Con
gest
ion
Win
dow
avg. cwnd = 2745 avg. cwnd = 3406
avg. red cwnd = 1853avg. blue cwnd = 1553
rtt = 162 ms
rtt = 42 ms
0 1000 2000 3000 4000 5000 6000 7000 8000 9000 1 .1040
1000
2000
3000
4000
5000
6000
7000
8000
9000
Time
Con
gest
ion
Win
dow
avg. cwnd = 4325 avg. cwnd = 6254
avg. red cwnd = 3422avg. blue cwnd = 2832
0 1000 2000 3000 4000 5000 6000 7000 8000 9000 1 .1040
1000
2000
3000
4000
5000
6000
7000
8000
9000
Time
Con
gest
ion
Win
dow
avg. cwnd = 4325 avg. cwnd = 6254
avg. red cwnd = 3422avg. blue cwnd = 2832
rtt = 324 ms
MesoNet CTCP simulation behavioragrees with empirical results
Other MesoNet congestion-controlalgorithms also agree with empiricalresults
May 20, 2009Innovations in Measurement Science
38
Measurement Science for Complex Information Systems
Comparing Alternate Congestion-Control Algorithms in a Large (up to 278,000 sources), Fast (up 192 Gbps) Homogeneous Network
Transmission Control Protocol (Reno)TCP7Scalable Transmission Control ProtocolScalable6Hamilton Transmission Control ProtocolHTCP5
High-Speed Transmission Control ProtocolHSTCP4
Fast Active-Queue Management Scalable Transmission Control ProtocolFAST3
Compound Transmission Control ProtocolCTCP2Binary Increase Congestion ControlBIC1Name of Congestion-Avoidance AlgorithmLabelIdentifier
Transmission Control Protocol (Reno)TCP7Scalable Transmission Control ProtocolScalable6Hamilton Transmission Control ProtocolHTCP5
High-Speed Transmission Control ProtocolHSTCP4
Fast Active-Queue Management Scalable Transmission Control ProtocolFAST3
Compound Transmission Control ProtocolCTCP2Binary Increase Congestion ControlBIC1Name of Congestion-Avoidance AlgorithmLabelIdentifier
Source & receiver under normal access routerNN
Source or receiver under fast access router and correspondent under normal access routerFN
Source or receiver under directly connected access router and correspondent under normal access routerDN
Typical
Source & receiver under fast access routerFF
Source or receiver under directly connected access router and correspondent under fast access routerDF
Fast
Source & receiver under directly connected access routerDDVery Fast
DefinitionFlow TypePath Class
Source & receiver under normal access routerNN
Source or receiver under fast access router and correspondent under normal access routerFN
Source or receiver under directly connected access router and correspondent under normal access routerDN
Typical
Source & receiver under fast access routerFF
Source or receiver under directly connected access router and correspondent under fast access routerDF
Fast
Source & receiver under directly connected access routerDDVery Fast
DefinitionFlow TypePath Class
7 Algorithms
3 Path Classes
May 20, 2009Innovations in Measurement Science
39
Measurement Science for Complex Information Systems
Input Factors & Network Parameters
6 Robustness Factors
RTTxCapacity/SQR(N)RTTxCapacityBuffer Sizing Algorithmx650100File Sizex5
12Propagation Delayx4Skewed (.1/.6/.3)Uniform (.33/.33/.33)Source Distributionx3
25005000Think Timex240008000Network Speedx1
Minus (-1) ValuePLUS (+1) ValueDefinitionIdentifier
RTTxCapacity/SQR(N)RTTxCapacityBuffer Sizing Algorithmx650100File Sizex5
12Propagation Delayx4Skewed (.1/.6/.3)Uniform (.33/.33/.33)Source Distributionx3
25005000Think Timex240008000Network Speedx1
Minus (-1) ValuePLUS (+1) ValueDefinitionIdentifier
Router Speeds12 Gbps24 GbpsDirectly Connected Access
2.4 Gbps4.8 GbpsFast Access1.2 Gbps2.4 GbpsNormal Access12 Gbps24 GbpsPOP96 Gbps192 GbpsBackbone
Minus (-1)PLUS (+1)Router
12 Gbps24 GbpsDirectly Connected Access
2.4 Gbps4.8 GbpsFast Access1.2 Gbps2.4 GbpsNormal Access12 Gbps24 GbpsPOP96 Gbps192 GbpsBackbone
Minus (-1)PLUS (+1)Router
Propagation Delays (ms)100416Minus (-1)2008112PLUS (+1)
MaxAvgMin
100416Minus (-1)2008112PLUS (+1)
MaxAvgMin
Buffer Sizes (packets)
Number of Sources
174,600278,000Minus (-1)PLUS (+1)
174,600278,000Minus (-1)PLUS (+1)
25,879162,764
1,302,110Max
14,55791,555
732,437Avg
6,47040,691
325,528Min
PLUS (+1)
36920791Access908505221POP
4,6542,6061,153BackboneMaxAvgMinRouter
Minus (-1)
25,879162,764
1,302,110Max
14,55791,555
732,437Avg
6,47040,691
325,528Min
PLUS (+1)
36920791Access908505221POP
4,6542,6061,153BackboneMaxAvgMinRouter
Minus (-1)
May 20, 2009Innovations in Measurement Science
40
Measurement Science for Complex Information Systems
Simulation Scenario & Long-Lived Flows
F0aI0a
K0a
Receiver Router
0.4 x 25 mins.E0aShort-distance flowL30.4 x 25 mins.C0aMedium-distance flowL2
0.4 x 25 mins.B0aLong-distance flowL1
Start TimeSource RouterDefinitionIdentifier
F0aI0a
K0a
Receiver Router
0.4 x 25 mins.E0aShort-distance flowL30.4 x 25 mins.C0aMedium-distance flowL2
0.4 x 25 mins.B0aLong-distance flowL1
Start TimeSource RouterDefinitionIdentifier
Long-Lived Flows
Scenario
May 20, 2009Innovations in Measurement Science
41
Measurement Science for Complex Information Systems
32 Conditions Simulated (26-1 OFF Design)Factor-> X1 X2 X3 X4 X5 X6
Condition -- -- -- -- -- --1 4000 2500 .1/.6/.3 1 50 RTTxCapacity/SQR(N )2 8000 2500 .1/.6/.3 1 50 RTTxCapacity3 4000 5000 .1/.6/.3 1 50 RTTxCapacity4 8000 5000 .1/.6/.3 1 50 RTTxCapacity/SQR(N )5 4000 2500 .3/.3/.3 1 50 RTTxCapacity6 8000 2500 .3/.3/.3 1 50 RTTxCapacity/SQR(N )7 4000 5000 .3/.3/.3 1 50 RTTxCapacity/SQR(N )8 8000 5000 .3/.3/.3 1 50 RTTxCapacity9 4000 2500 .1/.6/.3 2 50 RTTxCapacity
10 8000 2500 .1/.6/.3 2 50 RTTxCapacity/SQR(N )11 4000 5000 .1/.6/.3 2 50 RTTxCapacity/SQR(N )12 8000 5000 .1/.6/.3 2 50 RTTxCapacity13 4000 2500 .3/.3/.3 2 50 RTTxCapacity/SQR(N )14 8000 2500 .3/.3/.3 2 50 RTTxCapacity15 4000 5000 .3/.3/.3 2 50 RTTxCapacity16 8000 5000 .3/.3/.3 2 50 RTTxCapacity/SQR(N )17 4000 2500 .1/.6/.3 1 100 RTTxCapacity18 8000 2500 .1/.6/.3 1 100 RTTxCapacity/SQR(N )19 4000 5000 .1/.6/.3 1 100 RTTxCapacity/SQR(N )20 8000 5000 .1/.6/.3 1 100 RTTxCapacity21 4000 2500 .3/.3/.3 1 100 RTTxCapacity/SQR(N )22 8000 2500 .3/.3/.3 1 100 RTTxCapacity23 4000 5000 .3/.3/.3 1 100 RTTxCapacity24 8000 5000 .3/.3/.3 1 100 RTTxCapacity/SQR(N )25 4000 2500 .1/.6/.3 2 100 RTTxCapacity/SQR(N )26 8000 2500 .1/.6/.3 2 100 RTTxCapacity27 4000 5000 .1/.6/.3 2 100 RTTxCapacity28 8000 5000 .1/.6/.3 2 100 RTTxCapacity/SQR(N )29 4000 2500 .3/.3/.3 2 100 RTTxCapacity30 8000 2500 .3/.3/.3 2 100 RTTxCapacity/SQR(N )31 4000 5000 .3/.3/.3 2 100 RTTxCapacity/SQR(N )32 8000 5000 .3/.3/.3 2 100 RTTxCapacity
May 20, 2009Innovations in Measurement Science
42
Measurement Science for Complex Information Systems
Processor Time Requirements
6.61
1.51
3.46
110.5
Scalable
26.37(13.18x2)
6.5728.425.635.845.175.855.94Max. CPU time
(one run)
1.611.281.401.441.331.16Min. CPU time
(one run)
13.70(6.85x2)
3.3914.772.943.012.923.042.86Avg. CPU time
(per run)
219.1108.6472.594.296.493.497.291.5CPU time (32 runs)
TotalsHSTCPTotalsTCPHTCPFASTCTCPBIC
Compute Servers ws9-ws10Compute Servers ws11-ws14
6.61
1.51
3.46
110.5
Scalable
26.37(13.18x2)
6.5728.425.635.845.175.855.94Max. CPU time
(one run)
1.611.281.401.441.331.16Min. CPU time
(one run)
13.70(6.85x2)
3.3914.772.943.012.923.042.86Avg. CPU time
(per run)
219.1108.6472.594.296.493.497.291.5CPU time (32 runs)
TotalsHSTCPTotalsTCPHTCPFASTCTCPBIC
Compute Servers ws9-ws10Compute Servers ws11-ws14
(Units are CPU days)
Experiment required about 15 days of wall-clock time spread over 48 processors
1,548,371,719,08416,583,418,069Total All Runs11,917,420,154154,914,953Max. Per Condition3,146,870,57140,966,013Min. Per Condition6,912,373,74674,033,116Avg. Per Condition
Data Packets SentFlows CompletedStatistic
1,548,371,719,08416,583,418,069Total All Runs11,917,420,154154,914,953Max. Per Condition3,146,870,57140,966,013Min. Per Condition6,912,373,74674,033,116Avg. Per Condition
Data Packets SentFlows CompletedStatistic
Simulated Workload
May 20, 2009Innovations in Measurement Science
43
Measurement Science for Complex Information Systems
Characterizing the 32 Conditions Simulated
0
0.1
0.2
0.3
0.4
0.5
0.6
12 8 20 2 32 26 3 14 4 15 27 9 16 10 28 11 18 24 22 19 17 6 1 29 7 30 5 25 23 13 31 21
Condition
Ret
rans
mis
sion
Rat
e
0
0.0002
0.0004
0.0006
0.0008
0.001
0.0012
0.0014
0.0016
0.0018
0.002
12 8 20 2 32 26 3 14 4 15 27 9 16 10 28 11
Condition
Ret
rans
mis
sion
Rat
e
Max. = 0.0018
LN M
C
0
0.1
0.2
0.3
0.4
0.5
0.6
12 8 20 2 32 26 3 14 4 15 27 9 16 10 28 11 18 24 22 19 17 6 1 29 7 30 5 25 23 13 31 21
Condition
Ret
rans
mis
sion
Rat
e
0
0.0002
0.0004
0.0006
0.0008
0.001
0.0012
0.0014
0.0016
0.0018
0.002
12 8 20 2 32 26 3 14 4 15 27 9 16 10 28 11
Condition
Ret
rans
mis
sion
Rat
e
Max. = 0.0018
0
0.1
0.2
0.3
0.4
0.5
0.6
12 8 20 2 32 26 3 14 4 15 27 9 16 10 28 11 18 24 22 19 17 6 1 29 7 30 5 25 23 13 31 21
Condition
Ret
rans
mis
sion
Rat
e
0
0.0002
0.0004
0.0006
0.0008
0.001
0.0012
0.0014
0.0016
0.0018
0.002
12 8 20 2 32 26 3 14 4 15 27 9 16 10 28 11
Condition
Ret
rans
mis
sion
Rat
e
Max. = 0.0018
LN M
C
16 Uncongested and 16 Congested Conditions
3000 3500 4000 4500 5000 5500 6000 6500 7000 75000
1000
2000
3000
4000
5000
6000
7000
8000Evolution of Flows Condition 4 - TCP
Time
Act
ive
Flow
s
Sending Flows
Normal Congestion Control
Connecting Flows
Flows in Initial Slow Start
3000 3500 4000 4500 5000 5500 6000 6500 7000 75000
1000
2000
3000
4000
5000
6000
7000
8000Evolution of Flows Condition 4 - TCP
Time
Act
ive
Flow
s
Sending Flows
Normal Congestion Control
Connecting Flows
Flows in Initial Slow Start
Evolution of Flow States in Uncongested Condition 4
3000 3500 4000 4500 5000 5500 6000 6500 7000 75000
2 .104
4 .104
6 .104
8 .104
1 .105
1.2 .105
1.4 .105Evolution of Flows Condition 5 - TCP
Time
Act
ive
Flow
s
Sending Flows
Normal Congestion Control
Connecting Flows
Flows in Initial Slow Start
3000 3500 4000 4500 5000 5500 6000 6500 7000 75000
2 .104
4 .104
6 .104
8 .104
1 .105
1.2 .105
1.4 .105Evolution of Flows Condition 5 - TCP
Time
Act
ive
Flow
s
Sending Flows
Normal Congestion Control
Connecting Flows
Flows in Initial Slow Start
Evolution of Flow States in Congested Condition 5
Cluster Analysis Annotated with Congestion Level
May 20, 2009Innovations in Measurement Science
44
Measurement Science for Complex Information Systems
Cluster Analyses – Algorithm 3 (FAST) Stands OutTime Period 1 Time Period 2
Time Period 3 Aggregate Responses over 25 minutes
May 20, 2009Innovations in Measurement Science
45
Measurement Science for Complex Information Systems
Approach to Detailed Analysis of Individual Responses
Time Period 1Retransmission Rate
May 20, 2009Innovations in Measurement Science
46
Measurement Science for Complex Information Systems
Approach to Summarizing Detailed Analyses of Responses
Condition-Response Summary Time Period 1
Statistically significant and >10% Effect Filter
May 20, 2009Innovations in Measurement Science
47
Measurement Science for Complex Information Systems
Condition-Response Summaries – Algorithm 3 (FAST) Stands OutTime Period 1 – 10% Filter Time Period 2 – 30% Filter
Time Period 3 – 30% Filter Aggregate Responses – No Filter
May 20, 2009Innovations in Measurement Science
48
Measurement Science for Complex Information Systems
Why does Algorithm 3 (FAST) Stand Out?
FAST does not respond well to congestion, where too many flows compete for insufficient buffers – leading to rapid oscillation in flow congestion windows
0
50
100
150
200
250
300
350
4600 4700 4800 4900 5000 5100
Time
CW
ND
Evolution of congestion window under FAST for long-lived flow L2 during 500 measurement intervals within Time Period 2 under (the most congested) condition 21
May 20, 2009Innovations in Measurement Science
49
Measurement Science for Complex Information Systems
Other Congestion-Control Algorithms Adjust Less Rapidly
Evolution of congestion window under other congestion-control algorithms for long-lived flow L2 during the same500 measurement intervals within Time Period 2 under (the most congested) condition 21
0
50
100
150
200
250
300
350
4600 4700 4800 4900 5000 5100
Time
CW
ND
0
50
100
150
200
250
300
350
4600 4700 4800 4900 5000 5100
TimeC
WN
D
TCP Reno BIC
0
50
100
150
200
250
300
350
4600 4700 4800 4900 5000 5100
Time
CW
ND
CTCP
0
50
100
150
200
250
300
350
4600 4700 4800 4900 5000 5100
Time
CW
ND
HSTCP
0
50
100
150
200
250
300
350
4600 4700 4800 4900 5000 5100
Time
CW
ND
HTCP
0
50
100
150
200
250
300
350
4600 4700 4800 4900 5000 5100
Time
CW
ND
Scalable
May 20, 2009Innovations in Measurement Science
50
Measurement Science for Complex Information Systems
Summary of FAST Behavior under Congestion• Rapid oscillatory adjustment of congestion window
leads to:– Larger congestion-window increase rate– Higher retransmission rate (including for SYNs)– Larger number of flows pending in the connecting state
• Practical implications include:– Flows take longer to connect– Flows take longer to complete– Goodput is lower for flows transiting congested areas– Fewer (105 to 107, depending on condition) flows complete in a 25
minute period
May 20, 2009Innovations in Measurement Science
51
Measurement Science for Complex Information Systems
Future Work on Congestion-Control Algorithms
• Does MesoNet reveal the same behavior when modeling a smaller, slower network with a much lower initial slow-start threshold?
• How do the congestion-control algorithms compare in a relatively uncongested, heterogeneous network with a wider range of traffic classes (e.g., web objects, documents, service packs and movie downloads)?
• Does the sensitivity analysis change when considering all 20 MesoNet parameters?
May 20, 2009Innovations in Measurement Science
52
Measurement Science for Complex Information Systems
Summary of Presentation• Introduced NIST project to develop Measurement
Science for Complex Information Systems
• Showed an application of measurement science to compare seven alternate congestion-control algorithms for the Internet
• Identified a potential for FAST algorithm to behave undesirably under congested conditions– Explained the root cause for the potential undesirable behavior– Explained why other congestion-control algorithms are not
likely to exhibit the same potential undesirable behavior
May 20, 2009Innovations in Measurement Science
53
Measurement Science for Complex Information Systems
Backup Information
May 20, 2009Innovations in Measurement Science
54
Measurement Science for Complex Information Systems
Interactive Exploration of Multidimensional Data
Download from – http://math.nist.gov/mcsd/savg/software/divisa/
May 20, 2009Innovations in Measurement Science
55
Measurement Science for Complex Information Systems
Sample Papers Produced by the Project (1 of 2)
C. Dabrowski, “Reliability in Grid Computing Systems”, in Concurrency and Computation: Practice and Experience,Wiley-Blackwell, in press.
D. Genin and V. Marbukh, "Do Current Fluid Approximation Models Capture TCP Instability?“, submitted to ICC 2009,Dresden, Germany, June 14 -18.
C. Dabrowksi and F. Hunt, “Using Markov Chain Analysis to Study Dynamic Behavior in Large-Scale Grid Systems”,Proceedings of the 7th Australasian Symposium on Grid Computing and e-Research, Wellington, New Zealand, Jan. 2009.
C. Dabrowski and F. Hunt, Markov Chain Analysis for Large-Scale Grid Systems, NIST Technical Report (under review).
D. Genin and V. Marbukh, "Metastability in cellular networks with migrating users: emergence and implications for performance.“ GLOBECOM 2008, New Orleans, Nov. 31 - Dec. 4.
K. Mills and C. Dabrowski, “Can Economics-based Resource Allocation Prove Effective in a Computation Marketplace?", Journal of Grid Computing, Vol. 6, No. 3, September 2008, pp. 291-311.
F. Hunt and V. Marbukh, “Dynamic Routing and Congestion Control Through Random Assignment of Routes”, Proceedings of the 5th International Conference on Cybernetics and Information Technologies, Systems and Applications: CITSA 2008, Orlando FL, July 2008.
V. Marbukh and K. Mills, "Demand Pricing & Resource Allocation in Market-based Compute Grids: A Model and Initial Results", Proceedings of the 7th International Conference on Networking, IEEE, April 2008, pp. 752-757.
May 20, 2009Innovations in Measurement Science
56
Measurement Science for Complex Information Systems
Sample Papers Produced by the Project (2 of 2)
Marbukh and S. Klink, "Decentralized Control of Large-Scale Networks as a Game with Local Interactions: Cross-Layer TCP/IP Optimization", 2nd International Conference on Performance Evaluation Methodologies and Tools, Nantes, France, October 23-25, 2007.
V. Marbukh, "Utility Maximization for Resolving Throughput/Reliability Trade-offs in an Unreliable Network with Multipath Routing", 2nd International Conference on Performance Evaluation Methodologies and Tools, Nantes, France, October 23-25, 2007.
K. Mills, "A Brief Survey of Self-Organization in Wireless Sensor Networks", Wireless Communications and Mobile Computing, Wiley Interscience, Vol. 7, No. 7, September 2007, pp. 823-834.
V. Marbukh and K. Mills, "On Maximizing Provider Revenue in Market-Based Compute Grids", Proceedings of the 3rd International Conference on Networking and Services, Athens, Greece, June 19-25, 2007.
K. Mills and C. Dabrowski, "Investigating Global Behavior in Computing Grids", Self-Organizing Systems, Lecture Notes in Computer Science, Volume 4124 ISBN 978-3-540-37658-3, pp. 120-136.