- 1. 2012 eXtreme TCP Performance Testing GuideeXtreme TCP:
applicability scope, Intercontinental global tests and local
download tests EXtreme TCP (XTCP) is a technology that dramatically
improves network performance by maximizing network bandwidth
utilization. Provided below are some tests that attempt to cover
the scope of applicability of XTCP and show its performance in
comparison with standard TCP. 10.06.2012
2. ForewordThank you for your interest in eXtreme TCP (XTCP).
EXtreme TCP is a new technology developed byMainline Net Holdings,
Ltd. which dramatically improves networking performance by
allowingthe transport protocol to better utilize all available
networking resources under variousconditions. EXtreme TCP can be
used everywhere standard TCP is used.This document shows the
results of multiple tests of the performance of XTCP as compared
withstandard TCP and provides some simple instructions as to how
you can test XTCP yourself.The Automated Windows Tests..2The
Automated Linux tests.16 2 3. 1. Automated Windows testsThe tests
were performed by transferring a 64 MB file via FTP. Tests were
performed fordifferent sender and recipient destinations around the
world. The tests were performed onthe Internet under real world
conditions. Other than sending the 64 MB file, nothing wasdone to
influence or control traffic between the sender and the
recipient.Download rates during the transfer were recorded in bytes
per second and graphed againsttime. The yellow graph represents
standard TCP transfer rates; the green graph representsXTCP
transfer rates. Note that the performance graphs are defined only
while the 64 MB fileis being downloaded. Thus, if one graph ends
before the other, this signifies that one system(usually XTCP) was
able to download the entire file before the other.1Section 1 tests
were invoked to compare XTCP performance against standard MS
WindowsTCP implementation. Servers involved in this testing run MS
Windows Server 2003 or MSWindows Server 2008 OS.1 To better
simulate real world conditions the TCP and XTCP transfers were
performed consecutively and notsimultaneously. One would not
ordinarily perform both an XTCP and TCP transfer at the same time
by the samecomputer. Since temporal variations of bandwidth/latency
are essentially random, the consecutive operation does notcause any
lack of fairness. Furthermore multiple tries confirmed that the
tests results provided below are notaberrations.3 4. 1.1. From the
United Kingdom to New-York, USA.Notes: This shows the normal or
usually expected flow of transmission. XTCP detectedchannel
capacity due to packet loss during initial exponential growth of
send window. TCPwas not able to transmit anywhere close to channel
capacity, because standard TCP limitstransmission rates for
particular channel latencies In this example, XTCP transmitted the
filein around a tenth of the time required for TCP. The
non-saturated channel latency for thischnel was 75 ms.4 5. 1.2.
From the United Kingdom to Central America.Notes: This shows normal
flow of transmission. XTCP was able to detect channel capacitydue
to sufficient latency growth during network saturation. This
allowed XTCP to finish inless than half the time of TCP.
Non-saturated channel latency: 121 ms.5 6. 1.3. From the United
Kingdom to California, USA.Notes: This shows normal flow of
transmission. XTCP was able to detect channel capacity to
packetloss during initial exponential growth of send window. This
allowed XTCP to finish in less thantwelfth the time of TCP.
Non-saturated channel latency: 141 ms. 6 7. 1.4. From New-York, USA
to the United Kingdom.Notes: This shows normal flow of
transmission. XTCP was able to detect channel capacity due
tosufficient latency growth during network saturation. This allowed
XTCP to finish in around a tenththe time of TCP. Non-saturated
channel latency: 75 ms.7 8. 1.5. From New-York, USA to Central
America.Notes: This is probably an example of the worst case
scenario in terms of XTCP benefits over TCP.This environment is
rather advantageous for TCP, because the latencies are relatively
low. Lowlatencies allow TCP to better utilize the available
bandwidth. Furthermore, the connection is ratherstable, i.e., it
lacks sharp variations of available bandwidth and random dropped
packets which tendto create severe slowdowns for TCP. Indeed here
TCP is not stuck transmitting at a fraction of theavailable
bandwidth, as was the case for most other examples.Nevertheless,
even in this advantageous-for-TCP environment, XTCP manages to beat
TCP by about8%. This happens because the more responsive XTCP
algorithm is able to more accurately lock ontothe available
bandwidth. TCP, on the other hand, still features varying rates of
transmission withfrequent slowdowns. Non-saturated channel latency:
55 ms. 8 9. 1.6. From New-York, USA to California, USA.Notes: This
shows normal flow of transmission. XTCP was able to detect channel
capacity due tosufficient latency growth during network saturation.
As usual, TCP remained at a level significantlylower than the
channel capacity. XTCP performed more than ten times faster than
the standard TCPprotocol. Non-saturated channel latency: 76 ms. 9
10. 1.7. From Central America to United Kingdom.Notes: All tests
originating in Central America (Sections 1.7 1.9) possess one
commoncharacteristic: the outgoing channel bandwidth is
artificially limited. This means that the channelbandwidth is not
limited by the native capabilities of the networking hardware but
rather explicitlyshaped by additional equipment or software which
drops all packets transmitted at rates exceeding apreviously
defined artificial limit. Because these drops are artificial (i.e.,
not caused by limitations ofthe networking equipment), these drops
are not preceded by increases in latency or other suchindications
and are thus impossible to predict without existing knowledge of
the artificial bandwidthlimit. In other words, the network
characteristics remain stable right before the artificial limit
isreached because the artificial limit is usually much lower than
the hardware capabilities of thenetworking equipment. And because
the network characteristics remain stable, XTCP receives no
hintthat packet loss is impending and cannot limit throughput rates
accordingly. Thus, XTCP requiressome time to align itself with the
artificial limit and avoid unnecessary drops. During this time
XTCPexperiences relatively frequent drops. Nevertheless, these
drops seem to be significantly less thanthose experienced by
standard TCP, which similarly suffers from the artificial
limits.However tests taken in the reverse direction (to Central
America: Sections 1.2, 1.5, 1.12) do not seemto suffer from such
artificial limits. There exist several complicated explanations of
suchasymmetrical behavior but they are beyond the scope of this
document.The artificial limit is not the only problem here. The
channel is also moderately congested whichresults into unexpected
drops and latency variations. Several drops caused by explicit rate
shapingwere experienced until XTCP managed to detect the artificial
limit. Once XTCP detected the limit,further drops caused by
explicit shaping were avoided. Latency variations and random drops
causedXTCP to decrease transfer rates for several times during
transmission.10 11. On the other hand TCP was incapable of ever
reaching the artificial limit and thus transmitted at arate
significantly lower than the available bandwidth. TCP also suffered
severe and consistentvariation of transmission rates probably cause
by periodic dropped packets. Overall, XTCP was 60%faster than
TCP.Non-saturated channel latency: 121 ms.11 12. 1.8. From Central
America to New-York, USA.Notes: In this test channel bandwidth is
also artificially limited; however other networkcharacteristics
allowed XTCP to detect the limit after only couple of minor losses
at the verybeginning of transmission. Low channel latency allowed
standard TCP to utilize almost the entireavailable bandwidth.
However, TCP faced frequent drops caused by the artificial limit
and was notcapable of maintaining high transfer rates. Thus, TCP
suffered periodic performance degradations.The channel was
initially lightly congested which caused slight XTCP transfer rate
variation.Congestion-related packet losses in the first half of
XTCP transmission resulted into transfer ratedegradation but XTCP
quickly recovered from it. Non-saturated channel latency: 55 ms.12
13. 1.9. From Central America to California, USA.Notes: As noted in
Section 1.7 above an artificial bandwidth limit makes it harder for
XTCP (and TCPfor that matter) to detect the available bandwidth.
However, network latencies are relatively highwhich prevents TCP
from reaching the available bandwidth limit. XTCP has detected
transfer rateslimit after several drops caused by explicit shaping
and locked on it periodically probing the networkfor additional
available bandwidth. TCP was far from reaching the channel capacity
and sufferedseveral random losses during transmission.Our testing
has generally shown that even in the worst cases (as exemplified by
the sections 1.7-1.9and section 1.5) XTCP is usually able to
outperform TCP by a considerable margin and does notdegrade below
the performance of TCP.Non-saturated channel latency: 101 ms. 13
14. 1.10. From California, USA to United Kingdom.Notes: This shows
normal flow of transmission. XTCP was able to detect channel
capacity to packetloss during initial exponential growth of send
window. TCP, on the other hand transmitted at ratesmuch slower than
the channel capacity. This allowed XTCP to finish more than 13
times faster thanstandard TCP. Non-saturated channel latency: 141
ms.14 15. 1.11. From California USA to New-York, USA.Notes: This
shows normal flow of transmission. XTCP was able to detect channel
capacity to packetloss during initial exponential growth of send
window. This allowed XTCP to finish around 10 timesfaster than
standard TCP. Non-saturated channel latency: 77 ms. 15 16. 1.12.
From California, USA to Central America.Notes: This shows normal
flow of transmission. XTCP was able to detect channel capacity due
tosufficient latency growth during network saturation. TCP was
capable of utilizing the better part ofthe available bandwidth.
However, XTCPs advanced window sizing algorithm alongside
withaccurate bandwidth detection resulted into 42% performance
gain. Non-saturated channel latency:101 ms.16 17. 2. Automated
Linux testsUnix-based systems are well known for rich variety of
used TCP implementations. Previous sectiondescribed XTCP advantage
over MS Windows TCP implementation. Tests represented in Section
2were performed to compare XTCP to some other existing TCP
modifications.For those purposes a server running Ubuntu Linux
Server OS was deployed within the same subnetwith one of our
servers located at the United Kingdom. Testing process remained
exactly the sameexcept for the fact that one of the sender systems
was operated by Linux OS. Testing procedureinvokes successive
downloads from Linux system and then from system running XTCP to
the samerecipient. Both sender servers are located at the same
subnet (physically same LAN) which providesequal network conditions
and lets us say testing was absolutely fair.Three TCP
implementations were tested against XTCP: 1. TCP Vegas the ancestor
of proactive congestion avoidance idea, which is widely used in a
variety of present TCP modifications. 2. TCP Cubic the default TCP
modification used in most Linux systems. 3. HTCP advanced TCP
modification developed for high bandwidth-latency product
networks.17 18. 2.1.1 From the United Kingdom to New-York, USA. TCP
VegasNotes: This shows normal flow of transmission. XTCP was able
to detect channel capacity due topacket loss during initial
exponential growth of send window. XTCP was 5 times faster than
TCPVegas. Non-saturated channel latency: 75 ms.18 19. 2.1.2 From
the United Kingdom to Central America. TCP Vegas.Notes: This shows
normal flow of transmission. XTCP was able to detect channel
capacity due tosufficient latency growth during network saturation.
TCP Vegas was close enough to detectingavailable bandwidth at the
very start of transmission. However varying network latencies
hasprevented TCP Vegas from maintaining a good performance during
the whole transmission time.Non-saturated channel latency: 121 ms.
19 20. 2.1.3 From the United Kingdom to California, USA. TCP
VegasNotes: This shows normal flow of transmission. XTCP was able
to detect channel capacity due topacket loss during initial
exponential growth of send window. Due to high network latencies
TCPVegas was incapable of transmitting at high rates. Moreover,
latencies variation has prevented TCPVegas from reaching even this
low theoretical limit. As a result, XTCP has overperformed TCP
Vegasmore than 12 times. Non-saturated channel latency: 141 ms.20
21. 2.2.1 From the United Kingdom to New-York, USA. TCP CubicNotes:
This shows normal flow of transmission. XTCP was able to detect
channel capacity due tosufficient latency growth during network
saturation. XTCP has finished its transmission morethan 5 times
faster than TCP Cubic. Non-saturated channel latency: 75 ms.21 22.
2.2.2 From the United Kingdom to Central America. TCP Cubic.Notes:
This shows normal flow of transmission. XTCP was able to detect
channel capacity due tosufficient latency growth during network
saturation. TCP Cubic has shown its best and detectedthe available
bandwidth rather precisely. It, however, was always overshooting
the upper limitwhich in turn led to frequent losses and performance
degradations. XTCP behaved moreaccurately and provided a slight but
noticeable performance gain. Non-saturated channellatency: 121
ms.22 23. 2.2.3 From the United Kingdom to California, USA. TCP
CubicNotes: Network was initially highly congested during this
test. XTCP has suffered some randomlosses during bandwith
detection. However it managed to reach maximum possible transfer
ratesrather fast. TCP Cubic suffered moderate losses during all
transmission time and did not performanywhere close to XTCP.
Non-saturated channel latency: 141 ms. 23 24. 2.3.1 From the United
Kingdom to New-York, USA. HTCPNotes: This shows normal flow of
transmission. XTCP was able to detect channel capacity due topacket
loss during initial exponential growth of send window.
Non-saturated channel latency: 75ms.24 25. 2.3.2 From the United
Kingdom to Central America. HTCPNotes: This shows normal flow of
transmission. XTCP was able to detect channel capacity due
tosufficient latency growth during network saturation. HTCP has
precisely detected transfer rateslimit but has overshot it for
several times. It has slightly degraded HTCP performance.
Non-saturated channel latency: 121 ms. 25 26. 2.3.3 From the United
Kingdom to California, USA. HTCPNotes: This shows normal flow of
transmission. XTCP was able to detect channelcapacity due to packet
loss during initial exponential growth of send window.
HTCPexperienced several random losses but was capable to maintain
its transfer rates veryclose to theoretical TCP limit.
Non-saturated channel latency: 141 ms. MAINLINE NET HOLDING
LIMITED10.10.2010 26
