Transcript Fast TCP - Chinese University of Hong Kong

FAST TCP
Speaker:
Veune:
Date:
Time:
Ray
Room 1026
25th October, 2003
10:00am
Motivation

Demand for ultrascale networking
HENP (High Energy and Nuclear Physics)
 Data volumes of tens of Petabytes (1015) to
Exabytes (1018)
 Require Terabit/sec (1015 bit/sec or 1000Gbit/sec)


Scalability problem of TCP
Losses must be extremely rare
 TCP must induce loss
 Underutilization and oscillation

Scalability problem of TCP
extremely loss packet loss possibility


Rate = 1.3 * MTU / (RTT * sqrt(Loss))
MTU = 1500bytes, RTT = 10ms
Bandwidth
10Mbps
Loss rate
2.43 x 10-2
100Mbps
1Gbps
10Gbps
2.43 x 10-4
2.43 x 10-6
2.43 x 10-8
1Tbps
1Tbps (RTT:100ms)
2.43 x 10-12
3.80 x 10-14
Scalability problem of TCP
inevitable packet loss


TCP needs to create losses
Single bit network feedback signal
Scalability problem of TCP
Underutilization and oscillation


AIMD (1, 0.5)
At large window size (in excess of 10,000 pkts):
Halving window on loss event is too drastic
 Increasing window by one packet per RTT is too
conservative

FAST TCP
Achievement


CERN (European Organization for Nuclear
Research) sent 1.1 Terabytes of data at 5.44
Gbps
Full-length DVD film in 7 seconds !!
FAST TCP




Flow based vs Packet based
Network delay vs Packet loss
TCP-Vegas vs TCP-Reno
Stabilized Vegas
TCP Vegas
Techniques



New Retransmission Mechanism
Congestion Avoidance Mechanism
Modified Slow-Start Mechanism
TCP Vegas
New Retransmission Mechanism


Timeout
n duplicate ACKs
TCP Vegas
New Retransmission Mechanism


Check time record for
the first duplicate packet
non-duplicate ACKs first
or second after
retransmission

Catch other segment lost
previous to
retransmission
TCP Vegas
Congestion Avoidance Mechanism


Detect network delay by monitoring RTT
BaseRTT and ActualRTT
Source
Router
Dest.
TCP Vegas
Congestion Avoidance Mechanism

Expected = WindowSize / BaseRTT

Diff = Expected - Actual
Diff >> 0, decrease sending rate
 Diff = 0, increase sending rate


α< Diff < β
α
β
Diff
TCP Vegas
Congestion Avoidance Mechanism


Extra buffers occupied
BaseRTT: 100ms, segment size: 1KB
Expected = WindowSize / BaseRTT
α = 30KB/s, β=60KB/s
α=> 30KBps * 100ms / 1KB = 3
β=> 60KBps * 100ms / 1KB = 6
TCP Vegas
Congestion Avoidance Mechanism


α=β
Diff <α


Diff =α


increase one segment per RTT
no change in windows size
Diff >α

decrease one segment per RTT
TCP Vegas
Slow-Start Mechanism

TCP-reno
Send two segment for each ACK received
 Exponential growth every RTT


TCP-Vegas
Exponential growth every alternative RTT
 γthreshold


Diff >γ

Changes from slow-start mode to linear I/D mode
Stability of TCP Vegas
Network model

Set of L links with finite capacities c



c = (cl , l  L)
N sources indexed by r
Each source r uses a set of link defined by the L
 N routing matrix
Rlr = {
1 if source r uses link l
0 otherwise
Stability of TCP Vegas
Network model




For each link l, the congestion measure pl(t) is
call price
For each source r, it maintains a rate xr(t) in
packets/sec
Equilibrium forward delay from source r to link
l :  lr
Equilibrium backward delay from link l to
source r :  lr
Stability of TCP Vegas
Network model
x1(t)
x2(t)

p3(t)
p4(t)
p2(t)
Aggregate price source r observes in its path


p1(t)
qr(t) =  Rlr pl (t -  lr )
l
Aggregate source rate link l observes
 yl
(t) =  Rlr xr (t -  lr )
r
Stability of TCP Vegas
Network model

Tr denote equilibrium RTT


lr
+
lr
= Tr,
lL
Dynamical system of TCP Vegas
pl (t) =
( yl ( t ) – cl ) / cl
( yl ( t ) – cl ) / cl )+
if pl (t) > 0
if pl (t) = 0
xr (t) = 1/Tr 2(t) sgn( 1 – xr(t)qr(t) / rdr )
Tr (t) = dr + qr( t )
Where
sgn(z) = 1 if z > 0, -1 if z < 0 and 0 if z = 0
(z)+ = max { 0 , z }
Stability of TCP Vegas
Approximate model


xr (t) = 1/Tr 2(t) sgn( 1 – xr(t)qr(t) / rdr )
sgn(z)  2/ tan-1 (z)


  
xr (t) = (2/Tr 2(t))tan-1 (1 – xr(t)qr(t) / rdr )
Stability of TCP Vegas
Approximate model

In equilibrium, the source rate xr* and aggregate
price qr* satisfy
xr* qr* = r dr
Stability of TCP Vegas
Theorem 1



Suppose for all r, k0Tr  maxr Tr for some k0.
Let M be an upper bound on the number of links in
the path of any source, M  maxr l Rlr.
The Vegas model is locally asymptotically stable around
the equilibrium point (xr* , yl* , pl* , qr* ) if
maxr xr* Tr sinc  (ň / xr* Tr ) <  / Mk02



ň = 2/
Let (a) be the unique solution in ( 0, /2) of  tan = a as a strictly increasing
function of a
sinc  = sin /
Stability of TCP Vegas
Theorem 1





maxr xr* Tr sinc  (ň / xr* Tr ) <  / Mk02
() is strictly increasing
sinc() is strictly decreasing
LHS is strictly increasing in windows size xr* Tr
Theorem 1: Stability condition impose a limit on
max windows size
Stability of TCP Vegas
Corollary 2

maxr xr* Tr sinc  (ň / xr* Tr ) <  / Mk02
minr


qr* /Tr
sinc  (
ň

.
qr*
Tr
)
> Mk02
All source has the same target queue length, r dr = 
for all r
Corollary 2: LHS is strictly increasing in qr* / Tr ,
implying a lower bound on queueing delay
Stability of TCP Vegas
Corollary 3







Since () <  / 2, sinc () > 2 / , k0  1
Corollary 3: minr qr* / Tr > 2M / 
If M 2, then RHS bigger than 1, since Tr = dr + qr*
M=1
The stability condition cannot be satisfied if a source
has more than one link
Sufficient in multilink case
Necessary and sufficient in single-link-homogeneoussource case
Stability of TCP Vegas
Single link with homogeneous source



A single link of capacity c,
Shared by N homogeneous source,
with round trip propagation delay d
Stability of TCP Vegas
Single link with homogeneous source






From corollary 3: qr* / Tr > 2 / 
for all r
Tr / qr* < /2,
since Tr = d + qr*
d / qr* < (/2 – 1) =>
d < (/2 – 1) qr*
Since qr* =  / xr* =>
( N)/c
cd < (/2 – 1)  N
Conclusion: bandwidth delay product should be
small
Stabilized Vegas






xr (t) = (2/Tr 2(t))tan-1 (1 – xr(t)qr(t) / rdr )
xr (t) = (w/Tr 2(t))tan-1 r(t)(1 – xr(t)qr(t)/rdr -r(t) qr(t))
1 – xr( t ) qr( t ) /r dr
1 – xr( t ) qr( t ) /r dr -r( t ) qr( t )
r( t ) = (1 / ) ( Tr( t ) / qr( t ) )
r( t ) = (  / w ) ( xr( t ) Tr( t ) )
Stabilized Vegas




The gain r( t ) serves as a normalization to
qr( t )
Additional differential term r( t ) qr( t )
anticipates the future of qr( t )
Without: xr( t ) will increase if xr(t)qr(t)< rdr
Even xr(t)qr(t)/rdr is small, xr( t ) may decrease
if prices are rapidly growing
Stabilized Vegas

Stability condition for stabilized Vegas
cd < (




2 + 2a2
2 + a2
- 1 ) N
where
 = tan-1 ( (2)/(1-) )
Stabilized Vegas can choose a small
( a>0, (0,1) ) such that RHS can be larger for better
stability of the original Vegas cd < (/2 – 1)  N
Simulation Results
Reno/Reno Reno/Vegas
Regas/Reno Vegas/Vegas
Throughput (KB/s) 60/109
61/123
66/119
74/131
Throughput Ratio
1.02/1.13
1.10/1.09
1.23/1.20
Retranmissions (KB) 30/22
43/1.8
1.5/1.8
0.3/0.1
Retransmit Ratios
1.43/0.08
0.05/0.82
0.01/0.01
1.00/1.00
1.00/1.00
One-on-One (300KB and 1MB) Transfers
c = 200KB/s
50ms delay
Simulation Results





 = 20
N = 100 flows
Fixed packet size of 1KB
FIFO /w Droptail, queue capacity = 20000
( a ,  ) = ( 0.5 , 0.015 )
Simulation Results
c = 100 pkts/s and d = 10ms
Simulation Results
c = 1000 pkts/s and d = 10ms
Simulation Results
c = 100 pkts/s and d = 100ms
Simulation Results
Experimental Results


FAST TCP was first demonstrated publicly in during
the SuperComputing Conference (SC2002) in Baltimore,
MD, in November 16–22 2002
Caltech-SLAC research team






CERN
DataTAG
StarLight
TeraGrid
Cisco
Level(3).
Experimental Results
Experimental Results
Throughput and utilization
#flow
Throughput
Mbps
utilization
Delay
ms
Distance
km
Duration
s
Bmps
1015
Transfer
GB
1
925 (266)
95% (27%)
180
10,037
3,600
9.28
(2.67)
387 (111)
1,797 (931) 92% (48%)
180
10,037
3,600
18.03
(9.35)
753 (390)
2


7
6,123
90%
85
3,948
21,600
24.17
15,396
9
7,940
90%
85
3,948
4,030
31.35
3,725
10
8,609
88%
85
3,948
21,600
33.99
21,647
SC2002 FAST experimental result
Current TCP implementation in Linux v2.4.18 on Jan
27-28, 2003
FAST TCP
Conclusion

Problem of current TCP
Equilibrium
 Dynamic problem


FAST TCP
Equation-based control with queuing delay
 TCP Vegas
 Stabilized Vegas

Q&A
Thank you