Transcript FAST v3
FAST TCP
Cheng Jin
David Wei
Steven Low
netlab.CALTECH.edu
GNEW, CERN, March 2004
Acknowledgments
Caltech
Bunn, Choe, Doyle, Jin, Newman, Ravot, Singh, J.
Wang, Wei
UCLA
Paganini, Z. Wang
Martin, Martin-Flatin
Cottrell, Mount
Aiken, Doraiswami, Yip
Fernes
Wu
CERN/DataTAG
Internet2
Almes, Shalunov
SLAC
Cisco
Level(3)
LANL
FAST project
NSF RI
(2003)
NSF
STI
(2002)
HEP networks TeraGrid
Abilene
DummyNet
Abilene
HEP networks PlanetL
WAN
in Lab
Linux
TCP kernel
NSF
ITR
(2001)
Performance
Stability
Other
platforms
Fairness TCP/IP
IETF
GGF
Deployment
UltraLight
testbed
Experiment
Monitoring
Debugging
Implement
Noise
Random
-ness
Theory
Outline
Experiments
Results
Future plan
Status
Open issues
Code release mid 04
Unified framework
Reno, FAST, HSTCP, STCP, XCP, …
Implementation issues
Aggregate throughput
88%
FAST
Standard MTU
Utilization averaged over > 1hr
90%
90%
Average
utilization
92%
95%
1hr
1 flow
1hr
2 flows
6hr
7 flows
1.1hr
6hr
9 flows
10 flows
DataTAG: CERN – StarLight – Level3/SLAC (Jin, Wei, Ravot, etc SC2002)
FAST
Linux
Dynamic sharing: 3 flows
Dynamic sharing on Dummynet
capacity = 800Mbps
delay=120ms
3 flows
iperf throughput
Linux 2.4.x (HSTCP: UCL)
FAST
Linux
Dynamic sharing: 3 flows
Steady throughput
HSTCP
STCP
queue
FAST
loss
Linux
throughput
30min
Dynamic sharing on Dummynet
capacity = 800Mbps
HSTCP
delay=120ms
14 flows
iperf throughput
Linux 2.4.x (HSTCP: UCL)
STCP
queue
Room for mice !
FAST
loss
Linux
throughput
HSTCP
HSTCP
30min
STCP
Aggregate throughput
ideal
performance
Dummynet: cap = 800Mbps; delay = 50-200ms; #flows = 1-14; 29 expts
Aggregate throughput
large
window
8000
small
window
800pkts
Dummynet: cap = 800Mbps; delay = 50-200ms; #flows = 1-14; 29 expts
Fairness
Jain’s index
Dummynet: cap = 800Mbps; delay = 50-200ms; #flows = 1-14; 29 expts
Stability
stable in
diverse
scenarios
Dummynet: cap = 800Mbps; delay = 50-200ms; #flows = 1-14; 29 expts
Outline
Experiments
Results
Future plan
Status
Open issues
Code release
Unified framework
Reno, FAST, HSTCP, STCP, XCP, …
Implementation issues
Benchmarking TCP
Not just static throughput
Dynamic sharing, what protocol does to network, …
Tests to zoom in on specific properties
Throughput, delay, loss, fairness, stability, …
Critical for basic design
Test scenarios may not be realistic
Tests with realistic scenarios
Same performance metrics
Critical for refinement for deployment
Just started
Input solicited
What’s realistic for your applications?
Open issues: well understood
baseRTT estimation
route changes, dynamic sharing
does not upset stability
Small network buffer
at least like TCP
adapt a on slow timescale, but how?
TCP-friendliness
friendly at least at small window
tunable, but how to tune?
Reverse path congestion
should react? rare for large transfer?
Status: code release
Source release mid 2004
For any non-profit purposes
Re-implementation of FAST TCP completed
Extensive testing to complete by April 04
Pre-release trials
CFP for high-performance sites!
Incorporate into Web100
with Matt Mathis
Status: IPR
Caltech will license royalty-free if FAST TCP
becomes IETF standard
IPR covers more broadly than TCP
Leave all options open
Outline
Experiments
Results
Future plan
Status
Open issues
Code release mid 04
Unified framework
Reno, FAST, HSTCP, STCP, XCP, …
Implementation issues
Packet & flow level
Reno TCP
Packet level
W
W + 1/W
Loss: W
W – 0.5W
ACK:
Flow level
Equilibrium
Dynamics
pkts
(Mathis formula)
Reno TCP
Packet level
Designed and implemented first
Flow level
Understood afterwards
Flow level dynamics determines
Equilibrium: performance, fairness
Stability
Design flow level equilibrium & stability
Implement flow level goals at packet level
Reno TCP
Packet level
Designed and implemented first
Flow level
Understood afterwards
Flow level dynamics determines
Equilibrium: performance, fairness
Stability
Packet level design of FAST, HSTCP, STCP, H-TCP, …
guided by flow level properties
Packet level
Reno
AIMD(1, 0.5)
HSTCP
AIMD(a(w), b(w))
STCP
MIMD(a, b)
FAST
W
W + 1/W
Loss: W
W – 0.5W
ACK:
W
W + a(w)/W
Loss: W
W – b(w)W
ACK:
W
W + 0.01
Loss: W
W – 0.125W
ACK:
baseRTT
RTT : W W
a
RTT
Flow level:
Reno, HSTCP, STCP, FAST
Similar flow level equilibrium
MSS/sec
a = 1.225 (Reno), 0.120 (HSTCP), 0.075 (STCP)
Flow level:
Reno, HSTCP, STCP, FAST
Common flow level dynamics
window
adjustment
=
control
gain
flow level
goal
Different gain k and utility Ui
They determine equilibrium and stability
Different congestion measure pi
Loss probability (Reno, HSTCP, STCP)
Queueing delay (Vegas, FAST)
FAST TCP
Reno, HSTCP, and FAST have common flow level
dynamics
window
adjustment
=
control
gain
flow level
goal
Equation-based
Need to estimate “price” pi(t)
pi(t) = queueing delay
Easier to estimate at large window
k(t) and U’i(t) explicitly designed for
Performance
Fairness
Stability
Window control algorithm
Full utilization
regardless of bandwidth-delay product
Globally stable
exponential convergence
Intra-protocol fairness
weighted proportional fairness
parameter a
FAST tunes to knee
Goal:
• Less delay
• Less jitter
FAST
TCP
stabilized
oscillation
Window adjustment
FAST TCP
netlab.caltech.edu/FAST
FAST TCP: motivation, architecture,
algorithms, performance
IEEE Infocom March 2004
FAST TCP: from theory to experiments
Submitted for publication April 2003
Panel 1:
Lessons in Grid Networking
Metrics
Performance
Throughput, loss, delay, jitter, stability,
responsiveness
Availability, reliability
Simplicity
Application
Management
Evolvability, robustness
Constraints
Scientific community
Small & fixed set of major sites
Few & large transfers
Relatively simple traffic characteristics and
quality requirements
General public
Large, dynamic sets of users
Diverse set of traffic characteristics &
quality requirements
Evolving/unpredictable applications
Mechanisms
Fiber infrastructure
Lightpath configuration
Resource provisioning
Traffic engineering, adm control
Congestion/flow control
Months - years
Mintes - days
Service: sec - hrs
Flow: sec - mins
RTT: ms - sec
Timescale: desired, instead of feasible
Balance: cost/benefit, simplicity, evolvability