Transcript QoS Support in 802.11 Wireless LANs

Building a
connection-oriented internet
Malathi Veeraraghavan
Univ. of Virginia
• Outline
•
•
•
•
Problem statement
CHEETAH: an NSF-funded experimental project
Research problems
The “internet” name in the title
1
Problem statement
• How do we add on a complementary
connection-oriented internet to the
existing (connectionless) Internet?
– To allow end host applications to make a
reservation for bandwidth
– Anywhere from 10Mbps to 10Gbps
2
Motivation
• Analogy with transportation networks
Call to make a
reservation
(if only for part of
the distance:
airport-to-airport)
– “connectionless” roadways
• no reservation made
• “fair” but delay/jitter uncontrolled
– “connection-oriented” airlines
• reservation based
• better guarantees on delay/jitter
airport
airport
3
Motivation: eScience
NCSU
one 36-hour supernova
simulation creates 1TB
Supercomputers
4
Use Internet2 for 1TB download
•
•
Bottleneck link rate from NCSU to ORNL via Internet2/ESnet is 2.5 Gbps
But Prof. Blondin at NCSU only sees 300-400Mbps, Why?
5
Two reasons
• Disk access limitations at end hosts
• TCP-IP bandwidth sharing mode
– IP networks are connectionless – i.e., no reservations
– New transfers can simply start up
• impacts bandwidth available to ongoing transfers
– “Socialistic” resource sharing
Host
Host
IP router
Host
IP router
Host
Host
IP router
Host
Number of bytes
sent vs. time
• changing rate 6
Pros and cons of approach
• It allows an ongoing transfer to enjoy
bandwidth released by other
transfers as they complete
• On the other hand, as new transfers
start up within the duration of the
1TB transfer, its rate decreases
7
Coming back to the
problem statement
• How do we provide scientists a rateguaranteed connection for file
transfers
• At first glance, file transfers look
like an ideal app. for high-speed
circuits
– no burstiness
– can use as much as bandwidth as given
8
One answer: use optical fibers and circuit based gateways
Guaranteed rates + high bandwidth
Setup connection
(make reservation)
Transfer file
Release connection
Circuit
(release
resources)
signaling engine: dynamic call setup/release
Gigabit
Ethernet
interface
card
Control
Gigabit
Ethernet
interfaces
to hosts
Circuit
based
gateway
Time-division
or wavelength-division
multiplexing
optical interface
card
Circuit
based
gateway
Circuit
based
gateway
based
gateway
Circuit
based
gateway
Gigabit
Ethernet
interfaces
to hosts
• Gateways available that can crossconnect a Gigabit Ethernet
port to an equivalent-rate time-division or wavelengthdivision multiplexed signal dynamically
9
Networks deployed based on
this thinking (related work)
•
•
•
•
•
•
•
Canada’s Canarie: CA*net4
OptIPuter project: UCSD and Chicago
Chicago: OMNInet (Starlight PoP)
Dragon: DC area network
Netherlands: SURFnet
UK: UKlight
DOE UltraScience network
10
Our NSF-funded project:
CHEETAH
• Circuit-switched High-Speed End-to-End
Transport arcHitecture
• Participants:
–
–
–
–
Malathi Veeraraghavan, UVA
Nagi Rao, Bill Wing, Tony Mezzacappa, ORNL
John Blondin, NCSU
Ibrahim Habib, CUNY
• $3.5M project for three years, 2004-2007
Acknowledgment: NSF EIN grant ANI-0335190
11
Adding Cheetah into
existing optical NC
university network
Duke
15454
UNC
Wavelength
Division
Mux/Demux
(WDM)
optical
fiber
Internet2
connection
15454
15454
MCNC
Qwest PoP
Centaur
Lab
(John
Blondin’s
cluster
compter)
NCSU
15454
Circuitbased
gateway
15454
10 Gb/s Transponder
Level(3) PoP
15454
NLR
15808
National LambdaRail
Mark Johnson, MCNC
12
smj 10-28-04
National LambdaRail
Affordable
prices for
10Gbps
lambdas
13
Cheetah network
To DC – Dragon
NC
Implements
sig. protocols
To cluster computer
Circuit-based gateway
OC192 Control
card
card
ORNL
To Cray
Circuit based gateway
OC192 Control
card
card
GbE/
10GbE
card
GbE/10GbE
Ethernet
Switch
GbE/ 10GbEGbE/10GbE
10GbE
Ethernet
card
Switch
NLR
WDM
ORNL
WDM
10 Gbps lambda
NCSU
MCNC/NLR
10 Gbps lambda
NLR
SOX
Atlanta
GaTech
WDM
OC192 
GaTech
WDM
14
Connecting Cheetah to Dragon
and Ultrascience networks
Dragon
DOE Ultrascience
network (ORNL)
Cheetah
15
All this is fun, but
• What are the research problems?
What happens if the call gets blocked?
– Bandwidth sharing modes
• Low load performance
• Scheduled vs. immediate-request
• Long paths vs. short paths
– Mismatch between multitasking end
hosts and TDM circuits
16
What happens
if the call gets blocked?
• In TCP/IP networks, your new transfer
just joins in, perhaps receives small BW
until some other transfers complete
• In circuit-switched networks, your call is
accepted or rejected
– If rejected, what then?
– Call queueing – wastes resources
• because of multiple links
Acknowledgment: NSF ITR grant ANI-0312376
17
Practical answer: Leverage presence
of Internet path and fall back to it
• Use second NICs at hosts for circuit connectivity
leaving primary NIC for Internet access
Connectionless
Internet
End host
I
Circuit-Switched
Network
• Attempt circuit setup
• If rejected, fall back to
using TCP/IP
Two paths available
End host
II
Should we attempt a circuit
setup for ALL file transfers?
18
Expected delay on
TCP/IP path
Throughput B(p): approximately reciprocal of expected delay




Wmax
1


B( p )  min
,
 RTT
2bp
3bp 

2 
RTT
 T0 min 1,3
 p (1  32 p ) 

3
8 



• Main factors:
– Round-Trip Time (RTT) – main Tprop
– Prob. of packet loss on IP path, p,
– Bottleneck link rate
• Other terms:
– Wmax: receiver window size
– b= 2 (ACK-every-othersegment)
– T0: initial time-out
J. Padhye, V. Firoiu, D. Towsley, and J. Kurose, “Modeling TCP Throughput: A
Simple Model and its Empirical Validation,” Proc. of ACM SIGCOMM 98, Aug.
31 - Sep. 4, Vancouver Canada, pp. 303-314.
19
Mean TCP delays
Input parameters
Loss p
Rate r
Round- trip
prop. delay
T prop
Case 1
Case 2
Case 3
Case 4
Case 5
Case 6
0.0001 100 M bps
Case 7
Case 8
Case 9
Case 10
Case 11
Case 12
Case 13
Case 14
Case 15
Case 16
Case 17
Case 18
Case 19
Case 20
Case 21
Case 22
Case 23
0.001 100M bps
0.0001 1Gbps
0.001 1Gbps
0.01 100 M bps
0.01 1Gbps
0.1 100 M bs
0.1 1Gbps
Intermediate results
Queuing
RTT (ms)
delay plus
service time
0.1ms
5ms
50ms
0.1ms
5ms
50ms
0.2ms
0.1ms
5ms
50ms
0.1ms
5ms
50ms
0.1ms
5ms
50ms
0.1ms
5ms
50ms
0.1ms
5ms
50ms
0.1ms
5ms
0.26ms
0.02ms
0.026ms
0.38ms
0.038ms
0.68ms
0.068ms
Final result
Wmax
(pkts)
Mean delay
for a 1GB
file (s)
0.3
5.2
50.2
0.12
5.02
50.02
2.5
41
418
10
418
4168
82.25
89.45
396.5
8.25
39.6
395.7
0.36
5.26
50.26
0.13
5.03
50.03
0.48
5.38
50.38
0.138
5.038
50.04
0.78
5.68
50.68
0.168
5.068
3
43.8
418.8
10.8
419
4169
4
44.8
419.8
11.5
419.8
4169.8
6.5
47.33
422.33
14
422.33
82.93
135.4
1293
8.64
129.4
1287
92.41
471.7
4417
12.43
441.7
4387
283.56
2064.9
18424
61.07
1842.4
Impact
of propagation
delay
Low impact
of bottleneck
link rate
in wide-area
networks
Impact
of packet
loss rate
20
Delays incurred in using an
end-to-end circuit
• Circuit setup delay + File transfer delay
 sig
 sp
E (Tsetup) 
 (1 
)  (k  1)  Tsp  (1 
)  k  T prop
rs
2(1   sig )
2(1   sp )
msig
Ttransfer 
•
•
•
•
•
f
rc
msig: message length; rs: signaling link rate
Loads: sig and sp: sig. link and processor
Tsp: signaling protocol processing delay
k: number of switches; Tprop: r.t. prop. delay
f: file size rc: circuit rate
Acknowledgment: NSF ANI grant 0087487 for hardware signaling
21
Should the application attempt
a circuit setup or not?
• Mean delay if a circuit setup is attempted
E[Tcheetah ]  (1  Pb )( E[Tsetup ]  Ttransfer )  Pb ( E[T fail ]  E[Ttcp ])
Pb: call blocking probability in the circuit-switched network
If circuit setup fails, fall back to Internet path
22
Routing decision

if E[T

] attemptcircuit setup
if E[Tcheetah]  E[Ttcp ] use theT CP /IPpath
cheetah] 
E[Ttcp
 E[Tsetup]


if 
 E[Ttcp ]  Ttransfer use t heT CP /IPpat h
 1  Pb

 E[Tsetup]


if 
 E[Ttcp ]  Ttransfer at t emptcircuit set up
 1  Pb

23
Numerical results
link rate = 1Gbps
Tprop = 0.1ms
Tprop = 50ms
24
Crossover file sizes
When rc = 100Mbps and Tprop = 0.1ms
Measure of loading on
ckt. sw.
network
Pb = 0.01
Pb = 0.1
Pb = 0.3
TCP/IP path
rc = 1Gbps,
Tprop = 0.1ms
Ploss = 0.0001
Ploss = 0.001
Ploss = 0.01
22MB
9MB
1.2MB
24MB
10MB
1.4MB
30MB
12MB
1.8MB
25
Utilization considerations
• Example: in 50ms scenario, if we transfer a
100KB file over a 100Mbps path, transfer
time is only 8ms. Circuit utilization is
8/(50+8) = 13.7%
• Two opposing factors
– If the crossover file size (beyond which circuit
setup is attempted) is increased
• per-circuit utilization increases
• traffic load decreases (Pareto distribution of file
sizes), which means aggregate utilization decreases
26
Aggregate utilization ua
(1  Pb )  
 m / m!
ua 
, where Pb  m
k
m
  / k!
: traffic load
m: number of circuits
Pb: call blocking probability
k 0
Assuming file size follows Pareto
distribution
– Define fractional offered load
For a 1% call blocking probability Pb = 0.01

1
10
100
m
4
17
117
ua
24.8%
58.2%
84.6%

k  1
'
P( X   ) E[ X ] | X   ]   ( )
E( X )


 (fraction of )
40KB
81%
330KB
71%
80MB
51%
27
Plot of utilization u with
rc= 100Mbps, k=20
Pb=0.3
Pb=0.01
28
Research problems
• What are the research problems?
– What happens if the call gets blocked?
Bandwidth sharing modes
• Low load performance
• Scheduled vs. immediate-request
• Long paths vs. short paths
– Mismatch between multitasking end
hosts and TDM circuits
29
Connection-oriented bandwidth sharing
for file transfers
• This is a new problem
– Real-time (interactive) audio-video applications
generate data at a certain rate (constant or
variable)
• implication: application requests the required
bandwidth from the network, and answer is binary
(accept or reject); multiple classes
– File transfers: “any” bandwidth that the
network can provide could be acceptable
• implication: application requests a MAX bandwidth,
but the answer can be multi-level
30
Fixing the bandwidth for the transfer could be
a bad thing: low load problem
1
1
2
3
.
.
N
Packet
Switch
1
Capacity C
2
2
3
3
N
The lone remaining transfer enjoys
Each transfer
C/N capacity
fullgets
capacity
C
.
.
N
Circuit
Switch
1
Capacity C
2
3
N
Eachlone
transfer
is allocated
C/Ncontinues
capacity
The
remaining
transfer
with capacity allocation C/N
• Varying bandwidth list scheduling algorithm
– uses knowledge of file size to make varying bandwidth
allocations for transfer
– catch: requires circuit switches to be reprogrammed
multiple times within lifetime of a transfer (circuit)
31
Scheduled vs. immediate-request
calls
Session type requests:
• long holding times (2 hours)
• specific rate
• remote visualizations
• scientists participate in sessions
• best served with an advance reservation
Small files (e.g. 1 GB on 1 Gbps takes 8 sec)
• should be handled in immediate-request mode
File transfer requests:
• file sizes provided not holding times
• max rate specified but any rate can be allocated
• scientists not involved; just computers
Large files (e.g. 1 TB on 1 Gbps takes 2.2 hours)
• should be handled in scheduled mode
• should we allocate 10Gbps and finish in 800 sec?
• immediate-request? or scheduled?
32
Specific research activities
• Scheduling
– Design and compare algorithms for scheduling sessiontype and large file transfer requests
• Use preemption and repositioning of large file transfer
requests
• Immediate-request call admission algorithms
– Use Markov Decision Process (MDP) tools to balance
fairness and overall throughput
– Long-path and short-path calls
– Large files (high-BW) and short files (low-BW) calls
– Multi-level answer rather than binary accept/reject
• Both with Fixed bandwidth and Varying bandwidth
33
Research problems
• What are the research problems?
– What happens if the call gets blocked?
– Bandwidth sharing modes
• Low load performance
• Scheduled vs. immediate-request
• Long paths vs. short paths
Mismatch between multitasking end
hosts and TDM circuits
34
Mismatch between multitasking
end hosts and TDM circuits
File
transfer
Matlab
user space
Network kernel
protocols
Filesystem
network
card
Matlab
Network
protocols
File
transfer
Filesystem
network
card
Circuit-switched
network
•
Variability in sender:
•
Variability in receiver: if buffer not emptied out, data loss occurs
–
other processes (e.g. matlab) + disk access (disk head location)
35
Effects of mismatch in nature
of circuits and nature of hosts
• Choose a high circuit rate and receive
buffer can overflow causing losses
– impacts delay + utilization (retransmissions)
• Choose a low circuit rate and delay can be
high
• If sending rate is not matched exactly with
circuit rate
– circuit lies idle; utilization impacted
36
Fixed Rate Transport Protocol (FRTP)
• Set up a circuit at a carefully chosen rate
• Send data at that rate
– hard to meter out data at a fixed rate from a
multitasking sender when that rate is high
(Linux system time granularity: 10ms)
• No changes of sending rate
– i.e., no flow control or congestion control
• Packet losses recovered through
retransmissions
– no timers needed, just negative ACKs
• because of in-sequence delivery
37
Experimental results
CIRCUIT RATE CIRCUIT
RELATIVE
(Mbps)
UTILIZATION TRANSFER
(%)
DELAY
200
90
1.7
590
62
1.0
38
Current work
• Experimenting with RT schedulers to
schedule file transfer task in a set
rhythm
• Experimenting with file systems to
characterize file write time to collect
data to then determine circuit rate
and receive buffer size
39
Back to the outline
• Outline
• Problem statement
• CHEETAH: an NSF-funded experimental project
• Research problems
 The “internet” name in the title
40
Connection-oriented networks
• Many flavors
– Circuit switched
• Time Division Multiplexed (SONET)
– Equipment vendors: Sycamore, Ciena
– Network: Cheetah, UltraScience Net, CA*net 4
• Wavelength Division Multiplexed (WDM)
– Equipment vendors: Movaz, Calient, LambdaOptical
– Network: Dragon, OMNInet, Internet2 HOPI
41
Connection-oriented networks
• Many flavors
– Packet switched
• Multiprotocol Label Switching (MPLS)
– Equipment vendors: Cisco, Juniper
– Network: Internet2, ESnet
• Virtual Local Area Network (VLAN)
– Equipment vendors: Dell, Intel, Foundry, Extreme
– Network: Enterprise local area networks
– Just need to “enable” connection-oriented
network through already deployed boxes
42
Bandwidth sharing problem
in heterogeneous network
Request for 30Mbps connection
1Mbps
2, 30Mbps
b
Switch granularity
1, 150Mbps
10Gbps
a
f
d
5, 100Mbps
2, 50Mbps
c
1, 10Mbps
1, 50Mbps
1, 50Mbps
1, 500Mbps
e
51Mbps
1Mbps
• Problem:
– Tradeoff of fairness and utilization becomes
more difficult when these crossconnect
granularities are considered
43
Interconnecting these
networks
• Tricky business!
• Involves many levels of interworking
protocols
– User (data) plane
– Signaling protocols (for connection
setup/release)
– Routing protocols (for reachability,
topology, loading data dissemination)
44
But
• We need to solve this
internetworking problem for a true
connection-oriented service to
flourish!
Acknowledgment: DOE grant
45
Summary
•
•
•
•
Rich new set of research problems
Experimental challenges a plenty!
Real opportunity to deploy a network
Web site:
http://cheetah.cs.virginia.edu
46