Transcript CS 5865
Comparing the Accuracy of Network Simulators for Packet-Level Analysis using a Network Testbed
Chaudhry Usman Ali UNB, Fredericton
TESTING SIMULATOR ACCURACY
• •
Measure real network traffic.
Compare real network traffic with the simulator results.
Real Traffic Types of Traffic: CBR (Constant Bit Rate) data traffic.
(Simplicity)
FTP (File Transfer Protocol) session.
(Popularity and Dynamic behaviour)
Simulated Traffic CBR (Few external factors influence the model) FTP(Complex) HTTP(too many degrees of freedom)
METHODS OF SIMULATION
Discrete Event Analytical Simulation Produce predictions in the network at a low level (packet by-packet).
Accurate.
But SLOW !
Use mathematical models to produce their results.
Sacrifice Accuracy !
Faster speed !
NS-2 and OPNET use a hybrid approach.
SIMULATION TOOLS
•
OPNET MODELLER
•
NS-2
1. Finite State Machine + Analytical Model.
2. 400 special purpose modelling functions.
3. Not open source.
4. Model parameters can be altered. 1. Numerous models of Internet protocols.
2. NAM network animator. 3. Animations and graphs.
C++ :
per packet processing.
Otcl :
Simulation Scenario generation. 4. Custom applications and protocols can be created.
NETWORK TESTBED
5 PCs.
Client-Server Pair.
G- Traffic Generator.
S - Traffic Sink.
Hub 10/100 Mbit links.
* KURT
G. Flores-Lucio, M. Paredes-Farrera, E. Jammeh, M. Fleury, and M. Reed, OPNET-Modeler and NS-2: "Comparing the Accuracy of Network Simulators for Packet-Level Analysis using a Network Testbed". 3rd WEAS Int. Conf. on Simulation, Modelling and Optimization (ICOSMO 2003), Crete, vol. 2, pp. 700-707, 2003.
NETWORK REPRESENTATION
OPNET NS-2
G. Flores-Lucio, M. Paredes-Farrera, E. Jammeh, M. Fleury, and M. Reed, OPNET-Modeler and NS-2: "Comparing the Accuracy of Network Simulators for Packet-Level Analysis using a Network Testbed". 3rd WEAS Int. Conf. on Simulation, Modelling and Optimization (ICOSMO 2003), Crete, vol. 2, pp. 700-707, 2003.
EXPERIMENTS
CBR Traffic
1. Characterized by a fixed bandwidth across the network.
2. Typically used by applications such as video and audio.
3. CBR traffic stream can be generated by fixing the packet size and using the same inter arrival time between packets.
FTP Session
Share, transfer and transmit information between two computers.
Two types of test were performed for the FTP experiments.
1- FTP sessions using default settings.
2- Finely tuned FTP parameters.
PARAMETER TUNING
Reno to New Reno Window Scaling Timestamp Link Latency Max Segment Size Window Size
Fast Recovery capability + recover without retransmission.
Allows advertisement of window sizes bigger than 65 kB.
Imitates the echoing capability of the testbed in both directions.
Affects RTT.
Bigger Ethernet packets.
More range for sliding the window value.
G. Flores-Lucio, M. Paredes-Farrera, E. Jammeh, M. Fleury, and M. Reed, OPNET-Modeler and NS-2: "Comparing the Accuracy of Network Simulators for Packet Level Analysis using a Network Testbed". 3rd WEAS Int. Conf. on Simulation, Modelling and Optimization (ICOSMO 2003), Crete, vol. 2, pp. 700-707, 2003.
TRAFFIC SCENARIOS
IP Traffic monitored using tcpdump.
G. Flores-Lucio, M. Paredes-Farrera, E. Jammeh, M. Fleury, and M. Reed, OPNET-Modeler and NS-2: "Comparing the Accuracy of Network Simulators for Packet-Level Analysis using a Network Testbed". 3rd WEAS Int. Conf. on Simulation, Modelling and Optimization (ICOSMO 2003), Crete, vol. 2, pp. 700-707, 2003.
RESULTS
Scenario#
CBR1
Client –Server Load
2 Mb/s
Traffic G. – Traffic S. Load ROUTER`s PERSPECTIVE
0 Mb/s
NS-2 displays a more realistic CBR behaviour.
G. Flores-Lucio, M. Paredes-Farrera, E. Jammeh, M. Fleury, and M. Reed, OPNET-Modeler and NS-2: "Comparing the Accuracy of Network Simulators for Packet-Level Analysis using a Network Testbed". 3rd WEAS Int. Conf. on Simulation, Modelling and Optimization (ICOSMO 2003), Crete, vol. 2, pp. 700-707, 2003.
Scenario#
CBR2 CBR3
Client –Server Load
2 Mb/s 5 Mb/s
Traffic G. – Traffic S. Load ROUTER`s PERSPECTIVE
2 Mb/s 0 Mb/s
Lower throughput for the network testbed.
Scheduling problems?
G. Flores-Lucio, M. Paredes-Farrera, E. Jammeh, M. Fleury, and M. Reed, OPNET-Modeler and NS-2: "Comparing the Accuracy of Network Simulators for Packet-Level Analysis using a Network Testbed". 3rd WEAS Int. Conf. on Simulation, Modelling and Optimization (ICOSMO 2003), Crete, vol. 2, pp. 700-707, 2003.
G. Flores-Lucio, M. Paredes-Farrera, E. Jammeh, M. Fleury, and M. Reed, OPNET-Modeler and NS-2: "Comparing the Accuracy of Network Simulators for Packet-Level Analysis using a Network Testbed". 3rd WEAS Int. Conf. on Simulation, Modelling and Optimization (ICOSMO 2003), Crete, vol. 2, pp. 700-707, 2003.
Scenario# Client –Server Load
CBR4 5 Mb/s
CLIENT`s PERSPECTIVE Traffic G. – Traffic S. Load
6 Mb/s
ROUTER`s PERSPECTIVE
Sudden drop
SERVER`s PERSPECTIVE
Scenario#
FTP1 FTP3
Client –Server Load
10 MB File 10 MB File
Traffic G. – Traffic S. Load
0 Mb/s (CBR) 0 Mb/s (CBR)
ROUTER`s PERSPECTIVE Server to Client Client to Server Without tuning. With tuning.
G. Flores-Lucio, M. Paredes-Farrera, E. Jammeh, M. Fleury, and M. Reed, OPNET-Modeler and NS-2: "Comparing the Accuracy of Network Simulators for Packet-Level Analysis using a Network Testbed". 3rd WEAS Int. Conf. on Simulation, Modelling and Optimization (ICOSMO 2003), Crete, vol. 2, pp. 700-707, 2003.
Scenario#
FTP2 FTP4
Client –Server Load
10 MB File 10 MB File
Traffic G. – Traffic S. Load
6 Mb/s (CBR) 6 Mb/s (CBR)
ROUTER`s PERSPECTIVE Server to Client Client to Server Without tuning. With tuning.
G. Flores-Lucio, M. Paredes-Farrera, E. Jammeh, M. Fleury, and M. Reed, OPNET-Modeler and NS-2: "Comparing the Accuracy of Network Simulators for Packet-Level Analysis using a Network Testbed". 3rd WEAS Int. Conf. on Simulation, Modelling and Optimization (ICOSMO 2003), Crete, vol. 2, pp. 700-707, 2003.
Conclusions
• • Pure CBR type traffic. (
NS-2
performed better).
In one scenario
OPNET
performed “
better
”.
• • • • • • •
FTP Experiments NS-2
FTP simulation model only indicated general transfer rate rather than replicating the actual network flow.
OPNET
performed
closely
to the testbed results.
Simulation speed was fast in both simulators.
The learning curve was different and
steep
at times.
NS-2
: “script” generated.
OPNET
: User Interface.
Conclusions Contd..
• • • • In
FTP
session, simulators did not model the dynamic behaviour of
FTP
in standard form.
FTP
adapts
conditions.
its output to prevailing network NS-2 and OPNET performance.
did not always mimic this However, OPNET performed better.
Future Work
• Establishing a scheme to model HTTP in the simulators and the testbed.
REFERENCE
• G. Flores-Lucio, M. Paredes-Farrera, E. Jammeh, M. Fleury, and M. Reed, OPNET-Modeler and NS-2: "Comparing the Accuracy of Network Simulators for Packet-Level Analysis using a Network Testbed". 3rd WEAS Int. Conf. on Simulation, Modelling and Optimization (ICOSMO 2003), Crete, vol. 2, pp. 700-707, 2003.