Transcript www.cs.newpaltz.edu
Chapter II
net_py
Reliable or Unreliable
Do you need service that keeps packets in order and guarantees delivery?
Are your packets “individual” and so can arrive in any order or possibly even not at all?
Example: Suppose you plan to send a single packet and get a single reply?
In the above example, you would only need to worry about non-delivery (and hence non-reply).
net_py
UDP
Short, self-contained requests and responses.
Real-time traffic like voice.
Author says it is not used often but that is not to say it is not useful. A single server port can receive packets from millions of distinct clients over time with no additional memory allocation beyond original setup.
congested network routers tend to be more sympathetic to UDP traffic compared to TCP traffic since they know that the latter will be retransmitted if dropped by the router. You can add your own “99% reliability” more cheaply (performance) than using TCP's 100% reliability. easier to use net_py
Addresses and Ports
file descriptor
running program socket memory allocation
port number
UDP IP Ethernet
net_py
Addresses and Ports
Ports allow multiple programs to use the same TCP/IP stack Packets come in and go up the same stack. They are “demultiplexed” at the top of the stack by port number to different programs.
The pair (IP address:port number) is called a
socket
and identifies a program connected to the internet, running on some machine.
Example: www.mysite.com:8080
Every packet sent on the internet contains a quadruple
(sourceIP: source port, destIP: dest port)
that identifies the connection.
net_py
Port Ranges
Well-known: 0-1023 Registered: 1024-4915: used by large companies The Rest: above, used by us How to find a well-known or registered port number
>>> import socket >>> socket.getservbyname('domain')
Check out the IANA (www.iana.org) net_py
Python Virtual Environment
● ● ● ● It is a good idea to create your own
virtual environment
the various python programming projects found in this course.
for This gives you an easy way to install special python packages just for your own use and leave the main python install on your computer alone.
This is a useful skill for future python development you might do.
Here is an on-line tutorial; read it carefully and follow the instructions.
http://dabapps.com/blog/introduction-to-pip-and-virtualenv-python/ http://virtualenvwrapper.readthedocs.org/en/latest/ net_py
Using virtualenvwrapper
● ● ● ● The
virtualenv
tutorial suggests that inside each project directory (say
Project1
) there should be a project-specific virtual environment (
Project1/env
). There is an alternative approach suggested by
virtualenvwrapper
– an extension of
virtualenv
.
When using using
virtualenvwrapper
like: your file hierarchy looks
Projects: ArcGIS BenchMark GeoIP Memcached QGIS Rabbit .virtualenvs: ArcGIS hook.log.1 postmkvirtualenv premkvirtualenv Benchmark initialize postrmproject prermproject BenchMark Memcached postrmvirtualenv prermvirtualenv GeoIP postactivate preactivate get_env_details postdeactivate predeactivate hook.log QGIS postmkproject premkproject Rabbit README
All the executable code in
.virtualenvs
package.
net_py is from the
wrapper
Sockets
Python makes calls to the lower level operating system-level calls that implement the networking functionality The python interface is slightly OO Python's contact to these OS calls is through a program construct called a
socket
Sockets are “files” just like everything else and are accessed via a file descriptor, which in python you never see but in C it is all you see.
Sockets
are
the file descriptor We read and write to sockets the same way we do files net_py
import socket, sys s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) # AF_INET is a “family” type # SOCK_DGRAM means UDP and not TCP MAX = 65535 # packet size PORT = 1060 # port used by our server if sys.argv[1:] == ['server']: # are these two lists equal s.bind(('127.0.0.1', PORT)) # specify your local socket address while True: # since you are a server print 'Listening at', s.getsockname() # returns a list data, address = s.recvfrom(MAX) # wait for traffic print 'The client at', address, 'says', repr(data) # data.toString() s.sendto('Your data was %d bytes' % len(data), address) # send reply to original sender
net_py
elif sys.argv[1:] == ['client']: print 'Address before sending:', s.getsockname() # 0.0.0.0:0 means no port on any interface s.sendto('This is my message', ('127.0.0.1', PORT)) print 'Address after sending', s.getsockname() # bind is automatic, but only to a port number # 0.0.0.0:34567 means port number 34567 on any interface data, address = s.recvfrom(MAX) # overly promiscuous - see Chapter 2 # will accept a datagramfrom anyone and not just the server print 'The server', address, 'says', repr(data) else: print >>sys.stderr, 'usage: udp_local.py server|client'
net_py
elif sys.argv[1:] == ['client']: print 'Address before bind:', s.getsockname() s.bind('',55000) print 'Address after bind:', s.getsockname() s.sendto('This is my message', ('127.0.0.1', PORT)) print 'Address after sending', s.getsockname() # bind is automatic, but only to a port number # 0.0.0.0:55000 means port number 55000 on any interface data, address = s.recvfrom(MAX) # overly promiscuous - see Chapter 2 # will accept a datagramfrom anyone and not just the server print 'The server', address, 'says', repr(data) else: print >>sys.stderr, 'usage: udp_local.py server|client'
net_py
Ports
Remote access to a socket is through its port number.
Access from the program that opens the socket is through the file descriptor.
net_py
Socket
Author says “both IP address and port number start as all zeros – a new socket is a blank slate”. This is not entirely true.
recv() vs recvfrom():
recv()
: does not return sender address and typical of client code because clients typically go to a single server and so “know” where any data they receive comes from.
recvfrom()
returns sender address and is typical of server code because servers receive data from many clients and typically need to reply to a client request with a packet sent right back to the same client. The
socket.sendto()
function takes the address returned by
recvfrom()
. net_py
Congestion:
If you
timeout and resend
and the problem is that the server is down, you will add useless traffic to the network and cause congestion that will slow everyone down.
Best answer is each time you timeout
extend the timeout interval
so that eventually you are resending a packet only once per hour or more.
exponential backoff
:
delay *= 2
What about giving up?
best to give up only if you need a timely answer or not at all
What about trying forever?
weather icon example
net_py
More Complications:
Suppose a successful request/reply exchange takes 200 ms on average.
Client can use this information to set a minimum timeout delay to be at least 200 ms?
What does the server do with duplicate requests?
1) Reply again 2) Don't bother to repeat reply (How does the server know a previous reply got to the original sender?).
In both cases the server must keep track of request identifiers so it knows what requests have already been received.
If client also keeps track of request IDs and whether or not they have been replied to then the client can quietly drop duplicate replies.
net_py
reply lost optional net_py no need to pass off to higher level but must keep data ID to know it has already been forwarded
Reliable Code
#!/usr/bin/env python # Foundations of Python Network Programming - Chapter 2 - udp_remote.py
# UDP client and server for talking over the network
import random, socket, sys s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) MAX = 65535 PORT = 1060
# usage: udp_remote.py server [
if 2 <= len(sys.argv) <= 3 and sys.argv[1] == 'server':
# server side
interface = sys.argv[2] if len(sys.argv) > 2 else ''
# two single quotes
s.bind((interface, PORT)) # '' is the same as '0.0.0.0' == all interfaces print 'Listening at', s.getsockname() while True: data, address = s.recvfrom(MAX)
# server always needs client address
if random.randint(0, 1):
# flip a coin and reply only if heads
print 'The client at', address, 'says:', repr(data) s.sendto('Your data was %d bytes' % len(data), address) else:
# tails
print 'Pretending to drop packet from', address net_py
Reliable Code
# Usage: udp_remote.py client
elif len(sys.argv) == 3 and sys.argv[1] == 'client': hostname = sys.argv[2] s.connect((hostname, PORT)) # delay = 0.1 while True:
# initial retry delay
s.settimeout(delay)
no packets exchanged;
print 'Client socket name is', s.getsockname()
# keep resending if no reply
s.send('This is another message') print 'Waiting up to', delay, 'seconds for a reply'
# so recv() will block only so long
try: data = s.recv(MAX)
# blocking
except socket.timeout:
# timeout expired
delay *= 2
# double timeout delay time
if delay > 2.0:
# time to stop all this nonsense
raise RuntimeError('I think the server is down') else: break
# we are done after one success, # and can stop looping
print 'The server says', repr(data) net_py
Code Alternatives:
• See homework net_py
Connecting vs Implicit Connecting
• • In Listing 2-1 we used
sendto(msg,address)
on the client and an implicit binding happened when the first datagram was sent.
In Listing 2-2 we used
send(msg)
so the binding had to be done explicitly, and at the same time we indicate where the message is to be sent
s.connect(remote_host,remote_PORT) ...
s.send(data)
check
netstat -an | grep udp
at this point • In the first situation we could send to various different servers by modifying the address argument. In the latter we can only send to the address we originally connected to.
net_py
How things look after s.sendto(msg,address):
[pletcha@archimedes ~]$ python >>> import socket >>> s=socket.socket(socket.AF_INET,socket.SOCK_DGRAM) >>> s.sendto('my message',('wyvern.cs.newpaltz.edu',50000)) 10 >>> s.getsockname() ('0.0.0.0', 52011) [pletcha@archimedes ~]$ netstat -an | grep udp udp 0 0 0.0.0.0:55188 0.0.0.0:* udp 0 0 0.0.0.0:47503 0.0.0.0:* udp 0 0 0.0.0.0:52011 0.0.0.0:* udp 0 0 0.0.0.0:60386 0.0.0.0:* udp 0 0 192.168.122.1:53 0.0.0.0:* my port 52011.
net_py
Sneaking into the conversation
[pletcha@archimedes ~]$ python >>> import socket >>> s=socket.socket(socket.AF_INET,socket.SOCK_DGRAM) >>> s.sendto('my message',('wyvern.cs.newpaltz.edu',50000)) 10 >>> s.getsockname() ('0.0.0.0', 33897) >>> data,address = s.recvfrom(4000) >>> print data Fake reply
use the bound port number as your destination
[pletcha@archimedes ~]$ python >>> import socket >>> s = socket.socket(socket.AF_INET,socket.SOCK_DGRAM) >>> s.sendto('Fake reply',('127.0.0.1',33697)) 10
net_py
How things look after s.connect(address):
> python >>> import socket >>> s =socket.socket(socket.AF_NET,socket.SOCK_DGRAM) >>> s.connect((wyvern.cs.newpaltz.edu,50000)) [pletcha@archimedes ~]$ netstat -an | grep udp udp 0 0 0.0.0.0:55188 0.0.0.0:* udp 0 0 137.140.8.104:51400 137.140.4.187:50000 ESTABLISHED udp 0 0 0.0.0.0:47503 0.0.0.0:* udp 0 0 0.0.0.0:60386 0.0.0.0:* udp 0 0 192.168.122.1:53 0.0.0.0:* no one can send data to my port 51400 except what I connected to (I really didn't connect since no data was sent)
net_py
Exercise
•
Repeat the “sneak into the conversation” example running the client software of Listing 2-2 and see that your message never gets read by the application.
It is rejected by UDP What happens is that Ethernet and IP recognize the sneaky packet as intended for this machine. IP forwards the packet to UDP. UDP looks up the destination port and sees that it will only accept data from (wyvern, 50000)
• • Lesson to learn: If you want to use
sendto() connect()
followed by
send()
instead of , then you can use
recvfrom()
and look at each sender address to be sure it is an address you recognize.
Exercise: Think of a UDP application that would have you sending data to several destinations so you could expect answers back from them all.
net_py
Request IDs
• • • Sending a packet ID with every packet makes it easier to identify replies and know what request they are replying to.
(See homework) Packet IDs are some protection against spoofing. Client that doesn't call
connect()
can be exposed to unexpected (spoofing) traffic. If the unwelcome traffic doesn't know the packet ID it is not possible to fake a response.
net_py
Binding to Interfaces
• • • We have used
bind()
to listen on 127.0.0.1 or on ' ', which means all network interfaces.
We can specify an interface if we know its IP address (remember
/sbin/ifconfig -a
) Connecting to the College using VPN I have 4 network interfaces on my laptop net_py
4 Network Interfaces at once:
[pletcha@archimedes 02]$ ifconfig -a cscotun0: flags=4305
lo: flags=73
virbr0: flags=4099
wlan0: flags=4163
net_py
Ports are bound to Interfaces
[pletcha@archimedes 02]$ cat all_interfaces.sh #!/usr/bin/bash ./udp_remote.py server 127.0.0.1& sleep 1; ./udp_remote.py server 137.140.108.133& sleep 1; ./udp_remote.py server 192.168.122.1& sleep 1; ./udp_remote.py server & sleep 1; ./udp_remote.py server 192.168.1.124& [pletcha@archimedes 02]$ ./all_interfaces.sh Listening at ('127.0.0.1', 1060) Listening at ('137.140.108.133', 1060) Listening at ('192.168.122.1', 1060) Traceback (most recent call last): File "./udp_remote.py", line 13, in
net_py
local host vs remote host:
• • • Local packets can arrive destined for 127.0.0.1 but they can also arrive locally by using the machine IP address, say 192.168.122.1.
By binding to 127.0.0.1, external clients can not talk to you.
By binding to 192.168.122.1 both internal and external clients can talk to you. •
Lesson to Learn
: Binding means specifying both a port number and a network interface (or all interfaces), so the basic data structure is not just a port but an (IP address, port) pair, in other words, a
socket
.
net_py
Two Interface problem:
• • • Suppose a machine has two external interfaces – 192.168.122.1 and 192.168.1.124.
If we open a socket at (192.168.1.124,1060) but try to send data to (192.168.122.1, 1060) what will happen? Apparently it depends on the OS.
On my laptop I get:
[pletcha@archimedes 02]$ ps -ef | grep remote pletcha 28577 3450 0 18:07 pts/1 python ./udp_remote.py server 192.168.122.1
[pletcha@archimedes 02]$ ./udp_remote.py client 192.168.1.124
Client socket name is ('192.168.1.124', 48271) Waiting up to 0.1 seconds for a reply Traceback (most recent call last): File "./udp_remote.py", line 33, in
net_py
Fragmentation, ETC:
Read the last 4 pages of Chapter 2 yourselves.
net_py