Evolution of TCP / UDP

User Datagram Protocol (UDP)

• Strengths – – Lightweight Protocol No overhead Client rqst Server resp • Weaknesses – No reliability in UDP CS423 - Cotter 2

• •

Transmission Control Protocol (TCP)

Strengths – Reliable Data Transmission – Stream Protocol Client Server syn syn, ack Weaknesses – – Connection Overhead Transaction time: • 2 RTT + • • Server Processing time (SPT) + 1RTT for each additional data packet • Connection resources held until any possible lost packets have expired ( 2 * RTT max = 240 sec) ack data fin ack ack data fin CS423 - Cotter 3

Minimum TCP Transaction

• • Protocol allows sending data with syn – Not generally done – Must hold data until 3WHS complete Minimum transmission – 5 packets – 2 RTT + SPT Client syn, fin, data ack, fin ack Server syn, ack ack, fin data CS423 - Cotter 4

The Problem:

• Some applications (particularly Web apps) need both reliability and speed – Many connections (especially page loads) need only a few data packets – Early studies showed that median HTTP reply between 1770 and 958 bytes – May have many connections per session.

– Overhead of connection (syn, syn-ack, ack) increases the transaction time – Google study showed 4 to 40% improvement in page load time possible – Because ports cannot be reused for 240 sec, maximum transaction rate between a client and server is ~64000 ports/ 240 sec = 267 trans/sec – Problem is most pronounced when RTT is large CS423 - Cotter 5

• • • •

One Solution: TCP for Transactions (T/TCP)

Originally proposed in rfc 1644 (1994) Include data in original syn packet.

Maintain a Connection Counter (cc) in server that is passed by client to server as TCP option. Server uses this value to validate client, bypassing the need for 3-way handshake (3WHS) Because server can now validate each connection, connection delay reduced from 240 to 12 sec.

Client syn, cc, data, fin ack Server syn, ack, cc, data, fin CS423 - Cotter 6

T/TCP Initial Connection Setup

• • • Protocol requires setting up the connection count between each client and the server. Initial 3WHS includes CC request. Each succeeding connection uses CC and avoids 3WHS Protocol requires that CC increment monotonically.

CS423 - Cotter 7

T/TCP Protocol

• Initial 3WHS Client syn, CCnew, data Server syn, ack, CCecho fin, ack ack Data, fin ack ack • Succeeding data calls Client syn, cc, data, fin Server syn, ack, cc, data, fin ack CS423 - Cotter 8

T/TCP Problems !

• • • For most implementations, server initializes CC at 1 and increments by 1 for each new connection. By sniffing the connection, it is easy to anticipate and spoof the CC. Even without access to the path, picking a large number will usually succeed (CC is 32 bit number) This makes it very easy for an attacker to implement a syn attack (DoS). It is made more effective since server must store the data associated with the syn, using up more server resources with each new packet.

Protocol moved to historical status in 2011.

CS423 - Cotter 9

TCP Fast Open

• • • Researched originally at Google in 2011. – Published at CoNEXT 2011 – Objective was to improve web server performance Offered as IETF draft first in 2012.

– Currently in draft 08 (expires August, 2014) Implemented in Linux (since Kernel 3.6/3.7) CS423 - Cotter 10

TCP Fast Open

• • Use initial connection to establish relationship.

– Client requests TCP fast open cookie and sends data in original syn packet – Server creates cookie and returns to client in syn, ack packet. Cookie is an encrypted hash of client IP address and other data.

– Client ack packet completes 3WHS.

– Server processes client request and returns response.

Subsequent requests do not need 3WHS – Client request includes server cookie, data – – Server returns syn, ack Server returns response – Client completes handshake.

CS423 - Cotter 11

CS423 - Cotter

Fast Open Initial connect

12

Fast Open Subsequent Packets

CS423 - Cotter 13

Linux Implementation

• • Requires current kernel (at least 3.7 for client and server) Requires setting /proc/sys/net/ipv4/tcp_fastopen – Enable TCP Fast Open feature (draft-ietf-tcpm-fastopen) to send data in the opening SYN packet. To use this feature, the client application must use sendmsg() or sendto() with MSG_FASTOPEN flag rather than connect() to perform a TCP handshake automatically. The values (bitmap) are 1: Enables sending data in the opening SYN on the client w/ MSG_FASTOPEN. 2: Enables TCP Fast Open on the server side, i.e., allowing data in a SYN packet to be accepted and passed to the application before 3-way hand shake finishes. 4: Send data in the opening SYN regardless of cookie availability and without a cookie option. 0x100: Accept SYN data w/o validating the cookie. 0x200: Accept data-in-SYN w/o any cookie option present. 0x400/0x800: Enable Fast Open on all listeners regardless of the TCP_FASTOPEN socket option. The two different flags designate two different ways of setting max_qlen without the TCP_FASTOPEN socket option. Default: 1 Note that the client & server side Fast Open flags (1 and 2 respectively) must be also enabled before the rest of flags can take effect. CS423 - Cotter 14

:

Linux TCP Fast Open client (TFO_client.cpp)

void UDPecho(const char *host, const char *service) { charbuf[BUFFSIZE][LINELEN+1] = {"This is the first message", "This is the second message", "This is the third message", "This is the fourth message", "This is the last message" }; char inbuf[LINELEN +1]; int s, nchars, rchars; /* socket descriptor, character counters */ { for (index = 0; index < BUFFSIZE; index++) : s = socket(AF_INET, SOCK_STREAM, 0) ; nchars = strlen(buf[index]); CS423 - Cotter 15

Linux TCP Fast Open client (TFO_client.cpp)

: status = sendto(s, &buf[index], nchars, MSG_FASTOPEN, (const struct sockaddr *) &sin, sizeof(sin)); cout << "We just sent a datagram" << endl; rchars = recvfrom(s, inbuf, LINELEN, 0, NULL, NULL); : inbuf[rchars] = '\0'; //Add a null termination : cout << "We got back the following " << rchars << " characters: " << inbuf << endl; close (s); sleep(1); } /* end of while */ }/* end of UDPecho () */ CS423 - Cotter 16

Linux TCP Fast Open server (TFO_server.cpp)

int main(int argc, char *argv[]) { int qlen = 5; : sock = passiveTCP(service, qlen); ans = setsockopt(sock, IPPROTO_TCP, TCP_FASTOPEN, &qlen, sizeof(qlen)); while (keep_looping == YES) { ssock = accept (sock, (sockaddr *)&fsin, (socklen_t *)&fsinsize); sprintf (log_msg, "Just got a request from %s\n", inet_ntoa(fsin.sin_addr)); logData(log_msg); cc = recv (ssock, buf, LINELEN, 0); buf[cc] = '\0'; sprintf (log_msg, "Msg: %s\n", buf); logData(log_msg); strcat (buf, " answer"); send (ssock, buf, cc +7, 0); memset (buf, 0, LINELEN); close (ssock); }//end of while loop } CS423 - Cotter 17

Wireshark Trace – Initial connect (1) cookie request

CS423 - Cotter 18

Wireshark Trace – Initial connect (2) cookie

CS423 - Cotter 19

Wireshark Trace – Initial connect (3) data

CS423 - Cotter 20

Wireshark Trace – Initial connect (4) ack

CS423 - Cotter 21

Wireshark Trace – Initial connect (4) response

CS423 - Cotter 22

Wireshark Trace – Next connect (1) syn, cookie, data

CS423 - Cotter 23

Wireshark Trace – Next connect (2) syn,ack

CS423 - Cotter 24

Wireshark Trace – Next connect (3) response

CS423 - Cotter 25

Wireshark Trace – Next connect (4) fin, ack

CS423 - Cotter 26

Fast Open Client side Output

[bob@localhost client]$ g++ -o tfoc TFO_client.cpp

[bob@localhost client]$ ./tfoc 192.168.1.25 23456 Our target server is at address 192.168.1.25

Enter data to send...

We just sent a datagram We got back the following 32 characters: This is the first message answer We just sent a datagram We got back the following 33 characters: This is the second message answer We just sent a datagram We got back the following 32 characters: This is the third message answer We just sent a datagram We got back the following 33 characters: This is the fourth message answer We just sent a datagram We got back the following 31 characters: This is the last message answer [bob@localhost client]$ CS423 - Cotter 27

Fast Open Server Log

### Starting Server on port 23456 at Wed Apr 2 11:26:17 2014 Just got a request from 192.168.1.117

Msg: This is the first message Just got a request from 192.168.1.117

Msg: This is the first message Just got a request from 192.168.1.117

Msg: This is the second message Just got a request from 192.168.1.117

Msg: This is the third message Just got a request from 192.168.1.117

Msg: This is the fourth message Just got a request from 192.168.1.117

Msg: This is the last message CS423 - Cotter 28

Extended Fast Open Test

• Build Standard TCP Client – Send 100 packets, receive 100 responses • Build TCP Fast Open Client – Send 100 packets, receive 100 responses • Test Conditions – – Client and server on adjacent machines on same LAN Same code structure, same data sent and received.

• Compare Results – How many Packets required for each • • Standard: 1000 packets TFO: 801 – How much time required for each?

CS423 - Cotter • • Standard: .173333 sec TFO: .119198 sec 29

Summary

• • • Network Protocols continue to evolve TCP Fast Open offers the speed of UDP (after the first connection) with the reliability of TCP.

Protocol is still in experimental stage.

CS423 - Cotter 30

References

• • • • • • • TCP/IP Illustrated Vol 3, Richard Stevens – Chapters 1-12 (T/TCP) Original TCP Fast Open Paper – http://conferences.sigcomm.org/co-next/2011/papers/1569470463.pdf

– http://static.googleusercontent.com/media/research.google.com/en/us/pubs/archive/37517.

pdf IETF draft RFC – https://datatracker.ietf.org/doc/draft-ietf-tcpm-fastopen/ lwn.net (Linux Weekly News?) Article – https://lwn.net/Articles/508865/ Packet Pushers Article – http://packetpushers.net/tcp-fast-curious-a-look-at-tcp-fast-open/ Admin Magazine Article – http://www.admin-magazine.com/Articles/TCP-Fast-Open Linux Kernel documentation – https://www.kernel.org/doc/Documentation/networking/ip-sysctl.txt

CS423 - Cotter 31