Transcript Improving Web Systems

Introduction to Content Switch
C. Edward Chow
Department of Computer Science
University of Colorado at Colorado Springs
Outline of the Talk
•
•
•
•
•
•
•
•
What is a Content Switch?
What Services it Can Provide
Content Switch Example
Related Technologies
Content Switch Architecture and Basic Operations
TCP Delay Binding and Related Improvement
Content Switch Rule and Conflict Detection
Related Load Balancing Research Results
12/22/2000
Edward Chow
ACSD Project Status 2
Content Switch (CS)
• Route packets based on high layer (Layer 5/7) headers
and content.
• Examples:
– Direct Web traffic based on pattern of URLs, host
tags, cookies.
– Can Route incoming email based on email address;
Connect POP/IMAP based on login
• Web switches and Intel XML Director/accelerator are
special cases of content switch.
12/22/2000
Edward Chow
ACSD Project Status 3
What Services It Can Provide
• Enabling premium services for e-commerce, ISP, and
Web hosting providers
• Load Balancing and High Available Server Clusters:
Web, E-commerce, Email, Computing, File, SAN
• Policy-based networking, differential/QoS services.
• Firewall, Strengthening DoS protection, cache/firewall
load-balancing
• ‘Flash-crowd' management
• Email Spam Protection, Virus Detection/Removal
• Applet Authentication/Filtering
12/22/2000
Edward Chow
ACSD Project Status 4
F5 VRM Solution
Site II
losangeles.domain.com
Internet
Internet
Site I
newyork.domain.com
Router
3-DNS
BIG-IP
BIG-IP
Local DNS
GLOBAL-SITE
Webmaster
Site III
tokyo.domain.com
Server Array
User
london.domain.com
12/22/2000
Edward Chow
ACSD Project Status 5
Intel Netstructure
XML Director 7280
• Example of Rule:
Server1: create */order.asp & //Amount[Value >= 10000]
12/22/2000
Edward Chow
ACSD Project Status 6
Phobos In-Switch
• Only load balancing switch in a PCI card form factor
• Plugs directly into any server PCI slot
• Supports up to 8,192 servers, ensuring availability and
maximum performance
• Six different algorithms are available for optimum performance:
Round Robin, Weighted Percentage, Least Connections,
Fastest Response Time, Adaptive and Fixed.
• Provides failover to other servers for high-availability of the web
site
• U.S. Retail $1995.00
12/22/2000
Edward Chow
ACSD Project Status 7
E-Commerce Example: 1. Client
Client submits via HTTP/Post (or SOAP) the following purchase in XML:
<purchase>
<customerName>CCL</customerName>
<customerID>111222333</customerID>
<item><productID>309121544</productID>
<productName>IBM Thinkpad T21</productName>
<unitPrice>5000</unitPrice>
<noOfUnits>10</noOfUnits>
<subTotal>50000</subTotal>
</item>
<item><productID>309121538</productID>
<productName>Intel wireless LAN PC Card</productName>
<unitPrice>200</unitPrice>
<noOfUnits>10</noOfUnits>
<subTotal>2000</subTotal>
</item>
<totalAmount>52000</totalAmount>
</purchase>
12/22/2000
Edward Chow
ACSD Project Status 8
E-Commerce Example:
2. Content Switch
• Content switch receives the packet.
• Recognize it is a http post request from http request line
POST /purchase.cgi HTTP/1.1
• Recognize it is an XML document from the meta header
content-type: TEXT/XML
• Parsing XML content
• Extract values
of
tag sequences:
52000
purchase/totalAmount
CCL
purchase/customerName
• Rule 1 is matched and packet is routed to one of highSpeedServers.
Rule 1: if (xml.purchase/totalAmount > 5000) routeTo(highSpeedServers);
Rule 2: if (xml.purchase/customerName == CCL) routeTo(specialCustomerServers);
12/22/2000
Edward Chow
ACSD Project Status 9
No Free Lunch:
Penalty of Having Content Switch
•
Layer 4 Switching Layer 7 Switching
packet header extraction fixed short fields
varying length long fields
switch rule matching
hash table look up pattern matching
 Increased packet processing time.
• For XML Director/Accelerator, it needs to parse XML
document and match tag sequences.
 1-3? order of processing time
Size of XML Document (Bytes) XML Content Extract Time (ms)
600
14
7000
21
67104
53
12/22/2000
Edward Chow
ACSD Project Status 10
Related Technologies
• Application level solution:
Proxy server; Apache/Tomcat/Servlet; Microsoft NLB
• Kernel level layer 4 load balancing solution:
http://www.linuxvirtualserver.org/
– Joseph Mark’s presentation
– LVS-NAT(Network Address Translation) web page
– LVS-IP Tunnel web page
– LVS-DR (Direct Routing) web page
• Hardware solution: Cisco 11000, F5 (Big IP), Alteon Web
Systems, Foundry Networks (ServerIron),
Good information: Foundry ServerIron Installation and
Configuration Guide, May 2000.
12/22/2000
Edward Chow
ACSD Project Status 11
Basic Operations of Content Switching
CS: Content Switching
CS
Rules
Incoming
Packets
CS Rule
Editor
Packet Classification
Header
Content
Extraction
Network Path Info
Server Load Status
12/22/2000
CS
Rule Matching Algorithm
Packet Routing
(Load Balancing)
Edward Chow
Forward
Packet
To
Servers
ACSD Project Status 12
Content Switch Architecture
Apostolopoulos 2000
12/22/2000
Edward Chow
ACSD Project Status 13
Efficient Software Architecture
• Tasks: Million Packets with thousand of rules to match and
load balancing algorithms to run.
• How to assign tasks to the processors and threads?
– Packet Extraction
(Understand header formats, XML parsing)
– Content Switching Rule Matching
– Packet Routing
(Load Balancing, Bandwidth Control)
• How Much Packet Processing Should Controllers Do?
• What a controller can do?
• A Typical Parallel Processing Problem?
12/22/2000
Edward Chow
ACSD Project Status 14
TCP Delay Binding
client
server
content switch
SYN(CSEQ)
step1
SYN(DSEQ)
ACK(CSEQ+1)
step2
step3
ACK(DSEQ+1)
DATA(CSEQ+1)
ACK(DSEQ+1)
step5
step6
step7
step8
DATA(DSEQ+1)
ACK(CSEQ+LenR+1)
step9
step4
SYN(CSEQ)
SYN(SSEQ)
ACK(CSEQ+1)
ACK(SSEQ+1)
DATA(CSEQ+1)
ACK(SSEQ+1)
DATA(SSEQ+1)
ACK(CSEQ+lenR+1)
step10
ACK(DSEQ+ lenD+1)
DATA(?) 2nd request
ACK(?)
12/22/2000
step11
ACK(SSEQ+lenD+1)
lenR: size of http request.
.
lenD: size of return document
Edward Chow
ACSD Project Status 15
Lesson Learned in
Implementing TCP Delay Binding
• In our Linux 2.2 kernel-based content switch prototype, we
found client sends duplicate requests after step 3.
• It overloads the content switch and the real server.
• Reason:
– Client TCP time-out, retransmit
– Content switch printk() overhead, too many debug msgs
– It could happens when there are many content rules, slow
server response.
• Solution: content switch sends ack(CSEQ+LenR+1) to stop
retransmit.
12/22/2000
Edward Chow
ACSD Project Status 16
Pre-Allocate Server Scheme
client
content switch
SYN(CSEQ)
step1
SYN(SSEQ)
ACK(CSEQ+1)
DATA(CSEQ+1)
ACK(SSEQ+1)
DATA(SSEQ+1)
ACK(CSEQ+LenR+1)
Pre-allocated
server
SYN(CSEQ)
SYN(SSEQ)
ACK(CSEQ+1)
step3 DATA(CSEQ+1)
ACK(SSEQ+1)
step4
DATA(SSEQ+1)
step2
step5
ACK(SSEQ+ lenD+1)
ACK(CSEQ+lenR+1)
ACK(SSEQ+lenD+1)
• Guess routing decision based on IP/Port#/History
• Advantage:
• Faster than TCP delay binding.
.
• Possible direct route between client and server
• Reduce session processing overhead
no need to convert server sequence #
12/22/2000
Edward Chow
ACSD Project Status 17
Degenerated to TCP Delay Binding
Pre-allocated
If
Guess
Wrong
client
content switch
SYN(CSEQ)
DATA(CSEQ+1)
ACK(SSEQ+1)
SYN(SSEQ)
ACK(CSEQ+1)
step4
step5
step6
DATA(SSEQ+1)
ACK(CSEQ+LenR+1)
ACK(DSEQ+lenD+1)
server
step1
SYN(CSEQ)
step2
SYN(SSEQ)
ACK(CSEQ+1)
FIN(CSEQ+1)
step3a
SYN(CSEQ)
Right server
SYN(RSEQ)
ACK(CSEQ+1)
DATA(CSEQ+1)
ACK(SSEQ+1)
step4
DATA(RSEQ+1)
ACK(CSEQ+lenR+1)
step5
ACK(SSEQ+lenD+1)
Sequence # conversion needed
12/22/2000
Edward Chow
ACSD Project Status 18
Filter Process Scheme
client
content switch
SYN(CSEQ)
step1
SYN(DSEQ)
ACK(CSEQ+1)
DATA(CSEQ+1)
ACK(DSEQ+1)
step4a
step5
Filter Process
run on server
step2
step3
step4b SYN(CSEQ)
Migrate
(Data, CSEQ, DSEQ)
SYN(SSEQ)
ACK(CSEQ+1)
DATA(CSEQ+1)
ACK(SSEQ+1)
DATA(SSEQ+1)
ACK(CSEQ+lenR+1)
step6
DATA(DSEQ+1)step7
ACK(CSEQ+LenR+1)
step8
ACK(DSEQ+lenD+1)
12/22/2000
server
ACK(SSEQ+lenD+1)
Edward Chow
ACSD Project Status 19
Multiple HTTP Requests from One TCP Connection
NAT approach
server1
Content
Switch
client
Index.htm
uccs.jpg
server2
.
.
.
server9
• A keep alive TCP connection may include multiple HTTP “GET” requests.
• Content Switch examines each “GET” request and makes new routing decision.
• Content Switch establishes another connection with a different server based
on the routing decision.
• Those HTTP responses from different servers need to be interleaved and
seen by the user as if from the same server.
• Solutions: In order delivery (buffer requirement); Out of order delivery (seq# tracking)?
• Problems: Should we throw away earlier html requests if receive later requests?
12/22/2000
Edward Chow
ACSD Project Status 20
Multiple HTTP Requests from One TCP Connection
server1
client
Content
Switch
server2
.
.
.
server9
•
•
Can servers return documents directly to client in keep-alive session
case?
Can equivalent VS-Tunnel or VS-DR be implemented using Content
Switch?
12/22/2000
Edward Chow
ACSD Project Status 21
Content Switch Rule Survey
Survey shows that existing switches support
• rules in basic (condition action) or (action condition)
form
• some define condition as class, then specify the
action in separate statement or command
• simple single conditional term
• command line interface (to facilitate incremental
update?)
• Actions can include reject, forward, put in queue (for
bandwidth control, scheduling)
12/22/2000
Edward Chow
ACSD Project Status 22
Content Switch Rule Design
• Rule syntax generic to support all Intended features.
• Use simple C if statement syntax rule: if (condition) { action }
– Easy to read
– Allow optimization using c compiler
• Condition consists of multiple terms of
– variable relational_operator value
e.g. xml.purchase/totalAmount > 50000
smtp.to == “[email protected]”
cookie.name == “servlet1”
bitmatch(64, 8, 0xff) == 64
# above mean TTL=64 idea from netfilter universal filter
– suffix(variable, string) e.g. suffix(url, “gif”)
– regex(variable, pattern) e.g. regex(url, “/purchase”)
• Action consists of reject, forward(server| queue)
loadBalance(serverGroup, loadBalancingAlgorihtm)
12/22/2000
Edward Chow
ACSD Project Status 23
Efficient CS Rule Matching
• Brute force, strict priority: Rules are executed in
sequential manner.
• Efficient Rule Matching Method:
– Organize Rules so that rules can be skipped
based on existing content types.
– Utilize compiler optimization technique.
12/22/2000
Edward Chow
ACSD Project Status 24
Simple CS Rule Editor GUI
12/22/2000
Edward Chow
ACSD Project Status 25
Conflict Detection on
Content Switching Rules
• Detect conflicts among rules or rule set.
• Absolute conflict type:
r1: if (xml.purchase/customerName == “CCL”) {routeTo(r1)}
r2: if (xml.purchase/customerName == “CCL”) {routeTo(r2)}
• Potential conflict type:
r1: if (xml.purchase/totalAmount > 5000) {routeTo(quickServers)}
r2: if (xml.purchase/totalAmount >20000) {routeTo(superServers)}
• Algorithm: Build tree with the same variable, check operator and
value to see if they are the same or lead to potential conflict,
compare actions to decide conflict type or duplication.
• Editor can build these trees while a user enters rules and warns
about conflict right away.
12/22/2000
Edward Chow
ACSD Project Status 26
XML Tag Value Extraction
• A xmlContentExtract() is built to extract the tag values
of a list of unique tag sequences.
• It is based on clark cooper’s expat 1.0 xmlparser.
• Its argument include the pointer to an XML document,
the pointer to the array of strings (unique xml tag
squences we follow the xsl selector syntax), and the
number of sequences.
• It return the list of a structure node, with the tag
sequence, its attribute, and its value.
• Currently, it supports one attribute and tag sequece
needs to be unique.
12/22/2000
Edward Chow
ACSD Project Status 27
Status of UCCS ACSD Project
• A Linux-based content switch prototype is almost complete.
• It is based on Linux-2.2.16-3 and lvs.
• ip_forward.c, ip_masq.c, ip_vs.c are modified to implement
basic TCP delay binding.
• Preliminary tests had real server return web document and
discovered the client retransmission problem.
• ip_cs.c are added for most of the content switching functions.
• http header extraction and xml content extract code are being
integrated in for testing.
• A simple Java-based ruleEdit program was created for rule
editing.
12/22/2000
Edward Chow
ACSD Project Status 28
Related Load Balancing
Research Results
• Modified Apache status module to report
– Total bytes to be transferred by child processes
– Average document transfer speed
• Modified LB-DNS to receive server status and
bandwidth probing results.
• LB-DNS returns IP-address of the best server based
a weight contributed by both server load and
bandwidth.
• Modified WebStone benchmark to test the
performance of load balancing web server clusters.
12/22/2000
Edward Chow
ACSD Project Status 29
Load balancing Systems
Bandwidth Probe
Results
Statistics Gathering
Daemon
Modified Web Server
1
Server Delay
Server Ranking
/tmp/StatFile
Modified Web Server
n
12/22/2000
LBA: Modified
DNS
Edward Chow
Request for
Web pages
ACSD Project Status 30
Connection Rate: LBA vs.
Round-Robin
Server connection rate for 4 servers
Connections/sec
1000
800
600
400
200
0
1
2
3
4
5
6
7
8
9
10
11
12
load balancing system 418.2 656.6 907.9 420 636.7 322.6 711.6 420.5 638.3 670.6 683.4 899
327.6 327.6 327.6 327.6 327.6 327.6 327.6 327.6 327.6 327.6 327.6 327.6
round-robin
Update for LBA , per sec
Round robin only run once
load balancing system
12/22/2000
Edward Chow
round-robin
ACSD Project Status 31
Conclusion
• Content switch with generic rules can be easily configured for
wide-variety of value-added services:
– Load balancing/High Available server farm.
– Premium services
– Firewall
– Bandwidth control/Traffic shaping
• Require efficient SW/HW architecture and rule matching
algorithms to reduce processing overhead.
• Content rule design/conflict detection are important and
challenging.
• TCP delay binding can be improved.
• Servicing multiple requests in keep alive session introduces
interesting problem.
12/22/2000
Edward Chow
ACSD Project Status 32