Transcript slides
Design and Implementation of a Consolidated Middlebox Architecture Vyas Sekar Sylvia Ratnasamy Michael Reiter Norbert Egi Guangyu Shi 1 Need for Network Evolution New applications Evolving threats Performance, Security, Compliance Policy constraints New devices 2 Network Evolution today: Middleboxes! Type of appliance Data from a large enterprise: >80K users across tens of sites Just network security $10 billion Number Firewalls 166 NIDS 127 Media gateways 110 Load balancers 67 Proxies 66 VPN gateways 45 WAN Optimizers 44 Voice gateways 11 Total Middleboxes Total routers 636 ~900 3 Key “pain points” Narrow Management Management Management interfaces Specialized boxes “Point” solutions! Increases capital expenses & sprawl Increases operating expenses Limits extensibility and flexibility 4 Outline • Motivation • High-level idea: Consolidation • System design • Implementation and Evaluation 5 Key idea: Consolidation Two levels corresponding to two sources of inefficiency Network-wide Controller 2. Consolidate Management 1. Consolidate Platform 6 Consolidation at Platform-Level Today: Independent, specialized boxes Proxy Firewall IDS/IPS AppFilter Decouple Hardware and Software Commodity hardware: e.g., PacketShader, RouteBricks, ServerSwitch, SwitchBlade Consolidation reduces capital expenses and sprawl 7 Consolidation reduces CapEx Multiplexing benefit = Max_of_TotalUtilization / Sum_of_MaxUtilizations 8 Consolidation Enables Extensibility VPN Web Mail IDS Proxy Firewall Protocol Parsers Session Management Contribution of reusable modules: 30 – 80 % 9 Management consolidation enables flexible resource allocation Today: All processing at logical “ingress” Process Process (0.4(P) P) N1 Overload! Process (0.3 P) Process (0.3 P) N2 N3 P: N1 N3 Network-wide distribution reduces load imbalance 10 Outline • Motivation • High-level idea: Consolidation • CoMb: System design • Implementation and Evaluation 11 CoMb System Overview Network-wide Controller Logically centralized e.g., NOX, 4D General-purpose hardware: e.g., PacketShader, RouteBricks, ServerSwitch, Existing work: simple, homogeneous routing-like workload Middleboxes: complex, heterogeneous, new opportunities 12 CoMb Management Layer Goal: Balance load across network. Leverage multiplexing, reuse, distribution Policy Constraints Resource Requirements Network-wide Controller Routing, Traffic Processing responsibilities 13 Capturing Reuse with HyperApps HyperApp: find the union of apps to run HTTP: 1+2 unit of CPU 1+3 units of mem HTTP UDP IDS 2 1 HTTP = IDS & Proxy HTTP NFS Proxy common CPU 3 4 Memory UDP = IDS 3 1 3 1 Memory CPU Memory CPU NFS = Proxy Footprint on resource Need per-packet policy, reuse dependencies! CPU 1 4 Memory Policy, dependency are implicit 14 Modeling Processing Coverage HTTP: Run IDS < Proxy IDS < Proxy 0.4 IDS < Proxy 0.3 IDS < Proxy 0.3 HTTP N1 N3 N1 N2 N3 What fraction of traffic of class HTTP from N1 N3 should each node process? 15 Network-wide Optimization Minimize Maximum Load, Subject to Processing coverage for each class of traffic Fraction of processed traffic adds up to 1 Load on each node sum over HyperApp responsibilities per-path No explicit Dependency Policy A simple, tractable linear program Very close (< 0.1%) to theoretical optimal 16 CoMb System Overview Network-wide Controller Logically centralized e.g., NOX, 4D General-purpose hardware: e.g., PacketShader, RouteBricks, ServerSwitch, Existing work: simple, homogeneous routing-like workload Middleboxes: complex, heterogeneous, new opportunities 17 CoMb Platform Applications Policy Enforcer IDS … Proxy Core1 … Core4 Policy Shim (Pshim) IDS < Proxy Classification: HTTP NIC Traffic Challenges: Performance Parallelize Isolation Challenges: Lightweight Parallelize Challenges: No contention Fast classification 18 Parallelizing Application Instances App-per-core M1 M2 Core1 Core2 PShim HyperApp-per-core M3 Core3 PShim - Inter-core communication - More work for PShim + No in-core context switch M1 M2 Core1 PShim M2 M3 ✔ Core2 PShim + Keeps structures core-local + Better for reuse - But incurs context-switch - Need replicas HyperApp-per-core is better or comparable Contention does not seem to matter! 19 CoMb Platform Design Core-local processing Core 1 M1 Hyper App1 M2 Workload balancing Core 2 M3 M1 M4 Hyper App2 Hyper App3 PShim PShim PShim Q1 Q2 Q3 NIC hardware Core 3 M5 Hyper App4 PShim Q4 M1 M4 Hyper App3 PShim Q5 Parallel, core-local Contention-free network I/O 20 Outline • Motivation • High-level idea: Consolidation • System design: Making Consolidation Practical • Implementation and Evaluation 21 Implementation Network-wide Management Policy Shim Extensible apps Ported logic From Bro Click using CPLEX Kernel mode Click Standalone apps Protocol Memory mapped Or Virtual interfaces Session 8-core Intel Xeon with Intel 82599 NIC 22 Consolidation is Practical • Low overhead for existing applications • Controller takes < 1.6s for 52-node topology • 5x better than VM-based consolidation 23 Benefits: Reduction in Maximum Load MaxLoadToday /MaxLoadConsolidated Consolidation reduces maximum load by 2.5-25X 24 Benefits: Reduction in Provisioning Cost ProvisioningToday /ProvisioningConsolidated Consolidation reduces provisioning cost 1.8-2.5X 25 Discussion • Changes traditional vendor business – Already happening (e.g., “virtual appliances”) – Benefits imply someone will do it! – May already have extensible stacks internally! • Isolation – Current: rely on process-level isolation – Get reuse-despite-isolation? 26 Conclusions • Network evolution occurs via middleboxes • Today: Narrow “point” solutions – High CapEx, OpEx, and device sprawl – Inflexible, difficult to extend • Our proposal: Consolidated architecture – Reduces CapEx, OpEx, and device sprawl – Extensible, general-purpose • More opportunities – Isolation – APIs (H/W—Apps, Management—Apps, App Stack) 27