Transcript netseminar.stanford.edu

From the Cloud to SoNIC:
Precise Realtime Software Access
and Control of Wired Networks
Prof. Hakim Weatherspoon
Joint with Ki Suh Lee and Han Wang
Cornell University
Stanford University
April 17, 2014
The Rise of Cloud Computing
• The promise of the Cloud
– A computer utility; a commodity
– Catalyst for technology economy
– Revolutionizing for health care, financial systems,
scientific research, and society
SEATTLE
The Rise of Cloud Computing
• The promise of the Cloud
– ubiquitous, convenient, on-demand network access to a
shared pool of configurable computing resources (e.g.,
networks, servers, storage, applications, and services)
that can be rapidly provisioned and released with
minimal management effort or service provider
interaction. NIST Cloud Definition
SEATTLE
The Rise of Cloud Computing
• The promise of the Cloud
– ubiquitous, convenient, on-demand network access to a
shared pool of configurable computing resources (e.g.,
networks, servers, storage, applications, and services)
that can be rapidly provisioned and released with
minimal management effort or service provider
interaction. NIST Cloud Definition
SEATTLE
The Rise of Cloud Computing
• The promise of the Cloud
– ubiquitous, convenient, on-demand network access to a
shared pool of configurable computing resources (e.g.,
networks, servers, storage, applications, and services)
that can be rapidly provisioned and released with
minimal management effort or service provider
interaction. NIST Cloud Definition
• Requires fundamentals in distributed systems
– Networking
– Computation
– Storage
The Rise of Cloud Computing
• The promise of the Cloud
Switch
VM
App
VM
33KB
Guest OS
VM
VM
Xen-Blanket
VMM
App
Guest OS
– Networking
– Computation
– Storage
7/6/2015
Xen-Blanket
VMM
SoNIC
6
My Contributions
• Cloud Networking
–
–
–
–
–
SoNIC in NSDI 2013 and 2014
Wireless DC in ANCS 2012 (best paper) and NetSlice in ANCS 2012
Bifocals in IMC 2010 and DSN 2010
Maelstrom in ToN 2011 and NSDI 2008
Chaired Tudor Marian’s PhD 2010 (now at Google)
• Cloud Computation & Vendor Lock-in
–
–
–
–
Plug into the Supercloud in IEEE Internet Computing-2013
Supercloud/Xen-Blanket in EuroSys-2012 and HotCloud-2011
Overdriver in VEE-2011
Chaired Dan William’s PhD 2012 (now at IBM)
• Cloud Storage
–
–
–
–
Gecko in FAST 2013 / HotStorage 2012
RACS in SOCC-2010
SMFS in FAST 2009
Antiquity in EuroSys 2007 / NSDI 2006
The Rise of Cloud Computing
• The promise of the Cloud
Xen-Blanket
Switch
VMM
VM
VM
VM
– Networking
– Computation
– Storage
7/6/2015
VM
Xen-Blanket
33KB
VMM
SoNIC
8
Cloud Networking: Challenges
• Challenges remain: Performance – Packets are still lost
− Why is it so hard to move data between clouds over the wide-area?
− [NSDI 2014, 2013, 2008, FAST 2009, IMC 2010, DSN 2010]
7/6/2015
SoNIC
9
Cloud Networking: Challenges
• Uncover the ground truth
• Network changes inter-packet gap
– Traffic sent:
Packet
Interpacket gap
– Traffic received:
• Bursty traffic induced by packet chaining
• Inter-packet gaps are invisible to higher layers
(Software) Access to the physical layer is required
7/6/2015
SoNIC
10
Cloud Networking: Opportunities
• Why access the physical layer from software?
Application
Transport
Issue:
• Programmers treat layers 1 and 2 as black box
Network
Data Link
Physical
64/66b PCS
PMA
Opportunities
• Network Measurements
• Network Monitoring/Profiling
• Network Steganography
PMD
• Can improve security, availability, and performance of
the Cloud Networks
Cloud Networking: Opportunities
Understanding Cloud Networks via
Software-defined Network InterfaCe (SoNIC)
Application
Transport
Network
Data Link
Physical
Improve understanding of network
Improves security, availability, and
performance of the network
SoNIC: Software-defined NIC
• Access the PHY
– In real-time
– In software
64/66b PCS
PMA
PMD
• Separates
Netslice: Software Router [ANCS 2012]
• Enables Software-defined Networks
• Big Data-in-network solutions via
• Deep Packet Inspection (DPI) at 40Gbps
– what is sent (software)
– how it is sent (hardware)
12
Outline
•
•
•
•
Motivation
SoNIC
Network Research Applications
Conclusion / Research Agenda
7/6/2015
SoNIC
13
10GbE Network Stack
Application
Data
Transport
Network
Data Link
Preamble
Physical
64/66b PCS
Encode
Decode
Scrambler
Descrambler
Gearbox
Blocksync
PMA
Eth Hdr
64 bit
/S/
/D/
L3 Hdr
Data
L2 Hdr
L3 Hdr
Data
L2 Hdr
L3 Hdr
Data
Idle
characters
(/I/)
2 bit
syncheader
/D/
/D/
/D/
CRC
Gap
10.3125 Gigabits
/T/
/E/
16 bit
011010010110100101101001011010010110100101101001011010010110100101101
PMD
7/6/2015
SoNIC
14
10GbE Network Stack
Application
Data
Transport
Network
SW
Data Link
Physical
64/66b PCS
Encode
Decode
Scrambler
Descrambler
Gearbox
Blocksync
PMA
PMD
7/6/2015
L3 Hdr
Data
L2 Hdr
L3 Hdr
Data
L2 Hdr
L3 Hdr
Data
Packet i
Preamble
Eth Hdr
Packet i+1
CRC
Gap
HW
/S/
/D/
/D/
/D/
Packet i
/D/
/T/
/E/
Packet i+1
011010010110100101101001011010010110100101101001011010010110100101101
Commodity NIC
SoNIC
15
10GbE Network Stack
Application
Network
L2 Hdr
Data Link
Preamble
Eth Hdr
Physical
64/66b PCS
/S/
Packet i /D/
SW
Descrambler
Gearbox
Blocksync
L3 Hdr
Data
Transport
L3 Hdr
Data
Network
L2 Hdr
Data
DataCRC
Link
L3 Hdr
HW
IPG
Decode
Scrambler
Application
SW
Transport
Encode
Data
/D/ Packet/D/
i+1
IPD
Gap
Physical
64/66b PCS
Encode /T/ Decode /E/
/D/
Scrambler
Descrambler
Gearbox
Blocksync
HW
PMA
PMD
7/6/2015
011010010110100101101001011010010110100101101001011010010110100101101
PMA
SoNIC
NetFPGA
SoNIC
PMD
16
SoNIC Design
Application
Data
Transport
Network
Data Link
Preamble
Eth Hdr
L3 Hdr
Data
L2 Hdr
L3 Hdr
Data
L2 Hdr
L3 Hdr
Data
CRC
Gap
Physical
64/66b PCS
Encode
/S/
Decode
/D/
/D/
/D/
/D/
/T/
/E/
SW
Scrambler
Descrambler
Gearbox
Blocksync
HW
PMA
PMD
7/6/2015
011010010110100101101001011010010110100101101001011010010110100101101
SoNIC
SoNIC
17
SoNIC Design and Architecture
Application
Data
L3 Hdr
APP Data
Userspace
L3 Hdr
APP Data
Kernel
L2 Hdr TX MAC
L3 Hdr
Data
Transport
Network
L2 Hdr
Data Link
Preamble
Eth Hdr
RX MACCRC
Gap
Physical
64/66b PCS
Encode
/S/
Decode
/D/
/D/
SW
Scrambler
Descrambler
Gearbox
Blocksync
HW
PMA
PMD
7/6/2015
/D/
/D/
/T/
TX PCS
RX PCS
Gearbox
Blocksync
/E/
Hardware
011010010110100101101001011010010110100101101001011010010110100101101
Transceiver
Transceiver
SFP+
SoNIC
SoNIC
18
SoNIC Design: API
• Hardware control: ioctl syscall
• I/O : character device interface
• Sample C code for packet generation and capture
1: #include "sonic.h"
2:
3: struct sonic_pkt_gen_info info = {
4: .mode = 0,
5: .pkt_num = 1000000000UL,
6: .pkt_len = 1518,
7: .mac_src = "00:11:22:33:44:55",
8: .mac_dst = "aa:bb:cc:dd:ee:ff",
9: .ip_src = "192.168.0.1",
10: .ip_dst = "192.168.0.2",
11: .port_src = 5000,
12: .port_dst = 5000,
13: .idle = 12,
14: };
15:
16: /* OPEN DEVICE*/
17: fd1 = open(SONIC_CONTROL_PATH, O_RDWR);
18: fd2 = open(SONIC_PORT1_PATH, O_RDONLY);
7/6/2015
19: /* CONFIG SONIC CARD FOR PACKET GEN*/
20: ioctl(fd1, SONIC_IOC_RESET)
21: ioctl(fd1, SONIC_IOC_SET_MODE, PKT_GEN_CAP)
22: ioctl(fd1, SONIC_IOC_PORT0_INFO_SET, &info)
23
24: /* START EXPERIMENT*/
25: ioctl(fd1, SONIC_IOC_START)
26: // wait till experiment finishes
27: ioctl(fd1, SONIC_IOC_STOP)
28:
29: /* CAPTURE PACKET */
30: while ((ret = read(fd2, buf, 65536)) > 0) {
31: // process data
32: }
33:
34: close(fd1);
35: close(fd2);
SoNIC
19
Outline
• Motivation
• SoNIC
• Network Research Applications
– Measurement / traffic analysis
– Profiling / fingerprinting
– Covert channels
• Conclusion / Research Agenda
7/6/2015
SoNIC
20
Measurement / Traffic Analysis using SoNIC
• Uncover the ground truth
• Network changes inter-packet gap
– Traffic sent:
Packet
Interpacket gap
– Traffic received:
• Bursty traffic induced by packet chaining
• Inter-packet gaps are invisible to higher layers,
but not SoNIC
7/6/2015
SoNIC
21
Measurement / Traffic Analysis using SoNIC
• Precise end-to-end instrumentation platform
• Measurement at large scale
• Towards an open measurement platform
7/6/2015
SoNIC
22
Profiling / Fingerprinting using SoNIC
Profiling One Hop Through a switch
7/6/2015
SoNIC
23
Profiling / Fingerprinting using SoNIC
• Cisco Catalyst 6500 switch
• 1Gbps data (1518B)
Socket 1
APP0
0.1
Frequency (normalized)
Socket 0
1
0.01
0.001
0.0001
APP1
1e-05
TX MAC0
RX MAC0
TX MAC1
RX MAC1
TX PCS0
RX PCS0
TX PCS1
RX PCS1
TX SFP0
RX SFP0
TX SFP1
RX SFP1
7/6/2015
1e-06
109340
111340
113340
115340
117340
Interpacket gap (bits)
SoNIC
24
Profiling / Fingerprinting using SoNIC
• Router/ Switch Signatures
1
1
0.1
0.1
0.1
0.01
0.01
0.01
0.001
0.0001
1e-05
Frequency (normalized)
1
Frequency (normalized)
Frequency (normalized)
• Different Routers and switches have different response function.
• Improve simulation model of switches and routers.
• Detect switch and router model in real network.
0.001
0.0001
1e-05
1e-06
0
5000
10000
15000
20000
Interpacket gap (bits)
Cisco 4948
0.001
0.0001
1e-05
1e-06
1e-06
0
5000
10000
15000
Interpacket gap (bits)
Cisco 6509
20000
0
5000
10000
15000
Interpacket gap (bits)
IBM BNT G8264R
1500 byte packets @ 6Gbps
7/6/2015
SoNIC
25
20000
Profiling / Fingerprinting using SoNIC
• Router/ Switch Signatures
• Different Routers and switches have different response function.
• Improve simulation model of switches and routers.
• Detect switch and router model in real network.
NetFPGA 10G
1500 byte packets @ 6Gbps
7/6/2015
SoNIC
26
Profiling / Fingerprinting using SoNIC
End-to-End Profile of GENI Network
Modeling Network Elements
Testbed for Network System Theory and Queue Theory
Towards a Predictable Network
What is the
aggregate
effect?
1
0.1
1
0.01
Frequency (normalized)
0.1
1
0.1
0.001
Frequency (normalized)
Frequency (normalized)
0.0001
1e-05
0.01
0.001
0.0001
0.01
1e-05
0.001
1e-06
0.0001
0
5000
10000
15000
20000
Interpacket gap (bits)
1e-05
1e-06
0
5000
10000
15000
20000
Interpacket gap (bits)
1e-06
0
5000
10000
15000
20000
Interpacket gap (bits)
Stanford
U. Wash
Berkeley
Cornell
Princeton
UPenn
1
0.1
Frequency (normalized)
•
•
•
•
0.01
0.001
0.0001
1e-05
1e-06
0
5000
10000
15000
20000
Interpacket gap (bits)
7/6/2015
SoNIC
27
Profiling / Fingerprinting using SoNIC
• Challenges: Rogue routers
7/6/2015
SoNIC
28
Covert Channels in SoNIC
Create / Detect / Prevent Covert Channels in Layers 1 and 2
7/6/2015
SoNIC
29
Covert Channels in SoNIC
Application
• Hide transmission of data
Transport
Network
Data Link
Physical
• Storage Channel
– Writing/reading of a storage location
64/66b PCS
PMA
PMD
• Timing Channel
– Modulation of system resources
7/6/2015
SoNIC
30
Covert Channels in SoNIC
Application
Transport
Network
Data Link
• Existing Covert Channels
– TCP/IP headers, HTTP requests
– Packet Rate / Timing
Physical
64/66b PCS
PMA
PMD
• Increasing number of detection
techniques
• Covert Channels at the Physical layer
7/6/2015
SoNIC
31
Covert Channels in SoNIC
Sync
Application
Data Block
/E/
Network
/S/
Data Link
Physical
PMA
PMD
/S/
Start of Frame block
/T/
End of Frame block
/E/
Idle block
/D/
Data block
01
8
D0
16
D1
D2
C1
C1
D2
C1
40
48
D5
56
65
D6
D7
C6
D6
D6
C6
C6
C6
C6
C7
D7
D7
C7
C7
C7
C7
D4
D4
D4
C6
C6
D5
D5
C7
C7
C7
D6
/T/
/E/
Block Type
Transport
64/66b PCS
0
Block Payload
24
32
D3
D4
/T/
10
10
10
10
10
10
10
0x1e
0x33
0x78
0x87
0x99
0xaa
0xb4
C0
C0
D1
D0
D0
D0
D1
D1
10
10
10
10
0xcc
0xd2
0xe1
0xff
D0
D0
D0
D0
D1
D1
D1
D1
C2
C2
C3
C3
D3
C2
C2
D2
D2
D2
D2
D2
/D/
SoNIC
/D/
/D/
C5
D5
D5
D4
C3
C3
C3
Ethernet Frame
/S/
C4
C4
C4
C4
C4
D3
D3
D3
D3
C5
C5
C5
C5
C5
Gap
/D/
32
Covert Timing Channel in SoNIC
• Embedding signals into interpacket gaps.
– Large gap: ‘1’
– Small gap: ‘0’
Packet i
Packet i+1
Packet i
Packet i+1
• Covert timing channel by modulating IPGs at 100ns
• Overt channel at 1 Gbps, Covert channel at 80 kbps
• Over 9-hop Internet path with cross traffic (NLR)
• less than 10% BER (can mask BER w/ FEC)
• Undetectable to software endhost
7/6/2015
SoNIC
33
Covert Timing Channel in SoNIC
• Modulating IPGs at 100ns scale (=128 /I/s), over 4 hops
1
SoNIC
Kernel
CDF
0.8
3562 /I/s
3562 - 128 /I/s
3562 + 128 /I/s
BER = 0.37%
0.6
0.4
0.2
‘1’
‘0’
0
500
1500
2500
‘1’: 3562 + 128 /I/s
‘0’: 3562 – 128 /I/s
7/6/2015
3500
Interpacket delays (ns)
4500
‘1’: 3562 + a /I/s
‘0’: 3562 – a /I/s
SoNIC
34
Covert Timing Channel in SoNIC
• Prevent Covert Timing Channels?
3562 /I/s
1
CDF
0.8
0.6
0.4
0.2
0
500
7/6/2015
1500
2500
3500
Interpacket delays (ns)
SoNIC
4500
35
Covert Channels in SoNIC
• Challenges: Rogue end-hosts
7/6/2015
SoNIC
36
Outline
• Motivation
• SoNIC
• Applications
– Measurement / traffic analysis
– Profiling / fingerprinting
– Covert channels
• Discussion and Conclusion
7/6/2015
SoNIC
37
Overview of Collaborations and Resources
Mini-Cloud Testbed DURIP
– Funds for 16 SoNIC boards and
– Funds a small cloud: 38 nodes and 608 cores
– Funded by AFOSR
NSF Future Internet Architecture
– Collaboration with Cisco and other Universities such as Washington, Penn,
Purdue, Berkeley, MIT, Stanford, CMU, Princeton, UIUC, and Texas
DARPA CSSP
–
–
–
–
Funds research in three phases, we are currently in Phase 2
Requires Collaboration with non-DARPA DoD agency
Collaboration with AFRL
Collaboration with NGA
Exo-GENI
– Cornell PI into national research network
– Layer 2 access nationally
– Research in Software-Defined Networks (SDN) like OpenFlow
NSF CAREER and Alfred P. Sloan Fellowship
– Funds related basic research
38
SoNIC Contributions
• Network Research
– Unprecedented access to the PHY with commodity hardware
– A platform for cross-network-layer research
– Can improve network research applications
• Engineering
–
–
–
–
Precise control of interpacket gaps (delays)
Design and implementation of the PHY in software
Novel scalable hardware design
Optimizations / Parallelism
• Status
– Measurements in large scale: DCN, GENI, 40 GbE
7/6/2015
SoNIC
39
Concluding Remarks
• SoNIC responds to network at the center of the cloud
–
–
–
–
High precision network measurement
Profiles and characterizes switches and routers
Covert channel detection and prevention
Understand and create more available and secure networks
• Status:
– SoNIC platform is available
– DURIP grant has seeded and paid for a number of boards
• SDNM: Software Defined Network Measurement
– SoNIC enabled SDN/Openflow networks (e.g. GENI)
• Collaboration:
– Deployment in experimental networks
40
Questions
• Contact: [email protected]
• Website: http://fireless.cs.cornell.edu,
http://sonic.cs.cornell.edu, and
http://www.cs.cornell.edu/~hweather
7/6/2015
SoNIC
41
My Contributions and Paper Trail
• Cloud Networking
–
–
–
–
–
SoNIC in NSDI 2013 and 2014
Wireless DC in ANCS 2012 (best paper) and NetSlice in ANCS 2012
Bifocals in IMC 2010 and DSN 2010
Maelstrom in ToN 2011 and NSDI 2008
Chaired Tudor Marian’s PhD 2010 (now at Google)
• Cloud Computation & Vendor Lock-in
–
–
–
–
Plug into the Supercloud in IEEE Internet Computing-2013
Supercloud/Xen-Blanket in EuroSys-2012 and HotCloud-2011
Overdriver in VEE-2011
Chaired Dan William’s PhD 2012 (now at IBM)
• Cloud Storage
–
–
–
–
Gecko in FAST 2013 / HotStorage 2012
RACS in SOCC-2010
SMFS in FAST 2009
Antiquity in EuroSys 2007 / NSDI 2006