Overview of Mesh Networking Research @ MSR

Jitendra Padhye

Microsoft Research January 23, 2006

What are mesh networks?

• Multi-hop wireless networks • Mostly static nodes • Unplanned node placement • Applications: Disaster relief, Backhaul for city-wide wireless networks, Meeting mesh, Neighborhood Meshes, internet connection sharing • Many startups ….

Three main problems in mesh networking

• Capacity • Capacity • Capacity

Why is capacity a problem?

Source Mesh Router Destination With a single radio, a node can not transmit and receive simultaneously. A two-hop path has half the capacity of a one-hop path. Other interference patterns also possible.

Seminal Result by Gupta and Kumar (2000): Capacity = O(1/sqrt(n))

MSR’s research on Mesh Network Capacity

• Capacity estimation • Capacity improvement using multiple radios and other techniques • Feasibility study using realistic traffic

Mesh Network Capacity Estimation

• New framework for estimating capacity of multi-hop wireless networks – Gupta-Kumar result is asymptotic – Our framework calculates optimal capacity of a given mesh network for given set of flows 

MobiCom 2003 (Jain, Padhye, Padmanabhan and Qiu).

• Our framework requires knowledge of which links interfere with one another – Problem of “conflict graph” estimation – N nodes  O(N^2) links  O(N^4) pairs!

– We developed an approximation technique that takes O(N^2) time 

IMC 2005 (Padhye, Agarwal, Padmanabhan, Qiu, Rao and Zill)

Key Insight: Multiple radios necessary to improve capacity

Improving capacity using Multiple Radios

• Select best radio to send each packet using locally available information – Multi-radio unification protocol 

IEEE BroadNets 2004: Adya, Bahl, Padhye, Wolman and Zhou)

– Problem: sub-optimal in many cases • Optimize entire path for a given flow – Take into account interference and link capacity along entire path – Implemented in Mesh Connectivity Layer (MCL) 

MobiComm 2004: Padhye, Draves, Zill

• If second radio has very low bandwidth, can we use it to offload signaling?

– Simulation-based study of separating control and data into different frequency bands 

IEEE BroadNets 2005 (Kyasanur, Padhye, Bahl)

How do we know how much capacity is “enough”?

Feasibility study using realistic traffic

• Collect traffic traces from Microsoft’s wired network • Replay on mesh testbed • Study delay characteristics of replayed traffic • Conclusions: – Factors such as specific card brands, placement of servers have significant impact, routing metrics have less impact.

– 2-radio mesh network likely sufficient for supporting normal office traffic – Some large delay spikes.

•

MobiSys 2006 (Eriksson, Agarwal, Bahl, Padhye)

Ongoing work related to capacity:

• Capacity improvement using network coding • Use of directional antennas to reduce interference • Use of spectrum etiquettes and cognitive radios to improve spectrum utilization

Other challanges:

• Self-management – Network without administrator – is it possible?

– Engineering challenges such as automatic address assignment • Security and Fairness – Freeloaders – Information leakage by observing traffic – Malicious nodes can disrupt routing

Backup slides

Mesh Connectivity Layer (MCL)

Design & Implementation

Design Choice Multi-hop networking at layer 2.5

Framework – NDIS miniport – provides virtual adapter on virtual link – NDIS protocol – binds to physical adapters that provide next-hop connectivity – Inserts a new L2.5 header Why Layer 2.5?

– Works over heterogeneous links (e.g. wireless, powerline) – Transparent to higher layer protocols. • works equally well with IPv4 and IPv6 – ARP etc. continue to work without any changes Features – DSR-like routing with optimizations at virtual link layer – Link Quality Source Routing (LQSR) – Incorporates 5 different link selection metrics: – Hop count, RTT, Packet Pair, ETX, WCETT

Scope: Technical Problems we looked at

Range and Capacity – Off-the-shelf wireless hardware Is severely range limited – Throughput of 802.11 MAC degrades rapidly with the number of hops Our Solution: multi-radio meshbox, directional ant., NLDP, Interference management, Capacity-cal Routing – Network connectivity is highly dynamic – Classical single path & shortest path routing perform poorly in a dense network Our Solution: LQSR & MR-LQSR, WCETT (ETX, PacketPair, RTT,..) Security and Fairness – Mesh is susceptible to freeloaders and malicious users – Achieving “fairness” without topological and traffic information is difficult Our Solution: “ Windows certificate", greedy behavior detection, watchdog mechanism, intrusion detection Self Management – End users are non-technical – A no-network operator model is challenging Our Solution: M 3 , watchdog mechanism, data cleaning, liar detection, on-line network simulation, beacon stuffing, server placement Spectrum Management – Tragedy of the commons – Exploit spectrum white space Our Solution: Control channel, dual-frequency meshes, 700-900 MHz, Spectrum etiquettes

Impact of path length on throughput

Experimental Setup • 23 node testbed • One IEEE 802.11a radio per node (NetGear card) • Randomly selected 100 sender receiver pairs (out of 23x22 = 506) • 3-minute TCP transfer, only one connection at a time 10000 9000 8000 7000 6000 5000 4000 3000 2000 1000 0 0 1 2 3 4

Byte-Averaged Path Length (Hops)

5 If a connection takes multiple paths over lifetime, lengths are byte-averaged Total 506 points

.

6 Solution: Multi-Radio Meshes

Link Selection Metrics

Many metrics have been studied in literature – Hop count – – – – Round trip time Packet pair Expected data transmission count incl. retransmission Weighted cumulative expected transmission time – Signal strength stability – Energy related – Link error rate – Location related – … The ones in red are implemented in MCL

Link Selection Metric for Single Radio: ETX

• Each node periodically broadcasts a probe • The probe carries information about probes received from neighbors Advantages – Explicitly takes loss rate into account – Implicitly takes interference between successive hops into account – Low overhead • Each node can calculate loss rate on forward (P

f

) and reverse (P

r

) link to each neighbor • Selects the path with least total ETX ETX  1 (1  P f ) * (1  P r ) Disadvantages – PHY-layer loss rate of broadcast probe packets is not the same as PHY-layer loss rate of data packets  Broadcast probe packets are smaller  Broadcast packets are sent at lower data rate – Does not take data rate or link load into account Developed by De Couto et al @ MIT (2003)

Baseline comparison of Metrics

Single Radio Mesh

Experimental Setup • 23 node testbed • One IEEE 802.11a radio per node (NetGear card) • Randomly selected 100 sender-receiver pairs (out of 23x22 = 506) • 3-minute TCP transfer, only one connection at a time Median path length: HOP: 2, ETX: 3.01, RTT: 3.43, PktPair: 3.46

1600 1400 1200 1000 800 600 400 200 0 HOP ETX RTT ETX performs the best PktPair

Link Selection Metric for Multiple Radios: WCETT

State-of-art metrics (shortest path, Packet Pair, RTT, ETX) do not leverage channel, range, data rate diversity Multi-Radio Link Quality Source Routing (MR-LQSR) – Link metric: Expected Transmission Time (ETT)  Takes bandwidth and loss rate of the link into account – Path metric: Weighted Cumulative ETTs (WCETT)  Combine link ETTs of links along the path  Takes channel diversity into account – Incorporates into source routing Developed by Draves, Padhye et al @ MSR(2004)

Expected Transmission Time (ETT)

Given: – Loss rate

p

– Bandwidth B – Mean packet size S – Min backoff window CW min Takes bandwidth and loss rate of the link into account ETT  ET xmit  ET backoff where, ET xmit  S B(1  p) f(p)  1  i i    7 0 2 (i  1) p i ET backoff  CW min 2(1  f(p) p)

WCETT = Combines link ETTs

Need to avoid unnecessarily long paths bad for TCP performance - bad for global resources All hops on a path on the same channel interfere – Add ETTs of hops that are on the same channel – Path throughput is dominated by the maximum of these sums Given a

n

hop path, where each hop can be on any one of

k

channels, and two tuning parameters,

a

and

b

: WCETT   a* i n   1 ETT i   a   b* max 1  j  k X j  b where X j   ETT i hop i is on channel j Select the path with min WCETT

Baseline Comparison of Metrics

Two Radio Mesh

Experimental Setup • 23 node testbed • Randomly selected 100 sender-receiver pairs (out of 23x22 = 506) • 3-minute TCP transfer • Two scenarios: – Baseline (Single radio):  802.11a NetGear cards – Two radios  802.11a NetGear cards  802.11g Proxim cards

3500 3000 2500 2000 1500 1000 500 0

Median path length: HOP: 2, ETX: 2.4, WCETT: 3

Median Throughput of 100 transfers 1601 2989.5

1379 1508 Single Radio Two Radios 1155 844 WCETT ETX Shortest Path

WCETT utilizes 2 nd radio better than ETX or shortest path

Path Length and Throughput

Which metric is best?

• • • Experimental Setup 23 node testbed Randomly selected 100 sender receiver pairs (out of 23x22 = 506) 3-minute TCP transfer (transmit as many bytes as possible in 2 minutes, followed by 1 minute of silence) For 1 or 2 hop the choice of metric doesn’t matter 4000 3500 3000 2500 2000 1500 1000 500 0 3.5

3 2.5

2 1.5

1 0.5

0 A WCETT ETX HOP C WCETT D ETX E HOP A C D E

Testbed Configuration

F F

Comparison of Metrics

Wireless Office Scenario

23 node indoor testbed. Two radios (both 802.11a) per node. 11 active clients, 4 servers. 10000 Light Office Traffic 1 hour, 415 sessions, 19.72 MB total 1000 474 100 89 10 11 4 1 WCETT 120 6 4 ETX 82 179 5 3 8 3 HOP PKTPAIR 6 2 RTT 10000 Heavy Office Traffic 1 hour, 308 sessions, 587.5 MB total 1000 590 100 27 10 4 1 WCETT 862 ETX 31 3 943 30 3 HOP PKTPAIR RTT Relatively light traffic means performance is okay for all metrics. WCETT does better under heavy load (worst case delay)

Management:

Resiliency against Liars/Lossy Links

• •

Problem

Identify nodes that report incorrect information (liars) Detect lossy links • •

Assume

Nodes monitor neighboring traffic, build traffic reports and periodically share info.

Most nodes provide reliable information

Challenge

Wireless links are error prone and unstable • • •

Approach

Watchdogs Find the smallest number of lying nodes to explain inconsistency in traffic reports Use the consistent information to estimate link loss rates

Simulation Results Detect liars

1 0.8

0.6

0.4

0.2

0 NL=1 NL=2 NL=5 NL=8 NL=10 NL=15 NL=20 coverage false positive

Detect lossy links

1 0.8

0.6

0.4

0.2

0 NL=1 NL=2 NL=5 NL=8 NL=10 NL=15 NL=20 coverage false positive