Tiny Networking David Culler University of California, Berkeley Intel Research Berkeley http://webs.cs.berkeley.edu Vast Networks of Tiny Devices • Past 25 years of internet technology built.
Download ReportTranscript Tiny Networking David Culler University of California, Berkeley Intel Research Berkeley http://webs.cs.berkeley.edu Vast Networks of Tiny Devices • Past 25 years of internet technology built.
Tiny Networking
David Culler
University of California, Berkeley
Intel Research Berkeley http://webs.cs.berkeley.edu
Vast Networks of Tiny Devices
• •
Past 25 years of internet technology built up around powerful dedicated devices that are carefully configured and very stable
– –
local high-power wireless subnets at the edges 1-1 communication between named computers Here, ...
• • • • • •
every little node is potentially a router work together in application specific ways collections of data defined by attributes connectivity is highly variable must self-organize to manage topology, routing, etc and for power savings, radios may be off 99% of the time NEC Networking 11/14/2002
Directed Diffusion Concept
• •
Nodes express ‘interest’ in data with certain attributes (sinks) Establishes gradient from sources 11/14/2002 Estrin, Govindan, Heideman NEC Networking
Directed Diffusion Concept
• • • •
Nodes express ‘interest’ in data with certain attributes (sinks) Establishes gradient from sources Sources generate data Useful paths reinforced, others suppressed
• •
in-network aggregation nested queries 11/14/2002 NEC Networking
Huge Design Space: Application
•
Traffic
– –
any-to-any, all-to-one, one-to-all, collection of sub-groups, ...
steady low-BW steam of readings/findings, bursts of data, periodic logs, ...
•
Duration
–
years (hazard alarm), months (field season), when the big red button is pressed
•
Available infrastructure
–
power, base-stations, location
• •
Mobility, Stationary
–
all fixed, all mobile, mixture, changing environment Placement / Physical Topology / Scale
–
arbitrary, controlled, unknown
•
Network Services
–
time synchronization, localization, proximity, ...
=> Best way to get a networking solution now is to define your application context NEC Networking 11/14/2002
Design Space: Underlying Technology
• • • • • • •
Link (Radio) Technology
– – –
range, control of signal strength, noise tolerance channel capacity, coding, error rates single channel, multi-channel, tunable Device Density, Failures over time MAC (media access control)
– –
channel sensing, back-off, protocols, collision behavior, link-level acks time synch, energy aware Power management & Energy Constraints
–
scheduling, functional allocation Transmission Rate Control Topology Formation
–
hierarchical spine, geographic, static, dynamic Routing
– –
single path, multipath, passive participation explicit vs implicit nbhd detection NEC Networking 11/14/2002
Losing the forest for the trees
• • •
Systems side
–
clever fix to one particular aspect, but only looking at it in sliver of the design space Theory side
–
assuming cell coverage is a disk of radius r with sharp boundary
– –
within radius P connect = 1, outside P connect = 0 eg., unit disk-graphs for routing, maximal independent sets, min. dominating sets, leader election Wireless communication between small devices is inherently noisy, unpredictable, & non-deterministic
– – – – –
Radio signal fades with distance Interference Multipath (reflections) Collisions Mobility or not NEC Networking 11/14/2002
Surge Demo
11/14/2002 NEC Networking
Local Operations => Global Behavior
•
Nodes ‘sense’ network environment
–
uncertain, partial information
•
Packets directed to a “parent” neighbor
–
all other neighbors “hear” too
–
carry additional organizational information
•
Each nodes builds estimate of neighborhood
–
adjusted with every packet and with time
•
Interactively selects parent
•
Routes traffic upward
Collectively they build and maintain a stable spanning tree
takes energy to maintain structure
Predictable global behavior built from local operations on uncertain data
node # 17 6 ...
dept h 1 3 child?
parent ?
yes yes % link 90 75 goodnes s .7
.6
11/14/2002 NEC Networking
Wireless “Connectivity”
•
Controlled study on 13x13 array of Rene nodes
• •
Single transmitter Record fraction of packets received at each node
•
Many packets at each of several transmit power levels
complex fall-off over substantial range
‘range’ defined by CEP
Contours of probability of reception from center node for range of transmit power strengths
11/14/2002 with Ganesan, Woo, Krishnamacheti, Estrin NEC Networking
Alternative Perspective
• •
Develop algorithms that are built fundamentally upon a probabilistic connectivity structure
– –
Embrace noise, rather than fight it the network is really another sensor (and actuator) Utilize simple, local rules, rather than complex protocols
• •
Challenge: obtain predictable global behavior Challenge: interfaces for imperfect operation 11/14/2002 NEC Networking
Simple Epidemic Broadcast Schema
Local Rule: if (new mcast) then take local action retransmit modified request
• • • •
Should forms roughly breadth-first spanning tree Examples: Network wakeup, command propagation
–
Build spanning tree
»
record parent Naturally adapts to available connectivity Minimal state and protocol overhead => surprising complexity in this simple mechanism NEC Networking 11/14/2002
Network Discovery: Radio Cells
11/14/2002 NEC Networking
Network Discovery
11/14/2002 NEC Networking
Behavior at Scale
• • • •
Variations in connectivity With many nodes, likely that one far away will hear Long links tend to be asymmetric Structure dominated by contention 11/14/2002 NEC Networking
Final Tree
11/14/2002 NEC Networking
1000
Power Laws ?
1000 100 100 10 10 1 1 10
Cluster Size (1 + # children)
100 1 1 10
Cluster Size
• • •
Most nodes have very small degree (ave = .92) Some have degree = 15% of the population Few large clusters account for most of the edges 11/14/2002 NEC Networking
100
Open Territory => Many Children
•
Example: Level 1 11/14/2002 NEC Networking
Open Territory => Many Children
•
Example: Level 2 – variation in distance 11/14/2002 NEC Networking
Open Territory => Many Children
•
Example: Level 3 – long links 11/14/2002 NEC Networking
Importance of Asymmetric Links
• • •
Asymmetric Link :
– –
>65% successful reception in one direction <25% successful reception in the other direction 10%-25% of links are asymmetric Many long links are asymmetric
–
in large field it is likely that someone far away can hear you
–
what does this mean for protocol design?
11/14/2002 NEC Networking
Collisions are primary factor • •
Nodes out of range may have overlapping cells
–
hidden terminal effect Collisions => these nodes hear neither ‘parent’
–
become stragglers
• • •
As the tree propagates
– – –
folds back on itself rebounds from the edge picking up these stragglers.
Seen in many experiments Mathematically complex because behavior is not independent beyond singe cell NEC Networking 11/14/2002
Probabilistic Connectivity Model
•
Radio signal fades with distance in complex manner depending on environment
•
Radio receiver has complex behavior to extract signal
•
What matters to algorithms is whether packets are delivered or not
work directly with probabilistic communication model
but which one?
•
Calculate comm. rates for numerous transmit/receive pairs at range of distances 11/14/2002 NEC Networking
Fall-off of Probability of Comm.
Low Power 11/14/2002 NEC Networking High Power
Drive Simulation from Empirical Stochastic behavior
11/14/2002 clear transition silent NEC Networking
Example Cell Coverage from Model
feet 11/14/2002
Reception Model with Collisions
•
Second experiment with two nodes sending at once, record which nodes hear which one
• –
follows P success closely When does a second sender collide?
– – –
Clear comm. region => YES silent region => NO transition region => collides if would have communicated
•
Reception Model
–
Assume p ij is the probability of success (i->j), Probability for B to receive A’s message = p ab *
i=collider (1-p ib ) 11/14/2002 NEC Networking
Common Special Case: Data Gathering
• •
Collection of nodes take periodic samples Stream data towards a root node
• • • •
Root announces interest
–
depth = 0 Nodes listen to neighbors When hear neighbor with smaller depth
– –
start transmitting data to good neighbor with smallest depth set own depth to one greater and include with data Data transmission continuously reinforces / adjusts routes 11/14/2002 NEC Networking
Use Case Assumptions
• •
Application
– –
N-to-1 all data, no aggregation Each node generates small packets at regular interval
»
Appln phase shift on collision
– – –
Routed to a specific node, e.g, base station or root of request data rate below saturation 7x7 grid, 10 ft spacing => several hops Underlying
– –
link-level acknowledgement (may be used for retransmission) CSMA with simple channel sensing, fixed backoff, initial random delay
–
Receivers always attentive
»
may use sampled listening
• • •
Broadcast: choose parent = first contact Shortest Path: choose parent = closer (which one) Min path loss: choose parent = next step on min loss path NEC Networking 11/14/2002
Max-Path-Reliability routing
• each node maintains estimate of loss rate over entire path to root • select nbr on the minimum loss path as parent – P i [to root] through j = P[link i,j] * P j [to root] – assuming loss rate along path is independent of how packets enter the path • Subtleties – estimating link rates – transient error in link rate may lead to cycles – rate of updates – stability vs responsiveness – congestion – warm-up phase
60 65 70 99 95 95 95 P=0.86
NEC Networking 11/14/2002
Broadcast
•
overall success rate: 18.9% 11/14/2002 NEC Networking
SP50 Path reliability
•
overall success rate: 44.8% 11/14/2002 NEC Networking
MPR path reliability
•
overall success rate: 54.7% 11/14/2002 NEC Networking
Routing Distance Distribution
11/14/2002 MPR NEC Networking SD50
Distribution of success rates
11/14/2002 MPR NEC Networking SP50
Observations
• • •
Better success rate with more, better hops Partial information causes temporary loss
– – –
cycles introduced when link error estimates are poor tracks std. dev. of link error rates if difference between candidates not statistically significant, choose based on hop count
–
especially important during warm-up phase
»
also after topology changes Shortest Path will tend to use most marginal neighbors
–
although range is highly variable, for particular pairs of nodes at particular distances connectivity is very bimodal NEC Networking 11/14/2002
SP75 Path Reliability
•
overall success rate: 52.7% (vs 54.7% for MPR) 11/14/2002 NEC Networking
Multihop Path-Rate Contours
Min Path Loss 11/14/2002 Min Hop (75% nbr) NEC Networking
Hop-by-hop retransmission
•
Decent neighbors become good neighbors if you are willing to chat for a moment
• • •
may choose parent that is on “path of least transmissions”
–
expected number of transmissions = 1 / P(success) Distributed computation is estimate of SUM retransmissions of A few retransmissions make large difference in success rate NEC Networking 11/14/2002
11/14/2002
Overall Results
Success Rate to BS # Retrans / App packet Total Msg. Overhead/ pack. Recv. By BS Ave.
Hop Count MRP 83.94% Shortest Path Broadcast 85.79% 42.96% 1.03
0.82
5.76
8.36
7.13
13.13
6.12
6.18
4.92
NEC Networking
Fairness
11/14/2002 MRP SP75 NEC Networking Broadcast
Stability
• •
How often does the routing algorithm change?
# of parent changes per unit time MRP SP75 11/14/2002 NEC Networking
Stability (Broadcast)
11/14/2002 NEC Networking
Key Building Block: Link estimator
• • •
Nodes assess quality of link from packets they hear
– –
directed to them or snoop sequence number => infer losses between packets For each new packet (or empty window) revise estimate of link probability classic EWMA
–
P i+1 =
a
P i + (1-
a
) X i , where X i = 0,1 if loss, success 11/14/2002 NEC Networking
Link Probability Changes Dynamically
11/14/2002 NEC Networking
Revisit Classic Estimator
• •
Want estimator that is responsive to change, but stable with low error Candidates
– – –
EWMA stable, agile, flip-flop Moving Average ...
•
Best ended up being the own that we eyeballed
–
EWMA cascaded with average over a time window 11/14/2002 NEC Networking
Estimator driven from prob. generator
11/14/2002 NEC Networking
Empirical Trace
11/14/2002 NEC Networking
Bottom Line
• •
Can only estimate link rate to within ~10% Takes about 100 packets to settle
•
Design distributed algorithms with this kind of approximation of an inherently noisy world 11/14/2002 NEC Networking
So how about that demo?
11/14/2002 NEC Networking
Higher Level Network Services
• • • •
Time Synchronization
– –
many nodes with drift and offset variation classic:
» »
pairwise timesynch limited to half the variance in RTT NTP hierarchy to establish rough global time
–
local timesynch
»
deep integration into network stack (Hill)
»
multiple receivers, not send-rcv (Ellison, Girod)
–
global propagation: huge open distributed algorithm question Ranging & Localization Naming Data distribution, Aggregation, Storage
•
We’ve just begun to tackle the full problem NEC Networking 11/14/2002
Conclusion
• •
The networking part of the “future sensor networks” very different the power-hungry, sparse IP world
– – – –
severe constraints, noisy, dynamic, partial information Low power means you hardly ever listen self-organized and inherently distributed opportunity to work across layers of abstraction Presents a rich set of open problems
– – –
Probabilistic connectivity model is usable and powerful On-line distributed approximations to global optimization a whole new internet to invent
•
11/14/2002 Applications define valid assumption sets for network design
–
guide the front of progress in this huge space NEC Networking
Engineering Perspective
• • •
Low-hanging fruit
–
enough base stations that only need single hop Mid-range fruit
–
stationary, well-powered multi-hop backbone Research centroid
–
unstructured, ad hoc, multihop routing in energy starved environment 11/14/2002 NEC Networking