June 2008

Wireless Embedded Inter-Networking

Foundations Ubiquitous Sensor Networks

Self-Organized Network - Routing and Forwarding

David E. Culler

University of California, Berkeley

WEI Short Course - L8 Routing 1

June 2008 Embedded IP Architecture - Network

DHCPv6 UDP TCP Multicast Buffer Send Manager Forwarder Unicast

Queue Forwarding Table

Send Manager

Default

ICMPv6 Autoconf Router Routing Algorithm

Routing Table

Stateless Autoconf Data Ack

Addr

Media Management Control Remote Media

Period Neighbor Table Phase Pending RSSI

Link Stats

Success

Local Media

Sample Period Sample Phase WEI Short Course - L8 Routing 2

June 2008

The Basic Communication Primitive

• • • •

Transmit a packet Received by a set of nodes

– – – –

Dynamically determined Depends on physical environment at the time and what other communication is on-going And further constraints by the link layer Each selects whether

–

to retransmit Potentially after modification And if so, when WEI Short Course - L8 Routing 3

Wireless Multihop Communication

•

Upon each transmission, one of the recipients retransmits

– –

determined by source, by receiver, by … on the ‘edge of the cell’ June 2008 WEI Short Course - L8 Routing 4

140 120 100 80 60 40 20 0 0

Why Multihop Communication?

1 2 3 4 5 • • •

Power!

– –

to transmit D grows as D 3 or worse to route distance D grows linearly Bandwidth (spatial multiplexing)

–

With n nodes in a single cell, each gets at most 1/n bandwidth

–

Many small cells => many simultaneous transmissions.

Reliability (spatial diversity)

–

Individual links experience interference, obstacles, and multipath effects

–

Even short range “wireless wires” require human nurturing

»

IRDA, Bluetooth, WiFi, Cell

–

Provides spatial diversity and receiver diversity

»

rather than antenna diversity

–

Protocol level reliability WEI Short Course - L8 Routing June 2008 5

June 2008

WSN Communication Requirements

• Local neighbor communication (1 to few) • Dissemination (1 to many) • Data Collection and Aggregation (many to 1) • Point-to-point Transfers (1 to 1) • • • •

Reliably over lossy links At low energy (E = P*T)

–

Idle listening, management, monitoring

Adapting to changing conditions With very light memory footprint

WEI Short Course - L8 Routing 6

June 2008

Embedded Network Organization

IP Sensornet Sensor Node IP Network (powered) Border Router IP Device

WEI Short Course - L8 Routing 7

June 2008 Network Architecture – What forms the IP Link?

•

IP Protocols assume certain link properties

•

Many assume a full-broadcast domain

– – –

“Everyone can communicate with each other” Reflexive and transitive reachability Not ad-hoc, wireless networks

•

Examples

– – – –

IPv6 Neighbor Discovery IPv6 Address Autoconfiguration ICMPv6 Redirect DHCPv6 WEI Short Course - L8 Routing 8

Network Architecture – PAN = IP Link

Personal Area Network (PAN) Single IP Link • • •

PAN



IPv6 Link-Local Scope Emulate reflexive and transitive reachability Conceivable to run existing IP-based protocols unmodified

• •

June 2008 No IP-level visibility into wireless topology Must define subnetwork functionality WEI Short Course - L8 Routing 9

Network Architecture – Local Link = IP Link

Personal Area Network (PAN) Multiple IP Links

June 2008

• • • •

Radio Range



IPv6 Link-Local Scope IP-level visibility into link topology Routing metrics across other link technologies Utilize functionality defined by IP

•

Non-reflexive and non-transitive reachability WEI Short Course - L8 Routing 10

Multi-Hop Communication

PAN • • • •

Short-range radios & Obstructions => Multi-hop Communication is often required

– –

i.e. Routing and Forwarding That is what IP does!

“Mesh-under”: multi-hop communication at the link layer

–

Still needs routing to other links or other PANs “Route-over”: IP routing within the PAN 6LoWPAN supports both June 2008 WEI Short Course - L8 Routing 11

• • • • • •

IP-Based Multi-Hop

IP has always done “multi-hop”

–

Routers connect sub-networks to one another

–

The sub-networks may be the same or different physical links Routers utilize routing tables to determine which node represents the “next hop” toward the destination Routing protocols establish and maintain proper routing tables

– – –

Routers exchange messages with neighboring routers Different routing protocols are used in different situations RIP, OSPF, IGP, BGP, AODV, OLSR, … IP routing over 15.4 links does not require additional header information at 6LoWPAN layer Vast body of tools to support IP routing

– –

Diagnosis, visibility, tracing, management These need to be reinvented for meshing IP is widely used in isolated networks too

–

Broad suite of security and management tools WEI Short Course - L8 Routing June 2008 12

June 2008

Terminology

• • • • •

There is no single-hop routing!

IP routing allows hosts in one network (IP Link) to communicate with hosts in another Topology Formation: determining the connectivity graph, i.e., the network.

Routing: Protocols and Process for establishing what paths are used in communicating over that graph and setting up tables.

Forwarding: process of receiving messages on one interface, looking up the next hop, and transmitting them on that interface

•

Meshing: some combination of formation, routing, and forwarding that occurs at the link layer (L2) transparently to the network layer WEI Short Course - L8 Routing 13

Meshing vs Routing

• • • •

Conventional IP link is a full broadcast domain

–

Routing connects links (i.e, networks) Many IP links have evolved from a broadcast domain to a link layer “mesh” with emulated broadcast

– –

ethernet => switched ethernet 802.11 => 802.11s

Utilize high bandwidth on powered links to maintain the illusion of a broadcast domain 802.15.4 networks are limited in bandwidth and power so the emulation is quite visible.

• •

Routing at two different layers may be in conflict On-going IETF work in ROLL working group

–

Routing Over Low-Power and Lossy networks WEI Short Course - L8 Routing June 2008 14

June 2008

Classical View of Routing

• • • •

Connectivity between nodes defines the network graph.

–

Topology formation A Routing algorithm determines the sub-graph that is used for communication between nodes.

–

Route formation, path selection Packets are forwarded from source to destination over the routing subgraph

–

At each node in the path, determine the recipient of the next hop The selection at each hop is made based on the information at hand

– – –

Sender address, current address, destination address, information in the packet, information on the node.

Table driven, source based, algorithmic, … Who knows the route? Do you determine it as you go?

WEI Short Course - L8 Routing 15

June 2008

Basic Approaches

• •

Link state

–

Nodes shout (send) and listen (receive) to determine neighbor connectivity.

–

Each floods this information throughout (Link State Advertisement) so every node has a map (Link state data base) of the network.

– – –

Any node can determine the path or the next hop.

management protocol deals with changes in connectivity Classic Example: OSPF Distance vector

–

Nodes maintain routing information about “distance” and “direction” to destinations

–

Choose next hop by comparing the cost of routing through neighbors

»

Cost(dest D, neighbor b) = linkCost(b) + pathCost(b,D)

–

Management propagates routing information

»

Sequence numbers, etc.

–

Classic Example: RIP WEI Short Course - L8 Routing 16

What’s different in WSN?

• • •

There is no a priori network graph

–

It is discovered by sending packets and seeing who receives them.

–

The link relationship is not binary.

»

pairs of nodes communicate with some probability that is determined by many of factors.

–

It is not static.

The embedding of the “network” in space is important.

– –

Need to get information to travel between particular physical places.

But the “communication range” is not a simple function of distance.

addressing & naming

–

Flat EUID? Hierarchical IP? Topologically meaningful? Spatially meaningful?

WEI Short Course - L8 Routing June 2008 17

Topology Formation

• • •

Much of the “paper protocols” define connectivity graph with unit disk model

–

Link(A,B) iff dist(A,B) ≤ R OK for rough calculations, but not for protocol design

– – –

Nearby nodes may not be able to communicate.

Far away nodes may be able to communicate.

Nodes that communicated in the past may not be able to communicate in the future.

–

Nodes may have intermittent communication depending on external factors.

R

Connectivity is determined by communication

– – –

If B receives packet reasonably reliably from A, then A



B If A receives packet reasonably reliably from B, then A



B And if both are true, A



B WEI Short Course - L8 Routing June 2008 18

June 2008

Wireless Routing Protocols

• • – – – – –

Many wireless protocols in the IP context have been development in the IETF MANET (Mobile Ad Hoc Networking) working group in the context of 802.11 links carrying traditional TCP/IP point-to point traffic.

AODV – ad hoc on-demand distance vector OLSR – Optimized link state Routing DSDV - Destination Sequenced Distance Vector DSR – Dynamic Source Routing TDRPF - Topology Dissemination Based on Reverse-Path Forwarding Assume a fairly “classic” view of connectivity

–

Naïve radio WEI Short Course - L8 Routing 19

Neighbor Communication

June 2008 1 0 WEI Short Course - L8 Routing 1 20

June 2008

Fundamental Primitive

•

Transmit to whatever receivers happen to hear it

• •

This is the fundamental primitive that is buried underneath complex protocols like Bluetooth, but not made available.

It is what make it possible to build higher level protocols on the link, especially IP.

•

To determine connectivity,

– – –

Local broadcast Respond on-going protocol to estimate quality of the link

»

Packet reliability (sequence numbers, acks)

•

Note 802.15.4 acks only from a specific destination

»

RSSI, LQI, … WEI Short Course - L8 Routing 21

June 2008

Neighbor Communication in IPv6

• •

Route-over: this is simply transmission to the Link Local All Nodes multicast address – FF02::1 Mesh-under: the entire PAN is a single IP link (i.e., mesh under), this concept is lost.

• • •

At the link layer, the broadcast address is well defined But, in a low-power link, this is no longer the most basic communication primitive.

– –

Only receive if listening Link local broadcast requires either more wake up or more coordination than link local unicast Unicast/Multicast distinction remains WEI Short Course - L8 Routing 22

Simple Address-Free Flooding Protocol

• •

Root broadcasts a “new” message to local neighborhood Each node performs a simple rule if (“new” incoming msg) then take local action retransmit modified msg

•

No underlying routing structure required

–

The connectivity over physical space determines it.

June 2008 WEI Short Course - L8 Routing 23

Route Free “Flood”

June 2008 1 0 WEI Short Course - L8 Routing 1 24

June 2008

“Flooding”

• •

Route free dissemination is extremely useful in its own right

–

Disseminate information

»

Router advertisements, solicitaitons, …

– –

Network-wide discovery Join, … It is also the network primitive that most “ad hoc” protocols used to determine a route

– –

Flood from source till destination is reached.

Each node records the source of the flood packet

»

This is the parent in the “routing tree”

–

Reverse the links to form the path back WEI Short Course - L8 Routing 25

Data Collection in concept

June 2008 2 2 2 1 0 WEI Short Course - L8 Routing 1 2 2 26

The Problems

• •

Flood causes tremendous contention

– –

Many good links missed because of collistions Huge amount of noise Many links are not symmetric June 2008 WEI Short Course - L8 Routing 27

Flood Dynamics

• • •

Experimental Setup

–

13x13 grid of nodes

– – –

separation 2ft flat open surface Identical length antennas, pointing vertically upwards.

– –

Fresh batteries on all nodes Identical orientation of all nodes

–

The region was clean of external noise sources.

Range of signal strength settings Log many runs

Ganesan, Krishnamachari, Woo, Culler, Estrin and Wicker,

Complex Behavior at Scale: An Experimental Study of

Low-Power Wireless Sensor Networks , UCLA Computer Science Technical Report UCLA/CSD-TR 02-0013

June 2008 WEI Short Course - L8 Routing 28

Final Tree

June 2008 WEI Short Course - L8 Routing 29

Factors

• • •

Long asymmetric links are common

–

Many children Nodes out of range may have overlapping cells

–

hidden terminal effect Collisions => these nodes hear neither ‘parent’

–

become stragglers

• •

As the tree propagates

– – –

folds back on itself rebounds from the edge picking up these stragglers.

Redundancy

–

Geometric overlap => <41% additional area June 2008

Ni, S.Y., Tseng, Y.C., Chen, Y.S., Sheu, J.P.:

The broadcast storm problem in a mobile ad hoc network

. MobiCom'99

WEI Short Course - L8 Routing 30

Topology Reinforcement

June 2008 2 2 2 1 0 WEI Short Course - L8 Routing 1 2 2 31

Network Layer

Multicast Buffer Send Manager Forwarder Unicast

Queue Forwarding Table

Send Manager

Default

ICMPv6 Autoconf Router Routing Algorithm

Routing Table

Stateless Autoconf

June 2008

• • •

Router

– –

Populate the routing table with candidates By listening to Router Advertisement (RA) messages

»

In LoWPAN Link quality is advertised too

–

Manages entries in the forwarding table Forwarder

–

Recv datagram on interface, lookup next hop in FT, request transmission to that

–

Default route In IPv6, topology reinforcement is by RA

–

Plus whatever passive observation and link level piggy backing WEI Short Course - L8 Routing 32

June 2008

Routing – Baseline

•

Routing state very limited. Routing protocol messages very infrequent.

•

Reduce problem by:

– – –

Egress routing (default routes) Ingress routing (source routing) Any-to-any through border router or scoped discovery

IP Network (powered) Border Router

WEI Short Course - L8 Routing

IP Device

33

June 2008 Routing – Egress Routes

• • •

Distance-vector protocol rooted at Border Router Piggyback on ICMPv6 Router Advertisements

– –

Hop-count or other routing metric Version number to clear routing state List of next-hop candidates

– –

Top candidate used to configure default routes Select a random next-hop if default route fails

fd00::4 fd00::1 fd00::2

WEI Short Course - L8 Routing

fd00::3 fd00::5

34

Routing – Ingress Routes

•

Record route sent to subnet router anycast

•

Forward using IPv6 Routing extension header

–

Use 16-bit short compressed form to minimize overhead June 2008

fd00::4 1 | 3 | fd00::4 fd00::1 fd00::3 fd00::2

WEI Short Course - L8 Routing

fd00::5 fd00::4

35

June 2008 Routing – Any-to-any Routes

•

Route through border router by default

•

Utilize scoped discovery to for nearby neighbors

– –

Worst-case stretch (2D) with neighboring nodes New stretch 2D/(s+1), s = discovery scope

fd00::4 1 | 3 | fd00::4 fd00::1 fd00::3 fd00::2 fd00::4 fd00::5 Worst-case Stretch

WEI Short Course - L8 Routing 36

Achieving Reliability in the Face of Uncertainty

June 2008 WEI Short Course - L8 Routing 37

Abstracting Uncertainty

Signal June 2008 Noise Distance WEI Short Course - L8 Routing 38

Which node do you route through?

June 2008 WEI Short Course - L8 Routing 39

What does this mean?

•

Always routing through nodes “at the hairy edge”

–

Wherever you set the threshold, the most useful node will be close to it

• •

Topology determination is a continuous process of discovery and validation

–

and it must be done politely Connectivity is determined by communication

–

If B receives packet reasonably reliably from A, then A



B

– –

If A receives packet reasonably reliably from B, then A



B And if both are true, A



B WEI Short Course - L8 Routing June 2008 40

June 2008

Reliability Techniques

• • • • • • •

The IP end-to-end architecture assumes IP links are 99% reliable

–

Best effort says don’t try to push it to %99.999 on every link, put E2E reliability protocols over it when you need it, i.e., TCP

–

Not that every link should send and forget.

Good Low-power links are often <%90 reliable

–

0.9

h drops awfully quickly Need link-level acks

– –

15.4 acks not even good enough Filters out asymmetric links Need hop-by-hop retransmission

90% 80% 70% 60%

P

50% 40% 30% 20% 10% 0% 1 2 3 4 5

hops

6

Piggy-back net acks on link level Retransmission and rerouting

7 8

Provides the cross-layer visibility

9 10

WEI Short Course - L8 Routing 41

Routing

Selecting Bi-Directional Links and Forming Routes June 2008

Default Routes Discovering Links ICMPv6 Hdr Router Adv MHop Info Inferring a Connectivity Graph Low Routing Cost Routing Table

Prefix Next

High Confidence High Routing Cost Low Confidence Routing Table

Prefix Next

Selecting a Default Route Forwarding Table

Next

• Top candidate • Dynamic re-routing • Increasing confidence

Default WEI Short Course - L8 Routing 42

June 2008 High Quality Selection

•

Sort by:

– –

Link quality estimate confidence Path cost (including link quality)

•

Link success rate requires state

–

RSSI for insertion (no prior state)

•

Link success computed by link for each transmission

–

Dynamically alter default route to:

» » »

Refresh link quality estimates Discover lower path costs Re-routing naturally does this WEI Short Course - L8 Routing 43

Building Neighborhoods & Routes

• •

Node transmits to some unknown set Candidate nbrs are sources of incoming packets

• •

Estimate of inbound link reliability Occasionally announce inbound link states

– –

Provides reverse link estimate to outbound neighbors Basis for cost-based routing

•

Cost-based Parent Selection

– depth(me) = MIN nbr(me) depth(i) – loss(me) = MIN nbr(me) – trans(me) = MIN nbr(me) loss(i)*est(me,i) trans(i)+etrans(me,i) • What about nbrs that don’t fit in the table?

– FIFO, LRU, Frequency Taming the Challenges of Reliable Multihop Routing in Sensor Networks,

June 2008 44

Pragmatics

• • •

Maintain at least two (preferably three) candidate parents

– – –

Link level retransmission and rerouting Use acks to determine quality of the link Throw in a new candidate from time to time.

Do not record list of children

–

Insufficient memory to build the table.

Route update message on a trickle schedule June 2008 WEI Short Course - L8 Routing 45

Adding up the pieces - footprint

24038 ROM

* Production implementation on TI msp430/cc2420 CC2420 Driver 802.15.4 Encryption Media Access Control Media Management Control 6LoWPAN + IPv6 Checksums SLAAC DHCPv6 Client DHCPv6 Proxy ICMPv6 Unicast Forwarder Multicast Forwarder Message Buffers Router UDP TCP ROM 3149 1194 330 1348 2550 134 216 212 104 522 1158 352 0 2050 450 1674

WEI Short Course - L8 Routing June 2008

RAM 272 101 9 20 0 0 32 3 2 0 451 4 2048 106 6 50

46

… and reliability

Application Power Model June 2008

Data Rate Sensitivity (Router) Data Rate Sensitivity (Edge) Deployment Duty Cycle

WEI Short Course - L8 Routing

Deployment Reliability

47

Discussion

•

Upcoming

– –

Trickle protocol Extension of IPv6 to support this sort of routing June 2008 WEI Short Course - L8 Routing 48