Networked Systems in Developing Regions

Challenges and Opportunities

Lakshminarayanan Subramanian

Courant Institute of Mathematical Sciences New York University Joint work with many fantastic collaborators!

(Courant, NYU Med, NYU Wagner, UC Berkeley, Intel, Univ of Washington, Univ of Colorado, Amrita University, Aravind Eye Hospitals)

Disclaimers of a realist…

 “I am not a philanthropist”  “NGOs are great, but may not scale or be sustainable”  “Information and Communication Technologies (ICT) can only be an aid but cannot alleviate poverty”  “I believe in cell-phones, not yet in $100 laptops” 2

Causes of poverty

 Lack of awareness/opportunity  A negative earnings cycle  Serious healthcare problems  Prolonged debt  Inheritance from family/society 3

Strategies to eliminate poverty

 Jeffrey Sachs: “Provide aid to the underprivileged”  William Easterly: “2.3 trillion dollars of aid, nothing much to show. Promote homegrown solutions”  C.K. Prahlad: “Tap the fortune at the bottom of the pyramid”  Amartya Sen: “Promote development at the rural level” 4

The Bottom: A Brief Description

     3-4 billion people with per-capita equivalent purchasing power (PPP) less that US$2,000 per year Could swell to 6-8 billion over the next 25 years Most live in rural villages or urban slums and shanty towns—movement towards urbanization Education levels are low or no-existent (especially for women) Markets are hard to reach, disorganized, and very local in nature 5

The cost of being Poor

Bombay area: Credit (APR) Water (100 gal) Phone (cents/min) Diarrhea Meds Rice ($/kg) Dharavi (shantytown) 600-1000% Warden Road 12-18% $0.43

4-5 $0.011

2.5

Ratio 60-75x 37x 2x $20 $0.28

$2 $0.24

10x 1.2x

6

Even the Very Poor Spend

 Dharavi, one of the poorest villages in India:     85% have a TV 50% have a pressure cooker 21% have a telephone … but can ’ t afford a house  Even the poorest of the poor in Bangladesh:  devote 7 percent of income to communications services (GrameenPhone)  These are valid markets … 7

ICT: A Big Missing Piece

   Technology can impact everyone   “Bottom of the Pyramid” Not just Internet access:  Health, education, government, commerce Enable profitable businesses    Must be scalable and sustainable   Poor are a viable market Focus on income creation, supply chain efficiency Not charity, not financial aid Promotes stability, entrepreneurism and social mobility First World technology is a bad fit  New research agenda 8

How can ICT help?

    Communications  Awareness, access to external world, phone calls Healthcare  “Where there is No Doctor?”: Rural healthcare system    Telemedicine/consultation Continuing Medical Education for Health-workers Low-cost diagnostic tools Finance  Microfinance audit, insurance schemes Education  Educational modules, distance learning 9

Other important areas

 Commerce  Supply Chain  Agriculture  E-Governance 10

Where has ICT helped so far?



Very few ICT based efforts worldwide have led to a large-scale success

 ICT adoption successes  Cell-phone, Radio, TV  Why?

    Figure out what they need Never thrust a technology Economics not in favor Make change a gradual phenomenon!

11

A five-step approach

     1. Identify basic real-world problem 2. Where can ICT help? 3. Research challenges?

4. What is the appropriate solution?

5. Deployment?

12

ICT research challenges

Network connectivity is key!

  Traditional wire-line connectivity solutions are not economically viable!

Potential options  Develop new low-cost connectivity solution!

  Leverage existing low-bandwidth wireless solutions  Cellular, Satellite, CDMA450, WiMax

Intermittent links are a fact of life

 Budget constrained links    SMS Power outages Physical transportation links 14

Research Challenges

 Low-cost high-performance network connectivity   WiLDNet: WiFi-based Long Distance Networks Wireless Mesh Networks  Intermittent distributed systems  Leveraging the next-generation cell-phone  Redesigning applications to tolerate intermittency  “Interactive applications” in low bandwidth environments  What can you do using 30 Kbps bandwidth?

 Application specific research challenges  Security, User-interface, NLP, Image processing etc.

15

Research Challenges

 Low-cost high-performance network connectivity   WiLDNet: WiFi-based Long Distance Networks Wireless Mesh Networks  Intermittent distributed systems  Leveraging the next-generation cell-phone  Redesigning applications to tolerate intermittency  “Interactive applications” in low bandwidth environments  What can you do using 30 Kbps bandwidth?

 Application specific research challenges  Security, User-interface, NLP, Image processing etc.

16

Rural Connectivity in Developing Regions

Rural Clinic Village Kiosk Optical Fiber 3 Co m Town Village Kiosk School Rural Clinic

Current solutions:  Wired  Dial-up/DSL  Optical fiber    Cellular VSAT WiMax 17

WiFi-based Long Distance Networks

   WiLD links use

standard 802.11

radios Longer range up to 150km   Directional antennas (24dBi) Line of Sight (LOS) Why choose WiFi:     Low cost of $500/node  Volume manufacturing No spectrum costs Customizable using open-source drivers Good datarates  11Mbps (11b), 54Mbps (11g) 18

WiLD Deployments

Bay Area:

• 7 links up to 45km • Experimental testbed

India at Aravind Hosp:

• 12 links up to 15km • 9 Rural Clinics • 25,000 patients/yr • Scaling to 50 clinics

Ghana:

• 5 links up to 100km • Electronic Library

Aravind Network, Theni, India Other Projects: Digital Gangetic Plains

• 30 links • upto 39 km • 802.11 APs

Akshaya Wireless

• 400 nodes • 2-25 km • commercial APs

CRCNet,New Zealand

• 17 links • upto 13 km • Soekris SBCs 19

Akshaya: A Case study

    Joint project between the State of Kerala in India and Tulip IT Wireless IP network set up in the district of Malappuram – 630 eCenters in all Backhaul and last mile links are based on proprietary technologies 1 center for every 2000 families 20

Total Costs for network deployment

WiFi/WiMax is the most economically viable solution Fiber/WiMax is the least economically viable 21

Experience with WiLD Networks

   In the field, point-to-point performance is bad On a 60km link in Ghana  We get

0.6 Mbps

TCP vs

6 Mbps

UDP On a relay (single channel)  We get only

2 Mbps

TCP 22

WiLDNet Design Overview

 Fix 802.11 protocol problems  Replace CSMA -> TDMA  Enforce synchronization of multiple links  Variable channel loss  Adaptive loss recovery  Combine retransmissions and FEC 23

Design Constraints

 No hardware changes  Modify WiLD routers, not endpoints  Routers are inexpensive machines   low processing power low energy budget (solar)  We want to be spectrum efficient 24

Problem with 802.11: ACKs

 Low utilization    Large propagation delays Stop & wait inefficient RTS/CTS makes it worse  ACK timeouts   ACK doesn’t arrive in time Retransmissions until retry limit reached 25



Problem: Propagation Delay

Large propagation delay  high collision probability

A B

26

Design Choices for WiLDNet

 Use Sliding Window flow control    802.11 MAC ACKs disabled Packet batches sent every slot Slot allocation determined by demand  Replace CSMA with TDMA on every link  Alternate send and receive slots 27

Inter-Link Interference

Simultaneous Send B Simultaneous Receive B 1 A 2  Disable CCA C 1 A 2  12dB isolation C 1 A 2 Send & Receive B C 28

Implicit Synchronization for TDMA

   Every packet is time-stamped in TX slot Slots are offset because of propagation delay We don’t use explicit marker packets to signify end of TX slot*

∆ TX slot Sender RX slot

* 2P MAC protocol (Raman et al. Mobicom ’05)

Receiver

29

Channel Loss: From external traffic 

Strong correlation between loss and external traffic



Source (A) and interferer (I) do not hear each other A I B

30

Loss Recovery: Bulk ACKs + FEC

   Bulk ACKs:    Aggregate ACKs (bit-vectors) sent with every packet Use retransmissions for loss recovery Retry limit can be per-packet Adaptive FEC:   Sender performs encoding of packets proactively Packet level FEC Tradeoff of BW and Delay   Bandwidth efficient: use Bulk ACKs  TCP, bulk traffic Delay efficient: use Adaptive FEC  Voice, Video 31

WiLDNet Design Recap

 Replace CSMA with TDMA  Loose time synchronization   To eliminate inter-link interference Overcome variable channel loss  Adaptive loss recovery layer at link layer   Using Bulk ACKs: BW efficient Using FEC: Delay efficient 32

Evaluation: Multiple Hops outdoors

 2 hop network  19km, 1.5km

  WiLDNet: Similar throughput   Same channel OR Diff. channels More spectrum efficient 802.11

802.11

WiLDNet WiLDNet Channels Same Diff.

Same Diff.

Through-put for TCP (Mbps)

2.11

4.50

4.86

4.90

33

Deployment

34

Wireless Mesh Networks

 Rapidly deployable high performance wireless mesh networks  Current mesh networks have poor performance in multi-hop settings  What we are investigating    Multi-radio, Multi-channel Nodes with directional antennas Understanding multi-AP interactions  Visit the 12-node testbed in 715 Broadway!

35

Research Challenges

 Low-cost high-performance network connectivity   WiLDNet: WiFi-based Long Distance Networks Wireless Mesh Networks  Intermittent distributed systems  Leveraging the next-generation cell-phone  Redesigning applications to tolerate intermittency  “Interactive applications” in low bandwidth environments  What can you do using 30 Kbps bandwidth?

 Application specific research challenges  Security, User-interface, NLP, Image processing etc.

36

Intermittent Distributed Systems

 How do we build conventional distributed systems in intermittent environments?

    Connectivity is intermittent and unpredictable Net bandwidth is limited Potentially high delays Potentially lossy environments  Examples   A distributed system of cell-phones using GPRS links and SMS messages to communicate Web search from a rural cafe 37

Intermittent Web Search

 A typical search today involves 4-8 queries!

 Can we do web search in one round?

 What we have done    Change the query interface  Specify all that you know about what you are searching for Intermittent proxy issues multiple queries, prefetches and bundles response pages Local proxy enables search within retrieved bundle 38

Intermittent ATM

 How do you operate ATMs over intermittent environments?

 Our solution    Enable offline authentication Use “approximate consistency” results to split a bank balance into smaller entities Provide a “risk model” to enable extra cash availability at ATMs 39

Research Challenges

 Low-cost high-performance network connectivity   WiLDNet: WiFi-based Long Distance Networks Wireless Mesh Networks  Intermittent distributed systems  Leveraging the next-generation cell-phone  Redesigning applications to tolerate intermittency  “Interactive applications” in low bandwidth environments  What can you do using 30 Kbps bandwidth?

 Application specific research challenges  Security, User-interface, NLP, Image processing etc.

40

Low-bandwidth video streaming

 Multi-hop satellite network for distance learning /telemedicine  Very high delays, low bandwidth  Questions   Enhancing QoS on low bandwidth environments  OverQoS: Overlay based QoS On the fly transcoding  Can we develop an appropriate video codec that is easily “adaptable” to different rates?

41

Intermittent proxy

  Imagine a distributed system over a multi-hop intermittent/low-bandwidth network  Content distribution, distributed databases, video streaming, client/server transactions, web search Intermittent proxy    A generic optimization engine that performs resource allocation across flows on a low bandwidth/intermittent link Maintains “soft application state” to enhance performance Performs local hop-hop recovery 42

Research Challenges

 Low-cost high-performance network connectivity   WiLDNet: WiFi-based Long Distance Networks Wireless Mesh Networks  Intermittent distributed systems  Leveraging the next-generation cell-phone  Redesigning applications to tolerate intermittency  “Interactive applications” in low bandwidth environments  What can you do using 30 Kbps bandwidth?

 Application specific research challenges  Security, User-interface, NLP, Image processing etc.

43

Cell-phone based applications

   Cell-phone based Micro-finance    Use SIM as a cheap identity Use programmable smart-phones to provide low-cost authentication

Benefits: Reduce transaction costs and corruption

Cell-phone based Health record system  Health-workers in the field use cell-phones to enter health records  Need a distributed database syncronization/search mechanism which works over SMS-links Cell-phone based cheap Inventory management  Why need RF-ID based systems?

44

Low cost paper-watermarking

   Every piece of paper has an inbuilt random speckle pattern Can we extract this speckle pattern to watermark any paper?

 Use USB 60x microscope Applications  Verification of paper based records  Cheap inventory management 45

Watermarking initial results

Nearly 90% match Less than 20% match

46

Automated Diabetic Retinopathy

47

Medical Education Modules

 Can we create medical education modules to teach a healthcare worker to become a malaria expert?

 WiSE-MD modules for surgical education in NYU Med  Tailor modules for two focus areas:   Malaria and Infectious diseases Obstetrics and Gynecology 48

Other interesting problem areas

 Traffic Signaling Networks  User interfaces   Text-free interfaces for semi-literate or illterates Speech interfaces  Natural language processing   People interested in content in local language Language translation 49

Questions?

Thank you!

Synchronization in Bipartite Graphs

X %

A D B  2 global slots   send :X% of time receive :(1-X)% of time E C

1 - X %

51

Implicit Synchronization for TDMA

   Every packet is time-stamped in TX slot Slots are offset because of propagation delay We don’t use explicit markers based synchronization 58

Rural Connectivity in Developing Regions

Optical Fiber Rural Clinic Village Kiosk 3 Com Town Village Kiosk School Rural Clinic

 VSAT Wired  WiMAX  Optical fiber  DSL Cellular 59

CapEx for network deployment

WiFi/CDMA450 has the lowest deployment Largest cost for fiber is installation 60

OpEx for network deployment

WiFi/CDMA450 suffers from recurring spectrum lease cost Termination costs for a large portion of the OpEx 61

Conclusions for Akshaya

      WiFi with directional antennas + WiMax has most attractive economics.

WiFi/CDMA450 has lowest cost of deployment.

Largest cost for Fiber is installation.

Wireless backhaul (both WiFi and VIP) technologies have at most 1/8 the backhaul CapEx VS Fiber Largest component of the capital investment for providing connectivity is the cost of the end-user devices.

Cost of backhaul/access radio equipment on towers is miniscule 62

Technologies considered

WiFi (directional) Fiber WipLL Access Technologies WiMax WiFi (Omni) CDMA450 X X X VIP X

63

Implementation

   Driver modifications (Atheros madwifi)  Disable ACKs  Disable Carrier Sense Click Modular router framework   Creates virtual network interfaces Intercept packets and modify FEC encoding/decoding  Original packets sent first, redundant next  Decoding performed only on loss 64

Evaluation Overview

 How well does WiLDNet mitigate 802.11 protocol induced losses?

 Can multiple link synchronization in WiLDNet achieve spectrum efficiency?

 How does WiLDNet combine Bulk ACKs and FEC to achieve desired delay and bandwidth?

65

Evaluation: Single Hop

  TCP: Measured on indoor channel emulator Without channel loss: 2.5x improvement at 80km

WiLDNet constant at 6 Mbps

66

Evaluation: Single Hop

  TCP: Measured on indoor channel emulator With 10% channel loss: 2x improvement at 80km

WiLDNet constant at 5 Mbps

67

Evaluation: Single Hop outdoors

Link

B-R P-S Ghana

Distance (km)

8 45 65

802.11 CSMA TCP (Mbps)

One direction 5.03

3.62

2.80

Both directions

4.95

3.52

0.68

WiLDNet TCP (Mbps)

One direction 3.65

3.10

2.98

Both directions

5.86

4.91

5.51

  WiLDNet’s improvement increases with distance TDMA sends only 50% of time in one direction 68

Tradeoffs of Bulk ACKs and FEC

 

Delay increases with number of retries Overhead increases with increasing FEC Increasing retries

69

Summary and Conclusions

 Single hop:  Up to 8x better at 60km  5.5 Mbps vs 0.7 Mbps for CSMA  Multiple hops:  Can achieve optimal performance even with same channel operation  Can combine FEC and Bulk ACKs to achieve required delay and BW 70

Future Work

 Global scheduling  Future deployments   Aravind Eye Hospitals (India) Ghana, Guinea Bissau  Remote management  Network planning 71

Problem: Propagation Delay

 On Channel emulator with bi-directional UDP 72

Problem with 802.11: ACKs

 On Channel emulator with unidirectional UDP

Decreasing utilization with distance Not waiting for 802.11 ACKs Sharp drop at 110km

73