FRACTEL: Building (Rural) Mesh Networks with Predictable Performance On the Feasibility of the Link Abstraction in (Rural) Mesh Networks Dattatraya Gokhale (Indian Navy), Sayandeep Sen (U. Wisconsin-Madison), Kameswari Chebrolu (IIT Bombay), Bhaskaran Raman (IIT Bombay) Work done at IIT Kanpur Presentation at Infocom 2008, Phoenix, Apr 2008

FRACTEL Deployment wiFi-based Rural data ACcess & TELephony Cherukumilli Juvvalapalem Point-to-Point Links with Directional Antennas Landline: wired gateway to the Internet Point-to-Multi-Point link-sets using Sector Antennas 19 Km 19.5 Km Local- Gateway: gateway to LDN LACN: Local- ACcess Network at one of the villages (desired, not deployed) LDN: Long- Distance Network (deployed, in the Ashwini project) Jalli Kakinada Ardhavaram Kasipadu Alampuram Tetali Pippara Kesavaram Korukollu Polamuru Jinnuru Lankala Koderu Bhimavaram Tadinada IBhimavaram Need to support real-time apps.

Link Abstraction: Background Understanding link behaviour has implications on –Network Planning, Protocol Design, Application Design Packet Error Rate (%) Average RSSI (dBm) Link Exists Negligible error rates Link abstraction: –Either link exists or does not –That is, 0% packet reception, or ~100% –Abstraction holds in wired networks Link doesn’t exist. Steep change in Error Rate

DGP, Roofnet, FRACTEL LACN Typical link distances Network architecture Environ ment Multipath effects SNR or RSSI External interference Link abstra ction Long- distance mesh networks (e.g. DGP) Up to few tens of kms High gain directional & sector antennas on tall towers or masts Rural setting studied in depth Effect not apparent Has strong correlation with link quality Affects links performance Valid Rooftop mesh networks (e.g. Roofnet) Mostly < 500 m Mostly omni- directional antennas on rooftops Dense urban setting studied in-depth Reported as a significant component Not useful in predicting link quality Reported as not significant Not valid FRACTEL LACNs Mostly < 500 m Would like to avoid tall towers Rural, campus, residential To be determined

DGP, Roofnet, FRACTEL LACN Typical link distances Network architecture Environ ment Multipath effects SNR or RSSI External interference Link abstra ction Long- distance mesh networks (e.g. DGP) Up to few tens of kms High gain directional & sector antennas on tall towers or masts Rural setting studied in depth Effect not apparent Has strong correlation with link quality Affects links performance Valid Rooftop mesh networks (e.g. Roofnet) Mostly < 500 m Mostly omni- directional antennas on rooftops Dense urban setting studied in-depth Reported as a significant component Not useful in predicting link quality Reported as not significant Not valid FRACTEL LACNs Mostly < 500 m Would like to avoid tall towers Rural, campus, residential To be determined

DGP, Roofnet, FRACTEL LACN Typical link distances Network architecture Environ ment Multipath effects SNR or RSSI External interference Link abstra ction Long- distance mesh networks (e.g. DGP) Up to few tens of kms High gain directional & sector antennas on tall towers or masts Rural setting studied in depth Effect not apparent Has strong correlation with link quality Affects links performance Valid Rooftop mesh networks (e.g. Roofnet) Mostly < 500 m Mostly omni- directional antennas on rooftops Dense urban setting studied in-depth Reported as a significant component Not useful in predicting link quality Reported as not significant Not valid FRACTEL LACNs Mostly < 500 m Would like to avoid tall towers Rural, campus, residential To be determined

DGP, Roofnet, FRACTEL LACN Typical link distances Network architecture Environ ment Multipath effects SNR or RSSI External interference Link abstra ction Long- distance mesh networks (e.g. DGP) Up to few tens of kms High gain directional & sector antennas on tall towers or masts Rural setting studied in depth Effect not apparent Has strong correlation with link quality Affects links performance Valid Rooftop mesh networks (e.g. Roofnet) Mostly < 500 m Mostly omni- directional antennas on rooftops Dense urban setting studied in-depth Reported as a significant component Not useful in predicting link quality Reported as not significant Not valid FRACTEL LACNs Mostly < 500 m Would like to avoid tall towers Rural, campus, residential To be determined

DGP, Roofnet, FRACTEL LACN Typical link distances Network architecture Environ ment Multipath effects SNR or RSSI External interference Link abstra ction Long- distance mesh networks (e.g. DGP) Up to few tens of kms High gain directional & sector antennas on tall towers or masts Rural setting studied in depth Effect not apparent Has strong correlation with link quality Affects links performance Valid Rooftop mesh networks (e.g. Roofnet) Mostly < 500 m Mostly omni- directional antennas on rooftops Dense urban setting studied in-depth Reported as a significant component Not useful in predicting link quality Reported as not significant Not valid FRACTEL LACNs Mostly < 500 m Would like to avoid tall towers Rural, campus, residential To be determined

DGP, Roofnet, FRACTEL LACN Typical link distances Network architecture Environ ment Multipath effects SNR or RSSI External interference Link abstra ction Long- distance mesh networks (e.g. DGP) Up to few tens of kms High gain directional & sector antennas on tall towers or masts Rural setting studied in depth Effect not apparent Has strong correlation with link quality Affects links performance Valid Rooftop mesh networks (e.g. Roofnet) Mostly < 500 m Mostly omni- directional antennas on rooftops Dense urban setting studied in-depth Reported as a significant component Not useful in predicting link quality Reported as not significant Not valid FRACTEL LACNs Mostly < 500 m Would like to avoid tall towers Rural, campus, residential To be determined

DGP, Roofnet, FRACTEL LACN Typical link distances Network architecture Environ ment Multipath effects SNR or RSSI External interference Link abstra ction Long- distance mesh networks (e.g. DGP) Up to few tens of kms High gain directional & sector antennas on tall towers or masts Rural setting studied in depth Effect not apparent Has strong correlation with link quality Affects links performance Valid Rooftop mesh networks (e.g. Roofnet) Mostly < 500 m Mostly omni- directional antennas on rooftops Dense urban setting studied in-depth Reported as a significant component Not useful in predicting link quality Reported as not significant Not valid FRACTEL LACNs Mostly < 500 m Would like to avoid tall towers Rural, campus, residential To be determined

Experimental Methodology Two kinds of environment –Five locations on campus, One village location One transmitter, Multiple receiver positions Broadcast 6K 1400 byte pkts with 20ms gap. Hardware same as in DGP study, Roofnet study

Results: Err Rate vs RSSI For Interference free positions –If RSSI > Threshold Error rates are low and stable For Interference prone positions –Intermediate error rates Data Rate: 1Mbps

MIT Roofnet: Conclusions No correlation between SNR and Error Rate –Loss rate high at high SNR No correlation between lost packets and foreign packets observed Introducing delay spread causes high loss rates Multipath induced delay spread and not interference is the culprit

MIT Roofnet data: Fresh Analysis Data Rate: 1Mbps, Average RSSI > -80 dBm, 80% >Error Rate > 20%

MIT Roofnet data: Fresh Analysis Data Rate: 1Mbps, Average RSSI > -80 dBm, 80% >Error Rate > 20% Noise band as high as 16 dB, Ours ~ 2 dB

MIT Roofnet data: Fresh Analysis Data Rate: 1Mbps, Average RSSI > -80 dBm, 80% >Error Rate > 20% Roofnet Max Noise = -75 dBm, Ours = -94 dBm

MIT Roofnet data: Fresh Analysis Data Rate: 1Mbps, Average RSSI > -80 dBm, 80% >Error Rate > 20% What is the cause of increased noise level? Multipath does not cause high noise level Is it Interference ?

Understanding Interference 1.Does interference on affect noise levels ? 2.Can pkts. loss be related to no. of foreign pkts seen? 3.Can reported noise level be used to gauge the level of interference? 4.Can we estimate the link performance based on the average measured noise floor?

Understanding Interference 1.Does interference on affect noise levels ? 2.Does pkts. loss correlate with foreign pkts. ? 3.Use reported noise level to gauge level of interf.? 4.Estimate the link perf. based on the avg. measured noise floor?

Controlled Interference Expt Experimental Setup A and B Hidden Nodes to one another B’s power fixed at -75 dBm A’s power varied: -90, -85, -80, -75 dBm

Does Interference affect noise levels? Noise extends right up to -65 dBm RoofnetControlled Experiment Avg Received RSSI from A is -85 dBm and B is -75 dBm P1: Interference causes noise level to be high and variable

Can packet loss be related to foreign packets seen? As far as B’s packets are concerned: B’s loss = 18.3%; A’s loss = 99.2%  4 foreign pkts /sec P2 : Packet loss high even though number of observed foreign packets low Hidden node, receiver outside of interferer’s reception range

P3 : Packet loss can be low even though number of observed foreign packets is high IN RANGE NODE ‘R’ NODE ‘B’ NODE ‘A’ P2 P3 Support Roofnet’s observation but disprove conclusion Non-hidden-node case

Can the reported noise level be used to gauge the level of interference?

–Instantaneous noise levels show variability Large Noise Band

Can the reported noise level be used to gauge the level of interference? –Noise levels reported differ from known level P4: On this H/W, gauging level of interference is error prone Actual Interference Perceived Interference

Can link performance be estimated based on average noise floor? P5 : It is not possible to estimate the link quality based on reported noise floor From Roofnet Data

Operating near the RSSI threshold Village Location Variable Loss Rates

Operating near the RSSI threshold Variable Loss Rates Village Location No Interference

Operating near the RSSI threshold Variable Loss Rates Village Location No Interference What is the reason behind Intermediate error rates ?

Operating near the RSSI threshold Village Location Packet Error Rate (%) Average RSSI (dBm) RSSI Below Thresh.

Operating near the RSSI threshold Village Location Packet Error Rate (%) Average RSSI (dBm) RSSI Above Thresh.

Operating near the RSSI threshold Village Location Cannot distinguish between links with loss rates between 0-100% Cannot use routing metrics based on ETX or WCETT (going to be unstable) Small variation in RSSI  large variation in error rates

Design Implications Link abstraction: –Absence of external interference  can plan links Routing: –Metrics to distinguish between links with intermediate loss rates (e.g. ETX and WCETT) are likely unstable –Gauging interference can be error prone in interference-aware routing MAC: –CSMA/CA will lead to unpredictable loss rates, due to self-interference and hidden node cases

Conclusion Multipath and delay spread: –Not a likely factor in rural areas –May or may not be the main culprit in urban areas –Roofnet: extrapolates 900 MHz multipath measmts. Lesson: double-check measurements, analysis Open questions: –802.11g and a? –Future of unplanned deployments? For further information on FRACTEL: –

Backup slides

Operating near the RSSI threshold Village Location Cannot distinguish between links with loss rates between 0-100% Cannot use routing metrics based on ETX or WCETT In the absence of external interference Intermediate error rates due to operation in steep region Small variation in RSSI  large variation in error rates

Interference free locations: RSSI Stability Short-term stability (2-min expt): Mostly within 3-4 dB Hardware Quirk Person standing near antenna Band: Difference between the 95%-ile and 5%-ile values of RSSI

Long-term Stability Band variation depends on environment but within 4dB in most cases

Design Implication: Link Abstraction Absence of external interference  can plan links with predictable performance. Simplifies higher layer protocol design considerably RSSI threshold -79dBm RSSI variation 3-4dB Modified RSSI Threshold Value -75dBm

Design Implication: MAC CSMA/CA unsuitable in multihop mesh networks External interference can worsen CSMA/CA performance Roofnet data indicates considerable sources at interference range but not reception range –RTS/CTS will not help in this setting

FRACTEL: Ongoing Work Scope for TDMA-based mesh network –Lots of theory research, little in systems Systems issues in TDMA-based networks: –Interference mapping –Synchronization –Schedule dissemination, dynamic scheduling –Scaling For further information: –

Long Distance Networks (LDNs) "Long-Distance b Links: Performance Measurements and Experience'', Kameswari Chebrolu, Bhaskaran Raman, and Sayandeep Sen, MOBICOM 2006 Link abstraction holds Measurements on the DGP network, Kanpur, India

Local Access Networks (LACNs) Prior Studies: MIT Roofnet study –Outdoor WiFi mesh, Boston/Cambridge area –Most links have intermediate loss rates, between 0% and 100% –Multi-path (not external interference) is a major cause of losses –No Link Abstraction! –Work around: design of appropriate routing metrics

Experimental Methodology: Environment Two kinds of environment –Five locations on campus –One village location Link lengths: m One transmitter position Up to 6 receiver positions –Good – Avg. RSSI ≈ -70 dBm –Medium – Avg. RSSI ≈ -75 dBm –Bad – Avg. RSSI ≈ -80 dBm

FRACTEL Measmt. Study: IITK Source: Google Maps Dense buildings, academic area, student dormitories, campus housing, several trees

Understanding Interference 1.Does interference on affect noise levels ? 2.Can pkts. loss be related to no. of foreign pkts seen? 3.Can reported noise level be used to gauge the level of interference? 4.Can we estimate the link performance based on the average measured noise floor?

FRACTEL Measmt. Study: Amaur Source: Google Maps Dense buildings, 2-3 storey tall

Experimental Methodology Hardware –Laptops with Senao 2511CD Plus b PCMCIA cards –Antennas Sector Antenna – 17 dBi Omni Directional Antenna – 8 dBi Software –Linux – kernel –Modified HostAP driver – ver –Each experiment broadcasts byte pkts with 20ms gap at 4 different data rates Hardware same as in DGP study, Roofnet study

Controlled Interference Expt Experimental Setup A: 1400-byte packets, 2ms interval, 11Mbps B: 1300-byte packets, 2ms interval, 11Mbps B’s power fixed at -75 dBm A’s power varied: -90, -85, -80, -75 dBm

Can the reported noise level be used to gauge the level of interference? –Instantaneous noise levels show variability –Noise levels reported differ from known level P4: On this H/W, gauging level of interference is error prone