1. Tracking and Predicting End-to-End Quality in Wireless Community Networks (WCN) 4 th Int. Workshop on Community Networks and Bottom-up-Broadband, CNBuB 2015 August 26 th, 2015. Rome, Italy Pere Millán 1, C. Molina 1, E. Dimogerontakis 2, L. Navarro 2, R. Meseguer 2, B. Braem 3, C. Blondia 3 1 Universitat Rovira i Virgili, Tarragona, Spain 2 Universitat Politècnica de Catalunya, Barcelona, Spain 3 University of Antwerp – iMinds – MOSAIC, Antwerpen, België

2. Motivation [Link/End-to-End Quality] Prediction in [Wireless] Networks End-to-End Quality Prediction in Wireless Community Networks Experimental Methodology & Results Conclusions & Future Work OLSR Outline 2

3. Motivation 3

4. Community networks create measurable social impact providing the right and opportunity of communication 4

5. Community networks: – Large, heterogeneous, dynamic and decentralized structures Some challenges: – What is the effect of the asymmetrical features and unreliability of wireless communications on network performance and routing protocols? – Link quality tracking is a key method to apply when routing packets through an unreliable network. – Routing algorithms should avoid weak links whenever possible and as soon as possible. Motivation 5

6. Link quality (LQ) estimation/prediction approach increases the improvements in routing performance achieved through link quality tracking. – RT metrics do not provide enough information to detect degradation/activation of a link at the right moment. – Prediction techniques are needed to foresee link quality changes in advance and take the appropriate measures. Motivation 6

7. End-to-End Quality (EtEQ) or Path Quality extends the Link Quality (LQ) concept to the full communication path (sender-receiver) and is computed based on the quality (ETX) of the individual links that conform the communication path. Motivation 7

8. Main contributions: A detailed analysis of path properties and path ETX behavior in wireless community networks (WCN):  EtEQ prediction is possible and meaningful. Use of time series analysis to estimate EtEQ in the routing layer for real-world WCN. Clear evidence that EtEQ values computed through time series algorithms can make accurate predictions in WCN. A detailed analysis of prediction accuracy for the next step considering time of day and some steps ahead in future. In this work we present an analysis of End-to-End Quality tracking and prediction and differences with our previous LQ analysis 8

9. Prediction in WCN 9

10. FunkFeuer uses OLSR Routing Protocol: – Prediction of path changes can improve local node routing decisions, since it can provide the node with an estimation about the future local and remote events. – OLSR uses ETX to choose the next hop. ETX Metric: – Link: number of expected transmissions to send a packet over the link. – Path: sum of ETX values of links that form the path (≥ path Hops). Path ETX prediction: – Will allow more efficient routing decisions in an unstable environment. Background 10

11. Goals of Network Prediction: Routing Traffic Reduction: OLSRp, Kinetic Multipoint Relaying (KMPR). Energy Efficient Routing: LPR, MDR, E-DSR routing protocol. Link Quality prediction. Link Quality Prediction in WCN: LQ tracking: select higher quality links  max. delivery rate & min. traffic congest. LQ prediction: determine beforehand which links are more likely to change behavior. Idea: The routing layer can make better decisions at the appropriate moment. Link Quality Estimators (LQE) metrics: Measure quality of links between nodes based on physical or logical metrics. Physical metrics focus on the received signal quality: LQI, SNR, RSSI. Logical metrics focus on % of lost packets: RNP, ETX, PSR. To select the more suitable neighbor nodes when making routing decisions. Network Prediction 11

12. Compute Path Quality as aggregation of Link values: – MARA [28], EER [29], ETOP [30], EED/WEED [31]. Other relevant works: – MetricMap [32]: uses a learning-enabled method for LQ assessment. Also uses time series analysis to improve the routing protocol. – Maccari & Cigno [33]: quality of routes & techniques to select MPR nodes. – Cunha et al. [34]: detects path changes (NN4) and then remaps (DTRACK). – Millan et al. [13]: behaviour of LQ prediction in WCN, using Time-Series, several learning algorithms, to accurately predict future values. End-to-End Quality 12

13. Funkfeuer WCN (Austria): – 2.000+ links, OLSR-NG routing protocol. Open data set (Confine Project): – OLSR info, 404 nodes, 7 days, av. degree: 3.5, diameter: 18. – 1.032 links with variations in LQ (higher prediction accuracy using all links). 4 Prediction Algorithms: – Support Vector Machines (SVM), k-Nearest Neighbors (kNN), Regression Trees (RT), Rule-Based Regression (RBR). Time Series Analysis & Forecasting: – Training and test sets validation approach. – Weka: machine learning/data mining approach to model time series, encodes time dependency via additional input fields (“lagged” variables). Metrics and Plots: Mean Absolute Error (MAE). MAE = sum(abs(predicted - actual)) / N Boxplots: classic representations of a statistical distribution of values. Experimental Methodology 13

14. Results 14

15. Comparison of TS learning algorithms Time series analysis can be used to predict future End-to-End quality values? 4 classification algorithms : Support Vector Machines (SVM) k-Nearest Neighbors (KNN) Regression Trees (RT) Rule-Based Regression (RBR) 4 classification algorithms : Support Vector Machines (SVM) k-Nearest Neighbors (KNN) Regression Trees (RT) Rule-Based Regression (RBR) 15 Data sets: Training: 2016 instances (7 days) Test: 288 instances (1 day) Lag window: last 12 instances Data sets: Training: 2016 instances (7 days) Test: 288 instances (1 day) Lag window: last 12 instances BEST WORST High percentage of success

16. Learning algorithms: error variability 16 3 of 4 algorithms (RT, RBR, SVM) achieved a similar accuracy for most of the links Some outliers have high errors … that increase the average values t-test result: RBR is a good candidate to make predictions

17. EtEQ Prediction with RBR algorithm How can we reach a satisfactory level of prediction? 17 Same Data sets: Training: 2016 instances (7 days) Test: 288 instances (1 day) Lag window: last 12 instances Same Data sets: Training: 2016 instances (7 days) Test: 288 instances (1 day) Lag window: last 12 instances Some hop paths have high dispersion … but we successfully predict a big percentage of fluctuations

18. EtEQ Prediction with RBR: accuracy Real and predicted values are very close 18 Deviation remains below 0.5 throughout the whole prediction Why ETX error has an increasing trend? (8:00 – 8:30 am)

19. EtEQ Prediction accuracy (day/night) Future work: 2 different predictors (day/night)? 19 Less deviations/errors Data sets: Training: 2016 instances (7 days) Test: 144 instances (1/2 day) Lag window: last 12 instances Data sets: Training: 2016 instances (7 days) Test: 144 instances (1/2 day) Lag window: last 12 instances Day (12 am – 12 pm) Night (12 pm – 12 am) More deviations/errors

20. Prediction of some steps ahead Time series analysis and prediction can be used to predict the value of EtEQ some time steps ahead into the future? 20 Good results for the majority of steps ahead Average MAE grows very slowly We could predict successfully the EtEQ several steps ahead in time.

21. 21 Conclusions Future Work

22. Time series analysis is a promising approach to accurately predict LQs in Community Networks – Routing protocol performance can be improved by providing information to make appropriate and timely decisions to maximize the delivery rate and minimize traffic congestion. All algorithms achieved high percentages of success with average MAE per link between 2.4% and 5% when predicting the next value of EtEQ, being the Rule-Based Regression the best one. – RBR prediction shows an average absolute error less than 1. The error variability is similar for 3 of the algorithms: RT, RBR, SVM. – kNN performs worse due to outliers with larger errors. OLSR EtEQ Prediction: Conclusions 22

23. OLSR Future Work To extend this analysis to other community networks to evaluate if the observed behavior can be generalized. To identify which paths contribute most to the errors in the EtEQ prediction and to understand what factors make it more difficult to predict them. To study the impact of errors in routing decisions. To study a solution with 2 different predictors: Day/Night. To improve the prediction process by discarding those paths whose relation between EtEQ and prediction accuracy is above a certain threshold. 23

24. Thanks for Your Attention… Questions? 4th Int. Workshop on Community Networks and Bottom-up-Broadband, (CNBuB 2015)August 26th, 2015. Rome, Italy