1. TAPAS: A Research Paradigm for the Modeling, Prediction and Analysis of Non-stationary Network Behavior Almudena Konrad PhD Candidate at UC Berkeley almudena@cs.berkeley.edu Sahara Retreat, June, 2002 Advisor: Anthony Joseph

2. 2 Outline Introduction The Problem Existing Work Our Solution –Modeling of Network Measurements Through Data Preconditioning Modeling Methodology Applied to Wireless Networks Work in Progress Summary

3. 3 Introduction This talk will demonstrate that the modeling of network characteristics through data preconditioning, will provide accurate network models and optimal network protocol design compared to traditional approaches.

4. 4 The Problem Modeling of Network Characteristics: Application of Traditional Models Network characteristics experience –Complex patterns and dynamic time varying statistics Traditional models require stationary statistics –Non-time varying statistics Current approach generates poor model approximations

5. 5 Example of the Modeling Problem Modeling Wireless Error Design and evaluation of wireless protocols and applications –On “live” networks (eg:GPRS) –Simulators: accurate models for the error and loss process Traditional approach to channel modeling –Application of traditional DTMC to collected error traces Markov models require stationary statistics –Wireless channels experience time varying effects Traditional channel models –Bernoulli: Independent model –Gilbert: two-state DTMC –Higher order Markov: N states

6. 6 Existing Work on Modeling of Network Measurements Modeling IP Losses Bolot et al. (INRIA,1999) –Model loss process of audio packets to determine error control schemes –Use Gilbert model Yajnik et al. (University of Massachusetts, 1999) –Use third order Markov chain to model packet loss in multicast networks –Remove part of the data that experience non stationary error behavior

7. 7 (Continuation) Existing Work Modeling Wireless Errors Nguyen et al. (UC Berkeley, 1996) –WLAN error traces –Improve two-state Markov model Zorzi et al. (UC San Diego, 1998) –Use Gilbert model –Claim that higher order Markov models are not necessary –Results are drawn by applying models to artificial traces Willig et al. (Tech U of Berlin, 2001) –Special class of Markov models –Industrial WLAN traces –High complexity, 24 states

8. 8 Our Solution: Propose Modeling Methodology Modeling through data preconditioning Collect network characteristic trace Identify data patterns (stationary behavior) Precondition the data to fit traditional models –Associate a state with each pattern –Calculate probability distribution for each state –Determine transition probabilities among states Collected Trace Sub-trace 1 Sub-trace 2 Sub-trace 3

9. 9 Outline Introduction The Problem Existing Work Our Solution –Modeling Through Data Preconditioning Modeling Methodology Applied to Wireless Networks Work in Progress Summary

10. 10 Modeling Methodology Applied to Wireless Collect and analyze error & delay traces at the wireless layer …0000000 10101111 00000000000… Develop a modeling algorithm (MTA: A Markov-based Trace Analysis Algorithm) –Examine non-stationary behavior of a trace by doing pattern recognition –Precondition trace Divide non-stationary trace into stationary subtraces Characterize the transition between subtraces Apply Markov models to stationary subtraces Apply MTA to collected wireless traces –GSM, WLAN, GPRS, Sensor Networks

11. 11 The MTA Algorithm Divide wireless trace into two stationary sub-traces –Lossy and error-free states –Form lossy and error-free sub-traces –Show lossy sub-trace is stationary –Model lossy sub-trace as DTMC –Calculate best fitting distribution for the state length (EXP fitting) C …10001110011100….0 0000…0000 11001100…00 00000..000... Lossy Error-free C Trace: Lossy sub-trace: Error-free sub-trace:...10001110011100….0 11001100…00... … 0000…0000 00000..000... States:

12. 12 Applications of the Model Synthetic trace generation –Use to evaluate models by exploring distribution –Artificial traces can be used in simulators Develop feedback algorithm –Uses the MTA model statistics –Identify change of state => notifies the application –Allows application adpatation

13. 13 Outline Introduction The Problem Existing Work Our Solution –Modeling Through Data Preconditioning Modeling Methodology Applied to Wireless Networks –Applying the MTA Algorithm to GSM & WLAN error traces –Applying the MTA Algorithm to GSM delay traces –Evaluation Work in Progress Summary

14. 14 MTA Models for Error Traces GSM error trace: 576,021 frames with FER=0.058 WLAN error trace: 288,804 frames with FER=0.063 –(802.11b collected by Andreas Willig, TU of Berlin) MTA: –Lossy and error-free states –Form lossy sub-trace -> DTMC –GSM: Lossy state length distribution ~Exp(0.037) Error-free state length distribution ~Exp(0.04) –WLAN: Lossy state length distribution ~Exp(0.076) Error-free state length distribution ~Exp(0.059)

15. 15 Intuition for Stationary Behavior Error-free burst –mean: 115 –st deviation: 551 Error burst –mean: 6 –st deviation: 14 Long error-free bursts destroy error cluster statistics Non-stationary behavior Burst length analysis of “GSM Error Trace” Error-free bursts much longer than error bursts

16. 16 Apply the “Runs Test” to GSM trace Runs Test: Bendat and Piersol in 1986 –Compute median run value of the trace (run=error burst) –Divide trace into equal size segments –Plot histogram: runs not equal to median value in each segment –Too few or too many runs is a sign of non-stationarity Test for Stationarity : The “Runs Test” For stationarity: ~ 90% of the distribution must lie between boundary points (0.05% and 90%)

17. 17 Stationarity of “Lossy Sub-trace” The Runs Test Burst Lengths in “Lossy Sub-trace” “Lossy Sub-trace” is stationary

18. 18 Modeling “lossy sub-trace” as DTMC Order of the Markov chain Calculate transition probabilities accuracy

19. 19 Outline Introduction The Problem Existing Work Our Solution –Modeling Through Data Preconditioning Modeling Methodology Applied to Wireless Networks –Applying the MTA Algorithm to GSM & WLAN error traces –Applying the MTA Algorithm to GSM delay traces –Evaluation Work in Progress Summary

20. 20 MTA Model for GSM Delay Trace Multimedia applications tolerate a max delay –Max delay is application dependent –Packets arriving with delay greater than max value are discarted Model losses due to packet delays GSM delay trace: 2,580 UDP packets of video stream –UDP connection over reliable GSM link –Choose delay threshold of 2 sec Apply MTA to GSM delay trace

21. 21 Model Evaluation Metrics Traditional approach to model evaluation –Generate synthetic traces –Measure FER and throughput –Problem: Not correlated to error distribution Our approach: –Generate synthetic traces Calculate error burst distribution Compare traces’ error distribution to “original traces” –Protocol design under various models Calculate optimal frame size Calculate throughput for various frame sizes Optimal frame sizes are highly correlated with error distribution

22. 22 Evaluation: GSM Error Burst Distributions mean st dev max GSM: 6 14 126 MTA: 7 8 82 3rd M: 2.3 1.4 8 Gilbert: 1.8 0.4 4 GSM, MTA experience similar error burst characteristics Gilbert and 3 rd order don’t reproduce large error burst

23. 23 mean st dev max WLAN: 2.8 3.1 42 MTA: 3.2 2.8 33 3rd M: 2 1.2 8 Gilbert: 1.6 0.5 6 Evaluation: WLAN Error Burst Distributions GSM, MTA experience similar error burst characteristics Gilbert and 3 rd order don’t reproduce large error burst

24. 24 mean st dev max GSM: 20.5 12.7 38 MTA: 24 9.7 38 3rd M: 3.5 1.4 4 Gilbert: 1.7 0.7 2 Evaluation: GSM Trace Delay Burst Distributions Delay burst length => sequences of packets with delay greater than threshold (2 sec) MTA experience best burst characteristics

25. 25 Model Evaluation : Protocol Design Generate error traces from various models Calculate optimal RLP frame size (size that yields max throughput) Real GSM distribution => optimal size is 210 Bytes Traditional models’ distributions => wrong frame size Standard Error from GSM distribution EED = 48 Gilbert = 22 3rd Markov = 10 MTA = 8

26. 26 Outline Introduction The Problem Existing Work Our Solution –Modeling Through Data Preconditioning Modeling Methodology Applied to Wireless Networks Work in Progress –WSim Summary

27. 27 WSim: Wireless Simulator Implements error channel model based on data preconditioning models –Explores impact of high FER on applications –Tests various transport protocol configuration for different FER Provides feedback algorithm –UDP connection sends information on channel conditions from base station to the application Provides non reliable transport and radio link

28. 28 WSim Modules Sender Socket Interface RTP Packet UDP/IP/PPP Radio Link: Fragmentation/Reassembly... 30B Radio Frames Radio Link and Base Station 554B PPP Frames Fragmentation/Reassembly 554B PPP Frames... Feedback Message: Fragmented PPP Frame UDP/IP/PPP Receiver Socket Interface RTP Packet............ MTA Model Statistics Feedback Algorithm RL Module...... Sender Module Receiver Module

29. 29 Summary New modeling research methodology –Preconditioning of data to fit traditional models Apply modeling methodology to error and delay processes of wireless links –Develop the MTA algorithm to model error and delay processes in wireless More accurate models => more accurate emulations => better protocol design

30. 30 Work in Progress Provide a less threshold-oriented way to decide state changes –Wavelet analysis to do pattern recognition Define domain for which MTA works –Define FER, error density and distribution –When do current models break down? Apply MTA model to –GPRS networks traces (Ericsson Lab) –Sensor networks traces Improve predictive feedback algorithm –Use time series forecasting to predict future behavior Implement Wsim in Java –Evaluate MTA models –Explore ways to send feedback during high error periods

31. 31 Thank you :-) Tapas Web Page: http://www.cs.berkeley.edu/~almudena/tapas