Accuracy and Efficiency in Simulating VANETs Introduction and Motivation VANETs Simulation The CARLINK-UMA Scenario Results Conclusions and Future Work Accuracy and Efficiency in Simulating VANETs Enrique Alba, Sebastián Luna, and Jamal Toutouh Universidad de Málaga

Outline Introduction and Motivation VANETs Simulation The CARLINK-UMA Scenario Results Conclusions and Future Work Outline 1 Introduction and Motivation VANETs Simulation The CARLINK-UMA Scenario Results Conclusions and Future Work 2 3 In this presentation we want to through light on the problem of simulating VANETs. For this reason, I will introduce the problem of the VANET simulation. I will show the results obtained over the CARLINK-UMA scenario. Finally, I will present the conclusions and the future work come from this work. In the First Section I will introduce to the VANET simulation problem and the motivation of this presentation. In the Second Section I will summarize the different alternatives for VANETs simulation, focusing in the simulators used during the CARLINK project In the Third Section I will present the defined scenario, where both, simulations and real tests were carried out. In the Fourth Section I will show the achieved results, comparing them. Finally, in the Fifth Section I will present the conclusions and the future work come from this work 4 5 5 /13

Introduction and Motivation VANETs Simulation The CARLINK-UMA Scenario Results Conclusions and Future Work Introduction and Motivation The performance evaluation of the different VANET protocols and applications is done by using both outdoor experiments and simulations The evaluation through outdoors experiments has several drawbacks: neither easy nor cheap difficult to analyze in an inherently distributed and complex environment reproduction of all kinds of situations where they must act is not possible The simulation avoid the previous drawbacks. Nevertheless it presents the following one: the fidelity of the simulated results versus the real ones Nowadays are appearing new protocols and applications for VANETs, the test of these new protocols and applications CAN BE DONE by BOTH outdoor experiments and simulations. The VANETs evaluation through outdoor experiments is extremely difficult since: It is NEITHER easy nor cheap to have a high number of vehicles and a real scenario for VANET designers It is difficult to analyze the data generated during the real tests It is not possible to reproduce all kinds of situations where the VANETs must act The use of simulators avoids the previous drawbacks. However, it presents a new one, the fidelity of the simulated results versus the real ones. In the following I will present a comparison between the simulations and real tests. The goal is to better know the actual VANET used in CARLINK and the data/delay rates affecting the final applications. We present here a comparison between simulations and real tests The goal is to better know the actual VANET simulator used in CARLINK and the data/delay rates affecting the final applications /13

Introduction and Motivation VANETs Simulation The CARLINK-UMA Scenario Results Conclusions and Future Work VANET Simulation Three kinds of approaches in VANET simulation have been initially considered: Using an specially-designed VANET simulator Alternatives: TraNS, MOVE, CARISMA, and STRAW VanetMobiSim/Ns-2 Integrating a vehicular mobility model generator into an existing network (MANETs) simulator Mobility model generator: SUDO and VanetMobiSim Network simulators: Ns-2, GlomoSim, QualNet, and OpNet Nowadays, we identify three different approaches trying to through light on the complex problem of simulating VANETs in a trustworthy manner. The first one is using a specially-designed VANET simulators. As: TraNS, MOVE, CARISMA, and STRAW The second approach integrates a vehicular mobility model generator (as SUDO or VMS) into a MANET simulator as (Ns-2, Glomosim, Qualnet or Opnet) Finally, we found JANE, a MANET application programming framework which allows the developer to test the applications via simulations Finally, we considered the use of VMS combined with Ns-2 and JANE for the CARLINK simulations MANET applications programming framework which allows the developer to test the applications via simulations: Alternative: JANE JANE /13

Vehicular Data Transfer Protocol (VDTP) has been added Introduction and Motivation VANETs Simulation The CARLINK-UMA Scenario Results Conclusions and Future Work VANET Simulation. VanetMobiSim/Ns-2 VanetMobiSim/Ns-2 [ mobility model generator + network simulator ] VanetMobiSim: Ns-2: - Freely available Different kind of outputs Generates realistic models Macro-mobility features Micro-mobility features + Freely available Easily adding features Widely used Numerical output VMS/Ns-2 is a combination of VMS/Ns-2 which is a vehicular mobility model generator and Ns-2 which is a well know network simulator. We chose VanetMoviSim mainly because it s Freely available, the generated traces may have different outputs and it generates realistic models defined by different BOTH macro-mobility and micro-mobility features. Ns-2 have been chosen because it s freely available, too. It offers the possibility of adding new features (we add the VDTP definition). it is widely USED and VERIFIED, and it offers different numerical and statistical outputs. Vehicular Data Transfer Protocol (VDTP) has been added TARGET: making precise simulations of the real systems /13

VANET Simulation. JANE JANE: Java Ad hoc Network Environment Introduction and Motivation VANETs Simulation The CARLINK-UMA Scenario Results Conclusions and Future Work VANET Simulation. JANE JANE: Java Ad hoc Network Environment Open source Java-based middleware platform which is intended to assist ad-hoc network researchers in application and protocol design Three different execution modes: Simulation Mode: the complete environment is simulated Hybrid Mode: the devices and the ad-hoc network are simulated, but real users can interact with the simulation Platform Mode: the whole setting is real Drawback: It is not specialized for VANETs not provide realistic mobility models for the simulation of vehicular networks TARGET: programming high level applications Open source Java-based middleware platform which assists ad-hoc network researchers in application and protocol design It offers 3 different execution modes: 1.Simulation mode: The application is completely simulated 2.Hybrid mode: The user can interact with the simulator by using different defined interfaces 3.Platform mode: The application is executed over the real hardware platform The main drawback is that it is specialized for MANETs, so it not provides the realistic vehicular mobility models The main TARGET of JANE is to programming and test high level applications /13

The CARLINK-UMA scenario Introduction and Motivation VANETs Simulation The CARLINK-UMA Scenario Results Conclusions and Future Work The CARLINK-UMA scenario The scenarios are defined by their specific mobility models: File transfers between two cars by using ad-hoc operation mode of the IEEE 802.11b MAC Layer Standard PROXiM ORINOCO PCMCIA transceiver Range extender antenna gain: 7 dBi The CARLINK-UMA scenario defined to carry out the tests is defined as follows: The Scenario A, the vehicles are located separated by 100m over the same lane and they move in the same sense with a velocity of 30 Km/h. The Scenario B, the vehicles are in different lanes, separated by 500m, and they move in the opposite sense with a velocity of 30 Km/h. The cars use the ad-hoc operation mode of the WiFi standard protocol for the file transfers. Each car is equipped by a PROXIM ORINOCCO transceiver and a range extender antenna. /13

Test Xi The CARLINK-UMA scenario Introduction and Motivation VANETs Simulation The CARLINK-UMA Scenario Results Conclusions and Future Work The CARLINK-UMA scenario Each experiment is composed of different tests The tests consist in repeatedly transferring a file in specific scenarios: File type 1: traffic information documents (1 MB) File type 2: multimedia files (10 MB) The file transfers are carrying out by using the Vehicular Data Transfer Protocol (VDTP): Chunk size: 25Kbytes Retransmission time: 8 seconds Max number of attempts per packet: 8 The test are named as follows: Test A1 Test A2 Test B1 Test B2 It denotes the file type: File type 1 = 1 MB File type 2 = 10 MB The experiments are composed by different test. The test consist in continuously transferring a file in the previously presented scenario. We have used 2 types of file The file type 1 with 1 MB size represents traffic information documents. The file type 2 with 10 MB size represents multimedia files The file transfers have been carried out by using the VDTP protocol, setting it up as follows: - 25KB of Chunk size. - 8 seconds of retransmission time - and the maximum number of attempts of 8. The test are named as Test A1, Test A2, Test B1, and Test B2. Where the Uppercase letter describes the scenario and the number denotes the file type. Test Xi It describes the scenario: Scenario A Scenario B /13

Results Test A1 (same sense, 1 MB files) Average values: Introduction and Motivation VANETs Simulation The CARLINK-UMA Scenario Results Conclusions and Future Work Results Test A1 (same sense, 1 MB files) Average values: Now I will resent the results achieved. In the Test A1 the average transmission time is around 1 dot 6 seconds for Real Experiments and VanetMobiSim/Ns-2, and 1 dot 8 for JANE We can see that the Real test and VanetMobiSim/Ns-2 transmission time are close Accurate results Real Experiments JANE VanetMobiSim/Ns-2 Transmission Time 1.6 secs 1.8 secs Data Rate 626.9 KB/s 563.8 KB/s 609.7 KB/s /13

Results Test A2 (same sense, 10 MB files) Average values: Introduction and Motivation VANETs Simulation The CARLINK-UMA Scenario Results Conclusions and Future Work Results Test A2 (same sense, 10 MB files) Average values: In the test A2, the transmission time is around 1.7 seconds, and in this case, the transmission time achieved during the real tests and the simulations have an absolute difference of 0.6 seconds. Difference 0.6 secs Real Experiments JANE VanetMobiSim/Ns-2 Transmission Time 17.3 secs 17.9 secs 16.7 secs Data Rate 585.1 KB/s 564.4 KB/s 611.1 KB/s Difference 0.6 secs /13

Results Test B1 (opposite sense, 1 MB files) Average values: Introduction and Motivation VANETs Simulation The CARLINK-UMA Scenario Results Conclusions and Future Work Results Test B1 (opposite sense, 1 MB files) Average values: JANE: Only 3 completed transfers The results achieved during the Tests of Scenario B present the highest differences between the results of JANE simulator and the VMS/Ns-2 and the real tests results. In the Test B1 JANE finished just 3 transfers and and VMS/Ns-2 and real tests finished al the transfers with an average transmission time of 2.7 seconds. Accurate results Real Experiments JANE VanetMobiSim/Ns-2 Transmission Time 2.7 secs 1.8 secs 2.6 secs Data Rate 371.4 KB/s 563.7 KB/s 391.4 KB/s /13

Results Test B2 (opposite sense, 10 MB files) Average values: Introduction and Motivation VANETs Simulation The CARLINK-UMA Scenario Results Conclusions and Future Work Results JANE: No transfers completed Test B2 (opposite sense, 10 MB files) Average values: In the Test B2, All file transfers were successfully finished taking an average time of 20 seconds. However ,in the JANE simulations no one file transfer were successful. Accurate results Real Experiments JANE VanetMobiSim/Ns-2 Transmission Time 20.1 secs N/A 19.9 secs Data Rate 502.1 KB/s 513.3 KB/s /13

Results Global average results Introduction and Motivation VANETs Simulation The CARLINK-UMA Scenario Results Conclusions and Future Work s Results Global average results Mean data rate differences (absolute value in KB/s) between real and simulation results Analyzing the global average transmission times, we can see that the results of the real tests and the VMS/Ns-2 simulations are closer than the results achieved with JANE. Test A1 Test A2 Test B1 Test B2 JANE 63.1 KB/s 20.6 KB/s 192.3 KB/s N/A VanetMobiSim/Ns-2 17.2 KB/s 25.8 KB/s 20.0 KB/s 11.3 KB/s /13

Introduction and Motivation VANETs Simulation The CARLINK-UMA Scenario Results Conclusions and Future Work Conclusions The real world is quite difficult to simulate in a trustworthy manner. Our simulations can be changed by including: obstacles signal reflections etc. At the CARLINK-UMA Scenario an important quantity of data can be transferred with a transmission data rate always higher than 300 KB/s VanetMobiSim/Ns-2 is the most realistic simulator, if the goal is to model the VANET in a computer JANE is useful for programming/testing high-level applications As a future work: perform simulations of other scenarios in order to predict the performance of the communications at urban and highway environments (almost finished) combine the simulator (VMS/Ns-2) with optimization techniques in order to offer an optimal configuration of VDTP (in progress) The analysis of the results achieved during this work are that: The real world is quite difficult to simulate in a trustworthy manner. For this reason, our simulations can be changed by including obstacles, signals reflections, etc. In the VANET defined in the Carlink-UMA scenario, the transfers of 1 MB files take around 2 seconds and the transfers of 10 MB files take around 18 seconds. VMS/Ns-2 is the more realistic simulator, since its results are closer to the real world ones. JANE is useful for programming and testing high level VANET applications The defined future work after this study is: Performing new simulations in order to predict the performance of the communication at urban and highway scenarios. This work is almost finished. And Combining the VanetMobiSim/Ns-2 simulator with optimization techniques in order to offer an optimal configuration of the VDTP. Nowadays, this work is in progress.. /13

Thank you for your attention. Introduction and Motivation VANETs Simulation The CARLINK-UMA Scenario Results Conclusions and Future Work Thank you for your attention. Any questions? The analysis of the results achieved during this work are that: The real world is quite difficult to simulate in a trustworthy manner. For this reason, our simulations can be changed by including obstacles, signals reflections, etc. In the VANET defined in the Carlink-UMA scenario, the transfers of 1 MB files take around 2 seconds and the transfers of 10 MB files take around 18 seconds. VMS/Ns-2 is the more realistic simulator, since its results are closer to the real world ones. JANE is useful for programming and testing high level VANET applications The defined future work after this study is: Performing new simulations in order to predict the performance of the communication at urban and highway scenarios. This work is almost finished. And Combining the VanetMobiSim/Ns-2 simulator with optimization techniques in order to offer an optimal configuration of the VDTP. Nowadays, this work is in progress..