1. Discussion on Evaluation Metrics for 802.19.1
April 2009
doc.: IEEE /xxxxr0
Dec. 2009
Discussion on Evaluation Metrics for
Authors:
Name
Company
Address
Phone
Tuncer Baykas
NICT
Yokosuka Japan
M.Azizur Rahman --
Stanislav Filin
Zhou Lan
Junyi Wang
Yohannes Alemseged
Chin Sean Sum
Gabriel Villardi
Junyi Song
Chen Sun
Ha Nguyen Tran
Hiroshi Harada
Notice: This document has been prepared to assist IEEE It is offered as a basis for discussion and is not binding on the contributing individual(s) or organization(s). The material in this document is subject to change in form and content after further study. The contributor(s) reserve(s) the right to add, amend or withdraw material contained herein.
Tuncer Baykas, NICT
Tuncer Baykas, NICT

2. April 2009
doc.: IEEE /xxxxr0
Dec. 2009
Abstract
This presentation discusses evaluation metrics which can be used in standardization process and included in evaluation methodology document.
Tuncer Baykas, NICT
Tuncer Baykas, NICT

3. Dec. 2009
Possible Metrics
Complexity: What is the complexity per device and what is the setup complexity? Optimum throughput versus achieved throughput: Does the system find optimum throughput or stop in a local maxima? Total number of systems it can handle: Is it scalable? Stability: Under which circumstances, systems fluctuate? Information Requirement: How much information is necessary? Coexistence Overhead: How much radio resources is required by the coexistence system? (excluding the need of sharing the spectrum)(e.g. guard time, ) Protection from non systems: How much protection against non systems? Fairness: How fair is the system?
Tuncer Baykas, NICT

4. Possible Metrics from [1]
Dec. 2009
Possible Metrics from [1]
Medium occupancy: For each network it is the total time that that network is transmitting divided by the total simulation time. Successful medium occupancy: For each network it is the total time that that network is transmitting, and those transmissions are received properly at the intended destination, divided by the total simulation time Hidden node probability: For each network it is the probability of experiencing hidden node interference. Exposed node probability: For each network it is the probability of experiencing exposed node therefore missing a transmission chance.
Tuncer Baykas, NICT

5. Dec. 2009
Possible Metrics
The uplink and downlink packet error rates (PER): Median and %90 percent PER should be provided. The uplink throughput and the downlink throughput: For each of the link in the network. For throughput it is useful to focus not only on the median value but also the 10% value, since that is the throughput value at which 90% of the all stations exceed that throughput. Therefore, only 10% of the stations have a throughput less than the 10% throughput value. The uplink latency and downlink latency: for each of the link in the network (from top-of-the-MAC to top-of-the-MAC) For latency it is useful to focus not only on the median value but also the 90% value, since this is the latency value at which 90% of the stations have a latency that is less than this value. Therefore, only 10% of the stations have a latency more than the 90% latency value.
Tuncer Baykas, NICT

6. Conclusions We presented possible evalation metrics for 802.19.1.
Dec. 2009
Conclusions
We presented possible evalation metrics for
Next step should creating scenarios to evaluate different proposals.
Tuncer Baykas, NICT

7. Dec. 2009
References
[1] S. Shellhammer Coexistence metrics in the 3650 MHz frequency band IEEE P /0020r3 (A set of coexistence metrics are proposed for evaluation of coexistence of y and h in the 3650 MHz frequency band)
Tuncer Baykas, NICT