1 1 Detecting Network Neutrality Violations with Causal Inference Mukarram Bin Tariq, Murtaza Motiwala Nick Feamster, Mostafa Ammar {mtariq, murtaza, feamster,

2 2 Network Neutrality ISPs remain neutral in forwarding traffic irrespective of –Content. voice, video, data –Application. p2p, VoIP, VoD –Participants. Service providers, Google, Hulu, Youtube Not-neutral = Discrimination Focus of this paper –Detecting and Quantifying Discrimination

3 3 Net Neutrality Problem

4 4 Discrimination is Real Glasnost Project:

5 5 Discrimination can take many forms Blocking ports Disrupting connections, e.g., using TCP RST Throttling and prioritizing based on destination or service –Target domains, applications, or content Discriminatory peering –Resist peering with certain content providers

6 6 Problem Statement Identify whether a degradation in a service performance is caused by discrimination by an ISP –Quantify the causal effect Existing techniques detect specific ISP methods –TCP RST (Glasnost) –ToS-bit based de-prioritization (NVLens/NetPolice) –Shaping (ShaperProbe) Goal: Establish a causal relationship in the general case, without assuming anything about the ISPs methods

7 7 Causality: An Analogy from Health Epidemiology: study causal relationships between risk factors and health outcome NANO: infer causal relationship between ISP and service performance

8 8 Does Aspirin Make You Healthy? Sample of patients Positive correlation in health and treatment Can we say that Aspirin causes better health? Confounding Variables: correlate with both cause and outcome variables and confuse the causal inference Aspirin No Aspirin Healthy 40%15% Not Healthy 10%35% Sleep Diet Other Drugs Age Aspirin Health ?

9 9 Does an ISP Cause Service Degradation? Sample of client performances with correlation in ISP and service performance Can we say that Comcast is discriminating? Many confounding variables can confuse the inference Comcast No Comcas t Avg BitTorrent Download Time 5 sec2 sec Client Setup TimeofDay ContentLocation Comcast BT Download Time ?

10 Causation vs. Association (1) Baseline Performance Performance with the ISP Causal Effect = E(Real Download time using Comcast) of Comcast E(Real Download time not using Comcast) G 1, G 0 : Ground-truth values for performance (aka. Counter-factual values) Problem: Generally, we do not observe both ground truth values for the same clients. Consequently, in situ data sets are not sufficient to directly estimate causal effect.

11 Causation vs. Association (2) Observed Baseline Performance Observed Performance with the ISP Association = E(Download time using Comcast) with Comcast E(Download time not using Comcast) We can observe association in an in situ data set. In general,. How to estimate causal effect ( ) ?

12 Estimating the Causal Effect Two common approaches –Random Treatment –Adjusting for Confounding Variables

13 Random Treatment Treat subjects with Aspirin randomly, irrespective of their health Observe new outcome and measure association Association converges to causal effect if confounding variables do not change –Diet, other drugs, etc. should not change Aspirin Treated Not Aspirin Treated = = 0.55 H H H H SS S S HHH H H SSS S

14 Random Treatment (How to apply to the ISP Case?) Ask clients to change their ISP to an arbitrary one –Difficult to achieve on the Internet –Changing ISP is cumbersome for the users –Changing ISP may change other confounding variables, e.g., the ISP network changes.

15 Adjusting for Confounding Variables List confounders e.g., gender ={, } Stratify along confounder variable values and measure association Now association implies causation because there is no other explanation Strata HH H HH H HH H SS S H S S SS HH H H S S S S S HH HHH SS SS Treated Baseline Effect

16 Adjusting for Confounding (How to Apply to the ISP Case?) Challenges –What is the baseline? –What are the confounding variables? –How to collect the data? –Can we infer more than the effect? e.g., the discrimination criteria

17 What is the Baseline? Baseline: service performance when ISP is NOT used –We need to use some ISP for comparison e.g., compare ATT vs. Comcast –What if the one we use is not neutral Solutions: –Use average performance over all other ISPs –Use a lab model –Use service providers mode

18 Determine Confounding Variables (Using Domain Knowledge) Client Side –Client setup (Network Setup, ISP contract) –Application (Browser, BT Client, VoIP client) –Resources (Memory, CPU, Utilization) ISP Related –Not all ISPs are equal; e.g., location. Temporal –Diurnal cycles, transient failures

19 Collecting Data NANO uses a passive client side application –pcap library to infer application performance Throughput, Jitter, TCP loss, TCP Reset packets –Applications associated with traffic –Resource monitoring CPU, Memory, Network utilization Summarized performance stats sent to NANO server for analysis

20 Inferring the Criteria Label data in two classes: –discriminated (-) –non-discriminated (+) Train a decision tree for classification –Rules provide hints about the criteria Criteria: youtube traffic, greater than 1 MB is affected

21 Evaluation Experimental Setup ISP –5 ISPs in Emulab –2 Discriminating Service Providers –PlanetLab nodes –Divided in near (west coast) and far (mid and east coast) nodes –HTTP and BitTorrent service Discrimination –Throttling and discriminatory loss using Click routers

22 Experiments Experiment 1: Simple Discrimination –HTTP Web service –Discriminating ISPs drop packets –Server location is confounding: different ratios of near and far nodes in each ISP Experiment 2: Long Flow Discrimination –Two HTTP servers S 1 and S 2 –Discriminating ISPs throttle traffic for S1 or S2 if the transfer exceeds certain threshold –Server location remains confounding Experiment 3: BitTorrent Discrimination –Discriminating ISP maintains list of preferred peers –Has higher drop rate for BitTorrent traffic to non-preferred peers –Location of servers remains confounding Performance Metric: Throughput in all experiments

23 Result 1: Difficult to identify discriminating ISP without stratification Because location of server affect performance and is also correlated with ISP, it confounds the inference Overall throughput distribution in discriminating and non- discriminating ISPs is similar Simple Discrimination Experiment Long Flow Discrimination Experiment

24 Result 2: Discriminating ISPs are easy to identify with stratification Discriminating ISPs have clearly identifiable causal effect on throughput Neutral ISPs are absolved

25 Result 3: Inferring the discrimination criteria ISP throttles throughput of a flow larger than 13MB or about 10K packets cum_pkts not_discriminated cum_pkts > > discriminated NANO correctly infers the threshold for discrimination

26 Conclusions

27 Sufficiency of Confounding Variables

28 Deployment Status