22 October 2010 Eurecom, Sophia-Antipolis Thrasyvoulos Spyropoulos / Network Modeling (NetMod): Friday 1:30-4:45 Instructor: Thrasyvoulos Spyropoulos

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis : Undergraduate studies in Greece  National Technical University of Athens (NTUA)  Specialization: Telecommunications and Networking : MSc and PhD in Los Angeles, California  University of Southern California (USC)  Thesis: Perf. Analysis and Protocols for Wireless Networks : INRIA, Sophia-Antipolis  Post-doc at Planete group : ETH, Zurich  Senior Researcher/Lecturer ( ) 2010-present: EURECOM  Professor

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis Goal: to teach some mathematical tools that are valuable, when trying to understand real networks…in fact real systems!  STOCHASTIC PROCESSES  Learn to deal with Randomness  Markov Chains, Queueing Theory, Operational Laws, Scheduling  COMPLEX NETWORKS / NETWORK SCIENCE  Modeling Large Networks  Connectivity, Small-World Phenomena, Random Graphs, Information Diffusion, Sampling/Crawling  APPLICATIONS  MAC protocols, cellular networks, web search, social networks, data centers, cloud computing, virus infections, and others… 3

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  Many things in nature are random => same for networks  more precisely: increased complexity  described as randomness  Randomness in:  Propagation phenomena (coding, diversity)  Location and mobility of nodes (handoff)  Traffic/Service arrival patterns (cellular capacity allocation)  Next link to be clicked on a webpage (browser prefetch)  Size of files downloaded (cache sizing)  Computing job arrivals and duration (cloud computing)  Number of (facebook) friends per user (advertising) …… 4 Computer Science Approach Devise algorithm Deal with worst case Electrical Engineering Approach Optimize for probable cases Ignore rare events

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  Most networks can be modeled as a large graph 5 Network of Internet RoutersOnline Social Nets (FaceBook)Mesh Networks

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  Difficult to study/model a specific graph  Specific graph: an instance of a random graph with specific qualitative properties  “Complex/Social Network Analysis or Network Science”  the study of qualitative properties of large graphs/networks  Degree distribution, diameter, connectivity, clusters  (a) WHY do these properties arise? (Scientist)  (b) HOW can they be exploited? (Engineer)  Degree distribution  searching, security  Clustering  advertising, information spreading 6

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  (Complex) Networks as Random Graphs  Many interesting problems on Networks can be modeled as a Random Process on a (Random) Graph  routing,  connectivity prediction,  diffusion of information/viruses,  searching (e.g. in P2P),  medium access control (MAC), etc.), ……  A very relevant course! Similar courses in a few top schools  Performance Analysis classes from CalTech and Carnegie Mellon Univ  Complex/Social Networks classes from CalTech and Cornell University 7

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  PART I: Stochastic Processes and Queueing  “Performance Modeling and Design of Computer Systems” by Mor-Harchol-Balter – shared copies in library + printed notes  “Stochastic Processes” by Sheldon Ross – shared copies in library  “Introduction to Probability Models” by Sheldon Ross – copy in library  PART II: Complex Network Analysis  “Networks, Crowds, and Markets: Reasoning About a Highly Connected World” by D. Easley and T. Kleinberg – pdf freely available online  “Networks: An Introduction” by M. Newman – shared copy in library  Additional reference material (tutorials, articles) per topic 8

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  Regular Homeworks % of Grade  Among them 1-2 “lab” sessions  Midterm Exam (after Part I) % of Grade  Final Exam % of Grade  Participation --- extra credit!  Office Hours: TBD  Class Web Site: 9

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis What to expect from me:  To make the class entertaining  Many examples and application  Interaction Interaction Interaction!  To teach you key insights  Not just “tools”  when to use which tool  why it works What I expect from you:  To carefully/critically study the assigned material  To work hard on your homeworks  Interaction Interaction Interaction!!! 10

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  Introductory Probability Theory  Distributions (bernoulli, geometric, binomial, gaussian, poisson)  Expectations, Variance, etc.  Conditional probabilities and expectations  Independence and Correlation  Review Reference: “Introduction to Prob. Models” or “A First Course in Probability” by Sheldon Ross – available in library  (very!) Elementary Linear Algebra  Matrix multiplication  Solving Linear Systems  Eigenvalues  Check out Gilbert Strang’s online lectures for a refresher (excellent!)  A tiny bit of MatLab 11

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  Betting on the Roulette  18 red  18 black  2 green  John observes the roulette and counts 5 reds in a row Q: In the next roll should he bet on red or black? Q: What if John sees 20 reds in a row?? 12  John: “I should bet on black! 21 reds in a row are VERY unlikely!  James: “No, it makes no difference! Rolls are independent!” Q: Prob{20 reds in a row followed by a black}? Prob{21 reds in a row}? On August 18, 1913, at the casino in Monte Carlo, black came up a record twenty-six times in succession in roulette… There was a near-panicky rush to bet on red, beginning about the time black had come up a phenomenal fifteen times. …players doubled and tripled their stakes, led to believe after black came up the twentieth time that there was not a chance in a million of another repeat. In the end the unusual run enriched the Casino by some millions of francs.

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  A terrible crime has occurred in city A, and John is one suspect  A DNA matching that of John is found in the crime scene  This is the only evidence against John  Two DNAs matching have a 1 in a million chance  The prosecutor and jury conclude John is guilty Q: Were they right?  City A has about 10 million people. Q: What is the chance that John is innocent? A: John is innocent with a 90% chance!!! BAYES RULE: 13

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  Sensor node bootstrapping  Each node has a unique ID  Goal: each node needs to broadcast its ID to all other nodes Protocol  Node X picks a slot n uniformly in [1,N]  P{n = i} = 1/N  Broadcasts its ID in slot n  SUCCESS: no other node picked n  COLLISION: 2 or more nodes picked n  nodes fail and stay “off” Tradeoff: Low N  many collisions || High N  long delay Q: If 30 nodes, what is the minimum N  P{collision} < 10%? A: 200? 500? 1000? 5000? 14 t x.y.z N > 4500 (look up “birthday paradox”)

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  Given a system (e.g. Internet, a wireless LAN, server farm) --> we would like to know its performance  Throughput, delay, utilization, etc.  Compare protocol/policy performance: Is my new algorithm in fact better than the old one?  Identify bottlenecks: If I want to improve the performance of Eurecom’s WLAN, should I a) Install more Access Points? b) Propose a better channel selection algorithm? c) Other? 15

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  Simulations can answer these questions  But sometimes have to deal with huge problem/parameter space -Network size, transmission range, mobility model, buffer space, rate, scheduling policy, backoff value,… -Single simulation run takes long time (ns2 anyone?)  evaluating *all* scenarios prohibitively slow  Does not answer “Why?” -Quantitative results, but not necessary intuition  Measurements and Experiments often a more accurate way to evaluate performance in real(istic) setting  But lots of effort/time and usually only tiny platforms  Analysis can provide quick: performance prediction, insights, and ideas for improvement  Knowing analysis can make you good consultants ;) 16 “All models are bad! But some are useful” by statistician George Box

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  Your company has just got very positive publicity  The incoming load (arrival rate of requests) to the web server is expected to double as a result  Your boss tells you that you need to upgrade the server with a faster one (higher service rate μ) to ensure the same mean response time E[T] Q: How much should you increase the service rate? a) Double the server speed? b) More than double the speed? c) Less than double the speed? A: Correct answer is (c) 17 Job requests / per second Jobs served / per second 2x ? 

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis OPTION 1  3 slow cashiers  3 lines  randomly choose line and stay there  Each cashier: 10 customers per hour cust/hour 30 cust/hour OPTION 2  1 fast cashier  single line  30 customers per hour

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis cust/hour 19 Q: Which options has the smallest waiting time? - Option 1 or Option 2? A: Option 2 is 3x faster! Option 1 30 cust/hour Option 2

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis cust/hour Option 3 Q: Which options has the smallest waiting time? - Option 2 or Option 3? - Low load: Option 2 or Option 3? A: Similar delay (for high load) A: Option 2 up to 3x faster 30 cust/hour Option 2

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  How does all these relate to networking?? KHz OPTION 1: FDMA  Separate 200KHz channel to each  Flows do not compete 600KHz OPTION 2: CSMA  Each node senses the channel first  If idle  transmit pkt using 600KHz  If busy  queue (wait) Q: Which option would you prefer for data? Q: Which option would you prefer for voice?

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  What lies behind this simple box??  Searching the Web  Step 1: crawl all web pages and create index of {keywords-web pages}  Step 2: User enters keywords (e.g. “Network Modeling”)  Step 3: Google finds all web pages matching these keywords (these 3 steps are generic to almost every search engine)  Step 4: Return a list of matches ranked by importance. HOW???  Intuition 1: page important if many web pages refer to it  Intuition 2: page important if important web pages refer to it 22 Solution: PageRank algorithms solves an appropriate Markov Chain

22 October 2010 Eurecom, Sophia-Antipolis Thrasyvoulos Spyropoulos / the Celebrated (and Demonized!) Poisson Proccess

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis Iceland Volcano: Why could you not talk to airline cust. service? New Years Eve: Why can I not call my relatives? Problem 1: Call Center Dimensioning  Customers call randomly  Assume (for now!) duration of each call is fixed  N workers : if all busy, call is dropped  Question: What should N be to ensure at most 5% of calls are dropped?  Case 1: calls arrive regularly (one every X min)  Case 2: calls arrive in bursts (many together, then silence) 24

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis Problem 2: Internet Router Buffer Sizing  Packets arrive at a core router  Packets may belong to the same or different user/app  Need to be buffered before forwarded further  Question: How large should the buffer be (to ensure few drops) ? * Change picture with large switch! 25

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  Need to know/model the (random) arrival of “work” => to optimize the system!  Calls at a call center => to pick the number of employees  Calls to a base station (inside a cell) => to allocate frequencies  Packets at a router => to choose the right buffer size  (large) jobs at a cluster/supercomputer => to choose the number of CPUs  What might we need to know?  Average amount of work per min/hour/day  Probability of 3, 4, 5 customers arriving within T min  Probability that > N customers arrive within T min 26

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  Rate of events: λ  average number of events in an interval  Probability of n events in an interval  Examples well approximated by Poisson distribution  The number of deaths per year in a given age group.  The number of phone calls arriving at a call centre per minute.  The number of new sessions arriving at a web server per hour  The number of soldiers killed by horse-kicks each year in each corps in the Prussian cavalry …… 27

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis 28

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  Definition 1: A counting process viewpoint  Property 1 (“independent increments”): # of arrivals in non- overlapping intervals (e.g. N(T 1 ) and N(T 2 )) is independent  Property 2 (“stationary increments”): # of arrivals in [t 1,t 2 ] only depends on (t 2 -t 1 )  Property 3: # of arrivals N(t) in interval t is Poisson (λt) 29 T1T1 T2T2 time

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  Definition 2: a Renewal Process viewpoint  Inter-arrivals times are independent  Time T between arrivals (“renewals”) is exponential(λ) 30 T: exponential dt  Definition 3: “aggregate of many rare events”  Prob{1 event in dt} = λdt + o(dt) (independently of past events)  Prob{> 1 events in dt} = o(dt) (negligible as dt -> 0)

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  We can go (prove) from any definition to any other  Definition 1 (Poisson) => Definition 2 (Exponential)  Prob{T > t) = Prob{0 events in t} =>   T is exponential  Definition 1 (Poisson) => Definition 3 (rare events)    31

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  Number of arrivals N(t) in t  Binomial(n, p)  n = t/ δt  p = λδt + o(δt) If δt  0, such that np = λt: Then Binomial (n,p)  Poisson (λt) 32 δt P{arrival} = λδt t

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  Time to wait until the next arrival (T 1 ) is exponential  Time to wait until the n-th arrival (S n =T 1 +T 2 +…+T n )?  Sum of n independent and identically distributed (IID) exponential random variables  Gamma Distribution  How to get this?  Proof 1: Moment Generating Function  Proof 2: (CDF) F s (t) = Prob{S n ≤ t} = P{N(t) ≥ n} (Ross, Ch.2)  Proof 3: P{t < S n < t+dt} = P{n-1 events in t,1 event in (t,t+dt)} (Ross) 33 SnSn 0 t T1T1 T2T2

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  We are told that 1 arrival has occurred in the interval T  Question: When did it happen exactly?  NOTE: this is a conditional probability  Answer: Arrival is uniformly distributed: any instant in the interval is equally probable  P{S1 ≤ s} = s/T (0 ≤ s ≤ T) 34 1 arrivals in T 0 t S1S1 T

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  We are now told that n arrivals have occurred in T  (from before) each arrival is uniform in T  How is S 1 (1 st arrival), S 2 (2 nd arrival), etc. distributed?  Answer: Order Statistics of n IID random variables uniform in (0,T)  f(s 1,s 2,…,s N ) = n! / t n 35 n arrivals in T 0 t S1S1 S2S2 S3S3

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  Receives event readings with rate λ  must sent to a base station  To save battery power: (a) wireless card in sleep mode, (b) queue events during sleep mode, (c) wake up every T minutes and transmit all queued events  QoS: When an event is queued for  cost of queueing for t : c(t) = ct Q: What is the total cost incurred each period T? A: 0.5 c λT 2 Q: Assume battery consumption is a(T) = a/T. What is the optimal T? 36 Sensed data: Poisson (λ) t T2T sleep 0 wakeup

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  Assume Poisson arrivals with rate λ  A 2 nd random process is created as follows:  We accept each arrival with probability p < 1 (or reject with 1-p) 37 XXX X X X X accept with prob p  Question: what is the expected number of arrivals within T?  Answer: pλT  Question: what is the second process?  Answer: Poisson with rate pλ  Proof? T

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  2 independent Poisson arrival processes  Calls at “Free” service center  E.g. “my Internet connection not working” calls with rate λ 1  “TV box not working” calls with rate λ 2  Question: what is the process of total calls arriving at the service center?  Answer: Poisson with rate λ 1 +λ 2 38 rate λ 1 + rate λ 2 Poisson(λ 1 +λ 2 )

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  Definition: A stochastic process [X(t), t ≥ 0] is a compound Poisson process if:  [N(t), t ≥ 0] is a Poisson process  [Y i, i ≥ 1] is a family of IID random variables, independent of N(t)  Results 1) E[X(t)] = λtE[Y 1 ] 2) Var(X(t)) = λtE[Y 1 2 ] 39

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis A1: customers arrive at the Amazon web site can be modeled as a Poisson process with rate λ customers/min A2: The amount X each customer will spend is random with mean E[X] and independent of the arrival time. Q: What is the expected revenue per hour for Amazon? A: λ 60 E[X] 40

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  Memory-less property: simplifies models  No need to know/keep track of the past to predict future -Stationary behavior is sufficient!  Good approximation for aggregate “traffic” of many and independent sources  Palm-Khintchine Theorem  Why we don’t like it:  Not always true  Many workloads have “heavy-tailed” properties  memory 41