Networks and Distributed Systems a.k.a. G G Lakshmi Subramanian Jinyang Li

Class goals Help you –critically appreciate networks & systems research –learn creative problem solving (i.e. doing research) How? –Lectures/readings: discuss state-of-art work –Programming labs: play with real systems –A semester-long research project

Syllabus, grading etc. Class participation (20%) –Read assigned papers before class! Two labs (10%) One project (70%) –Team of 2-3 people (<= 1 Ph.D. student per group) –Start next week –Weekly (or once every two weeks) meetings

Who should take the class? Grad-level class –Satisfy M.S. requirement of a “project” course Pre-requisite: –Basic knowledge on networks Computer Networks (L. Peterson) An engineering approach to computer networking (S. Keshav) –Programming experience TCP/IP Illustrated (R. Stevens)

Misc. Office hours: –Jinyang: 715 Broadway Rm 705, Tue 5-6pm –Lakshmi: Rm 706 Mon 5-6pm –TA: Ja Chen

Next Generation Networks Jinyang Li

Emerging networks Wireless networks Sensor networks Overlays and P2P Delay tolerant networks (DTNs) …

Wireless networks

Wireless networks: why now? Proliferation of wifi-enabled devices Faster, cheaper radios and more powerful boxes

Wireless apps: urban mesh Provide cheap, ubiquitous Internet connectivity MIT Cambridge Roofnet Google Mountain View pole top network

Wireless apps: connecting rural villages Intel/UC Berkeley/NYU Tier project

Wireless apps: mobile, ad-hoc communication MIT CarTel

Wireless networks: challenges 1.Crappy links 2.Contention and self-interference 3.Frequent node/link failures 4.Many parameters Goal: Robust, high performance designs MAC layer Routing layer Transport layer

Challenge #1: crappy links Many asymmetric, lossy links

Challenge #2: contention Many nodes access the medium  collisions No way to explicitly detect collisions

Challenge #2: self-interference A multi-hop flow interferes at successive hops At most every third node can transmit

Challenges #3: dynamism Links/nodes fail and recover frequently Link qualities change over time Time (sec)

Challenge #4: (too) many tunable parameters Transmission power Transmission rate Directional vs. omni antennas Static vs. dynamic channel assignment One vs. multiple radios

Current state-of-art MIT Roofnet pair-wise node throughput (11Mbps b radios) # hopslatency (ms) throughput (kbps)

Sensor networks Beyond host-to-host communication

Sensor networks: why now? Technology is ready –Cheaper, smaller, more powerful sensors –Sense light, temperature, vibration, humidity, location, pulse, motion, vital sign etc. Monitor environment, collection information UCB Telos Xbow MicaZ Intel Dot

Sensor apps: understanding redwood forests UC Berkeley/Intel Research

Sensor apps: real-time patient tracking Harvard CodeBlue

Sensor-net challenges Different communication paradigm –host-to-host is the wrong fit –Data-centric Limited resources –Low radio bandwidth 250Kbps advertised, ~80Kbps in real life –Slow processor, tiny storage 8MHz CPU, 8K RAM –Limited energy

Overlays and P2P Distributed systems meet the Internet

Why p2p/overlay? A distributed system architecture: –No (minimal) centralized control –Nodes are symmetric in function Enabled by technology improvements Internet

Large scale wide-area systems Unmanaged (open p2p systems): – BitTorrent: >1M nodes – Skype: >5M users Managed –PlanetLab: 700 nodes over 336 sites –Akamai CDN: >10K nodes

What’s new here? Opportunities: –Huge aggregate capacity Network, storage, processing… –Geographic diversity Many apps: –File sharing –CDNs –VoIP –Streaming multicast –Usenet news –…

Challenges How to find data? How to deal with failures? –Nodes fail and recover –Network outage and partition (Open networks only) How to deal with selfish or malicious nodes? –provide data integrity –provide privacy or anonymity

Challenge #1: resource discovery Case study: file sharing Where is the file named “Hamlet”?

Challenge #2: churn What if the node with “Hamlet” goes down?

Challenge #3: selfish nodes Selfish nodes do not want to upload “Hamlet” I do NOT have Hamlet

Challenge #4: malicious nodes I HAVE junk named Hamlet Malicious nodes lie about their contents

Next week Naming and addressing Project ideas

Check out the Spring class “distributed storage systems” Distributed systems in a data-center Connected by LANs low loss and delay Provide infrastructural services for apps –Network file systems –Databases –Distributed data processing