1. Networks and Distributed Systems a.k.a. G22.3033-010G22.3033-010 Lakshmi Subramanian http://cs.nyu.edu/~lakshmi Jinyang Li http://cs.nyu.edu/~jinyang

2. Class goals Help you –critically appreciate networks & systems research –learn creative problem solving (i.e. doing research) How? –Lectures/readings: discuss state-of-art work –Programming labs: play with real systems –A semester-long research project

3. Syllabus, grading etc. http://www.cs.nyu.edu/courses/fall06/G22.3033-010 Class participation (20%) –Read assigned papers before class! Two labs (10%) One project (70%) –Team of 2-3 people (<= 1 Ph.D. student per group) –Start next week –Weekly (or once every two weeks) meetings

4. Who should take the class? Grad-level class –Satisfy M.S. requirement of a “project” course Pre-requisite: –Basic knowledge on networks Computer Networks (L. Peterson) An engineering approach to computer networking (S. Keshav) –Programming experience TCP/IP Illustrated (R. Stevens)

5. Misc. Office hours: –Jinyang: 715 Broadway Rm 705, Tue 5-6pm –Lakshmi: Rm 706 Mon 5-6pm –TA: Ja Chen (jchen@cs.nyu.edu)

7. Next Generation Networks Jinyang Li

8. Emerging networks Wireless networks Sensor networks Overlays and P2P Delay tolerant networks (DTNs) …

9. Wireless networks

10. Wireless networks: why now? Proliferation of wifi-enabled devices Faster, cheaper radios and more powerful boxes

11. Wireless apps: urban mesh Provide cheap, ubiquitous Internet connectivity MIT Cambridge Roofnet http://pdos.lcs.mit.edu/roofnet Google Mountain View pole top network http://wifi.google.com

12. Wireless apps: connecting rural villages Intel/UC Berkeley/NYU Tier project http://tier.cs.berkeley.edu

13. Wireless apps: mobile, ad-hoc communication MIT CarTel http://cartel.csail.mit.edu

14. Wireless networks: challenges 1.Crappy links 2.Contention and self-interference 3.Frequent node/link failures 4.Many parameters Goal: Robust, high performance designs MAC layer Routing layer Transport layer

15. Challenge #1: crappy links Many asymmetric, lossy links

16. Challenge #2: contention Many nodes access the medium  collisions No way to explicitly detect collisions

17. Challenge #2: self-interference A multi-hop flow interferes at successive hops 1 2345 At most every third node can transmit

18. Challenges #3: dynamism Links/nodes fail and recover frequently Link qualities change over time Time (sec)

19. Challenge #4: (too) many tunable parameters Transmission power Transmission rate Directional vs. omni antennas Static vs. dynamic channel assignment One vs. multiple radios

20. Current state-of-art MIT Roofnet pair-wise node throughput (11Mbps 802.11b radios) # hopslatency (ms) throughput (kbps) 1142451 226771 345362 450266 560210 6100272 783181

21. Sensor networks Beyond host-to-host communication

22. Sensor networks: why now? Technology is ready –Cheaper, smaller, more powerful sensors –Sense light, temperature, vibration, humidity, location, pulse, motion, vital sign etc. Monitor environment, collection information UCB Telos Xbow MicaZ Intel Dot

23. Sensor apps: understanding redwood forests UC Berkeley/Intel Research

24. Sensor apps: real-time patient tracking Harvard CodeBlue

25. Sensor-net challenges Different communication paradigm –host-to-host is the wrong fit –Data-centric Limited resources –Low radio bandwidth 250Kbps advertised, ~80Kbps in real life –Slow processor, tiny storage 8MHz CPU, 8K RAM –Limited energy

26. Overlays and P2P Distributed systems meet the Internet

27. Why p2p/overlay? A distributed system architecture: –No (minimal) centralized control –Nodes are symmetric in function Enabled by technology improvements Internet

28. Large scale wide-area systems Unmanaged (open p2p systems): – BitTorrent: >1M nodes – Skype: >5M users Managed –PlanetLab: 700 nodes over 336 sites –Akamai CDN: >10K nodes

29. What’s new here? Opportunities: –Huge aggregate capacity Network, storage, processing… –Geographic diversity Many apps: –File sharing –CDNs –VoIP –Streaming multicast –Usenet news –…

30. Challenges How to find data? How to deal with failures? –Nodes fail and recover –Network outage and partition (Open networks only) How to deal with selfish or malicious nodes? –provide data integrity –provide privacy or anonymity

31. Challenge #1: resource discovery Case study: file sharing Where is the file named “Hamlet”?

32. Challenge #2: churn What if the node with “Hamlet” goes down?

33. Challenge #3: selfish nodes Selfish nodes do not want to upload “Hamlet” I do NOT have Hamlet

34. Challenge #4: malicious nodes I HAVE junk named Hamlet Malicious nodes lie about their contents

35. Next week Naming and addressing Project ideas

36. Check out the Spring class “distributed storage systems” Distributed systems in a data-center Connected by LANs low loss and delay Provide infrastructural services for apps –Network file systems –Databases –Distributed data processing