1. PPSP Protocol Considerations and Tracker Protocol draft-gu-ppsp-tracker-protocol-01 Y. J. Gu, David A. Bryan, Y. Zhang, H. Liao IETF-78 Maastricht, PPSP Session

2. 1 My Thoughts… Trying to share a picture of what a PPSP deployment might look like –When possible, want to reuse protocols. Where can we and what? Given an architecture, what does the tracker protocol for that look like? –Short overview of the protocol details of BitTorrent –These are are own (very early!) thoughts, so may be wrong, but hope to stimulate discussion

3. 2 What are we trying to do? There are two basic protocols (or protocol operations) –Tracker protocol How to find other peers that share the information Focus of this talk, but need to discuss peer protocol a bit to make sense… –Peer protocol How to get information from those peers once you have found them But we seem to be looking at two different tasks –Offline/timeshifted media (essentially file sharing) –Streaming/realtime media

4. 3 Tracker Protocol Need some way to locate the peers that are sharing same content Don’t have a direct protocol to reuse from IETF Basic idea is that tracker functionality is on a single server, but could be distributed (DHT) –Note that this is essentially a client-server protocol. Could distribute as a DHT underneath, but the tracker-peer operation is basically C/S –I’ve heard proposal to use RELOAD, this works if the tracker is made up of a distributed set of peers –Anaheim meeting indicated interest in tracker as a server, so seems RELOADs only application here is possibly as an implementation detail underneath or as an alternate distributed implementation

5. 4 BitTorrent as a Model? Approach seems well suited to offline use case May not have all the information we want/need for streaming –Need to find peers “nearby” in the stream –Should tracker attempt to do this? –If not, many peers may need to be contacted to find one in “right place” (depending on window size, pause, etc) Possible issues –Security (or lack of) –HTTP approach is somewhat heavy (but easy) –Do we want to incorporate metadata into tracker (not offline in torrent file)? Need to specify syntax for these metadata files

6. 5 Some BT Basics BT’s primary purpose/design is for file sharing (not originally designed for live streaming) Peers that share a particular file cluster together to share portions of a file form a swarm

7. 6 BitTorrent Entities Peers: Hosts that hold some portion of the swarm are peers. Peers exchange blocks, and a set of blocks makes up a piece of the file –A Seed is a peer with the entire file Tracker: A central server that stores a mapping between a swarm and the peers participating in that swarm –Tracker doesn’t store which peers have which pieces, just list of the peers –Tracker is located offline…

8. 7 File and Metadata Original person sharing the file splits it up into pieces, and performs an MD5 hash on each piece The list of pieces and their hashes, and the location of a tracker that will serve peers sharing this file are placed into a metadata file called a torrent Original user places torrent on a web server, and subscribes to tracker with all chunks, as the initial seed

9. 8 Example Startup 1) File is chopped up, MD5 sum generated Tracker Chunk1 Chunk2 Chunk3 … 2) Torrent file lists chunks, sums, and tracker to use for swarm Tracker Chunk1 Chunk2 Chunk3 … 3) Torrent file is stored on web server T P 4) Peer connects to Tracker with entire file as seed

10. 9 Example Join Swarm Tracker Chunk1 Chunk2 Chunk3 … 1) Peer connects to web site and obtains torrent file to locate tracker P 2) Peer connects to Tracker to find other peers P T P P P P 3) Peer connects to other peers in swarm and exchanges chunks

11. 10 Peer Exchange Peers exchange blocks or chunks –Swap smaller bits than described in metafile, MD5 verify assembled chunks Simple gossip protocol –Generally unstructured, not a DHT –Connect to peers that may have desired chunks –Exchange bitfield msgs indicating which chunks it has –Request to ask for a chunk, returned in a piece msg –Once a piece is downloaded, advertised with have msg –Interested/not interested and choked/not choked are used in flow control –keep-alive message

12. 11 BitTorrent Protocol Details Regular HTTP is used to obtain the torrent from a web server The tracker protocol is also HTTP, essentially GET to ask for a list of peers/join swarm The peer protocol is a TCP wire protocol (binary)

13. 12 Our (Strawman) Proposal New -01 version coming soon –Yingjie Gu, David Bryan, Yunfei Zhang, Hongluan Liao Currently propose binary protocol (but open) –Light weight, aesthetic considerations –Could also use HTTP with XML or something similar Messages to connect to a tracker/disconnect –Credential verification, verify peer-ID later used by peer protocol –Credential issuance/peer-ID assignment not (necessarily) by tracker Messages to join/leave a swarm (and get list of peers) –Currently can store location in stream/get peers at this location…may be hard to implement Diagnostics between peers and tracker, keep-alive messages, query list of swarms from tracker Will describe in detail in later talk

14. 13 Peer Protocol Considerations New transport is out of scope. –Offline and Streaming scenarios Need to reuse existing protocols SRTP/RTP for streaming. ??? For offline Should work of LEDBAT be leveraged here? Lightweight gossip protocol between peers –Typical for BT is 20-50 peers, in an unstructured way –Is RELOAD suited for this, or will we need something lighter? Try a RELOAD usage and find out?

15. 14 What might this look like? P T P P P Offline/Time Shifted Scenario New peer protocol (BT based?) to find peers, get metadata (not specific chunks) Lightweight gossip protocol between peers to find chunks (RELOAD usage or new?) Existing transport protocol to obtain chunks from peer (leverage LEDBAT?)

16. 15 What might this look like? P T P P P Streaming Scenario New peer protocol (BT based?) to find peers, get metadata, where in stream peer is? Lightweight gossip protocol between peers to start/stop stream to other peers Existing transport protocol to stream the data (RTP/SRTP?)

17. 16 Protocol Reuse? May be many places where we can reuse existing protocols, but in some cases, using for things we haven’t done before –LEDBAT? –RELOAD for lightweight/gossip protocol (not DHT)? –New protocol for Tracker or HTTP with XML bodies or something similar?

18. 17 Tracker Protocol Proposal New -01 version released (quite a few changes) –Yingjie Gu, David A. Bryan, Yunfei Zhang, Hongluan Liao Key changes: –Changed names of several messages Name and semantic meaning not aligning –Added XML/HTTP Encoding Authors don’t view encoding as that important right not -- set of messages and semantic meaning is what is critical Encodings are proof of concept This is a basic overview, not a detailed description (can read draft shortly when we iterate for all details) Still very early work -- primary focus is exploring problems through design/early implementation A number of hard questions are being left open for WG input, and I’ll talk about those today This is by no means complete right now -- lots of work left to do!!! Authors are very interested in suggestions

19. 18 Messages CONNECT/DISCONNECT –Associate with server, verify credentials –DISCONNECT removes from all swarms, leaves system JOIN/LEAVE –Participate in a particular swarm (streaming or file for VoD) –Possibly a JOIN_CHUNK to allow for specifying where in a live streaming (but big can of worms…) FIND –Given a swarm, locate a number of peers –Leaving room to specify a criteria (where in stream (if we allow JOIN_CHUNK), capacity, possibly certain layers –Quality (ALTO in the tracker, or do peers do ALTO?)

20. 19 Messages, Continued STATs messages –STAT_QUERY/STAT_REPORT –Send info to tracker / query about other peers –Tracker can also poll peers KEEPALIVE –Limit on live time for peers to tracker, so if no requests in a certain time, refresh connections –Another option is to expire either CONNECT or JOIN and require a subsequent call… QUERY (do we need this?) –Search for swarms/list swarms (Tracker protocol or should this be something else?) –Not currently in the draft

21. 20 Open Issues/Considerations Binary vs. text encoding –Transport/security mechanism if not HTTP Need to define format for metadata describing the file Peer-IDs used in many messages, but assignment is offline. Do we want a version of connect that issues IDs? Response of list of peers depends on peer protocol used -- IP address vs. Peer-ID only NAT traversal needs to be considered

22. 21 Metafile Differs slightly for streaming/VoD VoD: –Needs to describe chunk format, number of chunks, break down (sizing), support for layered encodings, codec –MD5 sums of blocks (or collections of blocks) Streaming –Codec, chunk size, but likely some (number of chunks, MD5 sums don’t make sense) It is very important to get this right, but hard!

23. 22 Impact of Peer Prot. on Tracker Prot. If we use a DHT style peer protocol, with lookup, then at CONNECT time, the peers need to insert into the overlay –Tracker then only needs return peer-ID (after bootstrap to locate peer when connecting) If be use an unstrutured/gossip protocol, not clear this is ideal –Random connection to 20 peers in a system of millions likely means you need to provide an address –Or, just use a DHT for routing, nothing else?

24. 23 NAT Traversal Easy for managed systems -- server is placed in a reachable location. Issue: unmanaged systems: –Can’t we guarantee tracker in public space Bigger issue: distributed tracker –Peers may very well be behind NATs –Fully distributed RELOAD may solve some of this –If just a few “super-peers”, how do we decide promotion (NAT detection is provably difficult/impossible…)

25. 24 Conclusion This is early work -- much still to do Decoupling the design of the peer and tracker protocols may help with design, but some aspects are intertwined (for example peer list structure) Very much need advice drawn from existing implementations –Biggest question: Do we have the right commands?

26. PPSP Peer Protocol draft-gu-ppsp-peer-protocol-00 Y.J. Gu, David A. Bryan IETF-78 Maastricht, PPSP Session

27. 26 Overview Draft covers: –Requirements for Peer Protocol –Simple example of a possible flow –Discussion of open issues –Skeleton (currently empty) for new protocol Define binary and text strawman proposals such as in tracker protocol? Very early work (less developed than tracker protocol, but we will be extending/expanding) –Conversations with some vendors with deployment experience – including some since -00 Main emphasis currently around requirements and open issues

28. 27 Peer Protocol Requirements Location/Connection Requirements –Req 1: Once a client has a peerlist it SHOULD be able to locate peers and connect to them with no or minimal Tracker's assistance. –Req 2: The Peer Protocol MUST provide a mechanism for peers behind different NATs/Firewalls to connect with each other.

29. 28 Peer Protocol Requirements Information Exchange Requirements –Req 3: The Peer Protocol SHOULD enable peers to request/return/exchange peerlists. –Req 4: The Peer Protocol SHOULD enable peers to request/return/exchange data availability, e.g. bitmap of chunks. –Req 5: The Peer Protocol MUST be able to carry different data structures for different applications.

30. 29 Peer Protocol Requirements Transportation/Negotiation Requirements –Req 6: The Peer Protocol MUST be able to negotiate a transportation protocol that both peers can support. Security Requirements –Req 7: The Peer Protocol MUST guarantee peers' privacy. –Req 8: Peers SHOULD be able to verify the identity of remote peers.

31. 30 Protocol Overview Currently looking at peer protocol in two logical parts –Location Portion Locate and connect with Remote Peers –Signaling Portion Get additional peerlist (optional?) Exchange data availability (e.g. bitmap) Negotiate transport mechanism (e.g. protocol, port…) –Could be two protocols… –Actual transfer is yet another aspect, but may be out of scope here (beyond negotiation using signaling portion)

32. 31 Location Protocol Ties in intimately with tracker protocol The decision of identity type included in the peerlists in the tracker protocol will influence the choice of peer location mechanism –IP address, Peer-ID w/DHT, other? Effectiveness/decision on these options will influence the design of the tracker protocol.

33. 32 Candidate Protocols Candidate protocols include RELOAD, SIP, a new protocol, combination… –RELOAD In theory, works for unstructured, but untested. Could use structured simply to connect peers/create connections? –SIP Potentially heavier than RELOAD. No direct mechanism to support location by Peer-ID? –RELOAD w/SIP Use RELOAD to locate/connect, SIP to negotiate? –New protocol Single new protocol for location and negotiation? New for only some functions with reuse for others?

34. 33 NATs and Choice of Identifier Big issue in identifiers: NAT In current network, most peers are behind NAT. If IP addresses are used, ICE/STUN/TURN server functions may need to be deployed in PPSP system: –Tracker: Natural server/relay location, but will increase the burden on tracker and need long-term connection between peers and tracker. –3rd party server? If Peer-IDs are used (particularly reusing RELOAD) can potentially leverage ICE support, add tracker as a relay/STUN server when needed?

35. 34 Conclusions One of the biggest questions – and one that has strong bearing on the tracker protocol is the use of IP addresses or Peer-IDs (and a mechanism to locate them) NAT traversal is critical Need to identify set of operations, determine if reuse is possible Perhaps decide on identifier issue, then work on tracker protocol first?

36. 35 Acknowledgements We would like to acknowledge the following for providing advice/suggestions/questions: –Roni Even, Yunfei Zhang, Hongluan Liao, Ning Zong, Daniel De Vera, Matias Barrios

37. Backup Slides

38. 37 HTTP/XML Approach HTTP POST method used –May not be best approach Bodies are encoding in XML Tags defined in draft

39. 38 Encoding, Transport for Binary Currently, we have proposed a binary protocol –Light weight, low bandwidth for mobile devices –Basic ideas would apply to other encodings Right now, the transport is left unspecified -- looking for feedback from group –Since this is essentially a client-server like operation, preference is on a persistent secure connection approach (TLS/DTLS) May want a different approach if we do a distributed tracker –Is there a good reason to use DTLS (UDP)? Fragmentation mechanism borrowed from RELOAD (first/last bits, offset)

40. 39 Basic Shared Header 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | PPSP Tracker Protocol Token | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Version | Reserved | Method | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Transaction ID +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Fragmentation | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Message Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

41. 40 Example Specific Request (JOIN) 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | PeerID (160 bits...)... +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | SwarmID (128 bits...)... +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Expiration Time | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

42. 41 Messages/Responses Transaction ID used for correlation/retransmission Responses are numeric codes, optionally with bodies (for example when requesting a list of peers) Currently a pure request/response protocol –No need for anything else so far

43. 42 BitTorrent References Official Protocol Specification (very limited!) is at http://bittorrent.org/beps/bep_0003.html Unofficial Specifications (much more detailed) at theory.org: http://wiki.theory.org/BitTorrentSpecification