NoTorrent A Peer-to-Peer Web Caching System Modeled Partially After BitTorrent but Without the Need for.torrent Files Howie Vegter, hrv2101 Columbia University Fall 2005 CS 6901, Projects in Computer Science Advisors: Dr. Markus Hofmann, Dr. Henning Schulzrinne, Salman Abdul Baset

NoTorrent Architecture Overview Origin Server NoTorrent Tracker BrowserProxy Cache Server NoTorrent Client Client* Browser Proxy Cache Server NoTorrent Client Peer* *Note: A “client” and a “peer” both refer to a computer that is running the NoTorrent Client. I will say “client” when I mean my computer and “peer” when I am referring to the other NoTorrent Clients I am interacting with When the client requests a resource through the client proxy, we try to obtain the resource in this order: First try to get resource from Origin Server Then try to get it from client’s local cache Then try to get it from a peer

Requesting a Resource: First, Try the Origin Server Origin Server NoTorrent Tracker BrowserProxy Cache Server NoTorrent Client Client XML-encoded IHaveResourceMessage If origin server serves resource, Cache it locally Inform tracker client has the resource Send the resource to the browser If origin server fails to serve resource: Try checking local cache…

Requesting a Resource: Next, Try Local Cache BrowserProxy Cache Server NoTorrent Client Client Client could not get resource from origin server. Client may have obtained resource earlier (from peer or origin server before it went down), so check local cache. If resource in cache, Get resource from cache Send resource to browser If resource not in cache, Ask peers for the resource…

Requesting a Resource: Next, Ask Peers (slide 1 of 3) NoTorrent Tracker BrowserProxy Cache Server NoTorrent Client Client Browser Proxy Cache Server NoTorrent Client Peer XML-encoded PeerListRequestMessage XML-encoded ResourceRequestMessage Client could not get resource from origin server or cache, so we ask peers. 1. Ask tracker for list of peers with resource 2. Tracker responds with list 3. Ask a peer for the resource 4. Peer checks its cache 5. (see next slide)

Requesting a Resource: Ask Peers (slide 2 of 3) NoTorrent Tracker BrowserProxy Cache Server NoTorrent Client Client Browser Proxy Cache Server NoTorrent Client Peer If peer has resource: Peer sends resource to client Client proxy caches resource and informs tracker is has the resource Client proxy sends resource to browser XML-encoded IHaveResourceMessage byte[] resource

Requesting a Resource: Ask Peers (slide 3 of 3) NoTorrent Tracker BrowserProxy Cache Server NoTorrent Client Client Browser Proxy Cache Server NoTorrent Client Peer 1 Browser Proxy Cache Server NoTorrent Client Peer 2 X XML-encoded: Peer 1 Failed To Serve If peer does not have resource: Peer terminates connection Client informs tracker that peer failed to serve Client requests resource from another peer If no peers had resource, send error message to browser error

Messages are Encoded in XML MessagePeerListRequest: PeerListRequest NoTorrent uses JDOM to relatively easily parse XML.

Setting Up NoTorrent Unpack notorrent.tgz to get NoTorrent.jar, README, nttrack, and ntclient (the latter two are helper scripts) On tracker machine: Start tracker to listen for and handle messages from clients: java –cp NoTorrent.jar notorrent.tracker.Tracker or./nttrack On client machine: Start client proxy to handle requests from browser: java –jar NoTorrent.jar –ta TRACKER_ADDRESS –tp TRACKER_PORT or./ntclient Configure browser to use NoTorrent client proxy as its HTTP proxy

Lessons Learned and Encountered Roadblocks When basing a project on another project, make sure the base project is well-documented is really as close to the problem you are trying to solve as you thought. Standards (e.g. XML) and good libraries (e.g. JDOM) can simplify certain parts of the project. Specific Things I Learned About: Message encoding (XML, BEncoding). I had not anticipated that encoding would be a major part of system. TCP/IP sockets in Java, thread pools Python (although I ended up not using it) Roadblocks: HTTP 403 (Forbidden) Errors on certain pages on Google, NYTimes, Yahoo, etc. Yahoo search results include tracking info in the URLs and then redirect the user to the destination. This currently works without the NoTorrent proxy, but it only works partially with the NoTorrent proxy. Sample Yahoo search result URL (for columbia.edu): 2BGNvbG8DdwRsA1dTMQRwb3MDMgRzZWMDc3IEdnRpZANGNjY2Xzg1/SIG=11cn6o6 5u/EXP= /**http%3a//