Stream Spider Distributed Music Streaming Service with Sliced Music Files from Different Originating Hosts Justin Steffy and Michael D. Elder

Outline Service Operation Design –Peer to Peer Client and Service –Availability and Fault Tolerance –Consistency and Correctness Implementation –Foundations: Derby, Jetty, Eclipse RCP –Protocols (HTTP/XML) Demonstration

System Operation Register with service to find a set of peers with music to share Search for particular files on other peers Play music while streaming contents from multiple peers

Design Peer to Peer protocol –Peers can search and request files directly –Peers also service search and file requests –Two levels of peer Highly transient peers –Weak consistency –Sporadic availability Maintain state of peers on the network

Peers Types of Peers –Service Peer Executes and responds to file requests Requests and services music streams Maintains small “neighbor set” to service requests –Bootstrap Peer: Receives initial registration for new Peers Replicates available registrations Maintain consistency with other Bootstrap peers Can also act as a Service Peer

Diagram: Registration Register A List of Peers Register C Register B Peer B Peer A Peer C Bootstrap Peer List of Peers including A List of Peers including A & B

Diagram: File Search Bootstrap Peer Have files matching A? List of Files Peer A Peer C Peer B searches for songs that match A

Diagram: Music Streaming Bootstrap Peer Request Segment Octet Stream of data Peer A Peer C Request Segment Octet Stream of data Peer B

Availability and Fault Tolerance Availability does not depend on any one bootstrap peer –Replicate state among many peers –Loss of a bootstrap causes minimal effect Peers tolerate losses in neighbor set –When list gets low, request more peers –Losing a peer while streaming is handled

Important State Set of available peers (Shared) –Bootstrap Peers synchronize Peer locations –Service Peers request more peers as their neighbor set leaves or fails –Peer availability is volatile state, maintained relative to each Peer Spliced Music Streams –Recovery when an identified peer dies while servicing a stream

Consistency and Correctness Bootstrap Servers enforce Weak Consistency when sharing available Peers Correctness Criteria –Peers should be able to register with one of the active Bootstrap Peers –Peers should receive search results from active peers in its neighbor set –Peers should receive music splices upon request in order to stream music

Implementation Java and Eclipse –Eclipse UI –Database –Servlets –JUnit HTTP and XML basis for protocol –Object Models

Foundations Derby 10.1 –Embedded Java Database –Maintains local state at each Peer Jetty 6.1 –Embedded Java 2 EE Servlet Container –Act as protocol endpoints Eclipse Rich Client Platform (RCP) –Provides for rapid development of application architecture and UI Eclipse Modeling Framework (EMF) –Provides an object model abstraction and XML serialization JLayer 1.0 –MP3 playback Milestone development with unit tests (~20)

Protocols Based on HTTP and XML Registration –Accepts a requested number of peers –Answers set of available peers Search –Accepts a filename –Answers set of files that match a portion of the input Fragment –Accepts a file id and fragment range –Answers the portion of the file

Registration Request – HTTP Query String Response - XML …

Search Request – HTTP Query String Response - XML …

Fragment Request Request – HTTP Query String Response – Octet Stream –Beginning at start for length bytes

Consistency Protocols Status – Determine peer availability –Send status request to a peer –Peer returns a current load value Synchronization (Bootstrap only) –Keep list of other bootstrap peers –In round robin style, send all newest peers seen at this peer (HTTP Post) –Return the new peers seen

Bootstrap Synch POST with XML Peer List Return an XML Peer List Get last used time of Peer B Find all newer peers Construct XML and POST to Peer B Receive the response and add peers to our list Update the last used time of B Peer A Peer B Receive the Peer List Add peers to our list Find last used time of Peer A Gather all peers newer than time Send a Peer List Update the last used time of A

Key API Net API – Contains all server and state –Database Manager – Handles peer state –Bootstrap Manager – Synchs peer state –Peer Manager – Keep alive with other peers –Protocol Requests – Encapsulate requests Servlets – Started and forgot by Net API –One per request type –Run independently of rest of system

Architecture Core UI Net

Diagram: Metamodel

Diagram: Database

Diagram: Playback

Diagram: Net API

Diagram: Servlets

Diagram: Requests

Diagram: UI

Demonstration Normal Operation Failure of bootstrap Failure of service peer