A Gateway Publish Subscribe Design Architecture for Scalable Plug-In Optimization Jason Bryant System Capacity Sub Linear Super Linear Linear Performance

Objectives Scalability Peer to Peer Gateway Server implementation Plug-in Dissemination Protocols to Allow for Extensibility Security Approaches Distribute Centralized Server Costs Open Source Tool Implementation Encryption and Authentication Future Work Iterate to PhD via Expansion Isolate Research Opportunities Scalability Fixed Cost Designs Dissemination Protocols

Purpose Create the first iteration in a distributed publish and subscribe based system which relies on peer-to-peer gateway servers and plug-in optimizations. Determine areas of research interest.

Existing Solutions Publish Subscribe CLIENT CLOUD Server (Data Broker) Client AClient BClient AClient B

Existing Solutions Peer-to-peer

Data Popularity Centralized Publish and Subscribe: Suited for Low Popularity Low Scalability Peer-to-Peer: Suited for High Popularity High Overhead for Low Popularity Objects Popularity is Based on the Level of Match for Published Data to the Filters of all Subscribers

What is a Gateway Conceptual server Mirrored services Brokering for all connected client Distributed metadata channel Peer-to-Peer Features Data Sharing Data and metadata advertisements

Scalability Solutions Areas of bottlenecks Data Dissemination – Pull based One means of low-popularity distribution One means of high-popularity distribution Brokering – YFilter or other such super-scalable algorithms Bandwidth – Peer-to-peer for large objects and for popular objects (highly efficient) Processing – Peer-to-peer distributes overhead costs. Metadata Channels allow distribution Storage – Distribute using decentralization (TerraCotta and other clustered on-demand systems) In Memory databases Security One time authentication Pre-processing hashes for valid clients to unlock downloaded content

Coding Interfacing for Plug-in Support Abstract Connections Storage Transfer protocols Client and server implementations Core Protocol Handshake – Encryption and Authentication Subscribe – Filter Attachment MetadataAdvertisement – Inter-Gateway Advertisement MetadataRequest – Metadata Request. Inter-Gateway only. Metadata – Publish Initialization and Inter-Gateway Distribution DataAdvertisement – Ownership Receipt using objectID and data segments DataBlockRequest – Pull-based Download Request. Maximum limit at any one time per connection DataBlock – Object Data Segment sent as Response to Request InterestNotification – Gateway to End-Client Notification. Client begins actively seeking out data for object SimulationStart – Experimental Mode Only. Used to trigger simulations

Code Examples Refer to handout

Open Source and other Tools Learned and Used AIO4J Ember Keystores (SHA-1 with DSA) Authentication and Access Control (Password digests for handshakes) Berkeley Java DB Filter Brokering – Property Based

Future Work Areas of Research Experimentation Scalability Performance of Individual Plug-ins and Overall System Gateways Blur lines – Gateway Volunteers and Nominations Clients Fixed Cost downloads (Real-time computing) Long Tail vs. Heavy Tail Distributed Distributed Hash Tables for Peer Lookup Peer to peer tiers of data sharing Scalability and Low/High Popularity Optimization Interest Repositories Gateway Level Peer Level

Future Iterative Design

Conclusion Scalability Increased Number of Clients Throughput of Data Working Product Created Two Opposing Capabilities Combined in a Novel Way Future Research Areas Bounded

Time Spent on Project 25 Hours Research and Outlining 85 Hours Development and Testing 20 Hours on Write-ups and Overhead Total: 130 Hours

Thank you for your time Please ask any questions