Distributed Search with Rendezvous Search Systems Christof Leng Dipl.-Inform. Christof Leng Databases & Distributed Systems Department of Computer Science Technische Universität Darmstadt

Distributed Search: A Traditional View Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet" 2 | |  Implement distributed key-value lookup  e.g., a distributed hash table (DHT)  scalable & simple  Build everything else on top of it  keyword search, XPath, range queries  map every operation to a number of lookups  Downside:  implement every query language yourself  query processing not portable between DHTs  unforeseen side-effects & interactions  Experts needed for every application  application domain  P2P networking  information retrieval Application Developer P2P Networking Expert Information Retrieval Expert … need sophisticated queries …

Rendezvous Search  “Every query will meet each data item somewhere in the network”  don’t worry how it gets there  black box approach  Implement the match functionality  as if it was local  re-use existing libraries  Advantages:  separation of concerns  flexible  scalable (O(√n) search cost)  robust (O(√n) replicas) | | Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet" 3 Application Developer Rendezvous Search System Information Retrieval Library … need sophisticated queries … glue code

Example Application Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet" 4 | | P2P Wiki Lucene Keyword Search Lucene Keyword Search Rendezvous Middleware Overlay Network

Rendezvous Search Genealogy | | Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet" 5

Data Center Solutions | | Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet" 6

Grid Pro:  simple  efficient  flexible Con:  not fault-tolerant  not size-adaptive  not load-adaptive  two-dimensional structure Reference: L. A. Barroso, J. Dean, and U. Hölzle. Web Search for a Planet: The Google Cluster Architecture. IEEE Micro, 23(2):22–28, | | Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet" 7 Q Q Q Q Q Q Q Q Q Q Data Query D D D D D D D D D D D D Q Q R R Q Q Q Q Q Q Q Q Q Q Data Query D D D D D D D D Q Q R R Q Q Q Q Q Q

ROAR – Rendezvous on a Ring Pro:  simple mapping to 1 dim.  size-adaptive  load-adaptive  fault-tolerant Con:  centralized or  inefficient Reference: C. Raiciu, F. Huici, M. Handley, and D. S. Rosenblum. ROAR: Increasing the Flexibility and Performance of Distributed Search. In Proceedings of SIGCOMM’09, pages 291–302, ACM. | | Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet" 8

Structured Peer-to-Peer | | Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet" 9

Bit Zipper Pro:  z-order mapping  DHT-based = decentralized  fast tree-based replication Con:  relies on DHT’s reliability  no replica maintenance Reference: W. W. Terpstra, S. Behnel, L. Fiege, J. Kangasharju, and A. Buchmann. Bit Zipper Rendezvous—Optimal Data Placement for General P2P Queries. In EDBT’04 Workshop on Peer-to-Peer Computing and Databases, | | Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet" 10

Deetoo Pro:  2 rings for projection  includes replica maintenance Con:  maintenance overhead for 2 rings  no responsibility ranges (probabilistic approach)  additional overhead Reference: T. W. Choi and P. O. Boykin. Deetoo: Scalable unstructured Search built on a Structured Overlay. In HOTP2P - International Workshop on Hot Topics in Peer-to-Peer Systems, Los Alamitos, CA, USA, IEEE Computer Society. | | Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet" 11

Unstructured Peer-to-Peer | | Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet" 12

Ferreira et al. – Random Walks Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet" 13 | | Pro:  very resilient topology  proven correctness Con:  probabilistic  additional overhead  slow & fragile random walks  no topology protocol Reference: R. A. Ferreira, M. K. Ramanathan, A. Awan, A. Grama, and S. Jagannathan. Search with Probabilistic Guarantees in Unstructured Peer-to-Peer Networks. In Proceedings of P2P’05, pages 165–172, Washington, DC, USA, IEEE Computer Society.

BubbleStorm Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet" 14 | | Pro:  resilient & fast  size-adaptive  load-adaptive  topology & replica maintenance included Con:  probabilistic  additional overhead Reference: W. W. Terpstra, J. Kangasharju, C. Leng, and A. P. Buchmann. BubbleStorm: Resilient, Probabilistic, and Exhaustive Peer-to-Peer Search. In Proceedings of SIGCOMM’07, pages 49–60, New York, NY, USA, ACM.

Hautakorpi & Schultz Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet" 15 | | Pro:  unstructured solution on top of DHT  incremental deployment possible Con:  probabilistic  additional overhead  fragile Reference: J. Hautakorpi and G. Schultz. A Feasibility Study of an Arbitrary Search in Structured Peer-to-Peer Networks. In Proceedings of 19th International Conference on Computer Communications and Networks (ICCCN), pages 1–8. IEEE, Aug

Semi-Structured Peer-to-Peer | | Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet" 16

SplitQuest Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet" 17 | | Pro:  combines unstructured resilience with structured efficiency Con:  probabilistic  lacks analysis and prototype evaluation Reference: P. Lopes and R. A. Ferreira. SplitQuest: Controlled and Exhaustive Search in Peer-to-Peer Networks. In Proceedings of the 9th International Conference on Peer-to-Peer Systems, IPTPS’10, Berkeley, CA, USA, USENIX Association.

Comparison Data CenterStructured NameGridROARBit ZipperDeetoo ProsimpleadaptiveDHT add-onmaintenance Connot adaptivecentralizedmaintenance unclearprobabilistic | | Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet" 18 UnstructuredSemi-Structured NameFerreiraBubbleStormHautakorpiSplitQuest Prosimple resilient & adaptive increm. DHT add-onefficient & resilient Confragile & slowprobabilistic immature

Q Q Q Q Q Q Q Q Q Q Data Query D D D D D D D D Q Q R R Q Q Q Q Q Q Rendezvous Search & Cloud Computing Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet" 19 | |  similar problems  similar requirements  less research history could benefit from  size adaptivity  traffic adaptivity  fault tolerance  algorithm diversity Sharding Key-Value Store Scalability Replication D D D D D D Q Q Q Q Q Q M M Q Q Q Q Q Q

Conclusion Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet" 20 | |  Rendezvous Search is a powerful abstraction  scalable  flexible  robust  A wide range of solutions is out there  data center  unstructured P2P  structured P2P  semi-structured P2P  Ever needed decentralized but powerful search in a project?  don’t try to reinvent the wheel!

Thank you! ? Questions | | Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet" 21