Tycho: A General Purpose Virtual Registry and Asynchronous Messaging System Matthew Grove ACET Invited Talk February 2006

1 Outline Introduction and motivation, The architecture of Tycho, Implementation details, Benchmarking, Developing services for online games, Summary, Demo.

2 Introduction Tycho is a reference implementation of a combined extensible wide-area messaging framework with a built in distributed registry. –Tycho follows the publish-subscribe paradigm. –It provides services to allow produces and consumers to discover each other using a Virtual Registry (VR) made up of a network of mediators and then aid communication over both LAN and WAN. Tycho aims to simplify and speed application development by freeing developers from the need to use a combination of middleware to provide service discovery and messaging.

3 Design Philosophy Reuse existing software components, if possible, rather than reinvent functionality. Try to make use of existing infrastructure. Make Tycho simple to install, configure and use. Provide a ‘basic release’ with a choice to extend functionality with a further more sophisticated component.

4 The Two Parts of Tycho Message passing: –Provides asynchronous communications between Consumers, Producers and Mediators. Virtual Registry: –Boot-strapping: Allow Tycho mediators to discover each other and form the VR with minimal hardwiring. –Communications: Efficient routing of queries between Tycho Virtual Registries. –Caching: Keeping a temporary local copy of some information to reduce the amount of communications between peers.

5 Tycho’s Architecture

6 Tycho Implementation Java was used for portability and interoperability with other distributed systems, plus rapid development. The message passing services were implemented first as some of their functionality is used by the VR: –An initial prototype was developed over a few days using Java RMI. –This has been iteratively optimised and redeveloped to reduce latency and memory use, currently we use byte arrays for all data and Java sockets for communications.

7 The Tycho Library and API We provide a choice of implementations for each core service.

8 Tycho Core Services Transport handler, allows different protocols to be used for communications (HTTP, Sockets, IRC). Local store, for a mediator and VR information (JDBC, Java simple store). Boot service, used by the VR within a mediator to locate and join the rest of the VR (HTTP, IRC). Query Parser and result annotator, to support different query and markup languages (SQL, LDIF).

9 Example Tycho Setup

10 Tycho Benchmarks We performed two rounds of benchmarking: –Message Passing: These tests measured the performance of inter-client and inter-mediator communications for Tycho and NaradaBrokering. –Virtual Registry: These tests measured the performance of the VR (query routing and processing) in a range of different configurations:

11 Message Passing Tests (A) We measured the latency of communication for LAN and simulated WAN messaging between two clients varying message size (ping-pong tests). Then we measured the affect on scalability of message latency using many producers publishing to a single consumer and single consumer consuming from multiple producers. We used the DSG cluster to run the tests.

12 Ping-Pong - Latency Latency vs Message Length

13 Message Passing Tests - Summary Tycho achieves comparable or lower latency than NaradaBrokering in all of these tests. The performance of both systems does not drop off after a producer or consumer has become saturated. NaradaBrokering needs the JVM heap size to be increased as the number of clients increases (due to internal buffers): – Tycho used the default heap for all of the tests.

14 Virtual Registry Tests (B) We used 4 different configurations to measure the performance of the Tycho VR. For each test we varied one or more aspect of the configuration and measured the effect on client query latency: Number of records in a mediator (100,000 records), Number of simultaneous client queries (1000 clients), Numbers of mediators in the VR (1000 mediators), Different VR-interconnects between mediators. In order to make some comparisons between Tycho and R-GMA, MDS4, MDS2 and Hawkeye we attempted to reproduce tests used in a related paper.

15 Virtual Registry Tests - Summary The Java simple store performs best for small numbers of records, but HSQLDB and MySQL scale better with registry size. The response time increases linearly with number of clients. For routing queries in the VR, IRC has better performance than HTTP. As response size increases HTTP performs much better than IRC, this is due to the IRC code having to send multiple messages.

16 Comparison Tests Aims to compare Tycho’s VR to R-GMA, MDS4, MDS2 and Hawkeye. The tests measure the impact of increasing the number of clients simultaneously querying a registry containing 10 records. The set up was similar to a benchmark we had already performed but we altered some things to make it as close a match as possible: –We used 5 compute nodes instead of 8. –Each system has different sized responses which we matched by adding padding to Tycho registry information. –The main difference is CPU (2.8 Ghz vs 1.1Ghz).

17 Comparison Tests (2K) Response Time vs Number of Clients (2 Kbyte Responses)

18 Comparison Tests (10K) Response Time vs Number of Clients (10 Kbyte Responses)

19 Comparison Tests (19K) Response Time vs Number of Clients (19 Kbyte Responses)

20 VR Comparison Tests – Summary Tycho shows linear increase in response times regardless of message size and number of clients. Tycho had lower response times than R- GMA, MDS4 and MDS2 without caching. MDS2 with caching is basically OpenLDAP - which is mature and performs well. Hawkeye performs better than Tycho for less than 100 clients, but it does not scale well with larger number of clients.

21 Tycho Core – Future Work Some performance improvements: – Caching of local mediator queries to reduce response times, – The Simple store could be improved by making it more intelligent, – We could use a hybrid VR-interconnect to use IRC for query routing and HTTP for transporting large responses. More functionality can be added to provide more advanced services: – HTTPS and SSL Sockets for secure communications, – Support for transactions.

22 Tycho Applications We feel strongly that it is necessary to develop real world applications to further validate the implementation. A number of increasingly sophisticated applications have been developed and implemented: –Publishing and discovering Web-cams, –Remote resource discovery for the VOTechBroker project, –Binding components of the Slogger project together to publish system logs for analysis using Semantic Web technologies. An area that will stress and probably push any message-oriented middleware to its limits is online gaming.

23 Distributed Gaming Online distributed gaming has become increasingly popular with the wide spread uptake of broadband in homes. Games publishers have each tried to provide an infrastructure to support their online games. There is an opportunity to develop a standard set of services, based on a completed Tycho implementation, to support these games.

24 Distributed Gaming Services Discovery: The player requires a scalable service to index large numbers of game servers and allow searches based on user-defined parameters such as the current map being played. Content distribution: Scalable infrastructure to provide new or updated content from the content provider to the players. Single sign-on: Providing a unique identity for each player makes global high score tables and other statistic tracking possible: –User profiles (key bindings) could be stored server side to allow roaming.

25 Summary The initial reference implementation has been completed. Both the message-passing code and VR have been benchmarked and appears to perform well. The focus now is to review the current state of online game services and select a game framework to enhance using Tycho-based services.

26 Webcam Browser Demo

27 Project Web page: – The DSG Web page: – Links