1. Background Info The UK Mirror Service provides mirror copies of data and programs from many sources all over the world. This enables users in the UK to get mirrored files faster by downloading from a local source rather than transferring the file over a greater distance. It also reduces the load on the original source and means the files are still available even if the source is overloaded or down. The mirror service currently allows users to download software using http via the web interface, ftp or the rsync protocols. Our project is to create a custom bittorrent client to allow content on the mirrorservice to be “seeded” at high speed to other bittorrent users. Requirements Dependable – The solution must be resilient to cope with a high load and large number of connections Self Sufficient – The solution must be automated where possible to reduce admin overheads Scalable – The solution is required to run on 2 machines for load balancing and redundancy, with more possibly being added in future. Simple – The solution must be easy to set up and configure Secure – Parts of the solution will be accessible from the web and must therefore be as secure as possible. Maintainable – The solution must be written in a language the mirror service administrators are familiar with using supported libraries. Design Decisions Separate Processes – We decided to have multiple programs running simultaneously communicating via a message passing server. This solves several of our requirements. Dependable - If one program fails the others carry on running. For example if the program “seeding” $file crashed then all other files would still be available. Scalable – Programs can run on separate machines, communicating via a central server. Automated – New torrents are vetted on several criteria, torrents passing vetting need to be approved by an admin. After that the torrent will automatically be added and bandwidth re-balanced between all seeded files. Torrents are automatically removed after a substantial period of inactivity or when they get too old. Simple – All configuration files are stored as plain text using a clear syntax for easy editing by an administrator. The admin program can easily be scripted to enable further automation if necessary. Maintainable – Code is held in a subversion repository so that we have better control over our versions and releases. Alerts are sent to us by email when code is committed. This means the latest version of the code is always readily available. Spawned by Overlord Figure 1 Admin:: Command line interface for administrators of system Stats:: Module to control demand for torrent statistics Overlord:: Controls the handling of the torrents. (Starting, stopping etc) LCP:: Process that handles the specifics of seeding a torrent Vetting:: Checks torrents for illegal content Watcher:: Module to watch a folder. If a new torrent arrives module alerts system Server:: Communications hub – deals with all the internal messages between processes BitTorrent is a protocol designed for transferring files. It is peer-to-peer in nature, as users connect to each other directly to send and receive portions of the file. The key philosophy of BitTorrent is that users should upload at the same time they are downloading. In this manner, network bandwidth is utilized as efficiently as possible. BitTorrent is designed to work better as the number of people interested in a certain file increases, in contrast to other file transfer protocols. Overlord :: Showing download stats Team Adam Bagnall (ajb35) Adam Daley (ald7) Robert Jones (rmj2) Simon Rose (sr89) Ricardo Solano (rs86) Supervised by Phill Camp (P.S.S.Camp) Tim Hopkins (T.R.Hopkins) Admin :: Sample python code