Oxford eScience OxGrid: Virtualisation at Oxford Rhys Newman Manager of Interdisciplinary Grid Development, Oxford University Campus Grid Workshop – Edinburgh June 2005

Spare Computer Resources Yesterday’s analysis shows how the costs of a campus grid will be about £30k per week. Yesterday’s analysis shows how the costs of a campus grid will be about £30k per week. Still much cheaper than £6 million to buy the machines to do the same work, and still have to pay for the electricity!Still much cheaper than £6 million to buy the machines to do the same work, and still have to pay for the electricity! Assuming machines available with average age less than 3 years, there may be up to 500TB available Assuming machines available with average age less than 3 years, there may be up to 500TB available Still cannot quantify this figure precisely.Still cannot quantify this figure precisely. But storage is easier to manage with simpler technology, and does not have the associated running costs.But storage is easier to manage with simpler technology, and does not have the associated running costs.

Oxgrid Client A background “screen saver” to manage idle/spare resources. A background “screen saver” to manage idle/spare resources. Developed in Java for cross platform properties Developed in Java for cross platform properties A very useful attribute for heterogeneous computing environments.A very useful attribute for heterogeneous computing environments. Initially will be used to assess how much CPU time and storage can be harnessed without impacting users (machine owners). Initially will be used to assess how much CPU time and storage can be harnessed without impacting users (machine owners).

Oxgrid Client Most computers (90% or more) run Windows, so initial prototype designed for that, rather than YALA. Most computers (90% or more) run Windows, so initial prototype designed for that, rather than YALA. Oxgrid Client runs in the background and can be configured by the Oxgrid Icon in the system tray. Oxgrid Client runs in the background and can be configured by the Oxgrid Icon in the system tray. End user (host machine) acceptance: End user (host machine) acceptance: Does it interfere with normal work?Does it interfere with normal work? Does it cause any instability of the machine?Does it cause any instability of the machine? Is it easy to use and understand (provider’s perspective)?Is it easy to use and understand (provider’s perspective)?

Oxgrid Client

Oxgrid Client Advanced Schedule

Software Only: 1.Storable/Freezable 2.Relocatable over the network Emulation / Virtualisation A Virtual Machine can be frozen on command to provide OS level checkpointing! A Virtual Machine can be frozen on command to provide OS level checkpointing! A management system can monitor progress of jobs and continually match them to idle resources, moving them at will. A management system can monitor progress of jobs and continually match them to idle resources, moving them at will. host hardware host OS Emulator other software Emulated hardware Guest OS Grid Job

Fundamentals Moving a whole disk image of an OS around the network to find a CPU. Moving a whole disk image of an OS around the network to find a CPU. Storing whole disk images until free processing is available. Storing whole disk images until free processing is available. But technology is changing the design considerations. But technology is changing the design considerations. CPU x2 in 18 mths, NW x2 in 9 mths, Storage £/MB halving in 6 mths.CPU x2 in 18 mths, NW x2 in 9 mths, Storage £/MB halving in 6 mths. Systems desiged in the past are built around different assumptions – not necessarily the best suited in this new world. Systems desiged in the past are built around different assumptions – not necessarily the best suited in this new world.

OxGrid: Operation Researcher: needs lots of computer time/storage OxGrid Operation Centre Manages idle/spare resources “100 jobs in 24 hours please” Various Idle Resources Running OxGrid Client First 2 hours here Next 10 hours here 6 Hours here Final 4 hours here Need to finish in 24 hours Each job may only take 12 hours OxGrid turns many unreliable resources into one large reliable resource

OxGrid Central Management of virtualised resources around Oxford University. Central Management of virtualised resources around Oxford University. All key technologies and equipment needed already exists: All key technologies and equipment needed already exists: Networking, internet technologies.Networking, internet technologies. Virtualisation.Virtualisation. Idle machines and spare storage.Idle machines and spare storage. real users with real needs.real users with real needs. Activity does not fit into an existing department: Activity does not fit into an existing department: too much like product development for computer sciencetoo much like product development for computer science too much like product development for computer services.too much like product development for computer services.

Virtualisation as the Enabling Technology Virtualisation enables existing resources to be leveraged without complex security and software engineering constraints on users. Virtualisation enables existing resources to be leveraged without complex security and software engineering constraints on users. Virtualisation guarantees resource providers security and acceptable use by 3 rd parties (owner’s perspective). Virtualisation guarantees resource providers security and acceptable use by 3 rd parties (owner’s perspective). Core technology is open source, technique is industry standard and no longer suffers from performance issues: Core technology is open source, technique is industry standard and no longer suffers from performance issues: 10 machines at 80% is better than 1 at 100%!10 machines at 80% is better than 1 at 100%!

Interesting Opportunities With virtualisation computing becomes a liquid commodity…. With virtualisation computing becomes a liquid commodity…. A commodity can be traded….. A commodity can be traded….. A traded commodity can have derivative instruments….. A traded commodity can have derivative instruments….. Value can be derived for the main and derivative markets. Value can be derived for the main and derivative markets. What commercial interests could be here either as customers or providers? What commercial interests could be here either as customers or providers? + =

Coming soon…. If Oxgrid can use idle resources so well within Oxford, then it should be as easy to share machines between Universities……