SCI-BUS project Pre-kick-off meeting University of Westminster Centre for Parallel Computing Tamas Kiss, Stephen Winter, Gabor Terstyanszky

University of Westminster in SCI-BUS Leaders of SA3: Application and User Support Service Major contribution to task JRA2.7: Blender rendering community gateway (together with Laurea University)

Centre for Parallel Computing School of Electronics and Computer Science Research in Cluster, Grid and Cloud computing 5 academic staff, 5 researchers, 6 PhD students Well funded by European and UK research grants (nearly £1 million funding last year) Main research focus: User friendly interfaces/science gateways to Grids and Clouds Making Desktop and Service Grids interoperable (EDGeS, EDGI and DEGISCO projects) Grid workflow systems and their interoperability (SHIWA project) Application support – porting applications to clusters, Grids and Clouds (Westminster Grid Application Support Service)

The University of Westminster Local DG Over 1500 Windows PCs from 6 different campuses Lifecycle of a node: 1. 1.PCs basically used by students/staff 2. 2.If unused, switch to Desktop Grid mode 3. 3.No more work from DG server -> shutdown (green solution)

5 Service Grid Resources 96 node dedicated computing cluster (part of the UK National Grid Service)

6 The UK National Grid Service (NGS) Core members: Manchester CCLRC RAL Oxford Leeds HPCx Partner sites Cardiff Lancaster Westminster Queens University of Belfast University of Glasgow +17 affiliate sites Data clusters Compute clusters Supercomputer stable highly-available production quality Grid service for the UK research community Westminster

7 NGS P-GRADE GEMLCA Portal User friendly access to NGS resources portal website: portal.cpc.wmin.ac.uk Operated by the University of Westminster as NGS Partner Site Westminster

8 Supporting users and application developers to run their applications on the Grid developing the Grid application deploying on the Grid Grid W-GRASSsupport Requirements Domain expertise user knowledge & effort W-GRASS Westminster GRid Application Support Service

Rendering Portal Service for the Blender User Community

What is Rendering? Rendering: the process of generating an image from a model, by means of computer programs the process of calculating effects in a video editing file to produce final video output Model: description of three-dimensional objects in a strictly defined language or data structure geometry, viewpoint, texture, lighting, and shading information, etc.

The rendering problem Rendering images/animations on stand-alone PCs is very time consuming. Even relatively short rendering tasks can run for hours/days. How to speed up the rendering process? Parallelize the task and run on computer farms instead of standalone PCs frames can be rendered independently

What is Blender? 3D graphics application that can be used for rendering (amongst other tasks) Open source - GNU GPL license large community behind it actively developed under the supervision of the Blender Foundation Linux, MAC OS and Windows versions

Rendering portal implementation Based on P-GRADE portal 2.5 (latest release at the time) Totally stripped down version – very simple user interface No user certificate management - free access No settings or customisations Compute resources are completely hidden from user No workflow editor – specific workflows are created automatically Functionality: Create a new rendering task - upload input file Submit rendering workflow Download results

Rendering portal functionalities Creating new rendering job Give unique name, define frames to be rendered Select Blender input file Workflow created by pressing this button

Rendering portal functionalities Execute workflow – download results

Some statistics December 2008 – September registered user 9000 workflows 464,000 frames 28,000 CPU hours ~ 1167 CPU days 95 Gbyte input files – 180 Gbyte results Job submission on portal currently suspended to allow major upgrade

Issues addressed Dealing with the “tail” problem in Desktop Grids The problem: Whole workflow can be delayed by a few late jobs (due to job suspensions/interruptions). We term these late jobs the “tail” problem. Reduce the workflow makespan by replicating late jobs on the cluster.

Issues Addressed Security & Memory Demands The problem: Blender can execute potentially malicious Python scripts which can be embedded within input (.blend) files. Users typically requested huge memory resources on cluster nodes.

Virtualizing the DG The solution: Sandboxing. Use the Desktop Grid with Blender running in a system Virtual Machine (VM). Assign the guest OS physical memory from host and as much swap as desired (using virtual disks). Combination of Boinc and VirtualBox.