Distributed Video Rendering using Blender, VirtualBox, and BOINC. Christopher J. Reynolds. Centre for Parallel Computing, University of Westminster.

Material: The general concept and motivations. Choosing a system virtualization technology. Interactions with the hypervisor. Final comments.

Background: University of Westminster Approx 1600 node Windows & BOINC based DG. Runs applications from Enabling Desktop Grids for E-Science (EDGeS). 20 node GPU cluster under construction to run further EDGeS/DEGISCO applications. For most of 2009 we ran a public video/animation rendering service on a dedicated cluster (~256 Nodes).

Blender: Rastorization based rendering of frames; a Bag of Tasks (BoT). Fits the DG model so move from cluster to DG. Uses Blender, a well known animation/image rendering suite. Q: What are Blender’s requirements? A: Python. Q: What other considerations are required for a public service? A: Python scripts can be embedded within Blender input files (Security) Q: How to address these issues? A: Virtualization

Existing Virtualization Solutions Existing solutions use either process virtualization or system virtualization. Process virtualization (using Entropia VM) proposed by Calder et al [1] is tied to Windows as the host OS. VM Wrapper: Generic but more complex and incomplete (Windows Host Support). Marosi et al described proof of concept using QEMU based on features [2]. Q:QEMU/KQEMU now obsolete on Windows, so what next? A:Look at latest feature set and choose something else for a Lightweight solution.

QEMU/KQEMU vs VirtualBox FeatureVirtualBoxQEMUKQEMU X86 guest only.YesNoYes Requires admin privileges to run. No – Add user to allow execution of VBox DCOM service. No Requires admin to install. YesNoYes CPU performance overhead (x86) [3] 20%(SP) 10%(FP) Huge (emulation)100%(SP) 30%(FP) Supports disk overlay/cow images. Yes Guest to Host File I/O (Without Network) Yes (Guest Additions)No Monitor/Hypervisor interface. COMSockets Windows supportYesYes (Partial)Obsolete

Solution: A Lightweight VM Controller. Host/DG Node DG (BOINC) Server BOINC Core Client VM (Guest) Disk image (guest OS), VM config file, inputs. Instantiate VM and transfer inputs React to messages: suspend, resume, abort Transfer outputs VM Controller (standard BOINC job) Start VBox Hypervisor Transfer outputs

VirtualBox Hypervisor/Monitor Control VM Controller (C++ Boinc application) Win32 CreateProcess() or fork() to VboxManage cmdln tool. COM preferable. (COM)

VM Controller Two Threads. Worker launches unzipper (lzma), prepares VM config files, creates I/O shared directories, polls for Job completion. Monitor thread deals with messages and VM control. VM Controller (C++ Boinc application)

Worker Thread

Monitor Thread

VM Worker Script: Blender echo 15 > /proc/$$/oom_adj echo "Setting oom_adj to 15 for PID=$$" case "$1" in start) mkdir /mnt/share mount -t vboxsf Shared /mnt/share mkdir /mnt/share/Outputs swapon /dev/sda swapon /dev/sdb START_FRAME=`cat /mnt/share/input.txt | head -1 ` END_FRAME=`cat /mnt/share/input.txt | head -2 |tail -1 ` sudo –u blender_user sh -c "blender -b /mnt/share/input.blend -o /mnt/share/Outputs/Frame-# -F PNG -x 0 -s $START_FRAME -e $END_FRAME –a” echo $? > /mnt/share/Outputs/blender_return.log echo "Finished Rendering Application" ;;

Further Comments We adhere to the “one VM per Workunit” model. Our BOINC user account has restricted permissions. This means VirtualBox.xml and MachineDefinition.xml are loaded from C:\Windows\system32\config\systemprofile\.VirtualBox Single xml file location restricts us to one WU per node. Proof of concept, though proposed roll out soon (late 2010).

References [1]Calder, B., Chien, A., & Yang, J. W. D. (2005). The entropia virtual machine for desktop grids. In VEE ’05: Proceedings of the 1st ACM/USENIX international conference on Virtual execution environments (pp. 186–196). [2]Marosi, A., Kacsuk, P., Fedak, G., & Lodygensky, O. (2010). Sandboxing for Desktop Grids using Virtualization. In The 18th Euromicro International Conference on Parallel, Distributed and Network-Based Computing. [3] P. Domingues and F. Araujo and L. Silva. (2009) Evaluating the performance and intrusiveness of virtual machines for desktop grid computing. In IPDPS '09: Proceedings of the 2009 IEEE International Symposium on Parallel & Distributed Processing. Questions?