Transcript Slide 1

Distributed Video Rendering using Blender, VirtualBox,
and BOINC.
Christopher J. Reynolds.
Centre for Parallel Computing,
University of Westminster.
Material:
•
The general concept and motivations.
•
Choosing a system virtualization technology.
•
Interactions with the hypervisor.
•
Final comments.
Background: University of Westminster
•
Approx 1600 node Windows & BOINC based DG.
•
Runs applications from Enabling Desktop Grids for E-Science (EDGeS).
•
20 node GPU cluster under construction to run further EDGeS/DEGISCO
applications.
•
For most of 2009 we ran a public video/animation rendering service on a
dedicated cluster (~256 Nodes).
Blender: http://www.blender.org
•
Rastorization based rendering of frames; a Bag of Tasks (BoT).
•
Fits the DG model so move from cluster to DG.
•
Uses Blender, a well known animation/image rendering suite.
•
•
Q: What are Blender’s requirements?
A: Python.
•
•
Q: What other considerations are required for a public service?
A: Python scripts can be embedded within Blender input files (Security)
•
•
Q: How to address these issues?
A: Virtualization
Existing Virtualization Solutions
•
Existing solutions use either process virtualization or system virtualization.
•
Process virtualization (using Entropia VM) proposed by Calder et al [1] is
tied to Windows as the host OS.
•
VM Wrapper: Generic but more complex and incomplete (Windows Host
Support).
•
Marosi et al described proof of concept using QEMU based on features
[2] .
•
•
Q: QEMU/KQEMU now obsolete on Windows, so what next?
A: Look at latest feature set and choose something else for a Lightweight
solution.
QEMU/KQEMU vs VirtualBox
Feature
VirtualBox
QEMU
KQEMU
X86 guest only.
Yes
No
Yes
Requires admin
privileges to run.
No – Add user to
allow execution of
VBox DCOM service.
No
No
Requires admin to
install.
Yes
No
Yes
CPU performance
overhead (x86) [3]
20%(SP)
10%(FP)
Huge (emulation)
100%(SP)
30%(FP)
Supports disk
overlay/cow images.
Yes
Yes
Yes
Guest to Host File I/O
(Without Network)
Yes (Guest Additions)
No
No
Monitor/Hypervisor
interface.
COM
Sockets
Sockets
Windows support
Yes
Yes (Partial)
Obsolete
Solution: A Lightweight VM Controller.
Host/DG Node
Disk image (guest OS), VM config
1
file, inputs.
DG (BOINC)
Server
BOINC Core Client
6
2 Start
Transfer
outputs
VM Controller (standard BOINC job)
3
Transfer 5
outputs
4
React to messages:
suspend, resume,
abort
VBox Hypervisor
VM (Guest)
Instantiate VM
and transfer
inputs
VirtualBox Hypervisor/Monitor Control
VM Controller (C++ Boinc application)
•
•
(COM)
Win32 CreateProcess() or
fork() to VboxManage cmdln
tool.
COM preferable.
VM Controller
VM Controller
(C++ Boinc application)
•
•
•
Two Threads.
Worker launches unzipper (lzma), prepares VM config files, creates I/O
shared directories, polls for Job completion.
Monitor thread deals with messages and VM control.
Worker Thread
Monitor Thread
VM Worker Script: Blender
echo 15 > /proc/$$/oom_adj
echo "Setting oom_adj to 15 for PID=$$"
case "$1" in
start)
mkdir /mnt/share
mount -t vboxsf Shared /mnt/share
mkdir /mnt/share/Outputs
swapon /dev/sda
swapon /dev/sdb
START_FRAME=`cat /mnt/share/input.txt | head -1 `
END_FRAME=`cat /mnt/share/input.txt | head -2 |tail -1 `
sudo –u blender_user sh -c "blender -b /mnt/share/input.blend -o /mnt/share/Outputs/Frame-# -F PNG -x 0 -s
$START_FRAME -e $END_FRAME –a”
echo $? > /mnt/share/Outputs/blender_return.log
echo "Finished Rendering Application"
;;
Further Comments
•
We adhere to the “one VM per Workunit” model.
•
Our BOINC user account has restricted permissions.
•
This means VirtualBox.xml and MachineDefinition.xml are loaded from
C:\Windows\system32\config\systemprofile\.VirtualBox
•
Single xml file location restricts us to one WU per node.
•
Proof of concept, though proposed roll out soon (late 2010).
References
[1] Calder, B., Chien, A., & Yang, J. W. D. (2005). The entropia virtual machine for
desktop grids. In VEE ’05: Proceedings of the 1st ACM/USENIX international
conference on Virtual execution environments (pp. 186–196).
[2] Marosi, A., Kacsuk, P., Fedak, G., & Lodygensky, O. (2010). Sandboxing for
Desktop Grids using Virtualization. In The 18th Euromicro International
Conference on Parallel, Distributed and Network-Based Computing.
[3] P. Domingues and F. Araujo and L. Silva. (2009) Evaluating the performance and
intrusiveness of virtual machines for desktop grid computing. In IPDPS '09:
Proceedings of the 2009 IEEE International Symposium on Parallel & Distributed
Processing.
Questions?