Transcript Supporting Simulations on the Cloud using Workflows & Virtual

Supporting Simulations on the
Cloud using Workflows & Virtual
Machines
Gary Polhill
Macaulay Land Use Research Institute
Edoardo Pignotti
Computing Science, University of Aberdeen
Overview
 The SwarmCloud project
 Social simulation & the Cloud
 A virtualised simulation environment
 Facilitating access with workflows
 Workflows on the Grid/Cloud
 Conclusions
The SwarmCloud Project
 Goal: to enable the FEARLUS simulation model to be
run on National Grid Service infrastructure
 What’s so hard about that?
 FEARLUS is written in Obj-C for the Swarm libraries—
notoriously difficult to install
Numbers of pre-requisite libraries
 Can be sensitive to particular versions of libraries and Unix
commands

 Typical Grid use-cases involve running a script
 e.g. Model calibration
 Other software environments for social simulation
exhibit the same issues (barrier to adoption)
How can social simulation benefit from the
Cloud?
 Enabling large-scale distributed simulations
 e.g. Fraser et al. (2009 Science), Birkin et al. (2009
SSCoRe), Tang & Bennett (2009 US-IALE)
 More challenging because of managing agent
interactions across node boundaries
 Enabling multiple runs of smaller-scale models for:
 Exploring parameter space
 Experimenting with algorithmic variants
 Trying different scenarios
 Increasing sensitivity (more runs with different seeds)
 ‘Embarrassingly Parallel’
Software dependencies
 Swarm software
dependencies shown
 ‘worst-case’
 Java not immune
 Pre-requisite jars can be
required (with version
sensitivity)
 JRE version sensitivity
 Obvious problems for
programs/libraries using
JNI
The solution: A virtualised simulation environment:
simulationBox
 We used a lightweight version of Centos 5.2 to create
a virtualised simulation environment containing
Swarm, MASON and RePast
 Compressed image ~1GB in size
 http://www.simulationbox.net/
 The simulation environment was created using Sun’s
VirtualBox virtualisation software
 Binaries available under a Personal Use and Evaluation
Licence
 http://www.virtualbox.org/
Virtualisation on the Cloud
 The virtualisation approach relies on the Cloud host
making virtualisation facilities available
 The National Grid Service is testing a virtualisation
service based on Xen/EUCALYPTUS at Oxford
 Xen: Open Source standard for virtualisation
 http://www.xen.org/
 EUCALYPTUS (Elastic Utility Computing Architecture
Linking Your Programs To Useful Systems): Open-source
system for implementing clouds
Provides an authentication layer, and manages virtual
machines using ec2 tools
 http://www.eucalyptus.com/

Facilitating access: Workflows
 Virtualisation provides a replicable environment
for compiling and running a social simulation
 ‘User interface’ not very friendly
 Access via ssh
 Kepler workflow tool used to design simulation
experiments
Upper Deeside Case-study
Workflows on the CLOUD
 We have developed a number of Kepler
workflow activities based on the ec2 tools:
 Authenticate a user on the Cloud
 Upload simulationBox to a node manager
 Create instances of simulationBox on the Cloud
 Query for available simulationBox instances on the Cloud
 Compile a new simulation model on a simulationBox instance
 Run the model multiple time across different VM instances
 Collect and analyse results, saving them on the local machine
 Our Kepler libraries can be downloaded from
www.simulationbox.net
Compile a simulation model
Run an experiment
Conclusions
 For Cloud infrastructure supplying virtualisation
services it is possible to run social simulations with
specific computing requirements
 Access is facilitated through workflows that can be
shared amongst people
 Website: www.simulationbox.net
 ESSA 2009 Tutorial on simulationBox
 Large Scale Social Simulation
Special Interest Group (ESSA)
 SwarmFest 2010