Transcript MatlabCompilingWorkshopPowerPoint

MATLAB Compiling &
Running on CBI Cluster
®
May 25th, 2012 By: CBI Development Team
Overview
The Matlab Compiler & Deployment Tool
Compiling benefits
Designing your code for compilation
Compilation example( Example workflow )
Running compiled .exe on CBI cluster
Hands-on training

®
Compiler Toolbox Overview






Framework to package .m code as a native executable or shared
library.
Matlab® Compiler Runtime(MCR) provides access to Matlab®
functionality from the native operating system executable wrapper or
from a shared library.
No royalties/Matlab® licenses needed to use the exe + MCR
infrastructure.[30,31]
Multi-platform ( Linux, Windows, Mac )
Deployment Tool is a GUI front-end to the Matlab® compiler + build
tools command line (mcc,mbuild).
Most functions can be deployed. Some exceptions apply, mostly due
to toolbox functions that generate .m code dynamically at runtime.
Compiler Toolbox Overview

Version 4 of the Compiler Toolbox:
– An encrypted .m file + C wrapper + run-time interpretation
model[7],[9]
– Executable wrapper calls functionality within Matlab®
Compiler Runtime shared library.
– A great resource for enabling high throughput computing
with Matlab® codes, allowing many distributed jobs to be
submitted to the CBI cluster without using Matlab®
licenses.
– Performance: Should expect similar runtime to running
Matlab® from the command line.[7]
– GUI interface called Deploytool.
Deploytool
File--> New--> Deployment Project
Allows you to select
files to deploy
Deploytool is a GUI Tool that simplifies the process
of creating executable projects.
Key Benefits Summary
Minimize Matlab® + Toolbox license utilization in a grid setting

Deploy to users without a Matlab® license(MCR Installer +
Your Deployed Code)

Enable High Throughput Computing solutions

Intellectual property: Deploy proprietary closed source
algorithms securely

Deploy to multiple platforms( Linux, Windows, Mac)

Ability to use functionality from most toolboxes

Leaves open the possibility of speed improvements by
incorporating custom C/C++ code via the MEX interface.

Code Design:
Background on Internal Infrastructure
.c/.cpp
.m
mcc,mbuild,gcc
ELF_64
format
Executable
With
CTF
Archive
data
libmwmclmcrrt.so.7.15
Interface to shared modules:
MCLMCR,MCR, ...
The above diagram is based on research from the following sources: [3],[4],[17],[18]
Linux uses the
ELF format file
to create
a process
Code Design:
Background on Internal Infrastructure
Code Design Points

Convert your script code into functional form, or wrap your script with
a function.

Argument processing: use str2num function for numeric parameters

Output handling: Only 2 ways to get your output
– Standard output or File I/O



Toolbox utilization: Check whether your toolbox functions are
deployable.[13]
Must compile on the same OS version where software to be deployed
Do you dynamically generate Matlab® code? Generate your dynamic
code and then compile different versions of the dynamic code to
enable compiling your project.
Code Design Points


Avoid using relative paths, instead allow for user to provide absolute
path as a parameter or to provide for an absolute path to a parameter
file.
Using .mat data files within your deployed project: You can embed the
.mat data files within the Component Technology File (CTF) archive.
– RuntimeFolder = ctfroot;


Avoid calling help, edit, doc commands in compiled code( See
Compiler Toolbox User's Guide for more info )[1]
You can display figures in compiled version, however make sure to
disable figure display for running in batch mode on the cluster.
Example Workflow




Create a top level
function for your
project
Note how in the noncompiled
environment,
Matlab®
automatically treats
rows and cols as
numeric data types
In the compiled
version we can only
get results via
standard output or
via a file.
Will this work in the
deployed standalone
executable?
Why/Why Not?
Example Workflow

Key changes:

1) isdeployed check

2) str2num


3) sending results to
standard output
4) We could also have
sent the results to a
file... more on this later.
Example Workflow

Go to File, New, Deployment Project

Then, create a new deployment project
Example Workflow

When creating a deployment project, there are a few options:
– Standalone Application, C/C++ Shared Library, Java Package
– For grid deployment @ CBI Cluster, you will normally use the
Standalone Application option.
Example Workflow


Normally, the only file that needs to be added is the top level
function for the project.
Many more things can be added to the CTF archive in more
advanced setups.
Build
Button
Example Workflow

After the top-level function file is added, click on the build button.
The Build window will pop-up, and 2 directories will be created
within the current working directory: distrib, and src directories
Example Workflow
Packaging: Allows the inclusion of the MCR Installer within a single
installable package. However, the MCR is over 200 MB. This stage
is not needed to run @ CBI since the MCR is already installed at:
/share/apps/matlab2011a
Creating a package with the MCR installer included is a good idea, in
case you want to deploy it to a system that does not have the MCR.
( Note: As of R2012a, MCR can be downloaded directly from Mathworks @
http://www.mathworks.com/products/compiler/mcr/index.html )
Example Workflow:
Running Standalone exe @ CBI



How is the exe called?
Is it called directly?
An auto-generated .sh
script is created during
the deployment process.
It sets up the the
environment variable
containing the path to
the MCR shared library(
In Linux, its the
LD_LIBRARY_PATH
environment variable )
The wrapper script sets
environment variables
and then calls the
executable.
Example Workflow:
Running Standalone exe @ CBI

Creating a job script for the Sun Grid
Engine @ CBI


The script takes the 1st
parameter given to it and uses
it to update the
LD_LIBRARY_PATH
environment variable.
It then passes subsequent
parameters to the executable
as strings. ( Remember about
why we need to use str2num
in the Matlab® code )
Hands-On Example
Hands-On Example




High Throughput Mandelbrot
Set Image Generation via
Deployed Executable
Matlab® Application + Sun
Grid Engine running @ CBI
Cluster.
Compile as Standalone
executable
Submit multiple jobs to the
CBI Cluster, each with a
different set of parameters.
The Sun Grid Engine can
then schedule these jobs to
run concurrently.
Hands-On Example

High Throughput Mandelbrot Set image generation:
– This batch processing pattern is used by many types of
workflows.
– Running the same code with different
parameter sets.

Code Design Overview for Compilation
– Using parameter files
– Saving data as images

Deploying the Matlab® code packaged as a standalone executable

Running from Linux command line

Using the Sun Grid Engine to a submit a single job using the executable

Using the Sun Grid Engine to submit a set of jobs using the job array feature

We've successfully helped a number of users greatly reduce their project runtime expectations via this
Standalone executable + Grid deployment workflow. Can make infeasible projects possible!
CBI Laboratory
http://cbi.utsa.edu
Development team:
Zhiwei Wang, Director
David Noriega, Yung Lai, Jean-Michel Lehker, Nelson Ramirez
References
[1]http://www.mathworks.com/help/pdf_doc/compiler/compiler.pdf( Product documentation )
[2]http://www.mathworks.com/help/toolbox/compiler/bsfez67.html( dynamic code generation not supported )
[3]http://www.mathworks.com/help/toolbox/compiler/bsfey7f.html ( diagram#1, build process )
[4]http://www.mathworks.com/help/toolbox/compiler/f2-995712.html (diagram#2, build process )
[5]http://www.mathworks.com/help/toolbox/compiler/f13-1003481.html ( preferable to use a functional form )
[6]http://www.mathworks.com/support/solutions/en/data/1-QXFMQ/index.html?product=CO&solution=1-QXFMQ ( addpath info )
[7]http://www.mathworks.com/support/solutions/en/data/1-1ARNS/index.html?product=CO&solution=1-1ARNS ( performance vs regular matlab, changes from version
3 to 4)
[8]http://www.mathworks.com/help/toolbox/compiler/bs_kqkf.html( Compiler toolbox also works for MacOS)
[9]http://www.mathworks.com/help/toolbox/compiler/rn/bqnylfk-1.html (Version 4 differences from Version 3 of the Compiler Toolbox )
[10]http://www.linuxjournal.com/content/embedding-file-executable-aka-hello-world-version-5967( You can embed data within executables )
[11]ttp://www.mathworks.com/help/toolbox/compiler/rn/bqnylfk-1.html#f6-52806( Compiler version 4 overview )
[12]http://www.mathworks.com/products/compiler/requirements.html( Compiler supported on Linux,Mac,Windows )
[13]http://www.mathworks.com/products/compiler/compiler_support.html ( Compiler toolbox support information, cannot be compiled info for each toolbox )
[14]http://www.mathworks.com/support/compilers/R2011b/maci64.html#matlab ( Supported native compilers )
[15]http://www.mathworks.com/help/toolbox/compiler/btcdq67.html ( licensing terms of compiled apps )
[16]http://www.mathworks.com/help/toolbox/compiler/br2cqa0-20.html ( unsupported functions )
[17]http://www.mathworks.com/help/toolbox/compiler/f2-972343.html#bsic1of-2( diagram#3, build process)
[18]http://www.mathworks.com/help/toolbox/compiler/bsfey7f.html#bsfj66v-1(diagram#4, build process)
References
[19]http://www.mathworks.com/help/toolbox/compiler/f2-972343.html( C code integration )
[20]http://www.mathworks.com/help/toolbox/compiler/ismcc.html ( startup.m information )
[21]http://www.mathworks.com/products/compiler/demos.html( compiler demos on Windows )
[22]http://blogs.mathworks.com/loren/2011/04/21/deploying-multiple-c-shared-libraries/#2 ( Using deployed shared library from C++ )
[23]http://www.nmr.mgh.harvard.edu/martinos/itgroup/deploytool.html#TOC ( deploytool guide )
[24]http://www.linuxjournal.com/article/1060 ( Linux ELF file format )
[25]http://mathtools.web.cern.ch/node/20( Matlab compiler user guide from CERN )
[26]http://www.mathworks.com/support/tech-notes/1600/1608.html#Section_4 ( Deployment guides )
[27]http://www.mathworks.com/support/solutions/en/data/1-2YVMOV/index.html ( Web deployment guide )
[28]http://www.mathworks.com/support/solutions/en/data/1-WK5DJ/index.html?solution=1-WK5DJ( Web deployment guide)
[29]http://www.mathworks.com/support/solutions/en/data/1-30REEY/index.html?solution=1-30REEY( Web deployment guide)
[30]http://www.mathworks.com/products/compiler/ ( License terms )
[31]http://www.mathworks.com/help/toolbox/compiler/br5w5e9-2.html (License terms )
[32]http://www.mathworks.com/support/tech-notes/1600/1605.html#intro ( MEX interface, gcc version requirements )
Trademarks: http://www.mathworks.com/company/aboutus/policies_statements/trademarks.html