Transcript Advances in Generalization and Decoupling of Software Parts in a Scientific Simulation Workflow System Arne Bachmann Simulation- and Software Technology German Aerospace Center (DLR) ADVCOMP 2010,

Advances in Generalization and Decoupling
of Software Parts in a Scientific Simulation
Workflow System
Arne Bachmann
Simulation- and Software Technology
German Aerospace Center (DLR)
ADVCOMP 2010, Oct 26th, Florence/Italy
Slide 1
ADVCOMP 2009 > Arne Bachmann > Markus Kunde > et al. > 2009-10-13
DLR
German Aerospace Center
Research Institution
Space Agency
Project Management Agency
Slide 2
ADVCOMP 2010 > Arne Bachmann 2010-10-26
Locations and employees
6300 employees across
29 research institutes and
facilities at
 13 sites.
Hamburg 
Bremen 
 Neustrelitz
 Trauen
Berlin 
Braunschweig 
 Dortmund  Goettingen
Offices in Brussels,
Paris and Washington.
 Koeln
 Bonn
Lampoldshausen 
Stuttgart 
 Oberpfaffenhofen
Weilheim 
Slide 3
ADVCOMP 2010 > Arne Bachmann 2010-10-26
National and International Networking
Customers and partners:
Governments and ministries, agencies and organisations,
industry and commerce, science and research
World
Europe
Germany
Slide 4
ADVCOMP 2010 > Arne Bachmann 2010-10-26
Overview
Introduction
The „Chameleon“ Approach
Progress report 2009-2010
Conclusion
Outlook
Slide 5
ADVCOMP 2010 > Arne Bachmann 2010-10-26
Introduction
Interdisciplinary cooperation mostly within DLR
Goals: Simulation, construction, assessment, experiments
Global optimization of complex models
Reuse and integration of existing disciplinary tools
Slide 6
ADVCOMP 2010 > Arne Bachmann 2010-10-26
Introduction
Engineers collaborate in interdisciplinary projects
They share their expertise via problem-solving tools (= simulation)
But don't give away their sovereignty in their research field
They simply provide a service via a well-defined I/O (SOA)
Slide 7
ADVCOMP 2010 > Arne Bachmann 2010-10-26
Introduction
For problem-solving, a researchers can combine the published tools
Simply by dragging together a workflow at their PCs
The underlying framework takes care of all the infrastructural stuff
Service discovery
Configuration of tools and environment
Data management, workflow handling, logging
Data interfacing, integration and visualization
Slide 8
ADVCOMP 2010 > Arne Bachmann 2010-10-26
Examples of integrated pre-design tools
PANAM;
Mission simulation
Quelle: Studienarbeit Nr.
697, IFL Braunschweig
Lifting Line
HEIDI
ModGen
PAM-CRASH / SECT-MESH
Slide 9
ADVCOMP 2010 > Arne Bachmann 2010-10-26
The Chameleon Integration Environment
A simulation and integration environment
A software suite that enables you to
integrate any simulation tool
communicate data between tools
build up simulation processes and workflows
try out new ideas, experiment with your data and methods
A principle, methodology, architecture, philosophy?
Slide 10
ADVCOMP 2010 > Arne Bachmann 2010-10-26
The Chameleon Idea
Decouple all software parts of the simulation environment:
The disciplinary tools provided by engineers
The common data format
and accessing libraries / wrappers
CPACS
& Libraries
The integration framework / platform
Tool
Input
Output
Integration
framework
Slide 11
ADVCOMP 2010 > Arne Bachmann 2010-10-26
XML
<?xml version="1.0"
<cpacs>
<vehicles>
<aircraft>
<model uID="VFW-614">
<name>
VFW-614 – ATTAS
</name>
<description>
This is the VFW-614 –
ATTAS (D-ADAM)
</description>
…
</cpacs>
Slide 12
ADVCOMP 2010 > Arne Bachmann 2010-10-26
Data integration
Common Parametric Aircraft Configuration Scheme (CPACS)
XML-based data format
Structured, extensible, transformable
Hierarchical data structures
Since 2010:
Data concept
Parametric description, several information detail levels storable
Can be extended whenever new fields of science need to integrate
Dataset integrity by XML schema (XSD)
XSD allows for automatic validation of datasets
Integrated data format documentation within the schema → PDF/HTML
Validation strategies developed in recent Master Thesis
Slide 13
ADVCOMP 2010 > Arne Bachmann 2010-10-26
CPACS currently holds data concerning…
Aircraft…
Wings
Fuselage(s)
Engines
Systems
…
Aviation…
Missions
Airports
Fleets
Tools
Slide 14
ADVCOMP 2010 > Arne Bachmann 2010-10-26
Common Parametric Aircraft Configuration Schema
CPACS
cpacs
header
global
engines
wings
sections
fuselages
missions
section
transformation
elements
airfoilUID
Slide 15
ADVCOMP 2010 > Arne Bachmann 2010-10-26
Tool wrapping component
I/O converters from CPACS to custom XML I/O
Used by tools that have their own XML format
Wrappers from proprietary formats to XML
Used when tools are unmodifiable (no source)
Because one doesn't own rights
Because they aren't supported any longer
Because it's just often easier to write a little wrapper!
This two-stage wrapping shields both tools and the common dataset
definition from changes in the other
By providing a mapping mechanism for simple to complex cases
Slide 16
ADVCOMP 2010 > Arne Bachmann 2010-10-26
Coarse picture of the Chameleon Architecture (2009)
Aero
dynamics
Engine
performance
Climate
assessment
TIGL
Chameleon
TIXI
Integration framework „ModelCenter“
Slide 17
ADVCOMP 2010 > Arne Bachmann 2010-10-26
Coarse picture of the Chameleon Architecture (2010)
Aero
dynamics
Engine
performance
Climate
assessment
TIGL
Chameleon
TIXI
Integration framework „RCE“
Slide 18
ADVCOMP 2010 > Arne Bachmann 2010-10-26
The Chameleon Integration Environment
Why yet another framework?
Existing ones aren't flexible enough
With regards to dynamicity of data connections between tools
With regards to infrastructure
With regards to user-guidance and simplicity
Thus we put Chameleon on top of existing software integration systems
ModelCenter
http://phoenix-int.com
RCE
"Remote Component Environment"
http://rcenvironment.de
Slide 19
ADVCOMP 2010 > Arne Bachmann 2010-10-26
Framework abstraction layer
Chameleon comes with useful libraries for
Simple XML access for wrapping tools, written in C (TIXI)
Geometric library, written in C++ (TIGL)  STEP, IGES, VTK
Interfaces for C, C++, Fortran & Python included
Available for Linux/Windows, static/dynamic, 32bit and 64 bit
Java GUI components for
simple import/export of CPACS data
visualization of airplane geometry from within the framework
The combination of CPACS, ToolWrapper and Java components
make reusing the Chameleon suite in other frameworks easy
Under current development: JAR → OSGi; Swing → SWT
Slide 20
ADVCOMP 2010 > Arne Bachmann 2010-10-26
Status as of 2008
What it looked like:
Slide 21
ADVCOMP 2010 > Arne Bachmann 2010-10-26
Status as of 2009
What it looked like:
Slide 22
ADVCOMP 2010 > Arne Bachmann 2010-10-26
Status as of 2010
What it looks now:
Slide 23
ADVCOMP 2010 > Arne Bachmann 2010-10-26
Chameleon adapted to a new platform:
Remote Component Environment (RCE)
Realization of the Chameleon ideas on top of another framework
In-house development www.rcenvironment.de
Developed since 2006, open source at www.sourceforge.net
Successfully deployed e.g. in ship construction predesign
Advantages
Allows to run a workflow on a remote node
Allows to have local GUIs for remote components
Is free and open source
Offers a lot of functions at service level
Has an integrated dynamic help
Is modular, extendable, stable
Based on Eclipse RCP / OSGi
Slide 24
ADVCOMP 2010 > Arne Bachmann 2010-10-26
Operating System Independence
Currently Linux and Windows supported
Other OS possible due to Eclipse Rich Client platform
Slide 25
ADVCOMP 2010 > Arne Bachmann 2010-10-26
Detachable workflows
Start a workflow on a remote server, configure locally
Detach client (shut down computer)
Re-attach (or attach other client) and monitor remote workflow
Slide 26
ADVCOMP 2010 > Arne Bachmann 2010-10-26
Example: Integrated viewer component for CPACS
Slide 27
ADVCOMP 2010 > Arne Bachmann 2010-10-26
Example: Dynamic help capabilities in action
Slide 28
ADVCOMP 2010 > Arne Bachmann 2010-10-26
Example: Dynamic help capabilities in action
Slide 29
ADVCOMP 2010 > Arne Bachmann 2010-10-26
Effective advantages of RCE over our old system
Showed feasibility of applying our Chameleon approach to a completely
different grid computing platform: from ModelCenter to RCE
We have a working data management due to recent improvements in
RCE allowing any storage backends (In-memory, SQL, flat file, …)
Transparent access to remote tools‘ data files and metadata
Ability to transport and store large data via several data protocols
Typed and untyped asynchronous data transfer (data-driven parameters
as well as data streams via communication channels)
A complete framework with service discovery, security, extensibility
And open for everybody to try out and improve further!
Slide 30
ADVCOMP 2010 > Arne Bachmann 2010-10-26
Slide 31
ADVCOMP 2010 > Arne Bachmann 2010-10-26
Slide 32
ADVCOMP 2010 > Arne Bachmann 2010-10-26
Conclusion
Current drawbacks
Ease of build-up/collaboration over pure performance
Parallelization only in workflow and on node/cluster
No load-balancing
Largest advantages
No fixed data connections between tools due to flexible XML
Bunch of libraries to help engineers integrate and profit from
Chameleon and CPACS
Simple tool wrapping + language integration wrappers
Quick build-up and easy sharing of new project workflows
Wizards available, integration with expert systems easily done
Slide 33
ADVCOMP 2010 > Arne Bachmann 2010-10-26
Outlook
Implement generic provenance data recording
Allow for semi-automatic workflows
For distributed design and research teams
Pause, Resume, Manipulate and Monitor Values
Playback, Checkpointing
Concurrent Design Facility (CDF)
Concurrent Engineering Facility CEF
Success Stories Wanted 
Interested in trying out? Visit http://www.rcenvironment.de !
Contributions and cooperations welcome!
Slide 34
ADVCOMP 2010 > Arne Bachmann 2010-10-26
Thanks!
Questions?
Slide 35
http://www.walle-derfilm.de/
ADVCOMP 2010 > Arne Bachmann 2010-10-26
Kontakt
Arne Bachmann
Department Distributed Systems and
Component Software (SC-VK)
DLR Simulation and Software Technology
Köln-Porz / Braunschweig / Berlin
E-Mail: [email protected]
www.dlr.de/sc/abteilung/verteiltesysteme
Slide 36
ADVCOMP 2010 > Arne Bachmann 2010-10-26