Transcript Dynamic Data-Driven Application Systems (DDDAS)

Dynamic Data-Driven
Application Systems
(DDDAS)
What is DDDAS?
Dynamic Data Driven Application Systems
DDDAS is a new paradigm.
Old ways: Application simulations use static
data input and start up.
DDDAS (continued)
Application simulations are able to
RECEIVE AND RESPOND to ONLINE
physical data and measurements and/or
control the measurements.
Why DDDAS?

Computer models were not designed to deal
with dynamic conditions as the simulations
already started.

Example: simulating wild fire. We need to
know which community needs to be
evacuated. The wild fire simulation needs to
adjust based on the wind direction and other
factors.
Why DDDAS (continued)
Also fueled by advances in:
 Applications and algorithms for parallel and
distributed platforms.
 Computational steering and visualization.
 Computing.
 Networking.
 Sensors and data collection.
 System software technologies.
What DDDAS Needs
Three research areas:
 Application: data driven technology.

Algorithm: dynamic data injection and data
perturbation tolerance.

System software: support for dynamic environments.
Static applications can be changed into a more useful
dynamic data driven applications.
DDDAS General Properties
What’s new in DDDAS?
Feedback and control interactions between
computations and the physical measurement systems.
There are three ways to interact with DDDAS
computation:
 Human interaction.
 Physical system interaction.
 Computational infrastructure interaction (machines
and their connections).
DDDAS Interactions
DDDAS Key Characteristics
Key characteristics that need to be addressed:

Time dependency and/or real time aspect.

Data streams in addition to data sets.

Interactive visualization and steering.
DDDAS Example
Tsunami simulation.
(http://www.pgc.nrcan.gc.ca/geoscapevictoria)
Where we get the data from?
Tsunami can be caused by tectonic plate
movement, or sea floor quakes.
Data sources: GPS stations, sea floor sensors.
DDDAS Example 1 (continued)
DDDAS Example 1 (continued)
DDDAS Example 1 (continued)
DDDAS Example 2
Air traffic simulation.
(http://www.simlabs.arc.nasa.gov/cvsrf/atcs.html)
Where we get the data from?
We need the world location for all the aircrafts, as well
as their schedules. Outside data: weather simulations.
Data source: Aircraft transponders.
DDDAS Example 2 (continued)
DDDAS Example 3
Medical imaging and simulation.
(http://www.bitc.gatech.edu/bitcprojects/eye_sim/eye_surg_sim.html)
The Georgia Institute of Technology and the Medical College of
Georgia.
We need the real life model of the object of
interest. We can do this by using Intensity
Modulated Radiation Therapy (IMRT).
Simulating eye surgery. To blind or not blind.
DDDAS Example 3 (continued)
DDDAS Example 3 (continued)
DDDAS Researches: Past,
Present, and Future.
Reference taken from DDDAS.org Home Page,
NSF Official DDDAS Page, NSF 2000
Workshop on Dynamic Data-Driven Application
Systems, and Performance Engineering
Technology Notes.
DDDAS: Past and Present
Past Experience
February 17, 2000: Meteorologists missed
predicting the track and magnitude of a major
storm in January 24-25, 2000, that blanketed
major cities from South Carolina to New
England.
May 7, 2000: The National Park Service started
a controlled burn near Los Alamos National
Laboratory. Within a day, the fire was labeled
a wildfire.
What is being sought from
DDDAS?
DDDAS Capability: simulation applications that
can dynamically accept and respond to field
data and measurements, and can control such
measurements in a dynamic manner through
the symbiotic measurement and simulation
systems.
How DDDAS Affects Our
Present and Future
Examples of Applications benefiting from
DDDAS:

Engineering Design and Control: aircraft design, oil
exploration, semiconductor manufacturing, structural
engineering, computing systems hardware and
software design and runtime.

Medical: customized surgery, radiation treatment,
BioMechanics/BioEngineering
How DDDAS Affects Our Present
and Future (continued)

Economy: Production planning and control,
financial trading (stock market, portfolio
analysis).

Crisis Management and Environmental
Systems: Weather, hurricane/tornado
prediction system, floods, fire propagation,
transportation systems (emergency planning,
accident response)
Challenges in Enabling DDDAS
Capabilities

Application Simulations Developments

Algorithms

Computing Systems Support
Challenges in Enabling DDDAS
Capabilities (continued)

Applications:

Ability of the application to interface with measurement
systems.

The stream data might introduce new modalities to describe
the system, like a different level of the physics involved, in
cases where the analysis is about a physical system.

Ability to dynamically select the application components
depending on the dynamically streamed data.
Challenges in Enabling DDDAS
Capabilities (continued)

Algorithms:

Stable to dynamically injected data/ tolerant to
perturbations of dynamic input data.

Visualization with a human in the loop: feedback
to the simulations.

Handling data uncertainties: continuous
sensitivity analysis
Challenges in Enabling DDDAS
Capabilities (continued)

Computing Systems Support:

Support environments where the application
requirements change during the execution depending
on the streamed data/dynamic execution support on
heterogeneous environments.

Extended spectrum of platforms: Grid Computing and
beyond.
What is Grid Computing?
What is Grid Computing?
(continued)

Grid computing enables the virtualization of distributed computing
and data resources such as processing, network bandwidth and
storage capacity to create a single system image, granting users
and applications seamless access to vast IT capabilities.

Just as an Internet user views a unified instance of content via the
Web, a grid user essentially sees a single, large virtual computer.

At its core, grid computing is based on an open set of standards
and protocols that enable communication across heterogeneous,
geographically dispersed environments. With grid computing,
organizations can optimize computing and data resources, pool
them for large capacity workloads, share them across networks
and enable collaboration.
What is Grid Computing?
(continued)

Like the Web, grid computing keeps complexity hidden: multiple users enjoy a single,
unified experience.

Unlike the Web, which mainly enables communication, grid computing enables full
collaboration toward common business goals.

Like peer-to-peer, grid computing allows users to share files.

Unlike peer-to-peer, grid computing allows many-to-many sharing — not only files
but other resources as well.

Like clusters and distributed computing, grids bring computing resources
together.

Unlike clusters and distributed computing, which need physical proximity and
operating homogeneity, grids can be geographically distributed and heterogeneous.

Like virtualization technologies, grid computing enables the virtualization of IT
resources.

Unlike virtualization technologies, which virtualizes a single system, grid
computing enables the virtualization of vast and disparate IT resources.
Why DDDAS Now?
DDDAS has the potential to revolutionize
science, engineering, medicine, economy,
management systems, and etc.
We have technological progresses that has advances
the level of overcoming the challenges:
- computing speed (terascale, uni- and multiprocessor systems, Grid Computing, Sensor
Networks).
Why DDDAS Now? (continued)
- System software
- Applications (parallel and grid computing)
- Algorithms (parallel and grid computing, numeric
and non-numeric techniques, data assimilation,
and chaotic Monte-Carlo method).
Summary and What the Future
Holds

NGS: Next Generation Software (1998 - …)

Develops systems software supporting dynamic
resource execution
Summary and What the Future
Holds (continued)

SES: Scalable Enterprise Systems Program
(1999, 2000-2003)
 Geared toward commercial applications.

ITR: Information Technology Research (NSFwide project)
 Has been used as an opportunity to support
DDDAS-related efforts.
Summary and What the Future
Holds (continued)

Simultaneous advances on the models,
methods, and algorithms that underpin the
components – and on their systematic
integration to target strategic applications –
are crucial for realizing the potential of
DDDAS.

But hardware, software, and Cyber
Infrastructure alone are insufficient to achieve
this goal.
Case Study: Architecture of the DDDAS
Wildfire Model
Major Components of the Model

Coupled atmosphere/fire model



Data acquisition



runs on PDAs or cell phones in the field
Dynamic Data Assimilation control module




From the Internet: GIS maps, past fire information, weather
Field information: photos taken from aircraft, field sensors
Visualization and user interface


Legacy NCAR code
Combined with the latest techniques, such as OpenMP and Multigrid
Incorporates data from the field
Bayesian data assimilation
Data assimilation steers the data acquisition
Guaranteed secure communication infrastructure
The NCAR Coupled Atmosphere/Fire
Model

The interaction of fire and atmosphere is important



Traditional fire models cannot represent this interaction



Heat flux from the fire to the atmosphere produces fire wind
Wind facilitates the fire spread
Wildfires are difficult to model
Limited computational resources
An meteorological model is coupled with an fire spread model



The atmosphere model is based on the Clark-Hall Atmospheric Model
Fire model can be an empirical model, or a more realistic Stochastic
Reaction-Diffusion Equation Fire Model
Represents the important interaction between fire and atmosphere,
more accurate and closer to reality
Data Acquisition

Geographical, weather, and fire information available from the
Internet




Advancement of the fire front




Weather information: NOAAPORT broadcast, MesoWest weather stations,
and the Rapid Update Cycle (RUC) weather system by NOAA
Past fire information: the GeoMAC project at USGS
Fuel type: national database
infrared pictures taken from aircraft
GPS for aircraft position
3-axis inertial measurement to get the pointing direction of the camera
Field fire and weather information data

Sensors arbitrarily placed in the vicinity of a fire for point measurements of fire
and weather information
Visualization and User Interface

Simulation results (prediction) will be transferred to the
firefighters in the field

Need an easy and portable way to visualize the simulation result

Firefighters don’t want to carry a notebook when fighting fires

PDA (or cell phone) and Java is the natural choice

Limited computing power and memory

Needs Java-based graphic software
Why Dynamic Data Assimilation?

Wildfire is a complex process with lots of uncertainties, such as
wind, humidity, temperature

Neither empirical model nor physical model cannot represent the
wildfire very well

Lots of parameters cannot be measured accurately

All in all, the system is heavily non-linear and ill-posed

Sequential Bayesian data assimilation can be used to guide the
simulation
How Dynamic Data Assimilation works?

The state of the system at any time - physical variables and
parameters of interest at mesh points

Time-state vector x - snapshots of system states at different
points in time

The knowledge of the time-state of the system - probability density
function p(x)

p(x) is represented by a ensemble of time-state vectors x1, x2, …,
xn

Number of system states maintained = size of the ensemble *
number of snapshots

Thousands of simulation run simultaneously
Sequential Bayesian Filtering

Current state of the model


prior probability density –
p f (x)
Incorporate data from the field


Measurements – vector y
How the data is derived from
x – p(y|x)

Posterior probability density –

The system advances in time
from the posterior probability
density until new data arrives;
this process is called an
analysis cycle
p a ( x) 
p( y | x) p f ( x)
 p( y |  ) p
f
( )d
Standard Approach of Data Assimilation
by Ensemble Filter

Initialization: generate initial ensemble by a random
perturbation of initial conditions

Repeat the analysis cycle:

Advance ensemble states
to a target time by solving
the model PDEs in time

Inject data with time-stamps
equal to the target time:
modify ensemble states by
a Bayesian update
Overall Pictures
References






Geoscape Victoria http://www.pgc.nrcan.gc.ca/geoscapevictoria/
Air Traffic Controls Simulator
http://www.simlabs.arc.nasa.gov/cvsrf/atcs.html
Eye Surgery Simulation
http://www.bitc.gatech.edu/bitcprojects/eye_sim/eye_surg_sim.ht
ml
DDDAS by Dr. Craig Douglas http://www.dddas.org/
DDDAS by NSF
http://www.nsf.gov/cise/cns/darema/dd_das/index.jsp
Jan Mandel’s DDDAS Website http://wwwmath.cudenver.edu/~jmandel/dddas03/