Transcript The FLASH Code: From Design to Applications

The Center for Astrophysical Thermonuclear Flashes
The Flash Code
From Applications
to Design
Design to Applications
Tomek Plewa
on behalf of almost countless contributors
The ASCI Flash Center
Dept. of Astronomy & Astrophysics
The University of Chicago
July 22, 2002
http://flash.uchicago.edu
An Accelerated Strategic Computing Initiative (ASCI)
Academic Strategic Alliances Program (ASAP) Center
at The University of Chicago
What Is the Flash Center?
 Supported by the DOE ASCI/Alliances Program
 Over $1,000,000 p.a. budget
 20 core researches, 30 contributors at different levels
 Access to the most advanced computer technology
 Close partners at UofC, ANL MCS, UCSC, UIUC
 Collaboration with LLNL, LANL, LBNL, Sandia, ORNL
 Links to MPA Garching, Arizona, Palermo, Torino
The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes
The University of Chicago
What Is the FLASH Code?

Has a modern architecture
modular with interfaces
configurable with parameter/variable database

Is highly portable
across software platforms: most UNICES including Linux
across hardware platforms: MPI for intra- and inter-box communication
scalar cache-based systems
provides strong tests of software (operating system, compiler,
language interoperability) and hardware (network, storage)
parallel I/O with HDF5 for large data sets

Can solve a broad range of problems
adaptive mesh discretization in 3-D with PARAMESH, compressible hydrodynamics,
MHD, SRHD, elliptic operators, explicit diffusion, complex EOS, particle tracking

Future developments
•
•
•
•
•
extended framework through IBEAM (parallel solvers for large linear systems)
formal interface specification (for solvers and mesh component)
front tracking component
elements of TSTT/CCA forums
interoperability with other AMR software
The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes
The University of Chicago
Primary Applications



X-ray bursts on neutron star surfaces
Novae
Type Ia supernovae
The common elements:
• The underlying stars are compact
• Members of close binary systems
• Physical processes: hydrodynamics, complex EOS, gravitation, nuclear burning
• Radiation hydrodynamics important at late times
• Initial conditions involve long timescales (implicit solve)
• Rapid evolution during final event (explicit solve)
The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes
The University of Chicago
Primary Applications: Towards Understanding
What is environment for the explosion?
How does it form?
What happens during the explosion?
Where are complex elements produced?
How big is the Universe?
How old is the Universe?
The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes
The University of Chicago
Primary Applications: Examples
Flame-vortex interactions
Laser-driven shock instabilities
R-M instability
Wave breaking on white dwarfs
Cellular detonation
Helium burning on neutron stars
Type Ia supernova
Landau-Darrieus instability
Magnetic and hydrodynamic Rayleigh-Taylor instabilities
The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes
The University of Chicago
Primary Applications: Examples
Wave breaking on white dwarfs
3-D Rayleigh-Taylor instability
Flame-vortex interactions
The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes
The University of Chicago
Additional Applications
Jeans instability
Intracluster interactions
Non-relativistic accretion onto BH
Relativistic accretion onto NS
The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes
The University of Chicago
The FLASH 2+ Code: at a Glance
 General description
•
parallel block-structured adaptive mesh refinement code
 Solvers
•
•
•
•
hyperbolic: hydrodynamics, MHD, and SRHD
elliptic: self-gravity
parabolic: thermal conduction
ODE: nuclear burning
 Architecture
•
•
•
undergoes substantial changes
modular, fine-grained, SPMD (patch-based), efficient parallelism
cache-based scalar architectures (most of the market)
mostly F90 with elements in C; no language restrictions
 Testing
•
•
•
constantly developed
one of the finest and most matured elements
used to control development and prevent major design flaws
compile  run  compare
applied daily on several platforms, centralized database
The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes
The University of Chicago
The FLASH 2+ Code: Physics Modules
 Compressible hydro
• PPM, WENO, Tadmore central-difference
• MHD 2nd order TVD
• SRHD 2nd order Godunov, Roe solver, R-K stepping
 Source terms
• Nuclear burning – variety of reaction networks
 Gravitational field
• Externally imposed
• Self-gravity (multipole, multigrid, single level FFT)
 Diffusion
• Thermal
• Conduction
The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes
The University of Chicago
The FLASH 2+ Code: Component Model
FLASH Component
1. Meta-data (Configuration Info)
• Variable/parameter registration
• Variable attributes
• Module requirements
• Role in driver (?)
2. Interface Wrapper
• Exchange with variable database
FLASH Application
driver
Database
mesh
3. Physics Module(s)
• Single patch, single proc functions
written in any language
• Can be sub-classed
Collection of Flash Components
The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes
The University of Chicago
The FLASH 2+ Code: Application Builder
Configuration
Tool
(Setup)
Source
Terms
Materials
Gravity
MHD
Database Mesh
Driver
Hydro
Particles
I/O
The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes
The University of Chicago
Vis
The FLASH 2+ Code: Application Example
Sets solution variable
descriptions in database
Variable
Database
AMR Library
(PARAMESH)
Driver.f90
evolve();
timeStep();
Data Flow
Class accessor methods
Global memory
Parameter list
Evolve.f90
hydro3D( );
burn();
Framework
Hydro3d.f90
computeRHS(arglist);
eulerStep(arglist);
Burn.f90
burner(arglist);
Standard interfaces
Physics Modules
(easily interchanged)
PPM.f77
EulerStep.c
The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes
The University of Chicago
NetInt.f77
Structure of FLASH
Modules
Variable Database
Mesh
dBase_init()
dBaseGetData()
dBasePutData()
dBaseProperty()
init()
fill_guardcells()
test_refinement()
refine_derefine()
IO
MHD
Driver
Visualization
Particles
init()
write()
init()
init()
dBaseGetData()
dBasePutData()
dBaseProperty()
init()
render()
init()
advance()
Diffusion
Hydro
Gravity
Source_terms
Materials
init()
fill_guardcells()
test_refinement()
refine_derefine()
init()
tstep()
hydro3d()
init()
tstep()
grav3d()
init()
tstep()
src_terms()
eos3d()
eos1d()
eos()
Explicit
Implicit
constant
burn
point_mass
Split
Unsplit
iso13
Poisson
WENO
Multigrid
Multipole
Helmholtz
...
cool
PPM
Gamma
heat
The FLASH 2+ Code: Directory Structure
Mesh
Driver
steady
AMR
time dep
rk3
mesh
database
Solvers
mesh
database
HDF4.0
f77_unf
MPI
source terms
eos
composition
multipole
gamma
HDF5.0
mesh
database
Materials
poisson
multigrid
uniform
paramesh
strang
IO
hemlholtz
burn
nadozhin
aprox13
cool
heat
ppcno
Hydro
Gravity
explicit
implicit
planpar
split
unsplit
PPM
MUSCL
constant
PPM
Kurganov
WENO
Runtime
Visualization
MHD
Particles
active
passive
poisson
delta form
PVTK
2nd order
TVD
scripted
The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes
The University of Chicago
compiled
stir
The FLASH 2+ Code: Additional Features
 External libraries
•
MPI
• Paramesh
• HDF5
• pVTK
for parallel execution
for adaptive discretization
for efficient I/O
for remote visualization
 External tools
•
Python
• gmake
for configuration
for code compilation
 http://flash.uchicago.edu
•
Available with no major restrictions
• Looking to expand user base
• Support with short response time
The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes
The University of Chicago
Summary
 FLASH aspires to become a community code
 FLASH is freely available
 Major emphases
• Performance
• Portability
• Testing
• Usability
• Support (secured for at least next 5 years)
 Interest, skill, and organization guarantees success
 Future FLASH
• Implicit hydro solvers; Front tracking mesh component
• Solver and mesh interfaces
• FLASH component model
• FLASH developer’s guide
The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes
The University of Chicago
The FLASH Code: From Design to Applications
Questions and Discussion
The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes
The University of Chicago
The FLASH 2+ code
The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes
The University of Chicago