Transcript Coupling Climate and Hydrological Models
Coupling Climate and Hydrological Models
Interoperability Through Web Services
• • • • • • • Project Objective Motivation System Description – Components – – Frameworks System Driver Logical Workflow Data Flow Architecture Future Directions
Outline
Project Objective
The development of a two-way coupled, distributed, service based modeling system comprised of an atmospheric climate model using a standard ESMF interface and a hydrological model that utilizes standard interfaces from that domain.
Motivation and Background
• Hydrological impact studies can be improved when forced with data from climate models; hydrological feedbacks can affect climate • A technology and scale gap exists: – Many hydrological models have limited scalability, run on desktop computers, and have watershed-sized domains – Many climate models are highly parallel, run on high performance supercomputers and have global domains – However, scales are slowly converging (e.g. high resolution climate models, hydrological systems of greater extent), and this provides opportunities to explore new coupled model configurations and modes of coupling
Approach
• Leveraging frameworks (ESMF, OpenMI) that can operate as web services is a way to implement loose coupling between such systems – this has many advantages – Loose coupling can be implemented with minimal source code changes needed to run the models – Standard connectors between various frameworks and interfaces can be written, encouraging broad interoperability – A web service approach enables model components to be integrated with a heterogeneous network of other services (data retrieval and post-processing, etc.) – – The web service approach is well understood The approach leverages substantial investments in converting operational weather and water models to ESMF
System Description
• SWAT (hydrology model) runs on a PC • CAM (climate model) runs on a high performance computer • Wrappers for both SWAT and CAM provide an OpenMI interface to each model • Driver (OpenMI Configuration Editor) uses OpenMI interface to timestep through models via wrappers • The CAM run is invoked through ESMF web services • CAM output data is streamed to a CAM wrapper via ESMF web services Personal Computer Driver OpenMI SWAT CAM OpenMI Wrapper ESMF Web Services ESMF CAM Component High Performance Computer
Components: SWAT
• • • The hydrological model chosen for this project is the Soil Water Assessment Tool (SWAT) It is a river basin scale model developed to quantify the impact of land management practices in large, complex watersheds It was chosen for this project because it is widely used, open source, and has a standard interface (OpenMI)
Components: CAM
• The atmospheric model chosen for this system is the Community Atmospheric Model (CAM5), part of the Community Climate System Model (CESM1.0.3) • It was chosen because it is widely used, open source, and has a standard interface (ESMF)
Frameworks: Earth System Modeling Framework
• • • • ESMF is a high-performance, flexible, operational-quality software infrastructure that increases the ease of use, performance portability, interoperability, and reuse in Earth science applications It provides an architecture for composing complex, coupled modeling systems and can support fully unstructured, logically rectangular, and observational data structures It is highly portable (24+ platforms), highly scalable (tested to 16K+ processors), and includes 4500+ tests and examples Web services included in the ESMF distribution allow any networked ESMF component to be available as a web service.
Frameworks: OpenMI
• • • The OpenMI Software Development Kit (SDK) is a software library that provides a standardized interface that focuses on time dependent data transfer Primarily designed to work with systems that run simultaneously, but in a single-threaded environment [Gregerson et al., 2007] The primary data structure in OpenMI is the ExchangeItem, which comes in the form of an InputExchangeItem and an OutputExchangeItem (single point, single timestep)
The system driver
• • • Controls the application flow using a web service architecture Implemented using OpenMI’s Configuration Editor Convenient tool for the testing of the OpenMI implementations and model interactions
Personal Computer (Windows) • The Client contains the OpenMI and SWAT software, which run on a Windows platform.
Hardware Architecture
• The atmospheric model runs on a high performance platform, often split into Compute Notes and Login Nodes • Access to the Compute Nodes must be through the Login Nodes • Access to the Login Nodes is through the Virtual Server (Web Svcs) High Performance Computer Virtual Linux Server Login Nodes (kraken) Compute Nodes (kraken)
Software Achitecture Client
• Configuration Editor is the driver… it is used to link the models and trigger the start of the run.
• Hydrological model (SWAT 2005) is a version wrapped to work with OpenMI • Access to the atmospheric model (CAM) is done through “wrapper” code that accesses ESMF web services via an OpenMI interface Personal Computer (Windows) Wrapper OpenMI Configuration Editor CAM OpenMI … to Web Services OpenMI SWAT 2005
Tomcat/Axis2 SOAP Svcs Linux Server (Web Svr) • In some high performance systems, access to nodes can be restrictive. • Access to/from external systems can be controlled via “gateway” systems using web services.
• Running applications (such as CAM Component Svc) on Compute Nodes must be handled by a job scheduler.
Software Architecture Server
HPC Login Nodes Process Controller Registrar Job Scheduler Comp Svc CAM Comp Svc CAM HPC Compute Nodes Comp Svc CAM
Personal Computer (Windows) 1 Config Editor 1 CAM Wrapper OpenMI SWAT 2005
Logical Flow - Startup
When loading models into the Configuration Editor, each model is initialized. For CAM, this involves starting a “New Client” in the Process Controller, which submits a new CAM Component Service using the Job Scheduler. 3 HPC Login Nodes Process Controller 4 5 Registrar 2 Job Scheduler 7 Web Svcs Linux Server 6 1.
2.
3.
4.
Initialize New Client New Client Submit Job 5.
6.
7.
Status = SUBMITTED Instantiate Job (Comp Svc) Status = READY Comp Svc CAM HPC Compute Nodes
Personal Computer (Windows) Config Editor CAM Wrapper OpenMI SWAT 2005
Logical Flow - Status
The status of the CAM Component Service is checked often throughout the workflow. The status is stored in the Registrar, so it can be retrieved via the Process Controller.
2 HPC Login Nodes Process Controller 3 Registrar Job Scheduler 1 Web Svcs Linux Server 1.
2.
3.
Get Status Get Status Get State Comp Svc CAM HPC Compute Nodes
Personal Computer (Windows) 1 Config Editor 1 OpenMI SWAT 2005
Logical Flow - Initialize
Before the models can be run, they need to be initialized. For CAM, the Initialize call is sent to the CAM Component Service via Web Services and the Process Controller. The CAM Component Svc updates it’s status in the Registrar prior to and after Initialization.
HPC Login Nodes 3 Process Controller Registrar CAM Wrapper 2 Job Scheduler 4 5 6 Web Svcs Linux Server 1.
2.
3.
4.
Prepare Initialize Initialize Initialize 5.
6.
Status = INITIALIZING Status = INIT_DONE Comp Svc CAM HPC Compute Nodes
Logical Flow – Timestep (Run)
For each Timestep in SWAT, the trigger to run a timestep in Personal Computer (Windows) CAM is a Get Values request in the OpenMI Interface. The Run Timestep request is passed to the CAM Component Service and the Component Service sets the output data making it available for later retrieval (see Get Data).
1 Config Editor HPC Login Nodes 4 2 OpenMI Process Controller Registrar CAM Wrapper SWAT 2005 3 Job Scheduler 5 6 8 Web Svcs Linux Server 1.
2.
3.
4.
Get Values Get Values Run Timestep Run Timestep 5.
6.
7.
8.
Run Timestep Status = RUNNING Set Output Data Status = TIMESTEP_DONE Comp Svc CAM HPC Compute Nodes 7
Logical Flow – Timestep (Get Data)
Personal Computer (Windows) After each timestep run, the output data is then fetched from the CAM Component Service via the Web Services and Process Controller. The first time fetching data, a description of the data structure is requested. This description is then used for the remaining timesteps.
Config Editor HPC Login Nodes 2 OpenMI Process Controller 5 Registrar CAM Wrapper SWAT 2005 3 6 Job Scheduler 1 4 Web Svcs Linux Server 1.
2.
3.
4.
Get Data Desc* Get Data Desc* Get Data Desc* Get Data 5.
6.
Get Data Get Data * one time only Comp Svc CAM HPC Compute Nodes
Personal Computer (Windows) 1 Config Editor 1 OpenMI SWAT 2005
Logical Flow - Finalize
After all timesteps have completed, the models need to be finalized. For CAM, the Finalize call is sent to the CAM Component Service via Web Services and the Process Controller. The CAM Component Svc updates its status in the Registrar prior to and after finalization.
HPC Login Nodes 3 Process Controller Registrar CAM Wrapper 2 Job Scheduler 4 5 6 Web Svcs Linux Server 1.
2.
3.
4.
Finish Finalize Finalize Finalize 5.
6.
Status = FINALIZING Status = FINAL_DONE End Client (Next Slide) Comp Svc CAM HPC Compute Nodes
Personal Computer (Windows)
Logical Flow – End Client
After the Finalize call, the CAM Component Service is done, so the CAM Wrapper closes it out by calling End Client. This call results in the CAM Component Service completing it’s loop and exiting as well as the Process Controller removing all references to the client.
Config Editor HPC Login Nodes 2 OpenMI Process Controller 5 Registrar CAM Wrapper SWAT 2005 3 Job Scheduler 1 Web Svcs Linux Server 1.
2.
3.
4.
End Client End Client Kill Server Exit Service Loop 5.
Status = COMPLETED Comp Svc CAM HPC Compute Nodes 4
Logical Workflow
Driver Initialize Initialize Prepare Prepare SWAT/OpenMI ATM/OpenMI Wrapper ESMF Web Services NewClient Initialize ESMF Component ESMF_GridCompInitialize GetValues GetValues ValueSet RunTimestep GetData ESMF_GridCompRun ValueSet Finish Finish Finalize ESMF_GridCompFinalize Dispose Dispose EndClient
Data Flow One-Way Coupling
High Performance Computer
ESMF Component/CAM ESMF State GetDataValues
Personal Computer
GetValues SWAT/OpenMI Input Exchange Item CAM/OpenMI Wrapper Output Exchange Item • The data is pulled from the CAM Component to SWAT via the wrapper , initiated by the OpenMI GetValues call; this call is made once per timestep.
• Data is exchanged between CAM and SWAT using the OpenMI Exchange Item structures that handle the translation from grid to point values
Data Flow Two-Way Coupling
High Performance Computer
ESMF Component/CAM ESMF Export State ESMF Import State GetDataValues
Personal Computer
CAM/OpenMI Wrapper Output Exchange Item Input Exchange Item GetValues SWAT/OpenMI Input Exchange Item SetInputData Output Exchange Item • In two-way coupling, each model pulls the data from the other model using the OpenMI GetValues method. Extrapolation is used on the first timestep to break the deadlock between the two model requests.
• OpenMI Input and Output Exchanges items are again used to exchange and translate the data.
GetValues
Model Configurations
• CAM – Global Atmospheric Model – Model run: 1 day – – Timestep: 1800 sec Dynamical Core: finite volume – – Horizontal Grid: 10x15 Export data variables: • surface air temperature • precipitation • wind speed • • relative humidity solar radiation • SWAT – Hydrology science information provided by Jon Goodall of University of S. Carolina – – Lake Fork Watershed (TX) Watershed Area: 486.830 km2 – – Model run: 2 years, 1977 – 1978 Timestep = 1 day – Weather Stations: • wea62 (33.03 N, 95.92 W) • wea43 (33.25 N, 95.78 W)
Future Tasks
• • • • Continue development of two-way coupling Abstract data exchange within the ESMF wrapper code to accommodate configuration of different variables for different model implementations Explore coupling via web services provided by other frameworks and analysis and visualization tools Extend SWAT configurations for larger domains, and examine coupling to other hydrological and impacts models