Transcript CPL7 info
Note about Cpl7 and WRF Juanxiong He ARSC and IARC [email protected] The difference between Cpl6 and Cpl7 • Cpl6 uses MPH to organize the framework of the couple model, but Cpl7 uses MPI group communicator. • Cpl6 is concurrent, but Cpl7 is sequential. • Cpl6 is MPMD, but Cpl7 is SPMD. • In Cpl6 the communication process is embedded in the component integration process and the component is also responsible for data transferring. But in Cpl7 they are separated. All communication processes are taken over by Cpl7 and the component runs itself without taking care of data transferring. It’s better for model development. MPI_COMM_WORLD partition of the processors In RACM ICE ATM CPL LND 5 COMPONENTS 10 COMMUNICATION GROUPS OCN Partition of the processors in RACM is the similar as Cpl7 and CCSM4 ! Initialize call mpi_init(ierr) call mpi_comm_rank(MPI_COMM_WORLD, mype , ierr) call mpi_comm_size(MPI_COMM_WORLD, numpes, ierr) ! Create MPI communicator groups MPI code example call seq_comm_setcomm(ATMID,…) ! Atmoshere call seq_comm_setcomm(LNDID, …) ! Iand call seq_comm_setcomm(ICEID, …) ! Seaice call seq_comm_setcomm(OCNID, …) ! Ocean call seq_comm_setcomm(CPLID, …) ! Coupler call seq_comm_joincomm(CPLID,ATMID,CPLATMID) ! Atmoshere and coupler call seq_comm_joincomm(CPLID,LNDID,CPLLNDID) ! Land and coupler call seq_comm_joincomm(CPLID,ICEID,CPLICEID) ! Seaice and coupler call seq_comm_joincomm(CPLID,OCNID,CPLOCNID) ! Ocean and RACM Coupling flow • The model is controlled by the driver • The coupling process has three phases in each four components: importing, running and exporting • Two sub steps: at first, ocean, seaice and land finish importing, running and exporting; then, atmosphere imports, runs and export, since atmosphere needs the surface data from land, seaice and ocean. • Two implications at present: all the component has the same couple interval and their timesteps are equal to or less than couple interval OCN init ESMF Clock advance LND ICE init init ESMF Clock advance Begin ATM init ESMF Clock ESMF Clock advance advance CPLOCN receive Ocean run CPLLND receive Land run CPLICE receive Seaice run CPL init ESMF Clock advance Map, merg For Ocean send Reaarange and Map, merg Ocean import For Land send Rearrange and Land import Map, merg For Ice send Rearrange and Ice import OCN LND ICE ATM CPLOCN send CPLICE send receive CPL receive Rearrange and Ocean Export Accumulate For Ocean receive Rearrange Ice Export Acumulate For Ice CPLLND send Rearrange and Land Export Acumulate For Land OCN LND ICE CPL Merge For Atm ATM CPLATM receive Atm run send send Rearrange and Atm Import receive Rearrange and Atm Export CPLATM Stop time? Stop time? Stop time? Stop time? End Acumulate For Atm Stop time? CCSM4 clock mechanism • RACM has two sets of clocks. One is the ESMF clock, which controls the coupling process; the other is the internal clock in each components, which controls the integration. • Every pe has five ESMF clocks: Eclock_d (driver), Eclock_a (atmospher), Eclock_o (ocean), Eclock_l (land), Eclock_i (seaice). • The ESMF clock advances one step with the couple interval timestep. • Each component has its internal clock, which advances one step with integral timestep. • The component exports data and Cpl7 transfer data if the internal clock matches the ESMF clock. ! initialize seq_syncclock, Eclock_d, Eclock_i, Eclock_o, Eclock_l, Eclock_a call ESMF_Initialize() ! Advance clock call seq_timemgr_clockAdvance( seq_SyncClock) … ! Initialze internal clock call timemgr_init(…) Clock code example ! advance the internal clock call advance_timestep() ! Compare ESMF clock and internal clock call get_curr_date( yr, mon, day, tod ) ymd = yr*10000 + mon*100 + day tod = tod dosend = (seq_timemgr_EClockDateInSync( EClock, ymd, tod)) Data flow of the coupling process in each component • The component doesn’t import from or export to the coupler actively. It’s that the coupler transfers data into or out of the component. • By the union communication group of the coupler and the component, the coupler uses the MCT function “rearrange” to get and put the data. • The rearranged data has the same value and feature with the original except the palace of pe. So the rearrange process is very fast. • The coupler remaps or merges the incoming data for the sake of the coupling among different componet. Data flow of the coupling process in each component • Atmosphere: x2a_aa (import data) a2x_aa (export data) • Ocean: x2o_oo (import data) o2x_xx (export data) • Seaice: x2i_ii (import data) i2x_ii (export data) • Land: x2l_ll (import data) l2x_ll (export data) CPLLNDID CPLOCNID l2x_ll o2x_oo component exporting fields to CPL CPLATMID CPLICEID a2x_aa i2x_ii Rearrange l2x_lx a2x_ax o2x_ox a2x_ox preparing l2x_ax fiedls for atmosphere on CPL o2x_ax xao_ox i2x_ax Map i2x_ax xao_ax CPLID Merge Importing fiedls from CPL for atmosphere x2a_ax Rearrange x2a_aa CPLATMID ATMOSPHERE CPLLNDID component r2x_rr exporting fields to CPL r2x_rx CPLOCNID o2x_oo CPLATMID a2x_aa Importing fiedls from CPL for Ocean r2x_ox CPLID i2x_ii Rearrange a2x_ax o2x_ox a2x_ox preparing fiedls for ocean on CPL CPLICEID xao_ox i2x_ix Map i2x_ox X2o_ox fractions_ox Merge X2oacc_ox%data Rearrange x2o_oo CPLOCNID OCEAN CPLOCNID component exporting fields to CPL CPLATMID o2x_oo a2x_aa Rearrange a2x_ax o2x_ox Map preparing o2x_ix a2x_ix fiedls for seaice Merge CPLID on CPL X2i_ix Importing fiedls from CPL for seaice Rearrange X2i_ii CPLOCNID LAND CPLATMID component exporting fields to CPL a2x_aa Rearrange a2x_lx preparing fiedls for seaice on CPL Importing fiedls from CPL for seaice Map CPLID X2l_lx Rearrange x2l_ll CPLLNFID SEAICE if (iamin_CPLID .and. atm_prognostic) then call map_ocn2atm_mct ! o2x_ox ->o2x_ax, xao_ox->xao_ax call map_ice2atm_mct !l2x_ix ->i2x_ax call map_lnd2atm_mct !l2x_lx ->l2x_ax call mrg_x2a_run_mct ! l2x_ax+o2x_ax+xao_ax+i2x_ax+fractions_ax->x2a_ax a Code sample of Import endif and export of if (iamin_CPLATMID .and. atm_prognostic) then atmosphere couple data call map_atmx2atma_mct ! x2a_ax -> x2a_aa endif ! Rearrange and import call atm_run_mct( EClock_a, cdata_aa, x2a_aa, a2x_aa) !export a2x_aa if (iamin_CPLATMID) then call map_atma2atmx_mct !a2x_aa->a2x_ax ! Rearrange and export endif if (iamin_CPLID) then Some Changes of Cpl7 and WRF for the sake of RACM at present and in future • Reorganizing WRF upper structure to be fit for coupling. • Changing the the related Cpl7 atmosphere land couple subroutine and allow the LSM has a different grid. It has more flexible since the grids of WRF and LSM are often different. Any resolution change in LSM may bring arduous work of adjusting the parameters. • The decomposition of the grid in WRF has halo, but Cpl7 doesn’t consider it. Adaption of Cpl7 for halo will be necessary. • The couple time interval of the different model should be different for RACM. The couple time interval of VIC in RACM is one day. But the fine scale of NAPC needs get the data every one hour or even 15 minutes . Reorganizing of WRF integral process wrf_init ----- atm_init_mct(Eclock, cdata_a, x2a_a, a2x_a) Wrap Wrf_init with the related coupler data and initialize it wrf_run----- atm_run_mct(Eclock,cdata_a,x2a_a, a2x_a) (1) Wrap Wrf_run with atm_import_mct and atm_export_mct, make wrf_run run until the couple time is ok (2) reorganize the wrf_run into three parts and throw wrf land surface model: wrf_run1, undertaking dynamic integral; wrf_run2, calculating radiation; wrf_run3, calculating other physic process Destroy the related Wrf_finalize----- atm_final_mct() coupler data and wrf data