Transcript Document 7236994

Fleet Numerical Meteorology & Oceanography Center
Command Overview – Presented to
Committee Operational Processing Centers
13 May, 2009
CAPT John Kusters
Commanding Officer
Mike Clancy
Technical Director
Fleet Numerical…
Supercomputing Excellence for Fleet Safety and Warfighter Decision Superiority…
Outline
•
•
•
•
•
•
•
Introduction
Overview of Operations
Overview of POPS and A2
Cross Domain Solutions
NOGAPS Way Ahead
MILCON Project
Summary
Fleet Numerical…
Supercomputing Excellence for Fleet Safety and Warfighter Decision Superiority…
Operational Excellence
• The United States Navy is committed to excellence in operational
meteorology and oceanography (METOC)
– Fleet Safety
– Decision Superiority
– Information Assurance
• We are stakeholders and partners in National capability
– Key part of the Nation’s ocean & weather infrastructure
– Expect to be a major player in NUOPC
• Near-term goals (CNMOC)
– Tighter link between forecasts and decisions
– Predictive Oceanography
We are the Navy’s Operational Science Community
Fleet Numerical…
Supercomputing Excellence for Fleet Safety and Warfighter Decision Superiority… 3
3
FNMOC Mission
• We produce and deliver weather, ocean, and climate information for
Fleet Safety, Warfighting Effectiveness, and National Defense.
– Numerical Weather Prediction (NWP) is the core of our business.
• Global and Regional Operational Models
• Assimilate meteorological and oceanographic data worldwide
– ~6 million observations daily
• Scheduled and on-demand products
– 0 to 240 hour forecasts, updated at 6-hour interval
– Specific to Fleet and Joint operations
– 24x7 Operational Reachback Center
• Supporting National, Navy, DoD/Joint, and Coalition missions
• Direct Support for Global Submarine Weather
• Data fusion for planning and operations
Fleet Numerical…
Supercomputing Excellence for Fleet Safety and Warfighter Decision Superiority…
4
Focus on Strategy
Mission
We provide and deliver weather, ocean and climate information for
Fleet Safety, Warfighting Effectiveness, and National Defense.
Vision
To be the First Choice for environmental prediction and assessment in
direct support of National Defense
Values
People  Teamwork  Communications
Innovation  Technical Excellence
Basic Strategy
Best-Cost Provider
Fleet Numerical…
Supercomputing Excellence for Fleet Safety and Warfighter Decision Superiority…
Recent Accomplishments
•
Commenced FNMOC models Ops Run on Linux Cluster (operational since
October 2008), and retired legacy SGI supercomputers
•
Commenced operational support for the North American Ensemble Forecast
System (NAEFS), with NOGAPS Ensemble fields delivered daily to NCEP
•
Upgraded the NOGAPS Ensemble with Ensemble Transform (ET) initialization
•
Implemented the coupled air-sea version of the GFDN tropical cyclone model
•
Commenced Submarine Enroute Weather Forecast (SUBWEAX) mission at
SECRET and TS/SCI levels
•
Hosted and upgraded the Naval Oceanography Portal (NOP), the single
unclassified and classified operational Web presence for all CNMOC activities
•
Delivered the first spiral of the Automated Optimum Track Ship Routing System
(AOTSR)
•
Completed A76 source selection process for IT Services resulting in selection of
the government’s Most Efficient Organization (MEO) bid as the winning proposal;
implemented IT Services Department (ITSD) on February 1, 2009
Fleet Numerical…
Supercomputing Excellence for Fleet Safety and Warfighter Decision Superiority…
Operational Watch Presence
• CNMOC Oceanographic Operations Watch (COOW)
– Global and 24x7
– Administrative Reporting
– Coordinates and provides support for all time sensitive METOC RFIs
• Fleet Ops Reachback
– Oceanography Portal
– Fleet Products: AREPS, TAWS, Staff Briefs, Exercise Support
– Tailored Support: STRATCOM, NOPF Whidbey, NSW MSC
– Supporting all forward deployed METOC assets
• ISR Reachback
– TAM
– JFCC ISR
• Submarine Weather Direct Support
Fleet Numerical…
Supercomputing Excellence for Fleet Safety and Warfighter Decision Superiority…
7
Models
• Fleet Numerical operates a highly integrated
and cohesive suite of global, regional and
local state-of-the-art weather and ocean
models:
– Navy Operational Global Atmospheric
Prediction System (NOGAPS)
– Coupled Ocean/Atmosphere Mesoscale
Prediction System (COAMPS)
– Navy Atmospheric Variational Data
Assimilation System (NAVDAS)
15 km
– Navy Aerosol Analysis and Prediction System
(NAAPS)
10 km
– GFDN Tropical Cyclone Model
– WaveWatch 3 (WW3) Ocean Wave Model
5
– Navy Coupled Ocean Data Assimilation
km
(NCODA) System
0
– Ensemble Forecast System (EFS)
km
Fleet Numerical…
Surface Pressure and Clouds
Predicted by NOGAPS
60 m s-
1
l~30 km
Mounta
in
Wave
Obs ~28
km
40 m s-
1
20 m s-
1
Sierra
Owens
Valley
0 m s-1
Cross section of temperature and wind speeds from
Supercomputing Excellence for FleetCOAMPS
Safety and
Warfighter
Decision
Superiority…
showing
mountain
waves over
the Sierras
Satellite Products
•
•
•
•
•
•
SATFOCUS
SSMI and SSMI/S
Scatterometer
Tropical Cyclone Web Page
Target Area METOC (TAM)
Tactically Enhanced
Satellite Imagery (TESI)
SSM/I Wind Speed
Fleet Numerical…
SSM/I Water Vapor
Example SATFOCUS Products
SATFOCUS Dust Enhancement Product
Supercomputing Excellence for Fleet Safety and Warfighter Decision Superiority…
Models and Applications
Ocean Acoustic
Forecasting
Aircraft Routing
Visibility/Dust
Forecasts
Automated High
Seas / Wind Warnings
Aerosol
Modeling
Optimum Track
Ship Routing
Electro-Optical
Forecasts
NWP
Target Weapon
Systems
Search and
Rescue
(NOGAPS,
COAMPS)
Ice Forecasts
CEEMS
Ensemble
Models
Tropical Cyclone
Forecasts
Fleet Numerical…
Ballistic Wind
Computations
Long-Range
Planning
WRIP
Supercomputing Excellence for Fleet Safety and Warfighter Decision Superiority…
10
Overview of POPS and A2
Fleet Numerical…
Supercomputing Excellence for Fleet Safety and Warfighter Decision Superiority…
Primary Oceanographic
Prediction System (POPS)
• Program of Record (ACAT IAD Program in sustainment)
• Provides HPC platforms to run models and applications at the
UNCLAS, SECRET and TS/SCI Levels
• Has been traditionally composed of two Subsystems:
– Analysis and Modeling Subsystem (AMS)
• Models (NOGAPS, COAMPS, WW3, etc.)
• Data Assimilation (NAVDAS, NAVDAS-AR)
– Applications Transactions and Observations Subsystem (ATOS)
• Model pre- and post-processing (e.g., observational data prep/QC,
model output formatting, model visualization and verification, etc.)
• Full range of satellite processing (e.g., SATFOCUS products, TCWeb
Page, DMSP, TAM, etc.)
• Full range of METOC applications and services (e.g., OPARS,
WebSAR, CAGIPS, AREPS, TAWS, APS, ATCF, AOTSR, etc.)
• Hosting of NOP Portal (single Web Presence for Naval
Oceanography)
Fleet Numerical…
Supercomputing Excellence for Fleet Safety and Warfighter Decision Superiority…
New POPS Architecture Strategy
• Drivers:
– Knowledge Centric CONOPS, requiring highly efficient reach-back
operations, including low-latency on-demand modeling
– Growing HPC dominance of Linux clusters based on commodity
processors and commodity interconnects
– Resource ($) Efficiencies
• Solution:
– Combine AMS and ATOS functionality into a single system
– Make the system vendor neutral
– Use Linux-cluster technology based on commodity hardware
• Advantages:
– Efficiency for reach-back operations and on-demand modeling
– Capability to surge capacity back-and-forth among the AMS (BonD Tier
1) and ATOS (BonD Tiers 0, 2, 3) applications as needed
– Cost effectiveness of Linux and commodity hardware vice proprietary
operating systems and hardware
– Cost savings by converging to a single operating system [Linux]
• We call the combined AMS and ATOS functionality A2
(AMS + ATOS = A2)
Fleet Numerical…
Supercomputing Excellence for Fleet Safety and Warfighter Decision Superiority…
A2 Hardware Specifications
• A2-0 (Opal) Linux Cluster
–
–
–
–
–
–
–
232 nodes with 2 Intel Xeon 5100-Series “Woodcrest” dualcore processors per node (928 core processors total)
1.86 TB memory (8 GB per node)
10 TFLOPS peak speed
35 TB disk space with 2 GB per second throughput
Double Data Rate Infiniband interconnect at 20 Gb per
second
40 Gb per second connection to external Cisco network
4 Clearspeed floating point accelerator cards
• A2-1 (Ruby) Linux Cluster
–
–
–
–
–
–
–
146 nodes with 2 Intel Xeon 5300-Series “Clovertown” quadcore processors per node (1168 core processors total)
1.17 TB memory (8 GB per node)
12 TFLOPS peak speed
40 TB disk space with 2 GB per second throughput
Double Data Rate Infiniband interconnect at 20 Gb per
second
40 Gb per second connection to external Cisco network
2 GRU floating point accelerator cards
Fleet Numerical…
Supercomputing Excellence for Fleet Safety and Warfighter Decision Superiority…
A2 Middleware
• Beyond the hardware components, A2 depends on a complicated
infrastructure of supporting systems software (i.e., “middleware”):
– SUSE Enterprise Server Linux Operating System, soon to transition to
Red Hat Linux V5
– GPFS: the Global Parallel File System, commercial software that
provides a file system across all of the nodes of the system
– ISIS: Fleet Numerical’s in-house database designed to rapidly ingest and
assimilate NWP model output files and external observations
– PBSPro: job scheduling software
– ClusterWorx: System management software
– Various MPI debugging tools, libraries, performance profiling tools, and
utilities
– TotalView: FORTRAN/MPI debugging tool
– PGI’s Cluster Development kit, which includes parallel FORTRAN, C,
C++ compilers
– Intel FORTRAN Compiler
– VMware ESX server
Fleet Numerical…
Supercomputing Excellence for Fleet Safety and Warfighter Decision Superiority…
FNMOC HPC Systems
NAME
TYPE
CPUs
MEMORY
(GB)
PEAK
(TFLOPS)
OS
FS1
SGI ORIGIN 3900
256
256
0.7
TRIX
FS2
SGI ORIGIN 3900
256
256
0.7
TRIX
DC3
IBM p655+
256
512
1.9
AIX
ATOS2
IBM 1350s/x440s/x345s
438
645
2.2
Linux
CAAPS
IBM e1350s
148
272
0.7
Linux
A2-0 (Opal)
Linux Cluster
928
1800
10.0
Linux
A2-1 (Ruby)
Linux Cluster
1168
1170
12.0
Linux
A2-2 (Topaz)
Linux Cluster
280
1200
3.0
Linux
3730
6111
~30
TOTAL
Systems are linked directly to ~230 TB of disk space and ~160 TB of tape archive space.
FS = File Server / Legacy Cross Domain Solution
ATOS = Applications, Transactions, and Observations Subsystem
CAAPS = Centralized Atmospheric Analysis and Prediction System
A2 = Combined AMS / ATOS System
Fleet Numerical…
Supercomputing Excellence for Fleet Safety and Warfighter Decision Superiority…
16
Cross Domain Solutions
• In order to perform its mission, FNMOC fundamentally
requires a robust, high-bandwidth Cross Domain Solution
(CDS) system to allow two-way flow of data between the
UNCLAS and SECRET enclaves within its HPC environment
• FNMOC’s existing CDS system is nearing the end of it’s
lifecycle
– SGI computer hardware (FS1, FS2)
– Trusted IRIX (TRIX) multi-level secure operating system
• The DoD Unified Cross Domain Management Office
(UCDMO) has recently indicated that all legacy CDS
systems must be replaced by one meeting their approval.
• Existing UCDMO solutions do not meet FNMOC’s bandwidth
requirements (currently ~1 GB/sec, increasing to ~64
GB/sec by 2013)
Fleet Numerical…
Supercomputing Excellence for Fleet Safety and Warfighter Decision Superiority…
CDS Way Ahead
• FNMOC is currently partnered with the National Security
Agency (NSA) to achieve an acceptable CDS that meets
high-bandwidth requirements
– Seeking interim solution based on adaptation of one of the
existing 14 UCDMO approved solutions, allowing phased
migration to long-term solution
– Seeking long-term solution based on the NSA High Assurance
Platform (HAP) project
– Expect that resulting solution will also meet NAVO’s
requirements
• Joint NSA/FNMOC presentation at the Oct 2008 DISN
Security Accreditation Working Group (DSAWG) resulted
in approval of proposed CDS way ahead described
above
Fleet Numerical…
Supercomputing Excellence for Fleet Safety and Warfighter Decision Superiority…
NOGAPS Way Ahead
Fleet Numerical…
Supercomputing Excellence for Fleet Safety and Warfighter Decision Superiority…
UKMO Engagement
• CNMOC/FNMOC and NRL explored the possibility of a partnership
agreement with UKMO to obtain the UKMO Unified Model (UM) as a
new baseline for development of the Navy’s operational global NWP
capability.
• Exploratory discussions took place between CNMOC (RDML Titley,
Steve Payne), NRL (Simon Chang) and UKMO senior leadership in
Exeter Sept 2009
• NRL obtain Research License for the UM code and performed initial
testing on FNMOC HPC platforms in late 2008/early 2009.
• Issues:
– UM throughput performance is very poor on the FNMOC Linux Cluster
architecture
– Navy IA policies created significant impediments for working with the UM
code
– Business and Licensing issues are a concern
• NRL will test the UM on the IBM architecture at NAVO/MSRC
• Remote operations of a UM-based ensemble at NAVO/MSRC may be a
possibility
Fleet Numerical…
Supercomputing Excellence for Fleet Safety and Warfighter Decision Superiority…
Current Thinking on Navy Global NWP
• Fully support NUOPC
• Continue operational transition of most-promising nearterm NOGAPS upgrades:
– 4DVAR (NAVDAS-AR)
– Semi-Lagrangian
• Continue exploring the possibility of running operational
UKMO/UM based ensemble (NAVO/MSRC)
• Seek partnerships for development of a next-generation
global NWP model for operational implementation at
FNMOC ~ 2016-2018
• Link to growing Navy interest in: (1) the arctic, (2) climate
change, and (3) energy conservation
Fleet Numerical…
Supercomputing Excellence for Fleet Safety and Warfighter Decision Superiority…
Status and Milestones for 4DVAR
(NAVDAS-AR)
• NOGAPS/NAVDAS-AR top raised from 1.0 hPa to
0.04 hPa with NOGAPS T239L42 implementation
• Began assimilating AIRS, IASI and SSMI/S radiances
• Produced the following improvements relative to the
operational NOGAPS/NAVDAS (3DVAR) implementation:
– ~6 hours improvement in 500 mb height anomaly correlation at TAU120
– ~50 nm improvement in TC track performance at TAU120
(based on Aug/Sep 2008 cases)
• Final NAVDAS-AR Beta Testing began 16 Mar 2009 and
will complete in Jul 2009
• Expect operational implementation in Aug 2009
Fleet Numerical…
Supercomputing Excellence for Fleet Safety and Warfighter Decision Superiority…
Impact of NAVDAS-AR on NOGAPS
TC Track Skill
01 August 2008 – 17 September 2008
TC Track Error (nm)
350
300
250
200
NAVDAS (OPS)
NAVDAS-AR
150
100
50
0
Fleet Numerical…
24
48
72
96
120
150
116
86
68
50
Number of Forecasts
Supercomputing Excellence for Fleet Safety and Warfighter Decision Superiority…
23
MILCON Project
Fleet Numerical…
Supercomputing Excellence for Fleet Safety and Warfighter Decision Superiority…
MILCON Project
• $9.3M 14351 sq ft expansion/renovation of FNMOC
Building 700 (Computer Center Building)
–
–
–
–
–
Adds 2500 sq ft to Computer Center for NPOESS IDPS
Expands Ops Floor from 1550 to 3200 sq ft
Expands SCIF Ops Floor from 1947 to 3012 sq ft
Expands SCIF Computer Center from 875 to 2240 sq ft
Expands Auditorium from 47 seats to 100 seats
• Schedule
– Design/Build Contract awarded Sep 2008
– Groundbreaking May 27, 2009
– Completion approximately Oct 2010
Fleet Numerical…
Supercomputing Excellence for Fleet Safety and Warfighter Decision Superiority…
FNMOC MILCON Project
FNMOC Building 700 Computer Center
Fleet Numerical…
Supercomputing Excellence for Fleet Safety and Warfighter Decision Superiority…
Summary
• Several of our recent accomplishments of COPC interest:
– Achieved IOC for the A2 Linux Cluster, new host for the FNMOC models Ops
Run
– Commenced operational support for NAEFS, with NOGAPS Ensemble fields
delivered daily to NCEP
– Upgraded the NOGAPS Ensemble with ET initialization
– Implemented the coupled air-sea version of the GFDN TC model
• We have embraced Linux-based HPC as we continue to build out our A2
cluster to accommodate the full range of our operational workload
• We are partnered with NSA to achieve a DoD approved Cross Domain
Solution capable of meeting our operational throughput requirements
• Along with ONR, NRL, CNMOC, and OPNAV N84 we are exploring the
way-ahead for Navy Numerical Weather Prediction
• We will break ground on $9.3M MILCON project in June to expand and
renovate Building 700 Computer Center
Fleet Numerical…
Supercomputing Excellence for Fleet Safety and Warfighter Decision Superiority…
Questions?
Fleet Numerical…
Supercomputing Excellence for Fleet Safety and Warfighter Decision Superiority…
FY09 Objectives
Fleet Numerical…
Supercomputing Excellence for Fleet Safety and Warfighter Decision Superiority…
Top Five FY09 Objectives
• Implement the A76 Most Efficient Organization (MEO) as the
Information Technology Services Department (ITSD).
• Achieve Full Operational Capability (FOC) for the A2-0 (Opal),
A2-1 (Ruby) and A2-2 (Topaz) Linux Cluster systems.
• Achieve and maintain required Information Assurance (IA)
accreditations, and implement DoD-approved Cross Domain
Solution (CDS) for transferring data between classification levels.
• Design and install satellite processing systems in preparation for
launch of NPOESS and NPP.
• Increase skill of METOC model products through implementation
of new models, upgrades to existing models, and the assimilation
of additional observational data from new sources and
sensors.
Fleet Numerical…
Supercomputing Excellence for Fleet Safety and Warfighter Decision Superiority…
Additional FY09 Objectives
•
•
•
•
•
•
•
•
•
•
•
•
Execute the Building 700 MILCON Project while maintaining at least 98.5%
uptime for operations.
Maintain excellence in Submarine Enroute Weather Forecast (SUBWEAX)
support.
Achieve IOC for the Automated Optimum Track Ship Routing (AOTSR)
system.
Field and support next generation version of the Naval Oceanography Portal
(NOP).
Meet all requirements for the Weather Reentry-Body Interaction Planner
(WRIP) project.
Meet all requirements for the North American Ensemble Forecast System
(NAEFS) project in preparation for full engagement in the National Unified
Operational Prediction Capability (NUOPC) initiative.
Complete all Cost of War (COW) projects.
Maintain excellence in climatology support.
Implement Resource Protection Component of the new Weather Delivery
Model.
Develop a robust Continuity of Operations (COOP) plan for key capabilities.
Complete all Primary Oceanographic Prediction System (POPS) Program
Decision Board (PDB) actions.
Achieve Project Management culture aligned to the POPS program.
Fleet Numerical…
Supercomputing Excellence for Fleet Safety and Warfighter Decision Superiority…