Transcript NCL_and_NetCDF
The NCAR Command Language (NCL) and the NetCDF Data Format Research Tools Presentation Matthew Janiga 10/30/2012
Where Can I Learn More?
http://www.ncl.ucar.edu/index.shtml
Where Can I Learn More?
http://www.unidata.ucar.edu/software/netcdf/
What is NCL?
• • • NCL is a complete language with types, variables, operators, expressions, conditional statements, loops, and functions and procedures.
NCL can read many data formats: NetCDF, HDF, GRIB, ascii and csv, binary, WRF output, and more.
– With some additional work it can read GEMPAK, McIDAS AREA, and others.
NCL like MATLAB is an array based language. Arrays can be sent to functions (100s and growing) which perform computations, plotting functions (dozens and growing), or files (typically NetCDF but also ascii and binary).
What is NetCDF?
• • • NetCDF is a data format for storing arrays. Unlike ascii it also stores information describing the arrays.
– Attributes: strings or numbers storing: units, source locations, references, methodology, etc….
– Dimension coordinates: Arrays which describe the dimensions: lat, lon, time, level, etc. Dimensions coordinates can have their own attributes.
– NetCDF files are EASILY examined using GUIs (NCVIEW, IDV, Panopoly, etc.) or command line programs (ncdump, ncl_filedump).
– NetCDF files can also be created, read, or manipulated by functions in libraries written for Fortran, C, Java, and more!
How difficult is say… compiling a Fortran program that uses NetCDF?
– Just type "nc-config --libs“ on the command line and all flags will be given to you!
NetCDF is also fairly easy to read into programs other than NCL: R, MATLAB, etc.
– http://www.r-project.org/
NetCDF Java
The NetCDF Java is a jar file opened from the web that can convert many formats to NetCDF.
http://www.unidata.ucar.edu/downloads/netcdf/netcdf-java-4/index.jsp
NetCDF Operator
An efficient command line program for merging, splicing, and otherwise editing NetCDF files. I’ve found NCL to be much easier to use and powerful but this is another option.
http://nco.sourceforge.net
/
NCVIEW
NCVIEW is one of MANY programs that do this.
http://www.unidata.ucar.edu/softwar e/netcdf/software.html
NCVIEW is a simple GUI that lets you explore a NetCDF file: examine slices, time series, animate, and more. Look at the data you’ve generated before spending time to make a plot!
Advantages of NCL
• • • • • • Free, stable, and extremely well documented.
NCL is a fully fledged programming language not calls to compiled programs. Code yourself don’t “trick”.
NCL allows you to read in large amounts of data into memory (64-bit vars > 2GB) and perform computations there instead of constantly reading and writing to disk.
Creating your own procedures to call externally is exactly the same as writing a script (no Fortran or shell scripting necessary).
Handles many vertical and horizontal projections and conversions from one to the other.
NCL is very flexible, it can analyze any array of any dimension, type, with any coordinate description.
– Store and process: trajectories, tracks, hovmollers, time series, polar orbit swaths and other non-uniform projections, radar data, geostationary satellite data, surface obs, profiler data, radiosondes, and pretty much anything you can think of.
Disadvantages of NCL
• • • • • NCL has a bit of a learning curve. Learning how to fully utilize it and code using arrays takes time.
For some plots (plan plots and cross-sections) GEMPAK may be able to produce a similar plot with less code (assuming data is already in GEMPAK format).
GEMPAK is still heavily used in operations.
While there are lots of mathematical functions and procedures MATLAB has more types and variants.
Similarly R has more statistical tools.
NCL Eye Candy
And more…..
• • •
What’s Else can NCL Do?
NCL for EMACS OPeNDAP: The ability to grab what you want out of datasets directly from a server. Makes some local storage of data obsolete.
– Available for many datasets and reanalyses.
system and systemfunc – avail_files = systemfunc("ls -1") – time = systemfunc("date +'%s'") – system(“convert img.ps img.png &”) Use WRAPIT to send information to and from pre-compiled user-created or commercial Fortran or C programs – http://www.ncl.ucar.edu/Document/To ols/WRAPIT.shtml
http://www.ncl.ucar.edu/Applications/editor.shtml
More Features…
• • • • ParNCL - Parallel version of NCL for analyzing large datasets.
Earth System Modeling Framework regridding software.
(ESMF) NetCDF 4 – Ability to store and read groups “object-orientated variables”.
MJO CLIVAR Diagnostics – MATLAB still more flexible.
The Example Page
Documentation! The NCL webpage has 100s of examples each containing a picture of the plot and the code required to make it.
http://www.ncl.ucar.edu/Applications/
Function Documentation
Every function has an exhaustive documentation page showing how to use it, describing how the calculation is performed, and providing examples.
The Example Code
Perform the calculation Libraries: NCLs and your own.
Read data