Transcript WDSS-II Training Module IV

WDSS-II Training Module IV
Algorithms and Tools
General Notes


Output from WDSS-II applications may be
shared across multiple machines
Any application can use the output of another
application as input




The wg display is an example of this
It provides input/launch to the “Filter” algorithms
It uses products from other algorithms
Real-time and “data playback” modes are
essentially the same modes of operation
WDSS-II application types


Data ingest applications (“ingestors”)
Single-source algorithms


Multi-source algorithms


Usually single-radar applications
Combine input data from multiple sources of one
or more instrument types
General use tools

Data filters, objective analysis tools, data
remapping, data converters, verification tools, etc.
WDSS-II primary data types

LatLonGrid: geographic projection


RadialSet: cylindrical projection



Accommodates any number of radials with variable radial
widths
PolarGrid: an indexed RadialSet
DataTable: for point data



Equal spacing in degrees latitude and longitude
Trends
tracks
CartesianGrid: equidistant projection

equal spacing in N/S/E/W directions
Other types to be described in a later presentation
Data ingest

Data-ingesting programs read “raw”
data files and convert them to one of
the internal WDSS-II formats


New input types are easy to add
Maintains a consistent internal structure for
data sharing among applications
WDSS-II Real-time data flow
Legend
Single-radar products
satellite data*
RUC analysis data (grib)
*Satellite data
are required to
be in netcdf
format.
Applications
are in boxes
gribToNetcdf
w2cloudcover
nse1
Data sources
are in ovals
WSR-88D data (level 2)
ldm2netcdf
Dashed lines
represent
optional
inputs, data
sources, or
applications
Reflectivity Velocity Sp. W.
Other optional algorithms
Reflectivity OR
ReflectivityQC
w2hail
w2vil
w2qcnn
netssap
ReflectivityQC
swatScit2D
w2circ
Scit2D
(table)
AzShear
Divergence
CellTable
MESH
VIL
MesoTable
POSH
Comp. Ref.
TvsTable
MESHTracking
Echo Tops (H_*)
1If
AzShear layers
nse is not used as an input, then PolarHail.xml and ssaparm.dat should be updated twice
daily. It is highly recommended to use nse data if accurate hail guidance are desired.
The most-used single-source
algorithms

w2qcnn: quality control neural network




May use radar-only data, or radar plus cloud cover
information
Output: ReflectivityQC & ReflectivityQComposite
http://cimms.ou.edu/~lakshman/Papers/qcnnjam.pdf
w2circ: radial velocity derivatives


Produces rotational (AzShear) and divergent
(Divergence) shear fields for every tilt
Also produces layer maxima (e.g. 0-3 km MSL)
The most-used single-source
algorithms

nse: near-storm environment



Parameters are derived from the RUC model
analysis
Provides input to other algorithms
Output similar to SPC mesoanalysis web page
Other single-source algorithms



w2hail: hail grids and echo tops
w2vil: VIL and composite reflectivity
netssap: the original SSAP



MDA, TDA, SCIT, HDA, DDPDA
Requires copy of *.dat configuration files in working
directory
dealias: independent executable of WSR-88D
build 10 dealiasing

Note that dealiasing is usually done automatically in
data ingest process for WSR-88D data (ldm2netcdf)
WDSS-II Real-time data flow
Legend
Multi-radar products
AzShear[layer]
nse*
Scit2D
(x N radars;
or from
w2merger)
w2merger
Data sources
are in ovals
Reflectivity[QC]
(x N radars)
qcinfo
QCTimeInfo
w2merger
*If nse is not used as an input,
then MRScitHail.xml should be
updated twice daily. It is highly
recommended to use nse data if
accurate hail guidance are
desired.
Dashed lines
represent
optional
inputs, data
sources, or
applications
MergedAzShe
ar[layer]
scit3D
Applications
are in boxes
MergedReflectivity[QC]
MergedReflectivity[QC]Composite
VIL products
MR_Celltable
Reflectivity_X1C
w2segmotion
EchoTop_Y2
w2accumulator
(RotationTracks)
MESH Tracks (2
hr, 6 hr, etc)
HY2_Above_HX1 (“Height Above
Isosurface”)
ClusterTable
MESH /POSH / SHI (Hail)
MergedReflectivity[QC]Composite
Forecast (15,30,45,60 min)
Scit2D (from 3D grids)
Windfield
1isosurface(C); 2reflectivity
value (dBZ)
w2merger

Multi-radar data merging


Continuously updating





2D or 3D
The grid is updated each time data are received from
any source
Writes output at user-specified time intervals
Any resolution (Vertical/horizontal)
Also runs algorithms on the 3D data field
http://cimms.ou.edu/~lakshman/Papers/w2mer
ger.pdf
w2merger preparations: cache

Pre-compute the radars that will sample the
grid point (the “cache”)




Makes all computations faster
Beam blockage is considered
Use program “createCache” (once for each radar)
w2merger will create a cache on-the-fly if one is not
available, but:


It will not include terrain data
Data will not be processed until the cache creation is
complete (which might take a while)
w2merger preparations: cache

By default, the cache is stored in
~/.w2mergercache


A cache may be extracted from a cache with
larger spatial extents (“createCache –e”)


It might be big! If you are finished processing a
domain, you should delete it
Within NSSL: extract from /mnt/radararchive
Another option: createSubdomains – create
caches for all radars in the domain
w2merger preparations: cache

You may reduce the number of radars
that affect a point by running
“postprocessCache”

e.g. if you only want the 3 “best” radars to
impact the calculation at a point
Merging strategies

Different products may require different ways
of combination


Set through the ‘-C’ option
Some examples:




Reflectivity: ExponentialTimeAndDistance or Distance
AzShear: MagnitudeMaximum
Velocity: InverseVAD or MultiDoppler
Choose the most appropriate method for the
product you are merging.


There are others: see w2merger usage for list
If you need a different merging option, add it!
Running merging and
algorithms separately


Algorithms may be run each time
w2merger writes out 3D grids of
reflectivity data
If the merger is CPU-intensive or I/Ointensive, then run the algorithms
separately, perhaps on another machine

w2merger option “-C 10”
w2merger algorithms
(-a option)

Composite or VerticalMaximum


VerticalMinimum


vertical minimum product at each lat/lon
AbsMax or AbsoluteMaximum


vertical maximum at each lat/lon
abs-max product at each lat/lon. The result retains
the sign of the maximum.
VIL


vertical integrated liquid product at each lat/lon
(assumes that the 3D grid is a grid of Reflectivity)
Includes different integration strategies (e.g. along
storm tilt, VIL Density, etc)
w2merger algorithms
(-a option)

HDA


SCIT


produces SHI, POSH, and MESH at each lat/lon
(assumes that the 3D grid is a grid of Reflectivity).
creates 2D storm cell features from the multiradar grid (assumes a grid of Reflectivity).
LayerAverage or Isotherms

produces Reflectivity at various isotherms (0,-10
and -20C), ReflectivityBelowZero,
LowestReflectivity, etc.
w2merger supplemental
output

MergerInputRadarsTable

Provides information about the current
data streams




Age
Tile
VCP
Useful for determining which radars went
into the output
w2segmotion: storm segmentation
and motion estimation




Multiple scales
Can generate statistics based on storm
areas
Motion estimates feed back into
w2merger for time/space correction
http://cimms.ou.edu/~lakshman/Papers
/kmeans_motion.pdf
Mr. SCIT (Multi-radar storm
cell identification and tracking

“scit3D” executable


Use “-g” option for Scit2D features generated by
w2merger
Use “-t” option to ingest grid fields of various
parameters that should be added to the output
table




Environmental data from RUC analysis
Precipitation rate field
Etc.
Produces “MR_CellTable” output
w2accumulator

Take the:



Maximum
Minimum, or
Sum
of all tables or grids produced over a specified
time interval. E.g.:




2-hour max MESH = a hail swath
6-hour precipitation rate integration
4-hour max of 0-3 km Azimuthal Shear (“Rotation
Tracks”)
DataTable, RadialSet, or LatLonGrid
Other useful algorithms



w2cloudcover: estimate cloud cover
over a region using IR satellite and
surface temperature
w2vortdiv: compute vorticity and
divergence from a 2D wind field
w2alarm: collect statistics within an
earth-relative polygon for any grid
Data Converters




w2awipsnc: convert WDSSII netcdf grid
files to AWIPS format
w2cropconv: convert and remap any
WDSSII RadialSet or LatLonGrid to a
LatLonGrid
w2csv2table: convert a CSV file
(spreadsheet) to a WDSS-II DataTable
w2table2csv: vice versa
Data Converters

w2geotiff: convert a WDSSII netcdf file to a
geoTIFF file



A TIFF image file with geographic information tags
(for GIS)
w2grib2conv: convert a WDSS-II file to GRIB2
netcdf2ldm: convert a set of WDSSII netcdf
files to WSR-88D level II format

Can replace AliasedVelocity with Velocity,
Reflectivity with ReflectivityQC for example
Objective analysis / filters

w2smooth: smooth the data using one
of many strategies:






Gauss
Cressman
Percent (e.g. median)
Oriented
Ellipse
Various wavelets
Objective analysis / filters

w2threshold: Thresholds one field
based on another


Example, remove VIL in areas where the
IR temperature is > 250K
Various options to smooth (using
w2smooth internally) and/or segment field
Objective analysis / filters


w2oban: convert point data to a
LatLonGrid
w2morph: morphological filters



Dilate
Erode
w2contour: create contours of a data
field
File manipulation


w2get: copy a file via rssd
w2mirror: mirror all the files listed in an lb to a
different machine



Limits the number of users “hitting” a real-time
machine
w2simulator: simulate real-time data playback
w2stitcher: stitch together two different
domains into one larger one
Suggested exercise on archive
data




Download KTLX and KINX data from May 20, 2001
from 21:00 to 22:00 UTC from NCDC
Convert it into WDSS-II netcdf format
Run w2vil to produce VIL estimates in rapid-update
mode
Merge the VIL estimates using w2merger




What is a valid combination strategy here?
createCache before merging!
Compute VIL from merging reflectivity data
Compare the two VIL estimates

Find their difference field using w2scoregrid
Suggested exercise on realtime data

Connect to two adjacent radars that are currently
experiencing weather




Create cache for domain using createSubdomains.



Look at the 2DConUS index
Overlay the radarsites shapefile
Find LB names from the tensor list
Extract from /mnt/radararchive
Run w2vil, w2merger and w2scoregrid as described
before.
Set up a w2alg.conf to do this.
End of WDSS-II Training
Module IV

What to do next :





Practice running some algorithms and tools.
You will not be able to follow module 6 (writing a
WDSS-II algorithm) unless you are familiar with
how WDSS-II algorithms in general work.
Run both a single-radar algorithm and a multisensor algorithm.
Run both on archived cases and real-time cases.
Next module: Configuring WDSS-II