A Real-Time Automated Method to Determine Forecast Confidence Associated with Tornado Warnings

Using Spring 2008 NWS Tornado Warnings John Cintineo Cornell University Travis Smith Valliappa Lakshmanan Kiel Ortega NOAA - NSSL 24 April 2020 1



Background

Warning Decision Support System – Integrated Information (WDSS-II)    Uses merged, multi-sensor CONUS radar network combines model, lightning, and GOES satellite data Short-term severe weather forecasting products  Objective:  To examine how WDSS-II products can be used as predictors for issuing NWS tornado warnings.

 Assign objective probabilities to warnings based on varying the attribute threshold.

24 April 2020 2

Radar-derived products            Maximum Expected Size of Hail (MESH) Probability of Severe Hail (POSH) Severe Hail Index (SHI)

Vertically Integrated Liquid (VIL)

Area of VIL +30 Echo Tops of 50, 30, & 18dBZ 3-6 km &

0-2 km Azimuthal Shear

Lowest level max dBZ Reflectivity at 0C, -10C, & -20C Overall max reflectivity Height of 50dBZ above 253K isotherm Storm Environment Data    Environmental Shear Storm Relative Flow 9-11km AGL Storm Relative Helicity 0-3km   CAPE, CIN LCL min height  SATELLITE: IR band-4 min temp. (cloud tops)  Total of 23 products 3 24 April 2020

Methodology:

 Investigated archived NWS spring 2008 CONUS tornado warnings with WDSS-II radar-derived products  Each storm attribute maximum (or minimum) values computed every 1 minute of the warning  Compared attribute values from the issuance of the warning (initial values) and the expiration of the warning (lifetime max/min).

 Composite time series of each attribute  Warnings broken down by verified vs. unverified  Verification data obtained from the Storm Prediction Center’s storm data (preliminary).

 storm environment data provided by 20-km RUC model.

24 April 2020 4

Dataset:  2 May – 1 July for 0-2 km Azimuthal shear, VIL, Area of VIL +30, and reflectivity products  15 May to 1 July for 3-6 km Azimuthal shear  20 random days for Storm Environment attributes  NB: for 1 May – 10 May 0-2 km Azimuthal shear is replaced by 0-3km Azimuthal shear 1,617 Tornado Warnings Frequency of Hits = 0.256 (414 verified warnings) False Alarm Ratio = 0.744 (1,203 unverified warnings) Average Warning Duration: 38.6 mins 24 April 2020 5

Initial 0-2 km Azimuthal Shear

UNVERIFIED VERIFIED

Mean: 0.0053 s^-1 SD: 0.0044 s^-1 24 April 2020 Mean: 0.0078 s^-1 SD: 0.0053 s^-1 6

Lifetime Max 0-2km Azimuthal Shear

UNVERIFIED VERIFIED

Mean: 0.0078 s^-1 SD: 0.0051 s^-1 24 April 2020 Mean: 0.0109 s^-1 SD: 0.0055 s^-1 7

24 April 2020 0-2 km Azimuthal Shear 0.0090

0.0080

0.0070

0.0060

0.0050

0.0040

0.0030

0.0020

0.0010

0.0000

5 10 15 20 25 30 35 Time (mins) 40 45 50 55 60 65 VERIFIED UNVERIFIED 8

Probability that a warning verified, given an initial 0-2 km Az. Shear

P( Ver. | shear in bin ) for 0-2km Az. Shear 0.5

0.45

0.4

0.35

0.3

0.25

0.2

0.15

0.1

0.05

0 P(V | in bin) < 2 2 – 4 4 – 6 6 – 8 8 – 10 10 – 12 Shear bin ( x 10^-3 s^-1) 12 – 15 > 15 24 April 2020 9

Initial Vertically Integrated Liquid (VIL)

UNVERIFIED VERIFIED

Mean: 27.76 kg/m^2 SD: 20.46 kg/m^2 24 April 2020 Mean: 34.44 kg/m^2 SD: 18.66 kg/m^2 10

Lifetime Maximum VIL

UNVERIFIED VERIFIED

Mean: 37.00 kg/m^2 SD: 20.27 kg/m^2 24 April 2020 Mean: 46.35 kg/m^2 SD: 18.05 kg/m^2 11

Vertically Integrated Liquid (VIL) 38 36 34 32 30 28 26 24 22 20 5 10 15 20 25 30 35 Time (mins) 40 45 50 55 60 65 VERIFIED UNVERIFIED 24 April 2020 12

Probability that a warning verified, given an initial Vertically Integrated Liquid

( Ver. | VIL in bin ) 0.4

0.35

0.3

0.25

0.2

0.15

0.1

0.05

0 < 10 10 – 20 20 – 30 30 – 40 VIL bin (kg/m ^2) 40 – 50 50 – 60 > 60 P(V | in bin) 13 24 April 2020

CONDITIONAL PROBABILITY CONTINGENCY TABLE

Initial 0-2 km Az. Shear (s^-1) x < 0.004

0.004 <= x < 0.008

0.008 <= x < 0.012

x >= 0.012

Ver: 41 Unv: 238 Ver: 10 Unv: 61 Ver: 5 Unv: 34 Ver: 5 Unv: 19 PROB: 0.147

Ver: 29 Unv: 70 PROB: 0.141

Ver: 32 Unv: 49 PROB: 0.128

Ver: 18 Unv: 42 PROB: 0.208

Ver: 24 Unv: 26 PROB: 0.293

Ver: 15 Unv: 41 PROB: 0.395

Ver: 23 Unv: 69 PROB: 0.300

Ver: 26 Unv: 35 PROB: 0.480

Ver: 25 Unv: 27 PROB: 0.268

Ver: 0 Unv: 14 PROB: 0.250

Ver: 9 Unv: 36 PROB: 0.426

Ver: 10 Unv: 15 PROB: 0.481

Ver: 8 Unv: 9 24 April 2020 PROB: 0.000

PROB: 0.200

PROB: 0.400

PROB: 0.471

14

Summary

     Provide warning guidance for the NWS Once a NWS tornado warning is issued, WDSS-II can automatically assign a probability that it will verify, in real-time More years of warning data will lead to a better climatology of warning probabilities With more warning data, create a contingency table based on 3 or 4 of the best predictors Forecasters can use such probability data to reduce their FAR 24 April 2020 15

Future avenues of research

 Extend the data set to include past springs  Examine environment just outside the warning polygons (to capture the entire storm)  Compare spring and fall tornado warnings  Compare attributes in tornado and severe T-storm warnings  Compare warning data based on region  Investigate warnings issued in watches, and those outside of watches 24 April 2020 16

Summary

     Provide warning guidance for the NWS Once a NWS tornado warning is issued, WDSS-II can automatically assign a probability that it will verify, in real-time More years of warning data will lead to a better climatology of warning probabilities With more warning data, create a contingency table based on 3 or 4 of the best predictors Forecasters can use such probability data to reduce their FAR 24 April 2020 17

Acknowledgements

 Travis Smith  Lak  Kiel Ortega  Owen Shieh  This research was supported by an appointment to the National Oceanic and Atmospheric Administration Program through a grant award to Oak Ridge Institute for Science and Education.

Research Participation 18 24 April 2020

   

References

Erickson, S. A., Brooks, H., 2006: Lead time and time under tornado warnings: 1986-2004.

23 rd Conference on Severe Local Storms

Guillot, E., T. M. Smith, Lakshmanan, V., Elmore, K. L., Burgess, D. W., Stumpf, G. J., 2007: Tornado and Severe Thunderstorm Warning Forecast Skill and its Relationship to Storm Type.

Lakshmanan, V., T. M. Smith, K. Cooper, J. J. Levit, G. J. Stumpf, and D. R. Bright, 2006: High resolution radar data and products over the Continental United States.

22nd Conference on Interactive Information Processing Systems

, Atlanta, Amer. Meteor. Soc.

Lakshmanan, V., T. Smith, K. Hondl, G. J. Stumpf, and A. Witt, 2006: A real-time, three dimensional, rapidly updating, heterogeneous radar merger technique for reflectivity, velocity and derived products. Weather and Forecasting 21, 802-823.

 Lakshmanan, V., T. Smith, G. J. Stumpf, and K. Hondl, 2007: The warning decision support system - integrated information (WDSS-II). Weather and Forecasting 22, 592-608.  Ortega, K. L, and T. M. Smith, 2006: Verification of multi-sensor, multi-radar hail diagnosis techniques. 1st Severe Local Storms Special Symposium, Atlanta, GA, Amer. Meteo. Soc.

  Ortega, K. L., T. M. Smith, G. J. Stumpf, J. Hocker, and L. López, 2005: A comparison of multi sensor hail diagnosis techniques. 21st Conference on Interactive Information Processing Systems (IIPS) for Meteorology, Oceanography, and Hydrology, Amer. Meteo. Soc., P1.11 - CD preprints.

Witt, A., Eilts, M., Stumpf, G. J., Johnson, J. T., Mitchell, D. E., Thomas, K. W., 1998: An Enhanced Hail Detection Algorithm for the WSR-88D.

19 24 April 2020

BIN:

Verified Unverified 1174

5 10

860 1051

15 20

711 1027

25

769

30

792

35

473

40

469 3174 2244 2675 1819 2649 1971 2011 1161 1247

45

264

50

802 277

55

890 255

60

725 117

65

326

BIN:

Verified Unverified 24 April 2020

5 10

1176 1120 1047

15 20

936 1024

25

965

30

797

35

617

40

472 3089 2896 2733 2393 2613 2492 1978 1517 1169

45

367

50

977 293

55

821 343

60

852 142

65

332 20

Initial 3-6 km Azimuthal Shear

UNVERIFIED VERIFIED

Mean: 0.0054 s^-1 SD: 0.0041 s^-1 24 April 2020 Mean: 0.0076 s^-1 SD: 0.0046 s^-1 21

Lifetime Max 3-6 km Azimuthal Shear

UNVERIFIED VERIFIED

Mean: 0.0084 s^-1 SD: 0.0049 s^-1 24 April 2020 Mean: 0.0112 s^-1 SD: 0.0052 s^-1 22

3-6 km Azimuthal Shear 0.0090

0.0080

0.0070

0.0060

0.0050

0.0040

0.0030

0.0020

0.0010

0.0000

5 10 15 20 25 30 35 Time (mins) 40 45 50 55 60 65 VERIFIED UNVERIFIED

BIN:

Verified Unverified 951 2415

5 10

706 1739

15

860 2061

20

595 1429

25

851 2053

30

636 1535

35

642 1569

40

398 938

45

402 987

50

240 641

55

252 722

60

228 575

65

108 254 24 April 2020 23

Probability that a warning verified, given an initial 3-6 km Az. Shear

P( Ver. | shear in bin ) for 3-6 km Az. Shear 24 April 2020 0.5

0.4

0.3

0.2

0.1

0 < 2 2 – 3 3 – 4 4 – 5 5 – 6 6 – 7 7 – 8 8 – 9 10 – 12 9 – 10 12 – 15 > 15 Shear bin ( x 10^-3 s^-1) P(V | in bin) 24

Initial Max LL Reflectivity

Mean: 49.88 dBZ SD: 14.63 dBZ 24 April 2020 Mean: 55.09 dBZ SD: 11.02 dBZ 25

Lifetime Max LL Reflectivity

Mean: 56.86 dBZ SD: 10.39 dBZ 24 April 2020 Mean: 60.51 dBZ SD: 7.11 dBZ 26

Maximum Low Level Reflectivity 50 48 46 60 58 56 54 52 ma x_LL_d BZ ma x_LL_d BZ-UNV

BIN:

Verified Unverified 44 5 1 0 1 5 2 0 2 5 30 35 40 Tim e (m ins) 45 50 55 60 65 1187

5

3270

10

1125 3073

15

1051 2862

20

942 2516

25

1038 2765

30

983 2652

35

810 2098

40

636 1627

45

489 1264

50

374 1052

55

296 882

60

351 940

65

144 364 24 April 2020 27

Initial dBZ @ -20C

Mean: 46.58 dBZ SD: 14.21 dBZ 24 April 2020 Mean: 53.08 dBZ SD: 10.78 dBZ 28

Lifetime Max dBZ @ 20C

Mean: 52.75 dBZ SD: 11.97 dBZ 24 April 2020 Mean: 57.86 dBZ SD: 8.43 dBZ 29

24 April 2020 Maximum Reflectivity @ -20C 60 58 56 54 52 50 48 46 44 42 40 5 1 0 1 5 2 0 2 5 30 35 40 Tim e (m ins) 45 50 55 60 65 ma x_d BZ @ -2 0C ma x_d BZ @ -2 0C UNV

BIN:

Verified Unverified

5 10 15

1192 1127 1055

20 25

939 1034

30

982

35

807

40

630

45

487

50

374 3342 3127 2909 2549 2796 2688 2118 1629 1273 1051

55

294 878

60

348 944

65

142 368 30

Probability a warning verified, given a certain Az. shear P( V | shear in bin ) for 0-2km Az. Shear 0.7

0.6

0.5

0.4

0.3

0.2

0.1

0 0-0.001

0.002-0.003

0.006-0.007

0.004-0.005

0.01-0.011

0.008-0.009

0.014-0.015

0.012-0.013

0.018-0.019

0.016-0.017

0-2km Az. Shear bins 0-2km Az. Shear P( V | shear in bin) for 3-6km Az. Shear 0.3

0.2

0.1

0.8

0.7

0.6

0.5

0.4

0 0-0.001

0.002-0.003

0.006-0.007

0.004-0.005

0.008-0.009

0.01-0.011

0.014-0.015

0.012-0.013

0.016-0.017

shear bi n (s^-1) 3-6km Az. Shear 31 24 April 2020