Transcript Document

Hydrologic Data Assimilation:
Merging Measurements and Models
Steve Margulis
Assistant Professor
Dept. of Civil and Environmental Engineering
UCLA
CENS Technical Seminar Series
April 29, 2005
Hydrology and Water Resources Group
Outline
• Introduction and Motivation
• Data Assimilation and the Ensemble Kalman Filter
• Application: Soil Moisture Estimation via Assimilation of
Microwave Remote Sensing Observations into a Hydrologic Model
• Application to embedded sensor networks (?): Palmdale
Wastewater Irrigation Site (future collaboration with Tom Harmon)
Hydrology and Water Resources Group
Introduction
Hydrologic Observations:
Hydrologic Models:
Benefits:
Benefits:
• Provide important diagnostic
information about real conditions
• Representation of our knowledge of
physical processes (dynamics)
• Yield important validation and model
forcing databases
• Physical relationships between
observables and states/fluxes of interest
• Numerical tool for prediction
Limitations:
Limitations:
• Measurements generally sparse in
time and/or space
• Simplified abstractions of reality
 interpolation/extrapolation
• Subject to uncertainties in timevarying inputs/time-invariant parameters
 downscaling/upscaling
• Contain measurement error
• Often measuring states not fluxes
How can we combine the benefits of both in an optimal framework?
Hydrology and Water Resources Group
What is data assimilation?
Goal: Data assimilation seeks to characterize the true state of an environmental
system by combining information from measurements and models.
Typical measurements for hydrologic applications:
•
Ground-based hydrologic and geological measurements (stream flow, soil
moisture, soil properties, canopy properties, etc.)
•
Ground-based meteorological measurements (precipitation, air temperature,
humidity, wind speed, etc.)
•
Remotely-sensed measurements which are sensitive to hydrologically
relevant variables (e.g. water vapor, soil moisture, etc.)
Mathematical models used for data assimilation:
•
Models of the physical system of interest
•
Models of the measurement process
•
Probabilistic descriptions of uncertain model inputs and measurement errors
A description based on combined information should be better than one obtained from
either measurements or model alone.
Hydrology and Water Resources Group
Key Features of Environmental Data Assimilation Problems
State estimation -- System is described in terms of state variables (random
vectors), which are characterized from available information
Multiple data sources -- Estimates are often derived from different types of
measurements (ground-based, remote sensing, etc.) measured over a range of
time and space scales
Spatially distributed dynamic systems -- Systems are often modeled with partial
differential equations, usually nonlinear. Through discretization the resulting
number of degrees of freedom (unknowns) can be very large.
Uncertainty -- The models used in data assimilation applications are inevitably
imperfect approximations to reality, model inputs may be uncertain, and
measurement errors may be important. All of these sources of uncertainty
need to be considered systematically in the data assimilation process.
Hydrology and Water Resources Group
State-space Framework for Data Assimilation
State-space concepts provide a convenient way to formulate data assimilation
problems. Key idea is to describe system of interest in terms of following variables:
•
Input variables -- variables which account for forcing from outside the
system or system properties which do not depend on the system state.
•
State variables -- dependent variables of differential equations used to
describe the physical system of interest, also called prognostic variables.
•
Measurements -- variables that are observed (with measurement error)
and are either directly or indirectly related to states.
•
Output variables -- variables that depend on state and input variables,
also called diagnostic variables.
Hydrology and Water Resources Group
Basic Probabilistic Concepts in Data Assimilation
• Uncertain forcing (u) and parameter (a) inputs:
Postulated unconditional PDFs:
fu ( u) and fa (a )
• Uncertain States (y):
Derived (from state eq.) unconditional PDF:
fy ( y )
• Uncertain measurements (z):
Measurement PDF (error structure):
fz ( z )
• Knowledge of state after measurements included:
Characterized by conditional PDF:
(Bayes Theorem)
Hydrology and Water Resources Group
fy| z (y| z)
Components of a Typical Hydrologic Data Assimilation Problem
Time-varying input u(t)
(e.g. precip.)
True
Specified
(mean)
State y (t)
(e.g. soil moist.)
Hydrologic
system
Measurement
system
True
True
Random
fluctuations
Output zi
(e.g. radiobrightness)
Random
fluctuations
Specified
(mean)
Time-invariant input a
(e.g. sat. hydr. cond.)
Random
error, 
Measured
Data assimilation
algorithm
Means and covariances
of true inputs and
output measurement
errors
State Eq:
Estimated states and
outputs
y(t )  A [ y( ), α ,u(τ ), ( ), t, ] t  τ  0 ; y(0)  y0 (α)
Measurement Eq:
zi  M [ y,  i , ti ] ; i  1,...m
The data assimilation algorithm uses specified information about input uncertainty and
measurement errors to combine model predictions and measurements. Resulting
estimates are extensive in time and space and make best use of available information.
Hydrology and Water Resources Group
Characterizing Uncertain Systems
What is a “good characterization” of the system state y(t), given the vector Zi = [z1, ..., zi] of
all measurements taken through ti?
The posterior probability density p(y| Zi) is the ideal estimate since it contains everything
we know about the state y given Zi and other model inputs
p[y(t)| Zi]
y:
Std. Dev.
u and a .
p(y)
p(y | Zi)
Prior
Conditional
(Posterior)
Zi
Mode
Mean
y(t)
In practice, we must settle for partial information about this density
• Variational DA: Derive mode of p[y(t)| Zi] by solving batch least-squares problem
• Sequential DA: Derive recursive approximation of conditional mean (and
covariance?) of p[y(t)| Zi]
Hydrology and Water Resources Group
Monte Carlo Approach: Ensemble Filtering
Divide filtering problem into two steps – propagation and update. Characterize
random states with an ensemble (j = 1, … , J) of random replicates:
p[y(ti+1)|Zi+1]
p[y(ti)|Zi]
Evolution of posterior
probability density
Update with new
measurement (zi+1)
Propagate
forward in time
p[y(ti+1)|Zi]
ti
ti+1
Time
Update with new
measurement ( zi+1 )
y j(ti+1| Zi+1)
Evolution of random
replicates in ensemble
y j(ti| Zi)
y j(ti+1| Zi)
Propagate
forward in time
ti
ti+1
Time
It is not practical to construct and update complete multivariate probability density.
Ensemble filtering propagates only replicates (no statistics). But how should update
be performed?
Hydrology and Water Resources Group
The Ensemble Kalman Filter (EnKF)
Propagation step for each replicate (y j):
y j (ti1 | Zi )  A [ y j (ti ), α ,u(ti ), t ]
Update step for each replicate:
y j (ti 1 | Z i 1 )  y j (ti 1 | Z i )  K ( zi 1  M [ y j (ti 1 | Z i )])
meas. residual
K = Kalman gain derived from propagated ensemble sample covariance.
K=Cyz [Czz+C]-1
After each replicate is updated it is propagated to next measurement time. No
need to update covariance (i.e. traditional Kalman filter)—results in large
computational savings.
Hydrology and Water Resources Group
Application: Microwave Measurement of Soil Moisture
Land surface microwave emission
(at L-band) is sensitive to surface
soil moisture (~ 5 cm).
microwave emissivity [-]
1
sand
silt
clay
0.9
0.8
0.7
0.6
0.5
0
0.2
0.4
0.6
0.8
saturation [-]
Measurement Limitations:
• indirect measurement of soil moisture – inversion?
• sparse in time (~ 1 measurement per day) – interpolation?
• spatially coarse (~10s of kilometers) – downscaling?
• contains information about surface moisture only (want rootzone soil
moisture) – extrapolation?
Hydrology and Water Resources Group
1
Test Case: Application to SGP97 Experiment Site
•
Month-long experiment in central OK in summer 1997 (~10,000 km2 area)
•
Daily airborne L-band microwave observations (17 out of 30 days) to test
feasibility of soil moisture estimation from space
•
Ground-truth soil moisture sampling performed daily at validation sites
Can we use EnKF to map rootzone soil moisture fields
and associated surface fluxes from microwave
measurements?
Margulis et al., 2002; 2005
Hydrology and Water Resources Group
Key Features of Problem
• Off-the-shelf models
 Hydrologic: NOAH LSM
 Radiative Transfer: Jackson et al. (1999)
• Spatially-distributed states and
parameters
• Dealing with model nonlinearities
and input uncertainties
• Real-time (sequential) estimation
• Next generation satellite
observations (L-band passive
microwave)
Hydrology and Water Resources Group
Spatially variable model inputs
NOAH soil class
NOAH vegetation class
Meteor. Stations
RTM Inputs
Clay fraction
Sand fraction
El Reno
0
2
4
6
8
0 2 4 6 8 10 12
NOAH Model Inputs
50 km
0
0.05
0.1
0
0.2 0.4 0.6 0.8
Estimation region ~ 40 by 280 km (11 by 70 pixels--4 km resolution)
Hydrology and Water Resources Group
Illustrative Results: Sequential Updating
•
Right columns show
estimated error in
soil moisture fields
from ensemble
(Before update) (After update)
(Before update) (After update)
0.06
Day 169
Left columns show
estimated soil
moisture fields
before and after
assimilating Tb
Estimated Vol. Soil
Moisture Error
0.3
0.04
0.2
0.02
0.1
Day 179
•
Estimated Vol. Soil
Moisture
0.3
0.02
0.015
0.2
0.01
Information in
observations used
to not only update
mean fields, but
reduces uncertainty
0.1
0.03
Day 184
•
0.3
0.02
0.2
0.1
Hydrology and Water Resources Group
0.01
Illustrative Results: Downscaling
Observing System Simulation Experiments (OSSEs)
Used to Investigate Impact of Coarse Measurements
Microwave Observations (Tb in ºK):
4 km
12 km
40 km
270
260
0.4
250
True Vol. Soil
Moisture Field
0.3
Day 178
Generate obs. at different
meas. resolutions
240
230
220
0.2
210
200
0.1
190
180
Estimated Vol. Soil Moisture Fields:
Space-time averaged results
Volumetric
Soil
Moisture
0.4
Assim. Of Tb
4 km res.
Assim. Of Tb
12 km res.
Assim. Of Tb
40 km res.
Bias
0.004
0.004
0.004
RMSE
0.024
0.033
0.043
0.3
0.2
0.1
Hydrology and Water Resources Group
Illustrative Results: Interpolation
Comparison of Estimates to Real Ground-truth Time series
Volumetric Soil Moisture
0.5
CF08
0.4
Ground truth gravimetric
meas.
0.3
Individual replicates
Estimate (Cond. mean)
0.2
Open Loop (Uncond.
Mean)
0.1
0
175
180
185
Day of Year
ER
Precip.
170
Hydrology and Water Resources Group
190
195
Microwave obs. times
Illustrative Results: Extrapolation/Flux Estimation
Surface evaporation flux (latent heat) is a function of entire rootzone moisture,
not just surface. Is information in radiobrightness propagating to sub surface?
Note “spin-up”
effect of filter
during first 10
days
Over time,
information from
Tb about surface
conditions
propagates
downward
through rootzone
Hydrology and Water Resources Group
Summary of Results
Data assimilation (in this case using the EnKF) allows for merging of model
and data. Key benefits of this framework:
• inversion of electromagnetic measurement into estimates of
hydrologic states of interest (soil moisture)
• downscaling of coarse microwave radiobrightness measurement
resolution to estimation scale (similar potential for upscaling?)
• value added data products which are essentially continuous in
time/space (interpolation between sparse measurements)
• extrapolation/propagation of information to unobserved portions of
domain (subsurface states) via incorporation of model physics
• estimates of additional outputs of interest (e.g. fluxes) which are
difficult to measure directly
• estimates of uncertainty about mean estimate (via error propagation
through system)
Hydrology and Water Resources Group
CENS Example: Wastewater Reuse in Mojave Desert
•
Where does the County Sanitation
District (CSD) of Los Angeles put 4
million gallons per day of treated
wastewater in a landlocked region?
•
Can we use embedded sensors to track
infiltration plume, assess nitrate
concentrations, apply feedback control?
Reclaimed
wastewater
irrigation pivot plots
Palmdale, CA
wastewater
treatment plant
NO3clay
top soil
sand
NO3- sensor network
groundwater
(slide courtesy of Prof. Tom Harmon)
Hydrology and Water Resources Group
Distributed Monitoring and Adaptive Management Approach
• Monitoring network
design:
– How many sensors
can we get away
with?
– How do we optimally
place them?
mote
• Interpolating between
sensors/extrapolating to
depth:
– Distributed
parameter models
– Stochastic
approaches
image by Jason Fisher (Cal-CLEANER)
(slide courtesy of Prof. Tom Harmon)
Hydrology and Water Resources Group
Site characterization
• At the field scale:
– rigorous characterization sampling
being done
– geostatistical parameterization
techniques
450
400
Northing (m)
350
300
250
200
150
100
50
0
0
50 100 150 200 250 300 350 400 450
Easting (m)
ordinary kriging
(Ks)
indicator kriging
(probability Ks exceeds...)
(slide courtesy of Prof. Tom Harmon)
Hydrology and Water Resources Group
Proposed Research/Experiments
Data Assimilation (specifically the EnKF) proposed as a potential tool for
investigating these research and operational implementation questions
• Task 1: Model and EnKF Interface Design/Implementation
– Implementation of stochastic version of hydrologic flow/transport
model
– Input error model analysis using site characterization studies
– EnKF “wrapper” design
• Task 2: Network Design with Observing System Simulation
Experiments
– Model used to generate different measurement scenarios
– Evaluation of scenarios using OSSEs to determine optimal sensor
locations, sensor numbers, etc. (via minimization of state
estimation error)
Hydrology and Water Resources Group
Proposed Research/Experiments (cont.)
• Task 3: Real-time State and Parameter Estimation
– After network deployment, use as real-time state estimation tool
– Take advantage of early-life of sensors (accurate/stable error
structure) to calibrate model parameters
– Use real-time state estimates for feedback control
• Task 4: Real-time Network Monitoring and Maintenance
– What about degradation of sensor network over time?
– Once model parameters are estimated, can measurement error be
parameterized to detect changes in measurement error structure?
Hydrology and Water Resources Group
Summary
• Data assimilation provides a very general framework for merging
measurements and models
 inversion, interpolation/extrapolation, uncertainty
propagation, etc.
• In hydrology, these techniques have primarily been used in the
context of remote sensing due to limited availability of in-situ
measurements
• Problems where embedded sensor networks can be deployed are
ideal candidates for application of these techniques where the
ultimate goal is to maximize extraction of information content from
measurements.
Hydrology and Water Resources Group
Acknowledgments
Funding for Research:
NSF Water Cycle Research Grant
Collaborators:
Dara Entekhabi (MIT)
Dennis McLaughlin (MIT)
Hydrology and Water Resources Group
Some Helpful Data Assimilation References
•
McLaughlin, D., 1995: Recent developments in hydrologic data assimilation, U.S.
Natl. Rep. Int. Union Geod. Geophys. 1991-1994, Reviews in Geophysics, 33, 977984.
•
Margulis, S.A., D. McLaughlin, D. Entekhabi, and S. Dunne, 2002: Land data
assimilation and soil moisture estimation using measurements from the Southern
Great Plains 1997 field experiment, Water Resources Research, 38(12), 1299,
doi:10.1029/2001WR001114.
•
Evensen, G., 2003: The Ensemble Kalman Filter: theoretical formulation and practical
implementation, Ocean Dynamics, 53, 343-367.
Hydrology and Water Resources Group