Transcript No Slide Title

SYSTEMS
Identification
Ali Karimpour
Assistant Professor
Ferdowsi University of Mashhad
Reference: “System Identification Theory For The User”
Lennart Ljung
lecture 11
Lecture 11
Recursive estimation methods
Topics to be covered include:

Introduction.

The Recursive Least-Squares Algorithm.

The Recursive IV Method.

Recursive Prediction-Error Methods.

Recursive Pseudolinear Regressions.

The Choice of Updating Step.

Implementation.
2
Ali Karimpour Nov 2009
lecture 11
Introduction
Topics to be covered include:

Introduction.

The Recursive Least-Squares Algorithm.

The Recursive IV Method.

Recursive Prediction-Error Methods.

Recursive Pseudolinear Regressions.

The Choice of Updating Step.

Implementation.
3
Ali Karimpour Nov 2009
lecture 11
Introduction
In many cases it is necessary, or useful, to have a model of the system available
on-line.
The need for such an on-line model is required in order to:
• Which input should be applied at the next sampling instant?
• How should the parameters of a matched filter be tuned?
• What are the best predictions of the next few output?
Adaptive
• Has a failure occurred and, if so, of what type?
Adaptive control, adaptive filtering, adaptive signal
processing, adaptive prediction.
4
Ali Karimpour Nov 2009
lecture 11
Introduction
The on-line computation of the model must completed
during one sampling interval.
Identification techniques that comply with this requirement will be called:
• Recursive identification methods. Used in this Reference.
• On-line identification.
• Real-time identification.
• Adaptive parameter estimation.
• Sequential parameter estimation.
5
Ali Karimpour Nov 2009
lecture 11
Introduction
6
Ali Karimpour Nov 2009
lecture 11
Introduction
Algorithm format
General identification method:
ˆt  F (t , Z t )
Minimizing argument of some
function or…
This form cannot be used in a recursive algorithm, since it cannot be completed
in one sampling instant.
Instead following recursive algorithm must comply:
X (t )  H (t , X (t  1), y (t ), u (t ))
ˆ  h( X (t  1))
Information state
t
Since the information in the latest pair of measurement { y(t) , u(t) } normally is
small compared to the pervious information so there is a more suitable form
X (t )  X (t  1)  t QX ( X (t  1), y(t ), u (t ))
ˆ  ˆ   Q ( X (t  1), y(t ), u (t ))
t
t 1
t

7
Small numbers reflecting the relative information value in the latest measurement.
Ali Karimpour Nov 2009
lecture 11
The Recursive Least-Squares Algorithm
Topics to be covered include:

Introduction.

The Recursive Least-Squares Algorithm.

The Recursive IV Method.

Recursive Prediction-Error Methods.

Recursive Pseudolinear Regressions.

The Choice of Updating Step.

Implementation.
8
Ali Karimpour Nov 2009
lecture 11
The Recursive Least-Squares Algorithm
Weighted LS Criterion
The estimate for the weighted least squares is:
Where
9
Ali Karimpour Nov 2009
lecture 11
The Recursive Least-Squares Algorithm
Recursive algorithm
Suppose the weighting sequence has the following property:
Now
10
Ali Karimpour Nov 2009
lecture 11
The Recursive Least-Squares Algorithm
Recursive algorithm
Suppose the weighting sequence has the following property:
11
Ali Karimpour Nov 2009
lecture 11
The Recursive Least-Squares Algorithm
Recursive algorithm
Version with Efficient Matrix Inversion
To avoid inverting at each step, let introduce
Remember matrix inversion lemma
12
Ali Karimpour Nov 2009
lecture 11
The Recursive Least-Squares Algorithm
Version with Efficient Matrix Inversion
Moreover we have
We can summarize this version of algorithm as:
13
Ali Karimpour Nov 2009
lecture 11
The Recursive Least-Squares Algorithm
The size of the matrix R(t) will depend on the λ(t)
Normalized Gain Version
14
Ali Karimpour Nov 2009
lecture 11
The Recursive Least-Squares Algorithm
Initial Condition
A possibility could be to initialize only at a time instant t0
By LS method
Clearly if P0 is large or t is large, then above estimate is the same as:
15
Ali Karimpour Nov 2009
lecture 11
The Recursive Least-Squares Algorithm
Asymptotic Properties of the Estimate
16
Ali Karimpour Nov 2009
lecture 11
The Recursive Least-Squares Algorithm
Multivariable case
Remember SISO
Now for MIMO
17
Ali Karimpour Nov 2009
lecture 11
The Recursive Least-Squares Algorithm
Kalman Filter Interpretation
The Kalman Filter for estimating the state of system
Kalman Filter
The linear regression model
can be cast to above form as:
Now, let
Exercise: Derive the Kalman filter for above mention system, and show that it is
exactly same as the Recursive Least-Squares Algorithm for multivariable case.
18
Ali Karimpour Nov 2009
lecture 11
The Recursive Least-Squares Algorithm
Kalman Filter Interpretation
Kalman filter interpretation gives important information, as well as some practical hints:
19
Ali Karimpour Nov 2009
lecture 11
The Recursive Least-Squares Algorithm
Coping with Time-varying Systems
An important reason for using adaptive methods and recursive identification in practice
is:
• The properties of the system may be time varying.
• We want the identification algorithm to track the variation.
This is handled by weighted criterion, by assigning less weight to older measurements
20
Ali Karimpour Nov 2009
lecture 11
The Recursive Least-Squares Algorithm
Coping with Time-varying Systems
These choices have the natural effect that in the recursive algorithms the step size
will not decrease to zero.
21
Ali Karimpour Nov 2009
lecture 11
The Recursive Least-Squares Algorithm
Coping with Time-varying Systems
Another and more formal alternative to deal with time-varying parameters is that
the true parameters varies like a random walk so
Exercise: Derive the Kalman filter for above mention system, and show that it is
exactly same as the Recursive Least-Squares Algorithm for multivariable case.
22
Note: The additive term R1(t) in P(t) prevents the gain L(t) from tending Ali
toKarimpour
zero. Nov 2009
lecture 11
The Recursive IV Method
Topics to be covered include:

Introduction.

The Recursive Least-Squares Algorithm.

The Recursive IV Method.

Recursive Prediction-Error Methods.

Recursive Pseudolinear Regressions.

The Choice of Updating Step.

Implementation.
23
Ali Karimpour Nov 2009
lecture 11
The Recursive IV Method
Remember Weighted LS Criterion:
Where
The IV estimate for instrumental variable method is:
24
Ali Karimpour Nov 2009
lecture 11
The Recursive IV Method
The IV estimate for instrumental variable method is:
25
Ali Karimpour Nov 2009
lecture 11
Recursive Prediction-Error Methods
Topics to be covered include:

Introduction.

The Recursive Least-Squares Algorithm.

The Recursive IV Method.

Recursive Prediction-Error Methods.

Recursive Pseudolinear Regressions.

The Choice of Updating Step.

Implementation.
26
Ali Karimpour Nov 2009
lecture 11
Recursive Prediction-Error Methods
Analogous to the weighted LS case, let us consider a weighted quadratic predictionerror criterion
Where
so we have
the gradient with respect to θ is
27
Ali Karimpour Nov 2009
lecture 11
Recursive Prediction-Error Methods
Analogous to the weighted LS case, let us consider a weighted quadratic predictionerror criterion
Remember the general search algorithm developed for PEM as:
For each iteration i, we collect one more data point, so
now define
As an approximation let:
28
Ali Karimpour Nov 2009
lecture 11
Recursive Prediction-Error Methods
Analogous to the weighted LS case, let us consider a weighted quadratic predictionerror criterion
As an approximation let:
With above approximation and taking μ(t)=1, we thus arrive at the algorithm:
This terms must be recursive too.
29
Ali Karimpour Nov 2009
lecture 11
Recursive Prediction-Error Methods
This terms must be recursive too.
30
Ali Karimpour Nov 2009
lecture 11
Recursive Prediction-Error Methods
This terms must be recursive too.
31
Ali Karimpour Nov 2009
lecture 11
Recursive Prediction-Error Methods
This terms must be recursive too.
32
Ali Karimpour Nov 2009
lecture 11
Recursive Prediction-Error Methods
Family of recursive prediction error methods
• According to the model structure
• According to the choice of R
Wide family of methods
We shall call “RPEM”
For example, the linear regression
If we consider R(t)=I
This is recursive least square
method
Where the gain could be normalized so
33
Ali Karimpour Nov 2009
This scheme has been widely used, under the name least mean squares (LMS)
lecture 11
Recursive Prediction-Error Methods
Example 11.1 Recursive Maximum Likelihood
Consider ARMAX model
A(q) y(t )  B(q)u(t )  C (q)e(t )
where
and
(t, )   y(t 1) ... y(t  na ) u(t 1) ...u(t nb )  (t 1, ) ... (t nc , )T
Remember chapter 10
  a1 a2 ...an b1 b2 ...bn c1 c2 ...cn
a
b
c

T
By rule 11.41
This scheme is known as recursive
maximum likelihood (RML) 34
Ali Karimpour Nov 2009
lecture 11
Recursive Prediction-Error Methods
Projection into DM
The model structure is well defined only for   DM giving stable predictors.
In off-line minimization this must be kept in mind as a constraint.
The same is true for the recursive minimization.
35
Ali Karimpour Nov 2009
lecture 11
Recursive Prediction-Error Methods
Asymptotic Properties
The recursive prediction-error method is designed to make updates of θ in a direction
that “on the average” is modified negative gradient of
i.e.
36
Ali Karimpour Nov 2009
lecture 11
Recursive Prediction-Error Methods
Asymptotic Properties
Moreover (see appendix 11a), for Gauss-Newton RPEM, with
 (t )  1 / t
It can be shown that ˆ(t ) has an asymptotic normal distribution, which coincides with
that of the corresponding off-line estimate. We thus have
37
Ali Karimpour Nov 2009
lecture 11
Recursive Pseudolinear Regressions
Topics to be covered include:

Introduction.

The Recursive Least-Squares Algorithm.

The Recursive IV Method.

Recursive Prediction-Error Methods.

Recursive Pseudolinear Regressions.

The Choice of Updating Step.

Implementation.
38
Ali Karimpour Nov 2009
lecture 11
Recursive Pseudolinear Regressions
Consider the pseudo linear representation of the prediction
And recall that this model structure contains, among other models, the general linear
SISO model:
A bootstrap method for estimating θ was given by (Chapter 10, 10.64)
By Newton - Raphson method
39
Ali Karimpour Nov 2009
lecture 11
Recursive Pseudolinear Regressions
By Newton - Raphson method
40
Ali Karimpour Nov 2009
lecture 11
Recursive Pseudolinear Regressions
41
Ali Karimpour Nov 2009
lecture 11
Recursive Pseudolinear Regressions
Family of RPLRs
The RPLR scheme represents a family of well-known algorithms when applied to
different special cases of
The RPLR scheme represents a family of well-known algorithms when applied to
different special cases of
The ARMAX case is perhaps the best known of this. If we choose
This scheme is known as extended least squares (ELS).
Other special cases are displayed in following table:
42
Ali Karimpour Nov 2009
lecture 11
Recursive Pseudolinear Regressions
Other special cases are displayed in following table:
43
Ali Karimpour Nov 2009
lecture 11
The Choice of Updating Step
Topics to be covered include:

Introduction.

The Recursive Least-Squares Algorithm.

The Recursive IV Method.

Recursive Prediction-Error Methods.

Recursive Pseudolinear Regressions.

The Choice of Updating Step.

Implementation.
44
Ali Karimpour Nov 2009
lecture 11
The Choice of Updating Step
Recursive Prediction-Error Methods is based prediction error approach:
Recursive Pseudolinear Regressions is based on correlation approach:
45
Ali Karimpour Nov 2009
lecture 11
The Choice of Updating Step
Recursive Prediction-Error Methods (RPEM)
Recursive Pseudolinear Regressions (RPLR)
The difference between prediction error approach and correlation approach is:
Now we are going to speak about  (t ) R 1 (t ) that modifies the update direction and
determines the length of the update step.
We just speak about RPEM, RPLR is the same just one must change
46
Ali Karimpour Nov 2009
lecture 11
The Choice of Updating Step
Update direction
Easier
computation
Better convergence
rate
There are two basic choices of update directions:
47
Ali Karimpour Nov 2009
lecture 11
The Choice of Updating Step
Update Step: Adaptation gain
An important aspect of recursive algorithm is, their ability to cope with time varing
systems. There are two different ways of achieving this
48
Ali Karimpour Nov 2009
lecture 11
The Choice of Updating Step
Update Step: Adaptation gain
In either case, the choice of update step is a trade-off between
• Tracking ability
• Noise sensitivity
A high gain means that the algorithm is alert in tracking parameter changes but at
the same time sensitive to disturbance in data.
49
Ali Karimpour Nov 2009
lecture 11
The Choice of Updating Step
Choice of forgetting factor
The choice of forgetting profile β(t,k) is conceptually simple.
For a system that changes gradually and in a stationary manner the most common
choice is:
The constant λ is always chosen slightly less than 1 so
This means that measurements that are older than T0 samples are included in the
criterion with a weight e-1≈0.36% of the most recent measurement.
So T0 is the memory time constant.
So we could select λ such that 1/(1-λ) reflects the ratio between the time constant of
variations in the dynamics and those of the dynamics itself.
Typical choices of λ are in the range between 0.98 and 0.995.
For a system that undergoes sudden changes, rather than steady and slow ones, it is
suitable to decrease λ(t) to a small value and then increase it to a value close to 150
Ali Karimpour Nov 2009
again.
lecture 11
The Choice of Updating Step
Choice of Gain γ(t)
51
Ali Karimpour Nov 2009
lecture 11
The Choice of Updating Step
Including a model of parameter changes
Kalman Filter Interpretation
Remember
Now let
52
Ali Karimpour Nov 2009
lecture 11
The Recursive Least-Squares Algorithm
Kalman Filter Interpretation
In the case of a linear regression model, this algorithm does give the optimal trade-off
between tracking ability and noise sensitivity, in terms of parameter error covariance
matrix.
The case where the parameters are subject to variations that themselves are of a
nonstationary nature, [i.e. R1(t) varies with t] needs a parallel algorithm. (see
Anderson 1985)
53
Ali Karimpour Nov 2009
lecture 11
The Choice of Updating Step
Constant systems
54
Ali Karimpour Nov 2009
lecture 11
The Choice of Updating Step
Asymptotic behavior in the time-varying case
55
Ali Karimpour Nov 2009
lecture 11
Implementation
Topics to be covered include:

Introduction.

The Recursive Least-Squares Algorithm.

The Recursive IV Method.

Recursive Prediction-Error Methods.

Recursive Pseudolinear Regressions.

The Choice of Updating Step.

Implementation.
56
Ali Karimpour Nov 2009
lecture 11
Implementation
Implementation
The basic, general Gauss-Newton algorithm was given in RPEM and RPLR.
Recursive Prediction-Error Methods (RPEM)
Recursive Pseudolinear Regressions (RPLR)
Inverse manipulation is not suited for direct implementation, (d*d matrix R must be inverted)
We shall discuss some aspects on how to best implement recursive algorithm. By using matrix
inversion lemma
Here η (a d*p matrix)
represents either φ or ψ
depending on the
approcach.
But this is p*p57
Ali Karimpour Nov 2009
lecture 11
Implementation
Implementation
Unfortunately, the P-recursion which in fact is a Riccati equation is not numerically sound:
the equation is sensitive to round-off errors that can accumulate and make P(t) indefinite
Using factorization
It is useful to represent the data matrices in factorized form so as to work with better
conditioned matrices.
• Cholesky decomposition, which Q(t) is triangular
• UD-decomposition, which U(t) is triangular and D is diagonal
Here we shall give some details of a related algorithm, which is directed based on Householder
58
transformation (problem 10T.1). (by Morf and Kailath)
Ali Karimpour Nov 2009
lecture 11
Implementation
Using factorization
• Step 1: Let
Form (p+d)*(p+d) matrix
• Step 2: Apply an orthogonal (p+d)*(p+d) transformation T (TTT=I) to L(t-1) so that TL(t-1)
becomes an upper triangular matrix. (Use QR-factorization)
Let to partition TL(t-1) as:
59
Ali Karimpour Nov 2009
lecture 11
Implementation
Using factorization
60
Ali Karimpour Nov 2009
lecture 11
Implementation
Using factorization
• Step 3: Now L(t) and P(t) are:
61
Ali Karimpour Nov 2009
lecture 11
Implementation
Using factorization in summary
Now L(t) and P(t) are:
There are several advantages with this particular way of performing.
• The only essential computation to perform is the triangularization step.
• This step gives new Q and the gain L after simple additional calculations.
• Note that Π(t) is triangular p*p matrix, so it is easy to invert it.
• Condition number of the matrix L(t-1) is much better than that of P.
62
Ali Karimpour Nov 2009