Transcript Chapter 4 The Maximum Principle: General Inequality Constraints 4.1 Pure State Variable Inequality Constraints: Indirect Method It is common to require state variable to.

Chapter 4
The Maximum Principle: General Inequality
Constraints
4.1 Pure State Variable Inequality Constraints:
Indirect Method
It is common to require state variable to remain
nonnegative, i.e.,
i.e., xi(t)  0, i=1,2,…,n. Constraints exhibiting (4.1) are
called pure state variable inequality constraints. The
general form is
With (4.1): at any point where a component xi(t)>0, the
corresponding constraint xi(t)  0 is not binding and can
be ignored.
In any interval where xi(t)=0,we must have
so
that xi does not become negative.
The control must be constrained to satisfy
making fi  0, as a constraint of the mixed type (3.3) over
the interval. We can add the constraint
We associate multipliers ηi with (4.3) whenever (4.3)
must be imposed, i.e., whenever xi(t)=0. A convenient
way to do this is to impose an “either or” condition
ηi xi=0. This will make ηi=0 whenever xi >0.
We can now form the Lagrangian
where H is as defined in (3.7) and η=(η1, η2 ,…,ηn ).
We apply the maximum principle in (3.11) with
additional necessary conditions satisfied by η
and the modified transversality condition
where  is a constant vector satisfying
Since the constraints are adjoined indirectly (in this
case via their first time derivative) to form the
Lagrangian, the method is called the indirect adjoining
approach. If on the other hand the Lagrangian L is
formed by adjoining directly the constraints (4.1), i.e.,
where is a multiplier associated with (4.1), then the
method is referred to as the direct adjoining approach.
Remark 4.1 The first two conditions in (4.5) are
complementary slackness conditions on the multiplier
η. The last condition
is difficult to motivate. The
direct maximum principle multiplier  is related to η as
The complementary slackness conditions for
the direct multiplier  are   0 and x*=0. Since   0,
it follows that
Example 4.1 Consider the problem:
Solution. The Hamiltonian is
which implies the optimal control to be
When x=0, we impose
, in order to insure
that (4.10) holds. Therefore, the optimal control on the
state constraint boundary is
Now we form the Lagrangian
where 1,2, and  satisfy the complementary
slackness conditions
Furthermore, the optimal trajectory must satisfy
From the Lagrangian we also get
Let us first try (2)=  =0. Then the solution for  is the
same as in Example 2.2, namely,
Since (t)-1 on [0,1] and x(0)=1>0, the original
optimal control given by (4.11) is u*(t)=-1. Substituting
this into (4.8) we get x(t)=1-t, which is positive for t<1.
Thus
In the time interval [0,1) by (4.14), 2=0 since *<1,
and by (4.15) =0 because x>0. Therefore, 1(t)=-(t)
=2-t >0 for 0t<1, and this with u=-1 satisfies (4.13).
At t=1 we have x(1)=0 so the optimal control is given
by (4.12), which is u*(1)=0. Now assume that we
continue to use the control u*(t)=0 in the interval
1 t  2.
Figure 4.1: State and Adjoint Trajectories in Example 4.1
With this control, we can solve (4.8) beginning with
x(1)=0, and obtain x(t)=0 for 1t2. Since (t)  0 in the
same interval, we see that u*(t)=0 satisfies (4.12)
throughout this interval.To complete the solution, we
calculate the Lagrange multipliers.since u*=0, we have
1=2=0 throughout 1t2. Then from (4.16) we obtain
=-=2-t0 which, with x=0, satisfies (4.15). This
completes the solution.
Remark 4.2 In instances where the initial state or the
final state or both are on the constraint boundary, the
maximum principle may degenerate I.e., there is no
nontrivial solution of the necessary conditions,i.e.,
(t)0, t[0,T], where T is the terminal time.
4.1.1 Jump Conditions
There may be a piecewise continuous (t) satisfying a
jump condition
at a time at which the state trajectory hits its
boundary value zero.
Example 4.2 Consider Example 4.1 with T=3 and the
terminal state constraint
Clearly, the optimal control u* will be the one that
keeps x as small as possible, subject to the state
constraint (4.1) and the boundary condition
x(0)=x(3)=1. Thus,
We only compute the adjoint function and multipliers
that satisfy the optimality conditions. These are
4.2 A Maximum Principle: Indirect Method
Mixed constraints as in Chapter 3 and the pure state
inequality constraints
where h: En x E1  Ep. By the definition of function h,
(4.26) represents a set of p constraints hi(x,t)0,
i=1,2,…,p.It is noted that the constraint hi0 is called
a constraint of rth order if the rth time derivative of
hi is the first time a term in control u appears in the
expression by putting f(x,u,t) for after each
differentiation.
In case of first order constraints, h1(x,u,t) is as follows:
With respect to the ith constraint hi(x,t)0, a sub
Interval (1,2)  [0,T] with 1< 2 is called an interior
interval if hi(x(t),t)>0 for all t(1,2). If the optimal
trajectory satisfies hi(x(t),t)=0 for
for
some i, then
is called a boundary interval. An
instant
is called an entry time if there is an interior
interval ending at t=
and a boundary interval
starting at
. Correspondingly,
is called an exit
time if a boundary interval ends and an interior interval
starts at
.
If the trajectory just touches the boundary at time ,
i.e.,
and if the trajectory is in
the interior just before and just after ,then is called
a contact time. Entry, exit and contact times are called
junction times.
1
Entry
1
1
Exit
Contact
Full rank condition on any boundary interval
as follows:
where for
and
is
To formulate the maximum principle for the problem
with mixed constraints as well as first-order pure state
constraints, we form the Lagrangian as
H is defined in (3.7), u satisfies the complementary
slackness conditions stated in (3.9), and Eq satisfies
the conditions
The maximum principle sates that the necessary
conditions for u* to be an optimal control are that there
exists multipliers ,,,,,and , and the jump
parameters , which satisfy (4.29) that follows.
Note that the jump conditions on the adjoint variables
in (4.29) generalize the jump condition on H in (4.29)
requires that the Hamiltonian should be continuous at
if ht = 0.
Example 4.3 Consider the following problem with the
discount rate  0:
Solution. As one can see from Figure 4.2 and 4.3, the
optimal solution is:
Figure 4.2: Feasible State Space and Optimal State Trajectory
for Example 4.3
Figure 4.3: Adjoint Trajectory for Example 4.3
Note that at the entry time t=1 to the state constraint
(4.34), the control u* and, therefore, h1*=u*+2(t-2) is
discontinuous, i.e., the entry is non-tangential. on the
other hand, u* and h1* are continuous at t=2 so that
the exit is tangential.
With u* and x* thus obtained, we must obtain , 1 , 2,
, and  so that the necessary optimality conditions
(4.29) holds, i.e.,
From (4.41), we obtain (1-)=e- . This with (4.38) gives:
and
which, along with u* and x*, satisfy (4.29).
Note, furthermore, that  is continuous at the exit
time t=2. At the entry time so that (4.30) also holds.
4.3 Current-Value Maximum Principle: Indirect method
With the Hamiltonian H as defined in (3.33), we can
write The Lagrangian
We can now state the current-value form of the
maximum principle as given in (4.42)
4.4 Sufficiency Conditions
The sufficiency results can be stated in the indirect
adjoining framework. In order to do so, let us define the
Hamiltonian H and the Lagrangian Ld in the direct
method as
where d, d, and d are multipliers in the direct
formulation, corresponding to , , and  in the direct
formulation.
It can be shown that:
Theorem 4.1 Let
satisfy the necessary
conditions in (4.29) and let
If
is concave in (x,u) at each t [0,T], S in
(3.2) is concave in x,g in (3.3) is quasiconcave in (x,u)
h in (4.26) and a in (3.4) are quasiconcave in x, and b
in (3.5) is linear in x, then (x*,u*) is optimal.
Theorem 4.2 Theorem 4.1 remains valid if the
concavity of
in (x,u) at each t is replaced
by the concavity of the maximized Hamiltonian
in x, where
Theorem 4.1 and Theorem 4.2 are written for finite
horizon problems and remains valid if the
transversality conditions on the adjoint variables
(4.29) is replaced by the following limiting
transversality condition
Example 4.1 (Continued) First we obtain the direct
adjoint variable
It is easy to see that
is linear and hence concave in (x,u) at each t[0,2].
Functions
and
are linear and hence quasiconcave in (x,u) and x,
respectively. Functions S  0, a  0 and b  0 satisfy
The conditions of Theorem 4.1 trivially.