Transcript Optimization of Global Chassis Control Variables
A Conjugate Gradient-based BPTT-like Optimal Control Algorithm
Josip Kasać*, Joško Deur*, Branko Novaković*, Ilya Kolmanovsky**
* University of Zagreb, Faculty of Mech. Eng. & Naval Arch., Zagreb, Croatia (e-mail: [email protected], [email protected], [email protected]).
** Ford Research Laboratory, Dearborn, MI 48121-2053 USA (e-mail: [email protected]).
Introduction
In this paper a gradient-based algorithm for optimal control of nonlinear multivariable systems with control and state vectors constraints is proposed The algorithm has a backward-in-time recurrent structure similar to the backpropagation-through-time (BPTT) algorithm Original algorithm (
Euler method and standard gradient algorithm
) is extended with:
implementation of higher-order Adams methods
implementation of conjugate gradient methods
Vehicle dynamics control example maneuver – double lane change executed by using control actions of active rear steering and active rear differential actuators.
2
Continuous-time optimal control problem formulation
Find the control vector
u
(
t
)
R
m
that minimizes the cost function
J
0 0
x
t
0
f
F
0
x u
d t
subject to the nonlinear MIMO dynamics process equations
x
( )
x u x
R
n
0 with initial and final conditions of the state vector
x
x
0 ,
x
x
f
, subject to control & state vector inequality and equality constraints 0 0 3
Extending the cost function with constraints-related terms
Reduced optimal control problem - find
u
(
t
) that minimizes:
J
x
(
t f
)
t
0
f
F
x u
dt
subject to the process equations (only):
x
( )
x u x
x
0 , where penalty terms are introduced:
F
x
( ),
u
F
0
x
(
t
)
u
t
)
K E k q
1
h k
2
x
K V k p
1
g k
2
x
H
g k
x
x
(
t f
) 0
x
(
t f
)
K B k n
0 1
x k
x
2 ,
H
0, 1, if if
z
0 ,
z
0.
4
Transformation to terminal optimal control problem
In order to simplify application of higher-order numerical integration methods, an additional state variable is introduced:
x n
F
x u
x n
(0) 0,
n
n
0 1 The final continuous-time optimization problem - find the control vector
u
(
t
) that minimizes the terminal condition:
x
subject to the process equations:
x
( )
f x u x
where:
x
x
1
x
2
x n
0
x n
T
R
n
x
0 ,
f
2
n
0
F
T
5
Multistep Adams methods
1st order Adams method (Euler method): 2nd order Adams method: 3rd order Adams method:
f x
(
i
f x x
f
(
i
) ,
i
u
i
) ,
x
(0)
x
0 ,
i
0, 1, 2,...,
x
(
i
x
(0 )
x
0 ,
x x
3 2
x
1 ,
f
i
1 2
f
(
i
1) , 1, 2 , 3,...
x
(
i
x
(0)
x
0 ,
x
x
(1 ) 23 12
x
1 ,
f
16 12
f
(
i
x
( 2)
x
2 ,
i
5 12
f
(
i
2) , 2, 3 , 4,...
k
-th order Adams method:
x
(
i
x
( 0)
x
0 ,
x
j k
1
a
(
k j
)
f
(
i
x
(1)
x
1 ,..
.,
x
(
k j
1 ) ,
x
k
1 ,
i k
..
6
State-space representation of the Adams methods
3rd order Adams method:
x j
(
i x
2
n
j
(
i j
23 12
j
16 12 5
f j
12
j i
,
x
2
n
j
x
Initial conditions:
x j
(2)
x j
2 ,
x n
j
(2) 16 12
x
2
n
j
(2) 5 12
f j
(1)
f j
(1) , 5 12
f j
(0) , The fourth-order Runge-Kutta method is used (only) for calculation of the first
k
initial conditions for the
k
-th order Adams method:
k
1
k
2
k
3
k
4
x
(
i
x
i
u
t i
2
k u
1 ,
t i
2 , 2
k u
2 ,
t i
k u
3 ,
t i
2 , , 2
k
1 2
k
2 2
k
3
k
4 .
f
The Adams method (
i
) of
k
-th order requires only one computations in a sampling time
t i
The
f
(
i
) Runge-Kutta method in a sampling time
t i
of
k
-th order requires the
k
of the function computations of the function 7
State-space representation of the Adams methods
k
-th order Adams method:
x j
(
i x
(
k
1) (
i r
1, 2,
j
a
1
a r
1
j j
x
(
r
1)
x
,
k a k
2
j x j
Initial conditions:
x
1) , (
k r
1, 2,
k
l r
1
a l
,
k
1
j r l
) , The
k
-th order Adams discretization of continuous process equations:
x
(
i
u
x
(0)
x
0 where:
f
x
( )
a
1
x
1
x
2 ( )
x n
1
x n k
1
x T
R
( )
a
2 ( )
x n
2
a k f n
1
a k n T
Adams method provide same state-space formulation of the optimal control problem as Euler method:
x
(
i
Runge-Kutta method leads to:
x
(
i
u
i
),
u
(
i
1) 8
Discrete–time optimization problem
The final discrete-time optimization problem - find the control sequence
u
(0),
u
(1),
u
(2),…,
u
(
N
-1) that minimizes the terminal cost function:
J
x
x n
subject to the discrete-time process equations (
k
-th order Adams method):
x
(
i
u
x
(0)
x
0 Gradient algorithm:
u
u
u
J
, The cost function
J
time
x
(
N
) depends explicitly only on the state vector at the terminal implicit dependence on
u
(0),
u
(1),
u
(2),…,
u
(
N
-1) time state equations
x
(
i
f f
follows from the discrete-
f x u
9
Exact gradient calculation
Implicit but exact calculation of cost function gradient
J j
r nk
1
x r
J
x r
u j
The partial derivatives can be calculated backward in time for
i
=
N
-1 :
x
(
N
1),
u
(
N
1)
u
x r
j N
1
f r
u j
N N
1 1
N
1 for
i
=
N
-2 :
x
(
N
1)
f
x
(
N
2) ,
u
(
N
2)
N
2
u j
x r
N
2
f r
u j
N N
1 2
l nk
1
f r
x l
N N
1 1
u j
N
1 2
l nk
1
f r
x l
N N
1 1
f l
u j
N N
2 2 chain rule for ordered derivatives → back-propagation-through-time (BPTT) algorithm 10
The final algorithm for gradient calculation
Initialization (
i
=
N
-1 ):
J
u
(
N
1 ) ~
U
(
N
1 )
J
x
(
N
)
D
(
N
1 )
I
Backward-in-time iterations (
i
=
N
-2,
N
-3, ..., 1, 0 ):
D
(
i
)
D
(
i
1 ) ~
X
(
i
1 )
Y
(
i
)
D
(
i
) ~
U
(
i
)
J
u
(
i
)
Y
(
i
)
J
x
(
N
) and where:
J
u
J
u
J
x
x
J Y rj U rj
f r
u j
x r
u j
X rj
f r
x j
11
Conjugate gradient methods
w
(
k
1)
w
k
d d
(
k
1)
g
(
k
1)
k
d d
(
k
) – search direction vector
g
(
k
) – gradient
w
u
1 (0)
u
1 (1)
g
J
u
1 (0)
J
u
1 (1)
u m
(
N
2)
u m
(
N
1)
T
R
mN
J
u m
(
N
2)
J
u m
(
N
1)
T
R
mN
Standard gradient algorithm:
β k
=0 and
η k = const
.
Gradient algorithm with momentum:
β k
=
const
.
Standard method for computing
η k
is line search algorithm which requires one-dimensional minimization of the cost function. Computationally expensive method - require many evaluations of the cost function during one iteration of the gradient algorithm. 12
Learning rate adaptation
Learning rate adaptation (a modified version of SuperSAB algorithm):
k
d d
k
1
k
1 if if
g g
g
(
k
1)
g
(
k
1) 0 0 0
d
d
Fletcher-Reeves
:
Polak-Ribiere
:
Hestenes-Stiefel
:
Dai-Yuan
:
k
min
g
(
k
1)
T
g
g
(
k
1)
g
, max
k
min
g
(
k
1)
T
g
g
(
k
1)
g
g
, max
k
min
g
(
k
1)
T
d
g
(
k
g
(
k
1) 1)
g g
, max
k
min
d g
(
k
1)
T
g
(
k
1)
g
(
k
1)
g
, max 13
Vehicle dynamics control
x t
f
Active Front Steering D
T f
0
y t
1
T f
Power Plant
T i
Active Rear
r
Steering
r T r
2 D
T c
Central Differential
U V
CoG State Variables D
T r
Rear Differential
b c l x ti
i
i y i F xti V i y ti F yti x i U i
i T w i i -th tire
CoG
h g
2
R
3
y x z
4
t
/ 2
t
/ 2 14
1. State-Space Subsystem
1.1 Longitudinal, lateral, and yaw DOF U
Vr
1
M
F x
1
F x
2
F x
3
F x
4 ,
V
Ur
1
M
F y
1
F y
2
F y
3
F y
4 ,
r
b I zz
r
,
F y
1
F y
2
I zz c
F y
3
F y
4
t
2
I zz
F x
1
F x
3
t
2
I zz
F x
2
F x
4 ,
X Y
U
cos
U
sin
V
V U
,
V r X
,
ψ Y
- longitudinal and lateral velocity, - yaw rate, - vehicle position in the inertial system - yaw angle
F xi
,
M F yi
, - longitudinal and lateral forces - vehicle mass,
I zz b
- vehicle moment of inertia, - distance from the front axle to the CoG,
t c
- distance from the rear axle to the CoG, - track 15
1.2 The wheel rotational dynamics
i
1
I wi T i
R I wi F xti
,
j F xti T i I wi R - rotational speed of the i
-th wheel, - longitudinal force of the
i
-th tire, - torque at the
i
-th wheel, - wheel moment of inertia, - effective tire radius.
1.3 Delayed total lateral force (needed to calculate the lateral tire load shift): F
1
d F
1
d
F y
1
F y
2
F y
3
F y
4 ,
1.4 The actuator dynamics :
r
1
r
1
r
, D
r dt
1
T
D
r
1
T
D
T r
,
r
D
T r
T
- rear wheel steering angle, - rear differential torque shift, - actuator time constants.
16
2. Longitudinal and Lateral Slip Subsystem
i i
R
i
U i
,
U i V
arctan
U i i
,
U
1,3
V
1,2
t
2
r
,
br
,
U
2,4
V
3,4
3. Tire Load Subsystem
t
2
r
.
cr
.
F z
1,2
cMg
2
l h g F t
,
F z
3,4
bMg
2
l h g F t
,
l
- wheelbase
h g
- CoG height
4. Tire Subsystem
s i
i i
2 ,
F i
F D zi
sin
F xi
F xti
cos
i
F yti
sin
i
C
arctan
kBs i
,
F xti
i s i F yi
F xti
sin
i
F yti
cos
i F i
,
μ B
,
C, D
5. Rear Active Differential Subsystem
F yti
i s i F i
, - tire friction coefficient - tire model parameters
T
1,2
T b
4
T
3,4
T i
2 D
r T b
4 Δ
T r T i T b
- differential torque shift control variable, - input torque (driveline torque) and - braking torque 17
GCC optimization problem formulation
Nonlinear vehicle dynamics description: Control variables (to be optimized):
r
(ARS) Other inputs ( driver’s inputs ):
f
State variables :
U
,
V
,
r
,
i
(i = 1,...,4) , ,
X
,
Y
and D
T r
(TVD/ALSD) Cost functions definitions: Path following (in external coordinates): Control effort penalty :
J
2
t f i
1 0
u i
2 2
U i
max
dt
Different constraints implemented: control variable limit: vehicle side slip angle limit: boundary condition on
Y
and
dY
/
dt
: Reference trajectory
J
t
0
f
(
X R
X
) 2 (
Y R
Y
) 2
dt
D
T r
D
T r
max 1250 Nm max
Y
(0),
r
arctan
V U dt
0.
r
max 20 o 18
Simulation results – double line change maneuver
Front wheel steering optimization results for asphalt road ( and 2nd order Adams methods: = 1) using Euler 4 3 reference trajectory optimized trajectory (Euler) optimized trajectory (2-nd order Adams) 2 1 0 -1 0 20 40 60 X [m] 80 100 120 140 -1 -2 -3 0 2 1 0 1 2 3 Euler method 2-nd order Adams method 4 5 t [s] 6 1 0.5
0 -0.5
-1 0 1 2 3 t [s] 4 5 6 19
Simulation results – double line change maneuver
Optimization results for ARS+TVD control and order Adams methods : = 0.6 using Euler and 2nd 1 0 -1 3 2 -2 -3 -4 0 reference (Euler) no control (Euler) optimal control (Euler) reference (2-nd order Adams) no control (2-nd order Adams) optimal control (2-nd order Adams) 20 40 60 X [m] 80 100 120 140 1.5
1 0.5
0 -0.5
-1 0 1 2 3 t [s] Euler method 2-nd order Adams method 4 5 6 80 60 40 20 0 -20 -40 -60 0 1 2 3 t [s] Euler method 2-nd order Adams method 4 5 6 20
Comparison of gradient methods
Convergence properties for the double-line change example (M=400): 10 4 Gradient algorithm Momentum CG Dai-Yuan CG Hestenes-Stiefel CG Fletcher-Reeves CG Polak-Ribiere 10 3 10 2 0 1 0 -1 0 4 3 2 50 20 100 40 150 200 Number of Iterations 250 60 X [m] 80 100 300 350 400 reference trajectory Gradient algorithm Momentum CG Dai-Yuan CG Hestenes-Stiefel CG Fletcher-Reeves CG Polak-Ribiere 120 140 21
Comparison of gradient methods
Comparison of standard gradient algorithm (M=4000) with CG algorithms (M=400): 10 4 10 3 CG for M=400 Gradient algorithm (M=4000) Momentum (M=400) CG Dai-Yuan (M=400) CG Hestenes-Stiefel (M=400) CG Fletcher-Reeves (M=400) CG Polak-Ribiere (M=400) 10 2 0 500 1000 1500 2000 Number of Iterations 2500 3000 3500 4000 4 1 0 3 2 reference trajectory Gradient algorithm (M=400) Gradient algorithm (M=4000) CG Dai-Yuan (M=400) CG Fletcher-Reeves (M=400) -1 0 20 40 60 80 100 120 140 X [m]
for a similar level of accuracy the conjugate gradients methods are about 10 times faster then the standard gradient algorithm
22
10 4
Comparison of gradient methods
The number of iterations and computational time for the similar level of accuracy:
CG Fletcher-Reeves (M=250) Gradient algorithm Momentum CG Dai-Yuan CG Hestenes-Stiefel CG Fletcher-Reeves CG Polak-Ribiere Gradient method Gradient algorithm CG Dai-Yuan CG Hestenes Stiefel CG Fletcher Reeves CG Polak Ribiere Number of iterations
M
4000 Computational time 67 min 30 sec 170 170 250 400 2 min 54 sec 2 min 54 sec 4 min 17 sec 6 min 51 sec CG Polak-Ribiere and Momentum (M=400) 10 3 Gradient algorithm (M=4000) 10 2 0 CG Dai-Yuan and Hestenes-Stiefel (M=170) 50 100 150 200 Number of Iterations 250 300 350 400
for a similar level of accuracy the conjugate gradients methods Dai-Yuan and Hestenes Stiefel are about 23 times faster then the standard gradient algorithm
23
Sensitivity of the CG algorithm
CG algorithm contains four free parameters: For the parameters,
d
¯ ,
d
+ ,
η
0 ,
d
¯ ,
d
+ ,
β
max
β
max , tuning region is known in advance 0
d
d
1.15
1 max 1.2
Initial learning rate
η
0 is dependent on specific optimization problem CG method are less sensitive to the choice of
η
0 then the standard gradient algorithm 10 4 0 =0.1 (CG DY) 0 =1.0 (CG DY) 0 =10.0 (CG DY) 0 =0.1 (GA) 0 =1.0 (GA) 0 =10.0 (GA) 10 3 10 2 0 50 100 150 200 250 Number of Iterations 300 350 400 24
Conclusions
A back-propagation-through-time (BPTT) exact gradient method for optimal control has been applied for control variable optimization in Global Chassis Control (GCC) systems. The BPTT optimization approach is proven to be numerically robust, precise, and computationally efficient Recent model extensions:
model extension with roll, pitch, and heave dynamics
(
full 10-DOF model
)
use of more accurate tire model
(
full Magic formula tire model
)
introduction of a driver model for closed-loop maneuvers
The future work will be directed towards:
combined control/parameter optimization feedback systems optimization differential game controller
25