Transcript Document 7322700

Approximate Dynamic Programming and
Reinforcement Learning for Nonlinear
Optimal Control of Power Systems
Kumar Venayagamoorthy
University of Missouri-Rolla
Ronald Harley
Georgia Institute of Technology
ECS-0231632
ECS-0080764
November 4, 2003
Adaptive Critic Design: Nonlinear
Optimal Control
Plant
Informaton

k
J * (t ) 

Model Network
(Identifier) : To learn
Utility
Function ( U)
the dynamics of plant
U (t  k )
k 0
Optimal cost-to-go
function ( J)
Critic Networks :
To minimize the value (of derivatives)
of
J
Plant
with respect to the states
Control
Derivatives via BP
Model Network
Reinforcement Learning
Action Network :
To find optimal control
u
STATCOM Control
Simulation Results
100ms SC at PCC,
Line Voltage
V , Generator Terminal Voltage
Optimal control for FACTS devices
Internal control for static series synchronous
compensator (SSSC)
vs
is
vx
re
TurbineGovernor
vc
xe
Synchronous
Generator
SSSC
Inf. b us
idc
GTO
Control
Vdc
+
AVR Exciter
CONVC
vr
DHPNC
Series VSI
The simplified schematic of the SSSC (160 MVA, 15KV VL-L)
Optimal control for FACTS devices
Internal control for SSSC (CONVC)
+
iq*
iq

-
iq
+

+
id
+
Vdc*


ip
Vdc
kp 
kp 
+
V'dc
Vdc
ki
s v̂cq
PI-iq
id
id*
kp 
ki
s
ki
s v̂cd
PI-ip
PI-Vdc
ia
Real and reactive
current computation
Vdc
vˆcq
α  tan1( )
vˆcd
Vdc
Synchrnously
rotating ref erence
transf ormation
P*
Q*
m
2
2
vˆcd vˆcq
ib

va
Vector
phase-locked
loop
ic
GTO gate control
of series VSI
Vr
vb
vc
+

+

m
PI Based internal controller (CONVC) for the SSSC
Publication: N.G. Hingorani and L. Gyugyi, “Understanding FACTS-Concepts and
Technology of Flexible AC Transmission Systems”, IEEE Press, New York, 2000.
Optimal control for FACTS devices
Case study: 100 ms three phase short circuit test at
receiving-end (infinite-bus)
130
120
Uncompensated
110
100
CONVC
d [Degree]
90
80
DHPNC
70
60
50
40
30
20
0
1
Rotor angle
2
3
Time [s]
4
5
6
Optimal control for FACTS devices
External control for series capacitive reactance
compensator (SCRC)
vs
vr 0
Line #1
TurbineGovernor
Line #2

xe1
re2
xe2
Synchronous
Generator
AVR Exciter
1
1  sTC
re1
Filtering
K C sT W
1  sT W
XC
+

vc
Inf. bus
Voltage
Source
Inverter
X C*
1
1  sTC
is
GTO
+
XC
Internal Control
of SCRC
+
Damping
Controller
External Control
SCRC
Vdc
Schematic single-line diagram showing an SCRC with external controller
(160 MVA, 15KV VL-L)
Optimal control for FACTS devices
DHP based external controller (DHPEC)
vr
vs
Line #1
TurbineGovernor
AVR Exciter

Line #2
Synchronous
Generator
(DHPEC)
xe1
re2
xe2
XC*
XC
+
vc
is
Inf. bus
Voltage
Source
Inverter
+
DHP based
external controller
re1
GTO
XC
SCRC
+
Internal Control
of SCRC
Vdc
Schematic single-line diagram showing the DHP based external
controller (DHPEC)
Optimal control for FACTS devices
Case study: Step changes X*C [pu]
0.1
Fixed X*C
K C=1.5
0.08
DHPC
0.06
  [rad/s]
0.04
0.02
0
-0.02
-0.04
-0.06
-0.08
-0.1
0
5
15
10
Time [s]
Speed deviation
20
25
Application in Multi-Machine power
system
DHPEC-S
AREA 1
AREA 2
CONVEC-S
S2
13.2 kV
500 kV
13.2 kV
1
500 kV
500 kV
4
5
500 kV
Gen 3
1600 MVA
6
10
Line 1
FACTS
(SCRC)
Z1
Line 2
T4
Gen 1
5000 MVA
3
3
T6
13.8 kV
11
Industrial load
Line 3
Line 4
13.2 kV
2
T1
Line 5
Gen 2
2200 MVA
T5
200 km
DHPNC-G
S1
CONVC-G
115 kV
115 kV
7
8
Z2
T2
Large-scale multi-machine power system
T3
13.8 kV
9
Residential
load
A UPFC in the POWER SYSTEM
Governor
Pref

Turbine

Synch
Generator
1
2
UPFC
V1
Z1
V2
-
AVR

Shunt
Inverter
V
Neurocontroller
epd
Vd
c
Series
Inverter
epq
Infinite
Bus
+
V1

Verr
Shunt
Inverter
Control
Vdcerr
Series
Inverter
Control


Vdc
Qerr
Perr
Vdcref
Pref
Neuroidentifier
Pout,
Qout
V1ref
V1ref
Vdc
Z1
R2, L2
R1, L1
Exciter
Vr

Q ref
Qinj
Pinj
ed
eq
Neurocontroller
Q
Neuroidentifier
P
Responses of the Generator for a 180 ms 3- phase Short
Circuit at bus 2 at P=0.8 p.u & Q=0.15 p.u
250
Load Angle(°)
200
Load angle
150
100
50
0
PI
5
6
7
8
9
Time (sec)
10
1.1
11
1.08
Speed response
(pu)
Speed (Pu)
-50
UPFC
NC
1.06
1.04
1.02
1
0.98
UPFC
NC
0.96
PI
4
5
6
7
8
Time (sec)
9
10
11
Micro-Machine Research Lab. at the University
of Natal, Durban, South Africa
Terminal voltage in pu
Gen. #1: Trans. Line Impedance Increase
1.01
1
0.99
CONV_CONV
CONV_PSS_CONV
DHP_CONV
0.98
0.97
Load angle in degrees
10
10.5
11
11.5
12
12.5
13
Time in seconds
13.5
14
14.5
40
35
30
CON_CONV
CONV_PSS_CONV
25
20
10
10.5
11
11.5
12
12.5 13
13.5
Time in seconds
DHP_CONV
14
14.5 15