Transcript SMART-ISO: A Stochastic, Multiscale Model of the PJM Energy Markets Fields Institute August 14, 2013 PENSA Laboratory Princeton University http://energysystems.princeton.edu Slide 1

SMART-ISO: A Stochastic, Multiscale Model
of the PJM Energy Markets
Fields Institute
August 14, 2013
PENSA Laboratory
Princeton University
http://energysystems.princeton.edu
Slide 1
The PENSA team
Faculty
»
»
»
»
»
Warren Powell (Director)
Ronnie Sircar (ORFE)
Craig Arnold (MAE)
Elie Bou-Zeid (CEE)
Rob Socolow (MAE)
Staff/post-docs
»
»
»
»
»
»
»
Hugo Simão (Deputy Director)
Boris Defourny
Ricardo Collado
Somayeh Moazeni
Javad Khazaei
Michael Coulon
Marcos Leone Filho
Graduate students
»
»
»
»
»
»
DH Lee (COS)
Daniel Salas (CBE)
Yinzhen Jin (CEE)
Daniel Jiang (ORFE)
Harvey Cheng (EE)
Vincent Pham (ORFE)
Undergraduate interns (2012)
»
»
»
»
»
»
»
Austin Wang (ORFE)
Tarun Sinha (MAE)
Stephen Wang (ORFE)
Henry Chai (ORFE)
Ryan Peng (ORFE)
Christine Feng (ORFE)
Joe Yang (ORFE)
Energy
storage
Solar/gas
markets
CASTLE Labs
Wind
Electricity
SMART-ISO
modeling
Computational Stochastic Optimizationprices
and Learning
www.castlelab.princeton.edu
Grid
modeling
Building
mgmt
Northeast Reliability Councils and Interconnects
Research challenges
Electricity market questions:
» What impact will wind and solar have on electricity
prices?
» Where should we locate battery storage devices to have
the biggest impact on grid congestion?
» What is the value of different types of generators?
Algorithmic questions:
» How do we optimize the bidding of storage into the
grid?
» How do we optimize the charging/discharging of
energy into storage around the grid?
» How do we obtain robust solutions to the unit
commitment problem?
Energy markets
Sources of uncertainty
» Gaussian noise
• Errors in temperature/load forecasts
• Longer-term uncertainty in fuel prices (natural gas, oil and
coal)
» Rare/infrequent events
• Generator execution/failures
• Transmission failures
» Bursty/heavy-tailed noise
• Generation/loads from neighboring grids
• Heavy-tailed locational marginal prices
» Model uncertainty
• Behavior of demand response markets (e.g. fatigue)
• Uncertainty in government policies (cap and trade? Carbon
tax? Environmental regulations? Changes in electricity market
rules?)
Stochastic programming
Stochastic search
Optimal control
Model predictive control
Reinforcement learning
Q  learning
Temporal difference learning
Markov decision processes
Simulation optimization
Decision trees
Stochastic optimization models
The objective function


t

min  E   C  St , X t ( St )  
 t 0


T
Expectation over all
Cost function
random outcomes
State variable Decision function (policy)
Finding the best policy
Given a system model (transition function)
St 1  S M  St , xt ,Wt 1 ( ) 
» With deterministic problems, we want to find the best
decision.
» With stochastic problems, we want to find the best
function (policy) for making a decision.
Four classes of policies
1) Policy function approximations (PFAs)
» Lookup tables, rules, parametric functions
2) Cost function approximation (CFAs)
» X CFA ( St |  )  arg min x X
t

C
( St , xt |  )
t ( )
3) Policies based on value function approximations (VFAs)

» X tVFA ( St )  arg min x C ( St , xt )   Vt x  Stx ( St , xt ) 
t

4) Lookahead policies
» Deterministic lookahead:
X tLA-D (St ) = arg minC(Stt , xtt ) +
xtt , xt,t+1,..., xt,t+T
T
åg
t '-t
C(Stt ' , xtt ' )
t '=t+1
» Stochastic lookahead (e.g. stochastic trees)
X tLA-S (St ) = arg minC(Stt , xtt ) +
xtt , xt,t+1,..., xt,t+T
T
p(w ) å g
å
w
ÎWt
t '=t+1
t '-t
C(Stt ' (w ), xtt ' (w ))
Optimizing storage using a PFA
Battery arbitrage
Optimizing storage using a PFA
Grid operators require that batteries bid charge and
discharge prices, an hour in advance.
140.00
120.00
100.00
80.00

 Charge
Discharge
60.00
40.00
20.00
0.00
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59 61 63 65 67 69 71
We have to search for the best values for the policy
Charge
Discharge

and

.
parameters
Optimizing storage using a VFA
Energy storage with stochastic prices, supplies and demands.
Etwind
Dt
Pt grid
Rtbattery
wind
ˆ wind
Etwind

E

E
1
t
t 1
P grid  P grid  Pˆ grid
t 1
t
t 1
load
ˆ load
Dtload

D

D
1
t
t 1
battery
Rtbattery

R
 Axt
1
t
Wt 1  Exogenous inputs
St  State variable
xt  Controllable inputs
Optimizing storage using a VFA
Bellman’s optimality equation

Vt ( St )  min xt X C ( St , xt )   E Vt 1  St 1 ( St , xt ,Wt 1 )  | St 
 Etwind 
 grid 
 Pt

 D load 
 t

 Rtbattery 
 Eˆ twind

 xtwind battery 
1
 grid 
 wind load 
 Pˆt 1 
 xt

 ˆ load 
 x grid battery 
 Dt 1 
 t

 xtgrid load 
 battery load 
 xt



x 
X ( St )  arg min xt X  C ( St , xt )     f  f ( St ) 
f F



Optimizing storage policy
Optimizing storage policy
Optimizing storage policy
Optimizing storage policy
ADP exploiting convexity
120
100
Percent of optimal
80
60
40
20
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
ADP exploiting convexity
101
100
Percent of optimal
99
98
97
96
95
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
Time
Generation
Generation
Time
Generation
Generation
Generation Generation
Generation
Generation
Time
Generation
Time
Time
Time
Time
Tim
Time
Generation
Time
Robust unit commitment
We need to achieve a robust schedule:
Generation
3
1
2
Time
Lookahead policies
Lookahead policies peek into the future
The lookahead model
» Optimize over deterministic lookahead model
. . . .
t
t 1
t2
t 3
The real process
Lookahead policies
Lookahead policies peek into the future
The lookahead model
» Optimize over deterministic lookahead model
. . . .
t
t 1
t2
t 3
The real process
Lookahead policies
Lookahead policies peek into the future
The lookahead model
» Optimize over deterministic lookahead model
. . . .
t
t 1
t2
t 3
The real process
Lookahead policies
Lookahead policies peek into the future
The lookahead model
» Optimize over deterministic lookahead model
. . . .
t
t 1
t2
t 3
The real process
Lookahead policies
Probabilistic lookahead
» Here, we approximate the information model by using a
Monte Carlo sample to create a scenario tree:
» We can try to solve this as a single “deterministic”
optimization problem. This is a direct lookahead policy.
Lookahead policies
The lookahead model
We can then simulate this lookahead policy over
time:
. . . .
t
t 1
t2
t 3
The real process
Lookahead policies
The lookahead model
We can then simulate this lookahead policy over
time:
. . . .
t
t 1
t2
t 3
The real process
Lookahead policies
The lookahead model
We can then simulate this lookahead policy over
time:
. . . .
t
t 1
t2
t 3
The real process
Lookahead policies
The lookahead model
We can then simulate this lookahead policy over
time:
. . . .
t
t 1
t2
t 3
The real process
Four classes of policies
1) Policy function approximations (PFAs)
» Lookup tables, rules, parametric functions
2) Cost function approximation (CFAs)
» X CFA ( St |  )  arg min x C  ( St , xt |  )
t
3) Policies based on value function approximations (VFAs)

» X tVFA ( St )  arg min x C ( St , xt )   Vt x  Stx ( St , xt ) 
t

4) Lookahead policies
» Deterministic lookahead:
X tLA-D (St ) = arg minC(Stt , xtt ) +
xtt , xt,t+1,..., xt,t+T
T
åg
t '-t
C(Stt ' , xtt ' )
t '=t+1
» Stochastic lookahead (“stochastic programming”)
T
X tLA-S (St ) = arg minC(Stt , xtt ) + å p(w ) å g t '-tC(Stt ' (w ), xtt ' (w ))
xtt , xt,t+1,..., xt,t+T
w ÎWt
t '=t+1
The stochastic unit commitment problem
Day-ahead planning (steam)
Hour-ahead planning (gas turbine)
Real-time simulation (economic dispatch)
The timing of decisions
The day-ahead problem
Noon
Midnight
Midnight
Midnight
Midnight
Noon
Noon
Noon
The timing of decisions
The day-ahead problem
Noon to midnight:
Steam (1) on/off decisions determined the day before
Optimize within spinning reserve margins
Optimize on/off operation of gas turbines (2)
Constrained by aggregate DC power flow
(1) Includes also most combined cycle and some gas turbine generators
(the so-called slow generators)
(2) Includes all hydro and some combined cycle generators (fast)
The timing of decisions
The day-ahead problem
Midnight to midnight:
Optimize steam on/off decisions
Optimize within spinning reserve margins
Optimize on/off operation of gas turbines
Constrained by aggregate DC power flow
Steam on/off decisions are stored and implemented
The timing of decisions
The day-ahead problem
Midnight to, say, 4am the next day
Optimize steam on/off decisions
Optimize within spinning reserve margins
Optimize on/off operation of gas turbines
Constrained by aggregate DC power flow
No decisions are implemented. These are solved
only to minimize end-of-day truncation error.
The timing of decisions
Security-constrained economic dispatch
1pm
1:051:101:151:20
2pm
3pm
The timing of decisions
Security-constrained economic dispatch
2pm
1pm
3pm
1:05
2
1
3
Turbine 2
Turbine 1
» We need to call turbine 2 right now.
Turbine 3
The nesting of decisions
t
t'
SMART-ISO
Click on graphic to play video
SMART-ISO
Historical power generation during Jan 8-14 2010
Gas turbine
Steam
Nuclear
Pumped storage
Hydro
Combined cycle
SMART-ISO: Calibration
Simulated power generation during Jan 8-14 2010
Gas turbine
Steam
Nuclear
Pumped storage
Hydro
Combined cycle
SMART-ISO: Calibration
Real-time LMPs during 13-19 Jan 2010
SMART-ISO: Calibration
Real-time LMPs during 14-20 Oct 2010
SMART-ISO: Calibration
Real-time LMPs during 22-28 Jul 2010
SMART-ISO: Offshore wind study
Mid-Atlantic Offshore Wind
Integration and Transmission
Study (U. Delaware & partners,
funded by DOE)
29 offshore sub-blocks in 5
build-out scenarios:
»
»
»
»
»
1: 8 GW
2: 28 GW
3: 40 GW
4: 55 GW
5: 78 GW
SMART-ISO: Offshore wind study
Total amount of energy and peak power in the four
test time periods:
Time Period
Total Generated
Energy (MWhr)
Peak Generated
Power (MW)
13-19.Jan.2010
13,800,000
102,000
19-25.Apr.2010
11,000,000
75,000
22-28.Jul.2010
16,400,000
129,000
14-20.Oct.2010
11,000,000
78,000
SAILOR’S ENERGY
SMART-ISO: Offshore wind study
Observed wind levels for 14-20 Oct 2010
SMART-ISO: Offshore wind study
Predicted (WRF) wind levels for 14-20 Oct 2010
SMART-ISO: Offshore wind study
Observed wind levels for 22-28 Jul 2010
SMART-ISO: Offshore wind study
Predicted (WRF) wind levels for 22-28 Jul 2010
SMART-ISO: Offshore wind study
Uncovered
demand
SMART-ISO: Offshore wind study
Uncovered
demand
The unit commitment problem
A deterministic lookahead model
» Optimize over all decisions at the same time
24
min
 xt ' t '1,...,24
 C( x , y )
t '1
t'
t'
( yt ' )t '1,...,24
Steam generation
Gas turbines
» These decisions need to made with different horizons
• Steam generation is made day-ahead
• Gas turbines can be planned an hour ahead or less
The unit commitment problem
A stochastic lookahead model
» The decision problem at time t:
x t ,t '
y t ',t '
24
min E  C ( xt ,t ' , yt ',t ' )
 xt ,t ' t '1,...,24
t '1
( yt ',t ' )t ' 1,...,24
• xt ,t ' is determined at time t, to be implemented at time t’
• y t ',t ' is determined at time t’, to be implemented at time t’
» Important to recognize information content
• At time t, xt ,t ' is deterministic.
• At time t, y t ',t ' is stochastic.
The unit commitment problem
A stochastic lookahead model
» We capture the information content of decisions
Ft (St | )=
t  48
min E  C ( xtt ' , Y ( Stt ' ))
 xtt ' t '1,...,24

up
xtmax

x


Ltt '
,t '
t ,t '
down
xt ,t '  xtmax


Ltt '
,t '

t 't
Up-ramping reserve
Down-ramping reserve
• xt ,t ' is determined at time t, to be implemented at time t’

• y t ',t ' is determined at time t’ by the policy Y ( Stt ' )
» The challenge now is to adaptively estimate the ramping
constraints tt ' , and the policies Y  (Stt ' ).
The stochastic unit commitment problem
When planning, we have to use a forecast of energy from
wind, then live with what actually happens (in the simulation)
hour 0-24
ft ,t '  Wind forecast


x t ,t ' 


The stochastic unit commitment problem
The unit commitment problem
» Stepping forward observing actual wind, making small adjustments
hour 0-24
y t ',t '
The stochastic unit commitment problem
The unit commitment problem
» Stepping forward observing actual wind, making small adjustments
hour 0-24
The stochastic unit commitment problem
The unit commitment problem
» Stepping forward observing actual wind, making small adjustments
hour 0-24
The stochastic unit commitment problem
The unit commitment problem
» Stepping forward observing actual wind, making small adjustments
hour 0-24
The stochastic unit commitment problem
The unit commitment problem
» Stepping forward observing actual wind, making small adjustments
hour 0-24
The stochastic unit commitment problem
The unit commitment problem
» Stepping forward observing actual wind, making small adjustments
hour 0-24
The stochastic unit commitment problem
The unit commitment problem
» Stepping forward observing actual wind, making small adjustments
Hours 0-24
Hours 25-48
A nested, adaptive policy
t
t'
A nested, adaptive policy
t
t'
The unit commitment problem
Our hybrid policy
» The decision xtt ' is constrained by time-dependent lower
bounds  tt ' on the amount of fast ramping capacity,
which are adaptively updated during the simulation.
» The policy Y  ( Stt ' ) is constrained by the solution xt .
» Updates to  tt ' are based on stochastic gradients which

capture their impact on both xt and on yt 't '  Y ( Stt ' ) .
dF ( St |  ) dF ( St |  ) dxtt ' dF ( St |  ) dY  ( Stt ' ) dxtt '


dtt '
dxtt '
dtt '
dyt 't '
dxtt ' dtt '
» This produces a nested, adaptive policy which requires
solving sequences of deterministic problems.
A robust lookahead policy
Ramping reserve constraints
up
t ,t '
x
ptt
Virtual reserve at t+1
ptt '
Planned power
xtdown
,t '
t
t'
ptt '  Planned power level for time t ' when planning at time t
xtup,t ',t ''  "Virtual" up-reserve planned for time t '', for nested lookahead
indexed at time t ' (within lookahead for time t ).
xtdown
,t ',t ''  "Virtual" down-reserve planned for time t '', for nested lookahead
indexed at time t ' (within lookahead for time t ).
A robust lookahead policy
Ramping reserve constraints
Virtual reserve at t+2
ptt
ptt '
t
x
up
t ,t ' 1,t ' 1t , g
x
up
t ,t ' 1,t ' 1t , g
g
t'
 pt ,t '1,g
x
x
Planned power
up
t ,t ',t ' t , g
 pt ,t ',g   Up-ramping reserve for generation g
up
t ,t ',t ' t , g
 pt ,t ',g   Change in up-ramping reserve for g
 pt ,t '1,g    xtup,t ',t ' t , g  pt ,t ',g    up Lt Systemwide change in up-ramping
A robust lookahead policy
Ramping reserve constraints
ptt
ptt '
t
Planned power
t'
» We impose systemwide up- and down- ramping
constraints for each of the nested lookahead models.
» This is all solved within a single, “deterministic”
lookahead model solved as an integer program….
» …. a very large integer program.
SMART-ISO: Offshore wind study
Unconstrained grid: two buildout levels
Unconstrained Grid
Month-Year
Jan-10
Apr-10
Jul-10
Oct-10
Buildout
Level
% Steam % Nuclear
% Combined % Gas
Cycle
Turbines
% HydroPumped
Storage
% External
(Offshore
Wind)
Average Settlement
(LMP) Cost
($/MWhr)
% Avail
Wind
% Used
Wind
% Wind
Capacity
Factor
Total Demand
Shortage
(MWhr)
Peak Demand
Shortage (MW)
0
55.9
38.0
0.2
2.2
3.7
0.0
33.08
0
0
0
0
0
2
48.8
38.0
0.3
2.7
3.8
6.4
35.92
8
6.5
26.1
9072
4652
0
45.6
47.3
0.0
2.6
4.5
0.0
26.50
0
0
0
0
0
2
39.7
47.2
0.1
2.9
4.8
5.2
27.74
9.3
5.3
24.4
7470
4817
0
58.0
31.3
4.6
2.5
3.6
0.0
82.53
0
0
0
0
0
2
53.0
31.4
3.8
3.6
3.5
4.7
50.46
6
4.8
23.3
28082
9046
0
45.5
47.3
0.0
2.7
4.5
0.0
26.57
0
0
0
0
0
2
35.3
47.3
0.0
2.8
4.7
9.9
25.94
18.2
10.1
47.5
1328
2407
Time Period
Total Generated
Energy (MWhr)
Peak
Generated
Power (MW)
13-19.Jan.2010
19-25.Apr.2010
22-28.Jul.2010
14-20.Oct.2010
13,800,000
11,000,000
16,400,000
11,000,000
102,000
75,000
129,000
78,000
Missing .18 percent of load
Missing7 .012
percent
of load
percent
of peak
power!
3 percent of peak power
SMART-ISO: Offshore wind study
Comparing different levels of ramping reserves:
SMART-ISO: Offshore wind study
Constrained grid, different buildout levels
Constrained Grid
Month-Year
Buildout
Level
% Steam % Nuclear
% Combined % Gas
Cycle
Turbines
% HydroPumped
Storage
% External
(Offshore
Wind)
Average Settlement
(LMP) Cost
($/MWhr)
% Avail
Wind
% Used
Wind
% Wind
Capacity
Factor
Total Demand
Shortage
(MWhr)
Peak Demand
Shortage (MW)
0
55.2
38.0
0.5
2.7
3.7
0.0
33.42
0
0
0
0
0
1
53.3
38.1
0.5
2.6
3.6
2.0
33.41
2.3
2.1
26.4
198
538
2
51.6
38.1
0.4
2.7
3.5
3.7
31.92
8
3.7
26.1
1623
3338
3
50.8
38.1
0.5
2.8
3.4
4.4
32.31
12
4.4
27.5
5255
4460
4
50.4
38.2
0.4
2.9
3.3
4.8
34.31
16.9
4.8
28.6
12521
5138
5
49.7
38.2
0.3
2.8
3.3
5.7
32.02
25.4
5.8
30
9154
4114
Jan-10
Unconstrained grid:
Unconstrained Grid
Month-Year
Jan-10
Buildout
Level
% Steam % Nuclear
% Combined % Gas
Cycle
Turbines
% HydroPumped
Storage
% External
(Offshore
Wind)
Average Settlement
(LMP) Cost
($/MWhr)
% Avail
Wind
% Used
Wind
% Wind
Capacity
Factor
Total Demand
Shortage
(MWhr)
Peak Demand
Shortage (MW)
0
55.9
38.0
0.2
2.2
3.7
0.0
33.08
0
0
0
0
0
1
53.8
38.1
0.2
2.3
3.6
2.1
32.82
2.3
2.1
26.4
1054
1153
2
48.8
38.0
0.3
2.7
3.8
6.4
35.92
8
6.5
26.1
9072
4652
3
45.5
38.1
0.4
2.9
3.8
9.4
39.84
12
9.5
27.5
20898
8152
4
42.9
38.2
0.4
2.9
3.7
11.9
40.88
16.9
12.1
28.6
47721
10676
5
39.4
38.2
0.5
3.0
3.6
15.3
39.36
25.4
15.4
30
55704
12710
Are we there yet?
More realistic model of the PJM
scheduling process.
Parallel computation to accelerate
simulations.
More accurate modeling of LMPs
Development of real-time AC
power flow, and analysis of
accuracy of the DC approximation
Modeling of frequency regulation
process
We are looking for an opportunity
to incorporate a distribution grid.