Transcript MCP: Pitfalls & Common Mistakes Dr Jeremy Bass & RES Wind Analysis Teams (UK & USA) SENIOR TECHNICAL MANAGER AWEA Wind Resource &

MCP: Pitfalls & Common Mistakes
Dr Jeremy Bass & RES Wind Analysis Teams (UK & USA)
SENIOR TECHNICAL MANAGER
AWEA Wind Resource & Project Assessment Workshop
30 September – 1 October 2009, Minneapolis, MN, USA
1
OVERVIEW – What Do You Need for MCP?
high quality (target) site data
high quality, appropriately chosen,
reference site data
MCP software
the skills, knowledge & experience
to use software and interpret results
2
OVERVIEW – What Do You Need for MCP?
high quality (target) site data
high quality, appropriately chosen,
reference site data
MCP software
the skills, knowledge & experience
to use software and interpret results
3
1. TARGET SITE DATA: PITFALLS & COMMON PROBLEMS - HARDWARE
Need to avoid instrumentation issues, including:
• poor mast installation
• poor mounting of instruments (IEC; stub mounting)
• poor choice of instruments (anemometers, vanes etc)
• poor choice of data logger and/or configuration
• insufficient power!
• poor/lack of maintenance & record keeping
INSERT
IMAGE
INSERT
IMAGE
INSERT
IMAGE
4
1. TARGET SITE DATA: PITFALLS & COMMON PROBLEMS - HARDWARE
Need to avoid instrumentation issues, including:
• poor mast installation
• poor mounting of instruments (IEC; stub mounting)
• poor choice of instruments (anemometers, vanes etc)
• poor choice of data logger and/or configuration
• insufficient power!
• poor/lack of maintenance & record keeping
INSERT
IMAGE
INSERT
IMAGE
INSERT
IMAGE
5
1. TARGET SITE DATA: PITFALLS & COMMON PROBLEMS – QUALITY CONTROL - 1
Bin Averaged Ice-Free Wind Speed Ratio (Unheated/Heated) on SWEhrnM178
2
1.8
Windspeed Ratio (Unheated/Heated)
1.6
1.4
1.2
1
0.8
0.6
0.4
0.2
0
0
30
60
90
120
150
180
210
240
270
300
330
360
Wind Direction Bin (°N)
6
1. TARGET SITE DATA: PITFALLS & COMMON PROBLEMS – QUALITY CONTROL - 2
7
1. TARGET SITE DATA: PITFALLS & COMMON PROBLEMS – QUALITY CONTROL - 2
8
1. TARGET SITE DATA: PITFALLS & COMMON PROBLEMS – QUALITY CONTROL - 2
9
1. TARGET SITE DATA: PITFALLS & COMMON PROBLEMS – QUALITY CONTROL - 2
10
1. TARGET SITE DATA: PITFALLS & COMMON PROBLEMS – QUALITY CONTROL - 2
11
1. TARGET SITE DATA: PITFALLS & COMMON PROBLEMS – QUALITY CONTROL - 2
12
1. TARGET SITE DATA: PITFALLS & COMMON PROBLEMS – UNDERSTANDING (HARD)
13
1. TARGET SITE DATA: PITFALLS & COMMON PROBLEMS – UNDERSTANDING (HARD)
14
1. TARGET SITE DATA: PITFALLS & COMMON PROBLEMS – UNDERSTANDING (HARD)
15
1. TARGET SITE DATA: PITFALLS & COMMON PROBLEMS – UNDERSTANDING (HARD)
16
2. REFERENCE SITE DATA: PITFALLS & COMMON PROBLEMS – BACKGROUND
• the fundamental principle of MCP is that site climatology, over a 20 – 25
life, is stationary, i.e. statistics consistent over time
• reference site data must be consistent with this requirement – essential!
Monthly Mean Wind Speeds at Helsingborg, Sweden: 1996 - 2007
6
Monthly means
12 month rolling
Linear (Monthly means)
4
3
2
Jan-96
Apr-96
Jul-96
Oct-96
Jan-97
Apr-97
Jul-97
Oct-97
Jan-98
Apr-98
Jul-98
Oct-98
Jan-99
Apr-99
Jul-99
Oct-99
Jan-00
Apr-00
Jul-00
Oct-00
Jan-01
Apr-01
Jul-01
Oct-01
Jan-02
Apr-02
Jul-02
Oct-02
Jan-03
Apr-03
Jul-03
Oct-03
Jan-04
Apr-04
Jul-04
Oct-04
Jan-05
Apr-05
Jul-05
Oct-05
Jan-06
Apr-06
Jul-06
Oct-06
Jan-07
Apr-07
Jul-07
Oct-07
Monthly Mean wind speed / ms-1
5
Date
17
2. REFERENCE SITE DATA: PITFALLS & COMMON PROBLEMS – BACKGROUND
• the fundamental principle of MCP is that site climatology, over a 20 – 25
life, is stationary, i.e. statistics consistent over time
• reference site data must be consistent with this requirement – essential!
18
2. REFERENCE SITE DATA: PITFALLS & COMMON PROBLEMS – BACKGROUND
16.0
14.0
12.0
10.0
8.0
6.0
4.0
2.0
0.0
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
4.5
5.0
5.5
6.0
6.5
7.0
7.5
8.0
8.5
9.0
9.5
10.0
10.5
11.0
11.5
12.0
12.5
13.0
13.5
14.0
14.5
15.0
15.5
16.0
16.5
17.0
17.5
18.0
18.5
19.0
19.5
20.0
20.5
21.0
21.5
22.0
22.5
23.0
23.5
24.0
24.5
25.0
25.5
26.0
26.5
27.0
27.5
28.0
28.5
29.0
29.5
No Observations
• the fundamental principle
of MCP is that site climatology, over a 20 – 25
Frequency Distribution Comparison (Period 1)
life, is stationary, i.e. statistics consistent over time
20.0 must be consistent with this requirement – essential!
• reference site data
18.0
Wind Speed Bin (m/s)
Measured
Normal
25.0
20.0
15.0
10.0
5.0
0.0
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
4.5
5.0
5.5
6.0
6.5
7.0
7.5
8.0
8.5
9.0
9.5
10.0
10.5
11.0
11.5
12.0
12.5
13.0
13.5
14.0
14.5
15.0
15.5
16.0
16.5
17.0
17.5
18.0
18.5
19.0
19.5
20.0
20.5
21.0
21.5
22.0
22.5
23.0
23.5
24.0
24.5
25.0
25.5
26.0
26.5
27.0
27.5
28.0
28.5
29.0
29.5
No Observations
Frequency Distribution Comparison (Period 2)
Wind Speed Bin (m/s)
Measured
Normal
19
2. REFERENCE SITE DATA: PITFALLS & COMMON PROBLEMS – FIXED MASTS
Failure to:
• examine site photos/visit site
• inspect site records
• choose site which reflects ‘regional’
winds
• choose site with similar climatology
to target site
• choose site with good long-term mean
• choose site with long enough
concurrent period available?
• choose site with long enough historic
period available?
Last requirement can create problems…
20
2. REFERENCE SITE DATA: PITFALLS & COMMON PROBLEMS – FIXED MASTS
21
2. REFERENCE SITE DATA: PITFALLS & COMMON PROBLEMS – FIXED MASTS
22
2. REFERENCE SITE DATA: PITFALLS & COMMON PROBLEMS – FUTURE PROBLEM?
However:
• In US, ASOS masts recently reequipped with sonic anemometers
• In UK, UKMO has installed consistent
instrumentation at all stations
The problem:
• instrument changes may destroy
continuity
• can result in limited number of
reference sites being suitable
• very sparse networks of low quality
meteorological stations in many areas
Outcome: often little or no suitable
reference sites available!
23
2. REFERENCE SITE DATA: PITFALLS & COMMON PROBLEMS – FUTURE PROBLEM?
However:
• In US, ASOS masts recently reequipped with sonic anemometers
• In UK, UKMO has installed consistent
instrumentation at all stations
The problem:
• instrument changes may destroy
continuity
• can result in limited number of
reference sites being suitable
• very sparse networks of low quality
meteorological stations in many areas
Outcome: often little or no suitable
reference sites available!
24
2. REFERENCE SITE DATA: PITFALLS & COMMON PROBLEMS – MESO-SCALE DATA
As alternative, ‘virtual’ mast data may
be appropriate:
• NCEP/NCAR Re-Analysis 2 data (2.5
deg resolution; 6 hour time base)
• Meso-scale data
• WorldWind Atlas
• Others…
Don’t forget that:
• must fulfill same requirements as
fixed mast data!
• use with caution!
• last resort option!!
25
3. MCP – PRE-PROCESSING 1
Get to know your data:
• create time series plots of target and
reference site data
• are time series in-phase?
• do time series display the same gross
trends?
In practice the process of identifying a
good reference site is iterative!
26
3. MCP – PRE-PROCESSING 1
Get to know your data:
• create time series plots of target and
reference site data
• are time series in-phase?
• do time series display the same gross
trends?
In practice the process of identifying a
good reference site is iterative!
27
3. MCP – PRE-PROCESSING 1
Get to know your data:
• create time series plots of target and
reference site data
• are time series in-phase?
• do time series display the same gross
trends?
In practice the process of identifying a
good reference site is iterative!
28
3. MCP – PRE-PROCESSING 1
Get to know your data:
• create time series plots of target and
reference site data
• are time series in-phase?
• do time series display the same gross
trends?
In practice the process of identifying a
good reference site is iterative!
29
3. MCP – PRE-PROCESSING 2
• Create scatter plots of target and reference site data
– may give insight into choice of suitable MCP algorithm BUT...
– scatter plots often misleading and need a 3rd dimension (example)
• generally need to ensure that correlation coefficient, r, is  0.7
30
3. MCP – PRE-PROCESSING 2
• Create scatter plots of target and reference site data
– may give insight into choice of suitable MCP algorithm BUT...
– scatter plots often misleading and need a 3rd dimension (example)
• generally need to ensure that correlation coefficient, r, is  0.7
31
3. MCP – CHOICE OF ALGORITHM - 1
Assuming we have in-phase, suitably preaveraged QC’d data, what MCP algorithm
should we choose?
• several classes of algorithm:
– linear models (y = mx+c)
– non-linear models
– JPD-type models
– neural network models
• within each class, several choices
available
• all have strengths and weaknesses!
Choice might depend on what you want
to use MCP results for!
32
3. MCP – CHOICE OF ALGORITHM - 1
Assuming we have in-phase, suitably preaveraged QC’d data, what MCP algorithm
should we choose?
• several classes of algorithm:
– linear models (y = mx+c)
– non-linear models
– JPD-type models
– neural network models
• within each class, several choices
available
• all have strengths and weaknesses!
Choice might depend on what you want
to use MCP results for!
33
3. MCP – CHOICE OF ALGORITHM - 1
Assuming we have in-phase, suitably preaveraged QC’d data, what MCP algorithm
should we choose?
• several classes of algorithm:
– linear models (y = mx+c)
– non-linear models
– JPD-type models
– neural network models
• within each class, several choices
available
• all have strengths and weaknesses!
Choice might depend on what you want
to use MCP results for!
34
3. MCP – CHOICE OF ALGORITHM - 1
Assuming we have in-phase, suitably preaveraged QC’d data, what MCP algorithm
should we choose?
• several classes of algorithm:
– linear models (y = mx+c)
– non-linear models
– JPD-type models
– neural network models
• within each class, several choices
available
• all have strengths and weaknesses!
Choice might depend on what you want
to use MCP results for!
From Paul van Lieshout’s ‘Wind resource analysis
based on the properties of wind or “SKM Weibull’s
correlation methodology evaluated”’ paper at AllEnergy 2009 conference
35
3. MCP – CHOICE OF ALGORITHM – 2
• JPD methods, e.g. RES matrix method
36
3. MCP – CHOICE OF ALGORITHM – 2
• JPD methods, e.g. RES matrix method
37
3. MCP – CHOICE OF ALGORITHM – 2
• JPD methods, e.g. RES matrix method
38
3. MCP – CHOICE OF ALGORITHM – 2
• JPD methods, e.g. RES matrix method
39
3. MCP – DATA SUB-CATEGORISATION
• typically data decomposed into subcategories prior to applying MCP
• typical sub-category is wind direction
• 12 (or 16) directional sectors common
• not always good choice - inspection of
the wind rose may inform this
• inspection of diurnal trend may help
inform this choice (hourly)
– if pronounced trend, consider a
number of ‘time of day’ sectors
– trying to capture periods with
similar atmospheric stability
– see Andy Oliver & Kris Zarling’s
paper at AWEA 2009!
Regardless of sub-categorisation, need
enough data to populate all sectors
40
3. MCP – PREDICTION APPROACH & UNCERTAINTY ANALYSIS
Long Term Estimate (LTE)
Historic Estimate (HE)
Site Measurements (AAE)
Target Site
Historic Reference Measurements
Concurrent Period
Relationship (MCP)
Reference Site
2
2
Vn  VN  ({ M2 / NHE }  { M2 / n}   MCP
  Inst
)1 / 2
Time
2
Vn'  VN'  ({ M2 / N AAE}  { M2 / n}   %2   Inst
)1 / 2
3. MCP – PREDICTION APPROACH & UNCERTAINTY ANALYSIS
Long Term Estimate (LTE)
Historic Estimate (HE)
Site Measurements (AAE)
Target Site
Historic Reference Measurements
Concurrent Period
Relationship (MCP)
Reference Site
2
2
Vn  VN  ({ M2 / NHE }  { M2 / n}   MCP
  Inst
)1 / 2
Time
2
Vn'  VN'  ({ M2 / N AAE}  { M2 / n}   %2   Inst
)1 / 2
3. MCP – PREDICTION APPROACH & UNCERTAINTY ANALYSIS
Long Term Estimate (LTE)
Historic Estimate (HE)
Site Measurements (AAE)
Target Site
Historic Reference Measurements
Concurrent Period
Relationship (MCP)
Reference Site
2
2
Vn  VN  ({ M2 / NHE }  { M2 / n}   MCP
  Inst
)1 / 2
Time
2
Vn'  VN'  ({ M2 / N AAE}  { M2 / n}   %2   Inst
)1 / 2
3. MCP – SECOND STEP PREDICTIONS
44
3. MCP – SECOND STEP PREDICTION VS GAP FILLING
45
3. MCP – CONCURRENT PERIOD & SEASONALITY
Com parison of Norm alised York HE for All Sites Investigated
Sunflow er Electric
1.10
High Plains
• to avoid use only full integer periods
of data
Sleeping Bear
Somerset
1.05
Normalised HE Wind Speed
For most sites, the pattern of normal
seasonal variation means that the precise
choice of concurrent period will affect
the prediction
Hopkins
1.00
0.95
High variability initially, starting
to converge after 24 months
0.90
• not always practical!
0.85
0
• if have less than a year of data, try to
avoid extremes (e.g. summer/winter)
8760
17520
26280
35040
Num ber of hours
Com parison of Norm alised Matrix HE for All Sites Investigated
Sunflow er Electric
1.10
High Plains
Sleeping Bear
• Probably more significant for sites
with thermally, rather than pressure
driven, winds?
Somerset
1.05
Normalised HE Wind Speed
• can generally identify from data
whether significant
Hopkins
1.00
0.95
In first year, long-term predictions
can be in error by ± 5 - 10 %!
0.90
0.85
0
8760
17520
26280
35040
Num ber of hours
46
3. MCP – CONCURRENT PERIOD & SEASONALITY
Com parison of Norm alised York HE for All Sites Investigated
Sunflow er Electric
1.10
High Plains
• to avoid use only full integer periods
of data
Sleeping Bear
Somerset
1.05
Normalised HE Wind Speed
For most sites, the pattern of normal
seasonal variation means that the precise
choice of concurrent period will affect
the prediction
Seasonally-corrected
estimate is more stable
and shows less spatial
variability
High variability initially, starting
1.00
0.95
0.90
• not always practical!
to converge after 24 months
Method
shows promise!
0.85
0
• if have less than a year of data, try to
avoid extremes (e.g. summer/winter)
Hopkins
8760
17520
26280
35040
Num ber of hours
Com parison of Norm alised Matrix HE for All Sites Investigated
Sunflow er Electric
1.10
High Plains
Sleeping Bear
• Probably more significant for sites
with thermally, rather than pressure
driven, winds?
Somerset
1.05
Normalised HE Wind Speed
• can generally identify from data
whether significant
Hopkins
1.00
0.95
In first year, long-term predictions
can be in error by ± 5 - 10 %!
0.90
0.85
0
8760
17520
26280
35040
Num ber of hours
47
4. MCP – IDEAL PREDICTION STRAGEY
This might feature:
• a rigorous appreciation of
errors/uncertainty is crucial!
• the use of ‘portfolio’ MCP predictions
• predictions based on multiple
references sites, real and virtual
• consideration of whether predictions
are consistent with expectations
• ‘Round Table’ discussions amongst
colleagues
• some iteration is inevitable!
48
CONCLUSIONS
• if approached with diligence and care, MCP can be a vital tool
• it requires attention to detail at every stage of the site assessment
process, not just in MCP model building (tiny part overall!)
• you need to understand how to obtain:
–
high quality (target) site data
–
high quality, appropriately chosen, reference site data
• you need to understand the application and limitations of MCP software
• you need the skills, knowledge & experience to use it & interpret results
• experiments suggest that MCP success is far more to do with choice of
high quality reference site than it is to MCP algorithm!
49
50