Transcript Precision and Accuracy Agreement Indices in HSP

Precision and Accuracy
Agreement Indices in HSP
An Introduction to Rietveld Refinement
using
PANalytical X’Pert HighScore Plus v2.2d
Scott A Speakman, Ph.D.
MIT Center for Materials Science and Engineering
[email protected]
There are two different questions:
How good is my fit?
How good is my answer?
One question is easier to answer in HSP
One question is more important ...
What are the chances these are the same questions?
How Good is My Fit?
• HSP gives you many numerical indicators of
how good your Rietveld model has been refined
– Rp: residual of least-squares refinement
– wRp: weighted residual
– GOF: goodness of fit
• During refinement, you see a plot of Rp as it
improves (hopefully) during the refinement
• Numerically, you can consult Rp, wRp, and GOF
indices to judge how good your fit is
Residuals
RP
y y


y
io
io
  wi  yio  yic 

2
w
y
 i io

2
ic
Rwp



1/ 2
• Rp quantifies the difference between the observed and
calculated data points on a point-by-point basis
• Rwp weights the residual so that higher intensity data
points are more important than low intensity data points
– the rationale for using Rwp: fitting the diffraction peaks is more
important than fitting the background
– Rwp is unfavorable in some situations where the important
information is contained in the weakest peaks
• distortions in perovskites
• discerning between ordered and/or disordered system
• In general, you want Rp or Rwp to be less than 10%
Accounting for Data Quality
Rexp
 N P 

2 
  wi yio 
1/ 2
 Rwp 
GOF    

 Rexp 
2
2
• Rp and Rwp simply compare the calculated pattern to the data
– you will never have a perfect fit because all data contains noise
– HSP does not model noise
• Rexp evaluates the quality of the data
– data with more noise or low peak intensities will have a larger Rexp
– theoretically, your Rwp can never be better than the Rexp
• GOF compares Rwp to Rexp
– a GOF=1 means the model is as good as possible
– want GOF less than 4
– two ways to improve GOF
• better model or noisier data
• be careful your GOF isn’t good because your data is bad
How to find the agreement indices
• After refinement, the agreement indices are
shown in the bottom status bar
– this information disappears as soon as the mouse
shifts focus to something else
• ie as soon as the mouse floats over something else
• this makes this information very elusive- it disappears quickly
To Hunt Down the Agreement Indices
• Indices can be found in
the Object Inspector pane
for Global Parameters
• You can find help
references in
9.Algorithms > +Rietveld
Algorithm
Individual Phase Residuals
• Rp, Rwp, and GOF evaluate
how well the entire
Rietveld model fits the
entire data set
• RBragg attempts to evaluate
how well individual phases
are fit
– allows you to discern if
phase 1 is well fit and phase
2 is poorly fit
– Find RBragg in the Object
Inspector for each individual
phase
Other ways to evaluate your fit
• Visual Estimation
Counts
NIST640b
Silicon 100.0 %
10000
• Difference Pattern
• Equations can be
fooled, so you
must always use
your eyes
2500
0
30
40
50
60
70
Position [°2Theta] (Copper (Cu))
20
10
0
-10
-20
80
90
What do the agreement indices really tell us?
• Agreement indices evaluate how well the
calculated pattern produced by the Rietveld
model fits the experimental data
– does not account for poor or incorrect data
– the model may be wrong
• The only true evaluation of your results- does
this make sense?
– bond distances and angles
– difference Fourier maps
– what else is known about the sample
• complementary data
If the refinement produced significant changes in lattice
parameter or atom positions, then check the new bond
distances and angles for reasonableness
• go to Distances and Angles list
– Select the Phase from the drop-down
menu
– Click Calculate
• get options by clicking on … button
Evaluating Precision- the Estimated Standard
Deviation (ESD)
• The ESD is shown in the Deviation
Column in the Refinement Control
list OR in the Object Inspector for
individual parameters
• ESD quantifies the amount that a
parameter could change without
changing the Rwp of the refinement
– the amount of wobble or wiggle in that
parameter
– the smallest the error in that
parameter could be if everything else
is perfect
ESD is horribly misused in published literature
• ESD tells you how precisely the parameter is coupled to your refined
data
– a small ESD indicates that changes in the parameter will have a large
effect on the fit of the calculated pattern to your experimental data
• we have a precise fit
• we do not necessarily have a precise or accurate answer!
• example: we may not have allowed another variable to refine appropriately
– a large ESD indicates that changes in the parameter does not
significantly effect the goodness of the fitfit
• why?
– our measurement is not sensitive to that parameter
» Example: O atoms refined alongside Pb atoms
– our data is not good enough to refine that parameter
– we have too many correlating variables
• ESD is not a standard deviation nor does it give you error bars
• True standard deviation- refine data sets from several different
samples
Testing the Solution
• The numbers that HSP give you quantify how well the
calculated pattern fits the experimental data, nothing
more
• To test the solution
– Probe for false minima
• false minima in finding the lowest possible residual
• even the true minima might not be the correct answer
• Change a parameter, then refine again
– do you achieve the same answer?
– genetic fitting algorithms use this methodology to better avoid false
minima– may be added in HSP soon?
– Refine a second data set from the same sample
– Refine a data set from a different sample of the same population
What is the Real Answer?
• Question the First: is the solution realistic, reasonable,
and reproducible?
• Question the Second: does the answer agree with other
complementary data?
• Question the Third: how much accuracy do I really
need?
– how much time and resources am I willing to invest to get
accuracy?
• The key for QA and routine analyses- consistency
Testing your procedures- particularly for QPA
• for QPA analysis, in particular, you can make sure that
your procedure is valid by testing standard mixtures
– it is always easier to get the right solution when you already
know the answer!
• make standard mixtures of the phases found in your
sample
– use the exact same procedure for preparing the sample,
collecting the data, and refining the model
– test mixtures with a range of concentrations to evaluate the
ability to quantify a phase when it is present as the majority
phase and when it is present as a trace phase
Spiking to evaluate QPA
• if you don’t have single phase analogues for all of your materials,
then use one or two of your actual samples
– do the QPA with original sample
– add a known amount of a single phase specimen that is analogous to
one of the ingredients in your sample
– repeat the QPA
– add more of the standard specimen to your sample and repeat the QPA
again
• by making sure that the calculated amounts of the standard phase
tracks properly with your standard additions, you can gain
confidence in your refinement procedure
• by using several different levels of standard additions, you can avoid
errors from amorphous content (expected or unexpected)