Transcript Trend Data,” IVT Web Seminar, June, 28, 2007.
Trend Data
Lynn Torbeck Torbeck and Assoc.
Evanston, IL June 28, 2007 1
Overview
OOT vs. OOS Why trend?
How to get started Types of trends with examples OOT is relative Graphical tools Tend limits June 28, 2007 2
Why Trend Data?
Good business practice.
Early warning of possible Out Of Specification (OOS) results.
Gain process understanding.
Minimize risk of potential failures of product in the market.
Find the “gold in the hills” for process improvements.
June 28, 2007 3
Regulatory Basis for Trending
No specific regulation requirement 211.180(e) Annual Reviews FDA Form 483 for observations Establishment Inspection Reports Warning letters FDA presentations at conferences June 28, 2007 4
OOS Guidance Footnote
“Although the subject of this document is OOS results, much of the guidance may be useful for examining results that are out of trend (OOT).” How is OOT different than OOS?
How is OOT the same as OOS?
June 28, 2007 5
Out Of Specification - OOS
OOS is the comparison of one result versus a predetermined specification criteria.
OOS investigations focus on determining the truth about that one value.
Is the OOS result confirmed or not?
June 28, 2007 6
Out Of Trend - OOT
OOT is the comparison of many historical data values versus time.
OOT investigations focus on understanding non-random changes.
Is the non-random change confirmed or not?
June 28, 2007 7
OOS Guidance
Taking into account the differences between OOS and OOT, the guidance does provide a framework for OOT investigations: Responsibilities Philosophical basis General principles of investigations June 28, 2007 8
1. How to get started
Select the variable to be studied: Potency Yield Impurities Hardness Bioburden June 28, 2007 9
2. How to get started
Select a time period: At least one year if possible.
More than two preferred.
Do not go past a major change in the process. Use process knowledge to advantage.
Use the reportable result, the value compared to the specifications.
June 28, 2007 10
3. How to get started
Enter the data into analysis software: Excel Minitab Sigma Plot JMP StatGraphics Northwest Analytical SAS June 28, 2007 11
4. How to get started
Plot the data vs. time or lot sequence.
Look for non-random changes over time.
Determine if they are of practical importance.
Statistical significance is insufficient.
Do an impact and risk assessment.
June 28, 2007 12
What is Trending?
The several activities of: Collecting data, Recording it, Documenting it, Storing it, Monitoring it, Fitting models to it Evaluating it, and Reporting it.
June 28, 2007 13
What is a trend?
Any non-random pattern.
Short and long term patterns in data over time that are of practical importance.
June 28, 2007 14
Beneficial Trends
Desirable patterns in the data series.
Examples: A move toward the target or center of the specification.
More consistent with less variation.
Less likelihood of an OOS value.
A benefit to SSQuIP.
June 28, 2007 15
Beneficial Trend
1.2
1 0.8
0.6
0.4
0.2
0 12/10/200 2 6/28/2003 1/14/2004 8/1/2004 2/17/2005 9/5/2005 3/24/2006 10/10/200 6
Date
June 28, 2007 16
No Trend
Easier to define what a trend is not.
Random data Noise Stationary No ups, no downs No cycles No outliers June 28, 2007 99 98 97 Index 104 103 102 101 100 100 200 300 400 500 17
Neutral or No Trend
Neither beneficial or adverse Examples: Results that are always the same.
Stability data with a slope of zero.
Data in a state of “statistical control” on a control chart.
June 28, 2007 18
Process Control
Statistical Process Control, SPC Normal random data over time Due to common causes only Engineering Process Control, EPC Estimate departures from target Feedback to control point Physical changes to the process June 28, 2007 19
Adverse Trends
Undesirable patterns in the data series.
Examples: A movement away from the target.
Increased variability.
Increased probability of OOS.
An unexplained change to a beneficial trend.
A challenge to SSQuIP.
June 28, 2007 20
Out-of-Trend (OOT)
A change from an established pattern that has the potential of an adverse effect on SSQuIP or of becoming OOS.
Must be large enough to be of practical significance.
Statistical significance is insufficient to determine OOT.
June 28, 2007 21
Long Term Change
Not stationary around a fixed value Increasing or decreasing average.
Apparently will continue to get worse (or better) unless action is taken.
June 28, 2007 Increasing Trend .1 per step after 50 102 101 100 99 98 107 106 105 104 103 Index 10 20 30 40 50 60 70 80 90 100 22
The Aberrant Outlier
Stationary and random but with one very large value that could be a statistical outlier.
Generally assumed to be due to a “special cause.” An outlier Mu=100, Sigma=1.0
100 99 98 97 Index 105 104 103 102 101 10 20 30 40 50 60 70 80 90 100 June 28, 2007 23
Shift in the Average
Here the mean has increased from 100 to 104 at sample 51.
No other changes were made.
Variability is the same.
106 101 Mean Shift Mu=100 to 104 Sigma=1.0
96 Index 10 20 30 40 50 60 70 80 90 100 June 28, 2007 24
Variation Change
This is stationary around a fixed mean of 100%.
But, the standard deviation increased from 1.0 to 4.0.
110 Increasing Variability Mu=100, Sigma=1.0, 2.0, 3.0 & 4.0
100 90 Index 10 20 30 40 50 60 70 80 90 100 June 28, 2007 25
Cycles
A reoccurring cycle.
Stationary about a fixed mean.
The data are not independent.
Cycles 104 103 102 101 100 99 98 97 96 Index 10 20 30 40 50 60 70 80 90 100 June 28, 2007 26
Autocorrelated
Data are correlated with the previous data.
Not stationary.
Check different time lags, 1,2, ….
Autocorrelated 101 100 99 98 Index 105 104 103 102 10 20 30 40 50 60 70 80 90 100 June 28, 2007 27
OOT is Relative
Stationary White Noise mu=100%, S=1% 110 100 90 Index June 28, 2007 10 20 30 40 50 60 70 80 90 100 28
OOT is Relative
The importance of a trend is its size relative to the specification criteria.
A state of Statistical Control is desired but not necessary.
A state of Engineering Control is necessary to meet specifications.
Success is a marriage of the two.
June 28, 2007 29
A Little Humor (Very Little)
Lottery: A tax on the statistically challenged.
If you want three opinions, just ask two statisticians.
Statistics means never having to say you're certain.
http://www.keypress.com/x2815.xml
June 28, 2007 30
Trend Fitting
“The general process of representing the trend component of a time series.” A Dictionary of Statistical Terms . Marriott Depends very much on the type of data and the subject matter being studied.
Need to adapt the tools and techniques to our specific data and issues.
June 28, 2007 31
Tools of Trending
Summary statistics Averages, Medians Ranges, Standard Deviations, %RSD Graphical plots Distribution analysis - Histograms Outlier determination Regression analysis June 28, 2007 32
Graphic Tools
Line Plots vs. time.
Shewhart Control Charts.
Histograms.
Sector chart June 28, 2007 33
Line Plots vs. Time
Response on the vertical axis.
Time or batch # on the horizontal axis.
Usually connect the data points with a line, but optional.
Stationary Time Series Mu=100, Sigma=1.0
102 101 100 99 98 Index 10 20 30 40 50 60 70 80 90 100 June 28, 2007 34
Control Chart
Add ‘natural process limits’ to the line plot.
± 3 A chart for the response.
A chart for the variability.
103.5
102.5
101.5
100.5
99.5
98.5
97.5
96.5
Subgroup 0 4 1 0 3 2 I and MR Chart for Yield % 50 UCL=103 Mean=100 100 LCL=97 UCL=3.686
R=1.128
LCL=0 June 28, 2007 35
Control Chart Family
Individuals Averages Medians Standard deviations Ranges Number of defectives Fraction defectives Defects per units Number of defects June 28, 2007 36
Variation Change
A control chart will detect change in the variation.
110 100 90 Subgroup 0 10 5 0 I and MR Chart for Yield % 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 11 1 1 1 1 1 50 1 1 100 UCL=103 Mean=100 LCL=97 1 1 1 1 11 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 11 1 UCL=3.686
R=1.128
LCL=0 June 28, 2007 37
The Outlier
A control chart finds values outside the natural limits of the data.
The value is larger than would be expected by chance alone.
106 105 104 103 102 101 100 99 98 97 96 0 I Chart for Yield% 1 50 Observation Number 100 UCL=103 Mean=100 LCL=97 38 June 28, 2007
“Western Electric” Rules
1.
2.
3.
4.
One value outside 3 S limits.
Nine values in a row on one side of the average.
Six values in a row all increasing or decreasing.
14 values in a row alternating up and down.
June 28, 2007 39
“Western Electric” Rules
5.
6.
7.
8.
Two of three values greater than 2 S from the average.
Four of five values greater than 1 S from the average.
15 values in a row within 1 S of the average.
Eight values in a row greater than 1 S.
June 28, 2007 40
Histogram
Show the ‘shape’ of the distribution of data.
In this case it is Normally distributed.
20 10 0 96 97 98 99 100 Yield % 101 102 103 104 June 28, 2007 41
The Outlier
The outlier is clearly seen in the histogram.
20 10 Variation Change 0 97 98 99 100 101 102 103 104 105 106 Yield% June 28, 2007 42
Outlier Determination
Reference: USP 30 NF 25 Chapter <1010> “Analytical Data – Interpretation and Treatment” Page 392 “Outlying Results” Appendix C: Examples of Outlier Tests for Analytical Data.
June 28, 2007 43
Regression Analysis
99% Prediction Interval
220000 200000 180000 160000 140000 120000 100000 0 June 28, 2007 5 10
Months
15 20 44
Trend Limits
Numeric (or non-numeric) exceeded, indicates that an out-of-trend change has occurred.
criteria, that if Usually the ‘natural process’ variation AKA “Alert limits” Use Statistical Tolerance Limits See USP <1010> Appendix E June 28, 2007 45
Here, Trend This
20 Index June 28, 2007 40 30 100 200 300 46
A New Engineering Chart
Brings together for the first time : Comparison to the specification limits in place of the probability limits Divides the specification range into equal zones in place of 1, 2, & 3 sigma areas Uses cumulative scores Pharmaceutical Technology, April 2005 June 28, 2007 47
The New “Sector Chart”
SIALIC ACID EXAMPLE
3.915 3.695
Fail
3.298
4.04 3.87 4.147 3.938 4.167 3.9 3.927 3.81 3.9 4.033 3.853 4.142 3.958 3.77
Sector Weight Low High F 10 D 2 4.1
4.2
C 1 4 4.099
B A A B C D F 10 0 3.9 3.999 0 0 3.8 3.899
0 3.7 3.799
0 3.6 3.699
1 3.5 3.599
2 3.4 3.499
0 10 1 0 2 0 2 0 0 0 0 1 0 2 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 48 16 17
The New “Sector Chart” Rules
The first batch tally takes the weight of the sector it is in.
Subsequent batches have a cumulative tally of the previous tally plus the current sector weight.
If the tally reaches a value of, say, 10, an alert is given.
If the batch enters the A or B sectors, the tally is reset to zero.
June 28, 2007 49
The New “Sector Chart” Rules
Sectors A and B cover the center 50% of the specification range.
Sector F is outside the current specification.
Other weights can be set to fit the process and the degree of sensitivity needed.
June 28, 2007 50
Advantages of Sector Chart
No minimum sample size. Can start with one data point.
No assumptions about the data at all.
Identifies beneficial and adverse trends.
Weights and tally total are selected by scientific and empirical knowledge.
A decision is made with each new point.
Alerts quickly if a problem exists.
June 28, 2007 51
Justification for Sector Chart
If the process is well inside the specification, it need not be in a state of statistical control.
The focus is on OOT and SSQuIP not being out of “statistical” control.
Sensitivity of the chart is adjustable.
Can be use in parallel with other charts.
June 28, 2007 52
That’s All Folks
1.
2.
3.
4.
5.
Summary Points: OOT is not OOS OOT is non-random changes over time OOT is a statistical and graphical issue OOT is relative. Statistical significance is not sufficient.
Trend limits = Natural Limits June 28, 2007 53
References
Graphics: http://www.edwardtufte.com/tufte/ http://www.itl.nist.gov/div898/handbook/e da/section3/eda34.htm
Statistics http://www.itl.nist.gov/div898/handbook/in dex.htm
June 28, 2007 54
Software References
http://www.minitab.com/ http://www.systat.com/products/sigma plot/ http://www.nwasoft.com/ http://www.jmp.com/ http://www.statgraphics.com/ http://www.sas.com/ June 28, 2007 55