Trend Data

Lynn Torbeck Torbeck and Assoc.

Evanston, IL June 28, 2007 1

Overview

OOT vs. OOS Why trend?

How to get started Types of trends with examples OOT is relative Graphical tools Tend limits June 28, 2007 2

Why Trend Data?

Good business practice.

Early warning of possible Out Of Specification (OOS) results.

Gain process understanding.

Minimize risk of potential failures of product in the market.

Find the “gold in the hills” for process improvements.

June 28, 2007 3

Regulatory Basis for Trending

No specific regulation requirement 211.180(e) Annual Reviews FDA Form 483 for observations Establishment Inspection Reports Warning letters FDA presentations at conferences June 28, 2007 4

OOS Guidance Footnote

“Although the subject of this document is OOS results, much of the guidance may be useful for examining results that are out of trend (OOT).” How is OOT different than OOS?

How is OOT the same as OOS?

June 28, 2007 5

Out Of Specification - OOS

OOS is the comparison of one result versus a predetermined specification criteria.

OOS investigations focus on determining the truth about that one value.

Is the OOS result confirmed or not?

June 28, 2007 6

Out Of Trend - OOT

OOT is the comparison of many historical data values versus time.

OOT investigations focus on understanding non-random changes.

Is the non-random change confirmed or not?

June 28, 2007 7

OOS Guidance

Taking into account the differences between OOS and OOT, the guidance does provide a framework for OOT investigations: Responsibilities Philosophical basis General principles of investigations June 28, 2007 8

1. How to get started

     Select the variable to be studied: Potency Yield Impurities Hardness Bioburden June 28, 2007 9

2. How to get started

Select a time period: At least one year if possible.

More than two preferred.

Do not go past a major change in the process. Use process knowledge to advantage.

Use the reportable result, the value compared to the specifications.

June 28, 2007 10

3. How to get started

Enter the data into analysis software:        Excel Minitab Sigma Plot JMP StatGraphics Northwest Analytical SAS June 28, 2007 11

4. How to get started

Plot the data vs. time or lot sequence.

Look for non-random changes over time.

Determine if they are of practical importance.

Statistical significance is insufficient.

Do an impact and risk assessment.

June 28, 2007 12

What is Trending?

The several activities of:  Collecting data,        Recording it, Documenting it, Storing it, Monitoring it, Fitting models to it Evaluating it, and Reporting it.

June 28, 2007 13

What is a trend?

Any non-random pattern.

Short and long term patterns in data over time that are of practical importance.

June 28, 2007 14

Beneficial Trends

Desirable patterns in the data series.

Examples:     A move toward the target or center of the specification.

More consistent with less variation.

Less likelihood of an OOS value.

A benefit to SSQuIP.

June 28, 2007 15

Beneficial Trend

1.2

1 0.8

0.6

0.4

0.2

0 12/10/200 2 6/28/2003 1/14/2004 8/1/2004 2/17/2005 9/5/2005 3/24/2006 10/10/200 6

Date

June 28, 2007 16

No Trend

Easier to define what a trend is not.

Random data Noise Stationary No ups, no downs No cycles No outliers June 28, 2007 99 98 97 Index 104 103 102 101 100 100 200 300 400 500 17

Neutral or No Trend

Neither beneficial or adverse Examples:    Results that are always the same.

Stability data with a slope of zero.

Data in a state of “statistical control” on a control chart.

June 28, 2007 18

Process Control

Statistical Process Control, SPC   Normal random data over time Due to common causes only Engineering Process Control, EPC    Estimate departures from target Feedback to control point Physical changes to the process June 28, 2007 19

Adverse Trends

Undesirable patterns in the data series.

Examples:      A movement away from the target.

Increased variability.

Increased probability of OOS.

An unexplained change to a beneficial trend.

A challenge to SSQuIP.

June 28, 2007 20

Out-of-Trend (OOT)

A change from an established pattern that has the potential of an adverse effect on SSQuIP or of becoming OOS.

Must be large enough to be of practical significance.

Statistical significance is insufficient to determine OOT.

June 28, 2007 21

Long Term Change

Not stationary around a fixed value Increasing or decreasing average.

Apparently will continue to get worse (or better) unless action is taken.

June 28, 2007 Increasing Trend .1 per step after 50 102 101 100 99 98 107 106 105 104 103 Index 10 20 30 40 50 60 70 80 90 100 22

The Aberrant Outlier

Stationary and random but with one very large value that could be a statistical outlier.

Generally assumed to be due to a “special cause.” An outlier Mu=100, Sigma=1.0

100 99 98 97 Index 105 104 103 102 101 10 20 30 40 50 60 70 80 90 100 June 28, 2007 23

Shift in the Average

Here the mean has increased from 100 to 104 at sample 51.

No other changes were made.

Variability is the same.

106 101 Mean Shift Mu=100 to 104 Sigma=1.0

96 Index 10 20 30 40 50 60 70 80 90 100 June 28, 2007 24

Variation Change

This is stationary around a fixed mean of 100%.

But, the standard deviation increased from 1.0 to 4.0.

110 Increasing Variability Mu=100, Sigma=1.0, 2.0, 3.0 & 4.0

100 90 Index 10 20 30 40 50 60 70 80 90 100 June 28, 2007 25

Cycles

A reoccurring cycle.

Stationary about a fixed mean.

The data are not independent.

Cycles 104 103 102 101 100 99 98 97 96 Index 10 20 30 40 50 60 70 80 90 100 June 28, 2007 26

Autocorrelated

Data are correlated with the previous data.

Not stationary.

Check different time lags, 1,2, ….

Autocorrelated 101 100 99 98 Index 105 104 103 102 10 20 30 40 50 60 70 80 90 100 June 28, 2007 27

OOT is Relative

Stationary White Noise mu=100%, S=1% 110 100 90 Index June 28, 2007 10 20 30 40 50 60 70 80 90 100 28

OOT is Relative

The importance of a trend is its size relative to the specification criteria.

A state of Statistical Control is desired but not necessary.

A state of Engineering Control is necessary to meet specifications.

Success is a marriage of the two.

June 28, 2007 29

A Little Humor (Very Little)

Lottery: A tax on the statistically challenged.

If you want three opinions, just ask two statisticians.

Statistics means never having to say you're certain.

http://www.keypress.com/x2815.xml

June 28, 2007 30

Trend Fitting

“The general process of representing the trend component of a time series.” A Dictionary of Statistical Terms . Marriott Depends very much on the type of data and the subject matter being studied.

Need to adapt the tools and techniques to our specific data and issues.

June 28, 2007 31

Tools of Trending

Summary statistics   Averages, Medians Ranges, Standard Deviations, %RSD Graphical plots Distribution analysis - Histograms Outlier determination Regression analysis June 28, 2007 32

Graphic Tools

Line Plots vs. time.

Shewhart Control Charts.

Histograms.

Sector chart June 28, 2007 33

Line Plots vs. Time

Response on the vertical axis.

Time or batch # on the horizontal axis.

Usually connect the data points with a line, but optional.

Stationary Time Series Mu=100, Sigma=1.0

102 101 100 99 98 Index 10 20 30 40 50 60 70 80 90 100 June 28, 2007 34

Control Chart

Add ‘natural process limits’ to the line plot.

 ± 3  A chart for the response.

A chart for the variability.

103.5

102.5

101.5

100.5

99.5

98.5

97.5

96.5

Subgroup 0 4 1 0 3 2 I and MR Chart for Yield % 50 UCL=103 Mean=100 100 LCL=97 UCL=3.686

R=1.128

LCL=0 June 28, 2007 35

Control Chart Family

Individuals Averages Medians Standard deviations Ranges Number of defectives Fraction defectives Defects per units Number of defects June 28, 2007 36

Variation Change

A control chart will detect change in the variation.

110 100 90 Subgroup 0 10 5 0 I and MR Chart for Yield % 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 11 1 1 1 1 1 50 1 1 100 UCL=103 Mean=100 LCL=97 1 1 1 1 11 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 11 1 UCL=3.686

R=1.128

LCL=0 June 28, 2007 37

The Outlier

A control chart finds values outside the natural limits of the data.

The value is larger than would be expected by chance alone.

106 105 104 103 102 101 100 99 98 97 96 0 I Chart for Yield% 1 50 Observation Number 100 UCL=103 Mean=100 LCL=97 38 June 28, 2007

“Western Electric” Rules

1.

2.

3.

4.

One value outside 3 S limits.

Nine values in a row on one side of the average.

Six values in a row all increasing or decreasing.

14 values in a row alternating up and down.

June 28, 2007 39

“Western Electric” Rules

5.

6.

7.

8.

Two of three values greater than 2 S from the average.

Four of five values greater than 1 S from the average.

15 values in a row within 1 S of the average.

Eight values in a row greater than 1 S.

June 28, 2007 40

Histogram

Show the ‘shape’ of the distribution of data.

In this case it is Normally distributed.

20 10 0 96 97 98 99 100 Yield % 101 102 103 104 June 28, 2007 41

The Outlier

The outlier is clearly seen in the histogram.

20 10 Variation Change 0 97 98 99 100 101 102 103 104 105 106 Yield% June 28, 2007 42

Outlier Determination

Reference:      USP 30 NF 25 Chapter <1010> “Analytical Data – Interpretation and Treatment” Page 392 “Outlying Results” Appendix C: Examples of Outlier Tests for Analytical Data.

June 28, 2007 43

Regression Analysis

99% Prediction Interval

220000 200000 180000 160000 140000 120000 100000 0 June 28, 2007 5 10

Months

15 20 44

Trend Limits

Numeric (or non-numeric) exceeded, indicates that an out-of-trend change has occurred.

criteria, that if Usually the ‘natural process’ variation AKA “Alert limits” Use Statistical Tolerance Limits See USP <1010> Appendix E June 28, 2007 45

Here, Trend This

20 Index June 28, 2007 40 30 100 200 300 46

A New Engineering Chart

Brings together for the first time :    Comparison to the specification limits in place of the probability limits Divides the specification range into equal zones in place of 1, 2, & 3 sigma areas Uses cumulative scores Pharmaceutical Technology, April 2005 June 28, 2007 47

The New “Sector Chart”

SIALIC ACID EXAMPLE

3.915 3.695

Fail

3.298

4.04 3.87 4.147 3.938 4.167 3.9 3.927 3.81 3.9 4.033 3.853 4.142 3.958 3.77

Sector Weight Low High F 10 D 2 4.1

4.2

C 1 4 4.099

B A A B C D F 10 0 3.9 3.999 0 0 3.8 3.899

0 3.7 3.799

0 3.6 3.699

1 3.5 3.599

2 3.4 3.499

0 10 1 0 2 0 2 0 0 0 0 1 0 2 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 48 16 17

The New “Sector Chart” Rules

The first batch tally takes the weight of the sector it is in.

Subsequent batches have a cumulative tally of the previous tally plus the current sector weight.

If the tally reaches a value of, say, 10, an alert is given.

If the batch enters the A or B sectors, the tally is reset to zero.

June 28, 2007 49

The New “Sector Chart” Rules

Sectors A and B cover the center 50% of the specification range.

Sector F is outside the current specification.

Other weights can be set to fit the process and the degree of sensitivity needed.

June 28, 2007 50

Advantages of Sector Chart

No minimum sample size. Can start with one data point.

No assumptions about the data at all.

Identifies beneficial and adverse trends.

Weights and tally total are selected by scientific and empirical knowledge.

A decision is made with each new point.

Alerts quickly if a problem exists.

June 28, 2007 51

Justification for Sector Chart

If the process is well inside the specification, it need not be in a state of statistical control.

The focus is on OOT and SSQuIP not being out of “statistical” control.

Sensitivity of the chart is adjustable.

Can be use in parallel with other charts.

June 28, 2007 52

That’s All Folks

1.

2.

3.

4.

5.

Summary Points: OOT is not OOS OOT is non-random changes over time OOT is a statistical and graphical issue OOT is relative. Statistical significance is not sufficient.

Trend limits = Natural Limits June 28, 2007 53

References

Graphics:   http://www.edwardtufte.com/tufte/ http://www.itl.nist.gov/div898/handbook/e da/section3/eda34.htm

Statistics  http://www.itl.nist.gov/div898/handbook/in dex.htm

June 28, 2007 54

Software References

http://www.minitab.com/ http://www.systat.com/products/sigma plot/ http://www.nwasoft.com/ http://www.jmp.com/ http://www.statgraphics.com/ http://www.sas.com/ June 28, 2007 55