Transcript Slide 1

INITIALIZATION
AND ONGOING
ACCEPTANCE
TESTING
In order to use the contractor’s test
results for acceptance and payment,
ODOT must ensure that the contractor’s
test results compare favorably with their
own.
Favorable comparability will be
determined by means of paired testing
of split samples.
INITIALIZATION TESTING
 Minimum of 10 tests
 Include plant start-up testing
 1st Lot testing divided into 10 sublots
 Optionally, initialization can be made
up of the first 2 lots of 5 sublots
INITIALIZATION TESTING
Each set of asphalt testing includes:
✓ 1 AIR VOID TEST
✓ 1 AC CONTENT TEST
✓ 3 ROADWAY DENSITY TESTS
At the conclusion of the
sampling and associated
testing, a statistical “paired-t
test” will be performed by
ODOT using the pairs of
initialization test data.
If there is a significant bias:
ODOT’s test results will be relied
upon until the source of the bias
has been identified and corrected.
The lack of bias must then be
validated by further “paired-t
testing”
If Independent Evaluation requested:
The Contractor may request evaluation of
the testing bias via three-way split sample
testing with an independent laboratory.
ODOT will select the independent laboratory.
All initialization testing will be performed by
all three laboratories using split samples.
If ODOT testing shows a
significant bias:
● The Contractor will not be held
responsible for any additional costs
incurred by ODOT in conjunction with the
3-way split-sample testing
● The Contractor’s test results will be
relied upon for acceptance and pay
adjustments
If Contractor testing shows
a significant bias:
● The Contractor shall reimburse ODOT
for any additional costs incurred by
ODOT in conjunction with the 3-way splitsample testing
● ODOT’s test results will be relied upon
for acceptance and pay adjustments
If paired-t testing validates the Contractor’s
test methods, the Contractor’s test results
will be used for acceptance and pay
adjustment subject to:
● ODOT will perform ongoing testing at a
minimum frequency of 1 independent test
per lot.
● Ongoing process verification over time
will be performed using the F-test and the
t-test.
If ODOT’s testing-bias Control Charts
indicate an out-of-control condition
for testing bias:
Both the Contractor and ODOT will
immediately investigate the probable
cause.
If the probable cause is identified and
corrected or if the resulting bias is, in
ODOT’s judgment, not likely to exceed
the Allowable Testing Bias:
Acceptance and pay adjustments will
continue to be based on the Contractor’s
results
At ODOT’s discretion, errant test results may
be discarded. If replacement test results are
available, they may be substituted for
discarded test results
If the probable cause is not corrected or if
the resulting bias is, in ODOT’s judgment,
likely to exceed the allowable testing bias:
All subsequent testing will be paired
testing until the source of the
unaccepatable testing bias is corrected.
During this period, acceptance and pay
adjustments will be based on ODOT’s test
results.
GUIDELINES FOR INITIAL VALIDATION
OF CONTRACTOR’S TEST METHODS
● A paired test shall consist of 2
separate tests by ODOT and the
Contractor on a split sample
PAIRED TESTING
On split samples test:
● Air Voids
● Asphalt Cement Content
For Roadway Densities, perform paired
tests on the same cores. The first lab to
test the cores should make sure to ovendry the cores at 140°F instead of 230°F
GENERAL PROCEDURE
STEP 1
U Calculate X p, the average of the
differences between the paired tests
np
Xp 
Where:
 (X
i1
c
 Xd )
np
Xp = The average of the difference
between paired tests
np = Number of paired tests
Xc = The Contractor’s individual test
result for each sample.
Xd = ODOT’s individual test result for
each sample.
GENERAL PROCEDURE
STEP 2
U Calculate Sp, the standard deviation
of the differences between the
paired tests
np
 (X
Sp 
i 1
c
 X d  Xp )
2
np  1
Where:
Sp = The Standard Deviation of the
difference between paired tests,
np - 1 = Degrees of freedom, and
Xc, Xd, np, and Xp as previously defined.
GENERAL PROCEDURE
STEP 3
U Calculate tp, the paired-t test
statistic
Where:
Xp
t p  np
Sp
tp = The paired-t test statistic, and
np, Xp, and Sp as previously defined.
The vertical lines on either side of the
equation indicate the need to take the
ABSOLUTE VALUE of the equation
Simply put, if the result of the calculation is
NEGATIVE, change it to a POSITIVE number
GENERAL PROCEDURE
STEP 4
U Obtain the critical t-value, tcrit,
from Table 1 in Appendix A
using np-1 degrees of freedom
GENERAL PROCEDURE
STEP 5
U Compare the calculated paired-t
statistic (tp) to the critical t-value tcrit
U If tp is less than tcrit, there is no
statistically significant bias
U If tp is more than tcrit, there is a
statistically significant bias
If tp is more than tcrit :
Compare the testing bias (Xp) to the
associated allowable testing bias (ATB) from
the table in Appendix A:
AC Quality Char.
AC Content
Air Voids
Roadway Density
ATB
± 0.15
± 0.50
± 0.50
Units
% by weight
% by volume
% of Gmm
Note: ATB’s for aggregates are for control charts and
information only.
If the magnitude of Xp is less than the
magnitude of the corresponding ATB,
the Contractor’s bias is statistically,
but not practically significant.
Contractor test results shall then be
relied upon for acceptance and pay
adjustments
If the magnitude of Xp is greater than or
equal to the magnitude of the
corresponding ATB, the Contractor’s bias
is practically significant and unacceptable.
Contractor test results shall not be relied
upon until certified by a qualified
independent calibration specialist and
validated through further paired-t testing.
EXAMPLE - % Air Voids
TEST #
Xc
Xd
1
2
3
4
5
6
7
8
9
10
4.1
4.2
4.9
5.8
5.0
4.8
3.7
5.1
4.9
4.5
3.9
4.4
5.4
6.0
5.1
4.5
3.9
5.4
4.7
4.7
Step 1 – Average Differences
TEST #
Xc
Xd
1
2
3
4
5
6
7
8
9
10
4.1
4.2
4.9
5.8
5.0
4.8
3.7
5.1
4.9
4.5
3.9
4.4
5.4
6.0
5.1
4.5
3.9
5.4
4.7
4.7
Xc - Xd
0.2
-0.2
-0.5
-0.2
-0.1
0.3
-0.2
-0.3
0.2
-0.2
Xp = -0.1
Step 2 – Standard Deviation
TEST # Xc - Xd Xp
1
2
3
4
5
6
7
8
9
10
0.2
-0.2
-0.5
-0.2
-0.1
0.3
-0.2
-0.3
0.2
-0.2
Xc - Xd - Xp (Xc - Xd - Xp)2
-0.1
-0.1
-0.1
-0.1
-0.1
-0.1
-0.1
-0.1
-0.1
-0.1
0.3
-0.1
-0.4
-0.1
0.0
0.4
-0.1
-0.2
0.3
-0.1
S
0.09
0.01
0.16
0.01
0.00
0.16
0.01
0.04
0.09
0.01
(Xc - Xd - Xp)2 = 0.58
np
 (X
Sp 
i 1
c
 X d  Xp )
np  1
Sp
0.58
10  1
Sp = 0.25386
2
Xp
t p  np
Sp
- 0.1
t p  10
0.25386
tp = 1.246
Degrees of
Freedom
(np – 1)
Critical
t-value
(tcrit)
2
3
4
5
6
7
8
9
10
9.925
5.841
4.604
4.032
3.707
3.499
3.355
3.250
3.169
tp = 1.246 < tcrit = 3.250
Since tp < tcrit, there is no statistical
bias. Therefore the contractor’s
tests on this quality characteristic
are valid and will be relied upon for
acceptance and pay adjustments.
ONGOING
VALIDATION OF
CONTRACTOR’S
TEST METHODS
GUIDELINES FOR ONGOING VALIDATION
OF CONTRACTOR’S TEST METHODS
● Ongoing process verification over time
will be performed using the F-test and the
t-test.
● The F-test provides a method for
comparing the variances of the two data
sets.
● The t-test provides a method for
comparing the means of the two data
sets.
Contractor
Department
One independent
Contractor: Collect samples to split and
test ¼, discard ¼, store 2nd half.
ODOT: Collect one sample per lot,
independent of Contractor’s samples.
Test ¼, discard ¼, store 2nd half.
GUIDELINES FOR ONGOING VALIDATION
OF CONTRACTOR’S TEST METHODS
● First two lots do not yield enough data
for F-test and t-test.
● These lots will be validated by
comparing to D2S limits.
● If │Xc – Xd │≤ D2S limit, assumed from
same population – Contractor’s accepted.
● If │Xc – Xd │> D2S limit, assumed from
different populations – ODOT’s used.
D2S LIMITS
Quality Characteristic
D2S Limits
Asphalt Cement Content
0.30%
Air Voids (Lab Molded)
1.40%
Roadway Density
1.40%
THIRD - FIFTH LOT AFTER INITIALIZATION
● Using the F-test and t-test for process
verification.
● Use all data after Initialization,
including that used for D2S comparisons.
● By third lot after Initialization there
should be 15 contractor results and 3
ODOT results.
SIXTH LOTS & FOLLOWING
AFTER INITIALIZATION
● Still using the F-test and t-test for
process verification.
● Use the current lot results plus the
previous four for a total of five lots.
● Each of these analyses should include
25 contractor results and 5 ODOT results.
F-test
● Calculate the mean of each test set.
Xc
X


c
Xd
X


nc
d
nd
Xc = the mean of the Contractor’s results
Xc = individual Contractor result
nc = the number of Contractor results
Xd = the mean of the Department’s results
Xd = individual Department result
nd = the number of Department results
F-test
● Calculate the variance of each test set.
s
2
c
(X


c
 Xc )
nc  1
2
s
2
d
(X


d
 Xd )
2
nd  1
s2c = the variance of the Contractor’s results
nc - 1 = degrees of freedom of Contractor results
s2d = the variance of the Department’s results
nc - 1 = degrees of freedom of Department results
F-test
● Calculate the F-statistic.
s
F
s
2
c
2
d
or
s
F
s
2
d
2
c
Use the larger of the variances in the numerator
so that the F-statistic is always greater than one.
F-test
U Determine Fcrit from the F-distribution
table using a 0.01 significance level
U Compare the calculated F-statistic
to the critical F-value Fcrit
U If F is less than Fcrit, the data is
assumed from the same population.
U If F is more than Fcrit, the data is
assumed from a different population
t-test
2
p.
● Calculate the pooled variance, s
s (nc  1)  s (nd  1)
s 
nc  nd  2
2
p
2
c
2
d
Where nc + nd - 2 = pooled degrees of freedom
t-test
● Calculate the t-statistic.
t
Xc  Xd
2
p
s
p
s
s

nc nd
t-test
U Determine tcrit from the t-distribution
table using a 0.01 significance level
and the pooled degrees of freedom
U Compare the calculated t-statistic to
the critical t-value tcrit
U If t is less than tcrit, the data is
assumed from the same population.
U If t is more than tcrit, the data is
assumed from a different population
Both the F-test and the
t-test must show the
results to be from the
same population to
continue accepting the
Contractor’s results
If either test shows the
results to be from different
populations, ODOT will
test the remaining half of
the Contractor’s split
samples for acceptance
until the reason for the
difference is resolved.
EXAMPLE - % Asphalt Content
1st Lot after Initialization – D2S Limits
Test #
Xc
Xd
11
4.18
4.08
12
4.37
13
4.27
14
4.19
15
4.21
After initialization, ODOT and Contractor samples are no longer
split samples but obtained independently
Step 1: Compute Averages
Test #
Xc
Xd
11
4.18
4.08
12
4.37
13
4.27
14
4.19
15
4.21
Avg.
4.244
4.08
Step 2: Compute D2S Comparison
Xc - Xd  D2S
4.244- 4.08  0.30
(D2S=0.30 from Appendix A
Part 2 for Asphalt Cement
content)
0.164 0.30  True
● Since X c - Xd  D2S is true, we assume that the
test results come from the same population.
The Contractor’s results are accepted.
EXAMPLE - % Asphalt Content
2nd Lot after Initialization – D2S Limits
Test #
Xc
12
Xd
4.21
16
4.14
17
4.27
18
4.21
19
4.14
20
4.27
Step 1: Compute Averages
Test #
Xc
12
Xd
4.21
16
4.14
17
4.27
18
4.21
19
4.14
20
4.27
Avg.
4.206
4.21
Step 2: Compute D2S Comparison
Xc - Xd  D2S
4.206- 4.21  0.30
0.004 0.30  True
(D2S=0.30 from
Appendix A Part 2
for Asphalt
Cement content)
● Since Xc  Xd  D2S is true, we assume that the
test results came from the same population.
The Contractor’s results are accepted.
EXAMPLE - % Asphalt Content
3rd Lot after Initialization – Begin F & t
Test #
Xc
Xd
Test #
Xc
Test #
Xc
11
4.18
4.08
16
4.14
21
4.19
12
4.37
4.21
17
4.27
22
4.21
13
4.27
4.11
18
4.21
23
4.14
14
4.19
19
4.14
24
4.00
15
4.21
20
4.22
25
4.50
Step 1: Compute Mean for Each Set
Xc
X


Xd
X


c
nc
nd
d
63.29
Xc 
 4.2193333  4.22
15
12.40
Xd 
 4.1333333  4.13
3
Step 2: Compute s2 (Variance) for
Each Set
2
c
(X


2
d
(X


s
s
c
 Xc )
2
nc  1
d
 Xd )
nd  1
2
Step 3: Compute
2
sc
Test #
Xc
(Xc-Xc)2
11
4.08
0.0025
12
4.21
0.0064
25
4.50
0.0784
Sum
63.29
0.1797
Xc ≈ 4.22
Sum/(nc-1)
0.1797 / (15-1)
2
sc
0.01284
Step 4: Compute
2
sd
Test #
Xd
(Xd-Xd)2
11
4.08
0.0025
12
4.21
0.0064
13
4.11
0.0004
Xd ≈ 4.13
Sum
Sum/(nd-1)
0.0093
0.0093/(3-1)
2
sd
0.00465
Step 5: Compute F-statistic
s
F
s
2
c
2
d
or
s
F
s
2
d
2
c
F
Numerator
Denominator
Result
1
0.01284
0.00465
2.76
2
0.00465
0.01284
0.36
Note: Pick F with largest numerator (highest result).
Step 6: Pick Fcrit from F-distribution
(2-tailed, =0.01) Table
For this example,
contractor variance
was in the numerator
with 14 degrees of
freedom and ODOT’s
variance was in the
denominator with 2
degrees of freedom.
Read the table like this
Not like this!
n-1
13
14
15
2
199
199
199
3
43.3
43.2
43.1
n-1
2
3
13
8.19
6.92
14
7.92
6.68
15
7.70
6.48
Step 7: Perform F-test
F ≤ Fcrit ?
2.75 ≤ 199 = True
● Since F-test is true, we must perform
the T-test.
● Had F-test been false, the tests would not have
come from the same population. The
Contractor’s results would not be accepted.
2
p
Step 8: Compute Pooled Variance s
s (nc  1)  s (nd  1)
s 
nc  nd  2
2
p
2
c
2
d
0.01284(15 - 1)  0.00465(3  1)
S 
15  3  2
2
p
 s  0.0118
2
p
Step 9: Compute t-statistic
t
Xc  Xd
2
p
s
p
s
s

nc nd
t
4.22  4.13
0.0118 0.0118

15
3
 t  1.31
Note: Without rounding, t ≈ 1.25.
Step 10: Pick tcrit from t-distribution (2tailed, =0.01) Table
• Degrees of freedom = nc + nd – 2
DF = 15 + 3 – 2 = 16
DF
tcrit
15 2.947
16 2.921
17 2.898
Step 11: Perform t-test
t ≤ tcrit ?
1.31 ≤ 2.921 = True
● Since both t-test and F-test are true, we assume
that the test results came from the same
population. The Contractor’s results are
accepted.