Transcript No Slide Title

Is data quality more important than data quantity in the diagnosis
of occupational asthma from serial PEF records?
MOORE VC, BURGE CBSG, AND BURGE PS
Occupational Lung Disease Unit, Birmingham Heartlands Hospital, UK
ABSTRACT
Results: The table shows how the sensitivity and specificity
changed at each randomisation after data corruption.
Conclusion: The sensitivity and specificity for diagnosing
occupational asthma was little affected by reducing data
quality using Oasys analyses. This suggests that data quality
is less important than quantity in this situation
• Randomisation took place within the Oasys program. Each PEF
value within the record was randomly assigned a new number
ranging from +50L/min or -50L/min in increments of 10l/min from
the original value.
• Records were scored using the area between curves (ABC) score,
Oasys score and timepoint analysis within the Oasys program at
each randomisation.
• For sensitivity the cut off for the Oasys score was ≥2.51, the ABC
score ≥15 L/min/h and for timepoint ≥ 1 non-waking drop
Visit our website: www.occupationalasthma.com
4
Daily Max
3
4
300
4
4
3
Oasys 2b score for period
4
Infection?
280
Patient rested
Patient worked a day shift
260
Patient worked an afternoon shift
Patient worked a night shift
240
Patient worked
220
Patient recorded no data
Day excluded
200
c
e
F S S M T W T
15 16 17 18 19 20 21
16 17 18 19 20 21 22
November, 1996
Readings
7 8 7 5 7 1 9
Work Hours
6 6 5 6
Date
Additional
F S S M T W T F S S M T W T
22 23 24 25 26 27 28 29 30 01 02 03 04 05
23 24 25 26 27 28 29 30 01 02 03 04 05 06
December
7 8 8 8 1 9 9 7 8 7 9 7 8 9
6 6 5 6
6 6 5 6
360
Daily Min
F S S M T W T F S w
06 07 08 09 10 11 12 13 14
W
07 08 09 10 11 12 13 14 15
There are comments for day
Day is marked for exclusion
Missing waking reading(s)
Waking reading(s) Created
Peak Expiratory Flow (PEF) Litres / Minute
4
W
Normal Exposure
4
3
4
4
340
Patient rested
Patient worked a day shift
320
Patient worked an afternoon shift
Patient worked a night shift
300
Patient worked
280
Patient recorded no data
Day excluded
260
240
220
c
There are comments for day
e
Day is marked for exclusion
w
Missing waking reading(s)
W
Waking reading(s) Created
4
360
20%
380
4
4
3
Daily Min
340
Patient rested
Patient worked a day shift
320
Patient worked an afternoon shift
Patient worked a night shift
300
Patient worked
280
Patient recorded no data
Day excluded
260
240
220
c
There are comments for day
e
Day is marked for exclusion
w
Missing waking reading(s)
W
Waking reading(s) Created
200
Additional
8 6 7 9 8 7 8 7 0
6 6 5 6
3
Daily Min
Oasys 2b score for period
4
Infection?
Patient rested
Patient worked a day shift
300
Patient worked an afternoon shift
Patient worked a night shift
280
Patient worked
260
Patient recorded no data
Day excluded
240
220
200
160
F S S M T W T
15 16 17 18 19 20 21
16 17 18 19 20 21 22
November, 1996
Readings
7 8 7 5 7 1 9
Work Hours
6 6 5 6
Date
140
F S S M T W T
15 16 17 18 19 20 21
16 17 18 19 20 21 22
November, 1996
Readings
7 8 7 5 7 1 9
Work Hours
6 6 5 6
Date
Additional
W
3
c
There are comments for day
e
Day is marked for exclusion
w
Missing waking reading(s)
W
Waking reading(s) Created
180
160
F S S M T W T F S
06 07 08 09 10 11 12 13 14
07 08 09 10 11 12 13 14 15
Daily Max
3
320
180
F S S M T W T F S S M T W T
22 23 24 25 26 27 28 29 30 01 02 03 04 05
23 24 25 26 27 28 29 30 01 02 03 04 05 06
December
7 8 8 8 1 9 9 7 8 7 9 7 8 9
6 6 5 6
6 6 5 6
3
340
200
F S S M T W T
Date
15 16 17 18 19 20 21
16 17 18 19 20 21 22
November, 1996
Readings
7 8 7 5 7 1 9
Work Hours
6 6 5 6
4
Daily Mean
360
Oasys 2b score for period
4
Infection?
180
By Whole Record Mean
Daily Mean
380
Oasys 2b score for period
4
50%
Daily Max
3
Daily Min
Infection?
160
8 6 7 9 8 7 8 7 0
6 6 5 6
Daily Max
3
Daily Mean
380
Daily Mean
320
20%
400
3
3
By Whole Record Mean
D.V.
20%
400
340
50%
By Whole Record Mean
RANDOMISATION 3
Peak Expiratory Flow (PEF) Litres / Minute
D.V.
D.V.
20%
F S S M T W T F S S M T W T
22 23 24 25 26 27 28 29 30 01 02 03 04 05
23 24 25 26 27 28 29 30 01 02 03 04 05 06
December
7 8 8 8 1 9 9 7 8 7 9 7 8 9
6 6 5 6
6 6 5 6
F S S M T W T F S
06 07 08 09 10 11 12 13 14
07 08 09 10 11 12 13 14 15
8 6 7 9 8 7 8 7 0
6 6 5 6
Additional
F S S M T W T F S S M T W T
22 23 24 25 26 27 28 29 30 01 02 03 04 05
23 24 25 26 27 28 29 30 01 02 03 04 05 06
December
7 8 8 8 1 9 9 7 8 7 9 7 8 9
6 6 5 6
6 6 5 6
F S S M T W T F S
06 07 08 09 10 11 12 13 14
07 08 09 10 11 12 13 14 15
8 6 7 9 8 7 8 7 0
6 6 5 6
W
W
Normal Exposure
Normal Exposure
320
320
PEF (L/Min)
PEF (L/Min)
260
320
300
Definite occupational asthma
300
Immediate reactions
280
280
Definite occupational asthma
Immediate reactions
Definite occupational asthma
Immediate reactions
300
280
Opinions And Comments
Average Hour from Waking for Rest and Day Shift days
Average Hour from Waking for Rest and Day Shift days
Opinions
And Comments
320
Opinions And
340 Comments
Definite occupational asthma
300
Immediate reactions
PEF (L/Min)
Normal Exposure
Average Hour from Waking for Rest and Day Shift days
280
260
PEF (L/Min)
Average Hour from Waking for Rest and Day Shift days
Opinions And Comments
260
260
240
240
220
220
240
240
220
00-02 02-04 04-06 06-08 08-10 10-12 12-14 14-16 16-18 18-20
16 13 15 14 14 13 14 13 13 12 8 11 11 10 12 9 11 0 0 0
12
94
149
131
96
36
-25
ABC Score: 35 l/min/hour
Drop Drop Drop Drop
Hours From Waking, Number of Readings And Areas (Day Shifts
) (Rest
)
Oasys score 3.73
ABC score 35 L/min/h
Timepoint drops: 4
220
200
200
00-02 02-04 04-06 06-08 08-10 10-12 12-14 14-16 16-18 18-20
16 13 15 14 14 13 14 13 13 12 8 11 11 10 12 9 11 0 0 0
48
135
195
157
86
31
-14
ABC Score: 46 l/min/hour
Drop Drop Drop Drop
Hours From Waking, Number of Readings And Areas (Day Shifts
) (Rest
200
00-02 02-04 04-06 06-08 08-10 10-12 12-14 14-16 16-18 18-20
16 13 15 14 14 13 14 13 13 12 8 11 11 10 12 9 11 0 0 0
24
117
186
146
69
17
-4
ABC Score: 40 l/min/hour
Drop Drop Drop
Hours From Waking, Number of Readings And Areas (Day Shifts
Oasys score 3.73
ABC score 46 L/min/h
Timepoint drops: 4
Occupational
asthma n=36
Mean age
Sex
Mean FEV1 (% of predicted value)
% Non – specific reactors
(Methacholine PD20 <2000µg
or Histamine PD20<8mg)
Inhaled corticosteroids
Mean diurnal variation in PEF
(l/min) as % of whole record mean
43.3
21m, 15f
82.5
67.6
82.4
22.3
) (Rest
00-02 02-04 04-06 06-08 08-10 10-12 12-14 14-16 16-18 18-20
16 13 15 14 14 13 14 13 13 12 8 11 11 10 12 9 11 0 0 0
28
117
137
112
93
12
-23
ABC Score: 34 l/min/hour
Drop Drop Drop
Hours From Waking, Number of Readings And Areas (Day Shifts
)
) (Rest
)
)
Oasys score 3.44
ABC score 34 L/min/h
Timepoint drops: 3
Oasys score 3.92
ABC score 40L/min/h
Timepoint drops: 3
DEMOGRAPHICS
METHODS
• Serial PEF records from 36 workers confirmed to have occupational
asthma through specific inhalation challenge testing (to assess
sensitivity) and 44 from those with asthma/occupational asthma
who were not at work during their PEF record (to assess specificity)
were used.
50%
By Whole Record Mean
RANDOMISATION 2
D.V.
50%
180
Methods: PEF measurements from 36 specific inhalation
challenge positive workers and 44 non-occupational
asthmatics were used. Every PEF measurement was
randomly changed to be up to +50L/min or -50L/min from
the original value in increments of 10 L/min. Records were
randomised 3 times and compared at each randomisation to
the original using the Oasys score, area between curves
(ABC) score and timepoint analysis.
RANDOMISATION 1
Peak Expiratory Flow (PEF) Litres / Minute
RANDOMISATION 0
Peak Expiratory Flow (PEF) Litres / Minute
Serial peak expiratory flows (PEF) are a cheap and effective
way to confirm the diagnosis of occupational asthma.
Records are susceptible to fabrication and poor quality.
There is a tradeoff between accepting only blows fulfilling
quality standards with reduced data quantity and accepting
all blows irrespective of quality. We have investigated the
effect of randomly reducing quality on the diagnostic
sensitivity and specificity using the Oasys system.
SERIAL PEF RECORD CHANGES AT EACH RANDOMISATION
Nonoccupational
asthma n=44
48.6
31m, 13f
80.7
50.0
90
23.1
SENSITIVITY & SPECIFICITY AT
EACH RANDOMISATION
0
Sensitivity &
(Specificity) of
75 (84)
Oasys score
Sensitivity &
(Specificity) of
69 (93)
ABC score
Sensitivity &
(Specificity) of
64 (84)
Timepoint analysis
Randomisation number
1
2
3
67 (73)
72 (68)
86 (86)
61 (89)
55 (86)
69 (93)
58 (84)
44 (86)
55 (86)
CONCLUSIONS
• Data quality only minimally affected the sensitivity and specificity of the scoring systems in Oasys
• Data quantity is likely to be more important than data quality
• Cheap meters without quality control are just as useful as more expensive ones with quality management,
making PEFs for occupational asthma diagnosis possible in all departments rather than just specialist centres