Transcript Slide 1
Perspectives on QA Audits/
Responses
Martin Beran – QA
John Roncoroni – FRD
Van Starner – SD
IR-4 National Education Conference
Phoenix, AZ - March 1, 2006
Martin Beran
IR-4 Western Region
Quality Assurance officer
U.C. Davis
Those Darn Findings
• The Pseudo-Antagonistic and PolyUnsaturated Architecture of QA Findings
• I Have No Idea What QA is Asking Me?
• How Am I Supposed to Respond?
• Who Gets a Copy of My Responses?
QA Obligations
• Researchers Need to
Know What the Issue Is
• Findings Should be Clear
• Avoid Accusatory Tones
• Offer Solutions to
Problems
These Findings Say it All
• You did not have your SOPs on site at
the time of the application. According to
the GLP regulations, SOPs should be
immediately available. I recommend a
copy of your current SOPs be placed in
your truck.
One More:
• Part 4E, Test Substance Records - I cannot
seem to locate test substance storage
records for the period of 2/11 – 3/29/05. Test
substance storage records are needed from
the date of receipt to the date of the last
application. Please insert additional records
if available into the FDB.
Findings that are Weak Sauce
• Luke Skywalkers’s training records did not
include any reference to GLP training.
• Any impromptu GLP training Skywalker
might have had should be added to Part 2B
of the FDB. This can be in the form of a
copy of his ongoing Jedi training record or a
separate sheet of paper with any informal
GLP training documented.
Weak Sauce Continued
• Your Crop Destruct statement is
insufficient.
• Please explain how and when the
treated apples were destroyed. You
simply stated that the row was taped off.
Please document the measures you
took to ensure that the apples did not
enter the food chain.
Responses
• Did they answer the
question
• Did they provide the
necessary information
• Has the FRD carried
his corrections over to
the FDB
Responses
• I get a lot of these:
• OK
• Noted
• Study director will
Address
• How dare you
Note to Study Director
• The plot layout consists of one row of 14 plants.
The Protocol states that samples should not be
taken from hops on the ends or the edges and they
must be taken from at least five hills. Since the
researcher has only one row to sample from, this
seems to be a Protocol Deviation. Did you mean to
write in the Protocol that hops could not be taken
from plot edges? This means that plots need to be
more than two rows of hop hills. No response
needed from the researcher.
More Notes:
• The Chicory samples from the TA01 trial seemed to
be quite different in appearance from the –DS05
samples. The Tatooine root samples were at most
finger thickness compared to the Death Star
samples which were round as a turnip. Even
though Amendment #4 mentioned a potential
difference in residue, the physical difference in the
Tatooine samples was stark enough to be brought
to your attention.
Routing FDB Findings
QA
Field
Research
Director
Regional
Field
Coordinator
Regional
QA
Coordinator
HQ
Routing CPI Findings
QA
Field
Research
Directo r
Reg ional
Field
Coo rdina to r
HQ
Additional Points
• Sending Corrected
Pages Along with
the Audit
• The Quality Control
Review
• Communicate with
the Auditing QA
Response To My Responses?
John Roncoroni
IR-4 Western Region
Field Research Director
U.C. Davis
Finding #1
• The GLP status of the test substance
was not know “at the time of receipt,” and
there was no record of a communication
to the Study Director, who sent an email
on July 28. Suggest changing to “No”
(not known at time of receipt). Most time
there is a GLP assurance in the letter
from the supplier, but not in this case.
My Response
• “Technically I was taking the registrant’s
word that they would actually send a
Certificate of Analysis to Dan Kunkel at HQ see page 3 part 4. Section 13 of the
protocol states that ‘the registrant will
provide a Certificate of Analysis to IR-4
headquarters.’ Not necessarily to the FRD.”
Registrant Letter
• “This test substance, which was obtained
from production lot no. ABC123, has been
found to have an active ingredient
concentration of 9.3%. A certificate of
quality has been forwarded to the attention
of Dr. Dan Kunkel at IR-4 Headquarters.”
Finding #2
Part 6, page 14:
48.4 x 233.823 = 11317.033
Please correct this and the GPA and send
the corrected page back to HQ.
My Response
• The numbers in part 6 page 14 (and
actually on the page before also) are:
48.4 x 2333.823 = 112957.03
(reviewer is off by a factor of 10) !
Finding #3
• Part 6 page 28 of 30:
Show % deviation from Protocol
rate. This is asked for in instructions for
this page.
My Response
• Clarification added:
Note: I think that 100.39% indicates
that I was 0.39% over the protocol rate
and is clear for the reviewer to
recognize.
Finding #4
• Part 5 page 14 of 20:
The non-applicable ID# has been
lined out but there is no error code.
My Response
• This is a field ID that is on a page that is
copied – my answer is I have found in
the past that there really is no
appropriate error code - I will use WE for
wrong entry.
Finding #5
• Page 5, pages 7-9:
On page 7, there is a notation of the GPS
reading used for the water valve used as
a permanent landmark. If you will be
relying on that reading, versus the valve
itself, we should add the GPS unit to the
list of GLP exceptions in Part 1C. If the
valve has some kind of unique ID, that
could be used for future trials.
My Response
• I am relying on the valve as the
permanent marker. The valves are far
enough apart that the GPS will identify
this valve as opposed to others.
Van Starner
IR-4 Headquarters
Study Director
Rutgers Univ.
Current IR-4 Guidance
• San Antonio Results Document (9/27/02):
Q: How do we correct raw data that we no longer have in our
possession?
A: Make the correction on a copy of the raw data and forward with the
responses to HQ QA (updated as per new procedures). Contact the
Study Director if there are any questions about responding to QA
findings, or contact the QA who performed the audit with questions of
intent.
• IR-4 Advisory #2003-01 (4/25/03):
“Responding to FDB QA Findings”
Current IR-4 Guidance
• Instructions from QA auditor in audit cover
letter:
“Return the audit with your original responses and any
corrections, clarifications or altered copies of your data
pages to IR-4 HQ QA. The HQ QA will then forward
the responses and any corrected pages to the Study
Director and Testing Facility Management. Please
keep a copy of the audit for yourself, and send copies
of the responses to your Regional Field Research
Coordinator. Only one mailing, your original
response, needs to be sent to HQ.”
Who is the target for your
responses?
QA?
Field Coordinators?
Management?
EPA?
The Study Director!
Why?
By 40 CFR Part 160.33, the Study Director:
• Represents the single point of study control
• Has overall responsibility for the technical conduct of
the study
• Is responsible for interpretation, analysis,
documentation, and reporting of study results
So……..
• When you receive an audit for review
and response:
Don’t take it personally that QA has “dinged”
your work, BUT….
Consider how you can best respond to
assist the Study Director in finalizing the
study!
And Remember!
Before a final report can be finalized,
signed and submitted to EPA, EVERY QA
finding must be completely addressed to
the satisfaction of the SD, QA and
Management!
FINDING: “Meteorological data not complete;
last data is for 31 July 2003. Please update
meteorological data.”
RESPONSE: None – FRD only signed the
bottom of the page of audit findings.
FINDING: “Part 6H – a brief description of
procedure used in cleaning spray equipment is
asked for here, not the SOP followed. Please
provide this information.”
RESPONSE: “Prior to use, equipment is
washed with clean potable water. Tanks are triple
rinsed and flushed as are the lines, nozzles,
strainers, and nozzle screens. Equipment is
inspected for leaks or obvious wear prior to
calibration and use. Clamps are ………………….”
FINDINGS: “Part 6C – incomplete error
documentation; all cross-out entries need to be
initialed and dated and the error code added.
Lined out spaces need to be initialed and dated
on Part 5G, 5H, 7B, 7D. Part 7B – crop destruct;
entry needs to be initialed and dated.”
RESPONSE: For all three: “Acknowledged”
FINDING: “Parts 4 and 9 contain several
photocopies that lack one or all: Field ID #,
date, initials, location of originals”
RESPONSE:
“No
action needed, thanks
for the suggestion.”
OR
“No response necessary from Field Research
Director – Study Director should address.”
FINDINGS: “ Please line-out unused parts
of Parts 1C, 2A, 4C (page 1 and 3), 5G, and
5H.” A minor error is seen in Part 6, page 27
of 35. Total pass time was 41.5 seconds. 41
seconds has been used in calculations. This
has given a lower % deviation of +3.65. It
should be +4.76%.”
RESPONSE: For both: “OK”
FINDING: “Part 7A, sample collection/storage;
you failed to mention whether row ends were
sampled from, and whether five or more hills were
sampled from. This needs to be added to the
sampling description.”
states: ‘Avoid
sampling from plot edges/ends,’ so we complied
with the protocol.”
RESPONSE:
“Protocol
FINDING: “Part 6C, pg. 2 and 4 of 32: please
cross out field ID of trial that does not belong in
this FDB. In the future, only one field ID per
page, please.”
– not written rule.
If it’s not written, it doesn’t exist.”
RESPONSE:
“Opinion
FINDING: “Part 7B. Crop destruction
section needs to be filled out with appropriate
information.”
RESPONSE:
“Acknowledged.”
Final Thoughts
- Overall, responses to QA audits have been improving
significantly, as has the consistency of QA findings, etc.
- There is still room for improvement on both sides, and
it’s through training events like these that we can reemphasize, face-to-face, the need to communicate,
provide clear direction, and use a neutral (vs
adversarial) tone in order to expedite responses and the
entire review process.