Transcript Methods validation: A kit manufacturer perspective bioMerieux

AOAC & Industry Perspectives:
Validation and Verification of
Alternative Microbiological Methods
Ronald Johnson
Senior Manager
Scientific Affairs - bioMerieux Industry
AOAC INTERNATIONAL Board of Director
Roles at bioMerieux and AOAC
bioMerieux (bMx)
Project Management: AOAC Validations
North America Scientific Liaison: Government/Industry
AOAC
AOAC International Board Director (BOD)
Past Chair - AOAC Research Institute Board Director
Past Chair - AOAC RI Advisory Council
Study Director- bioMerieux method validations
Method author: Journal of AOAC, Inside Laboratory Magazine
AOAC INTERNATIONAL
 Non-profit organization with international membership
 Consensus organization - Headquarters Gaithersburg, MD
 Established in 1884 to review, validate &
publish analytical methods. Formerly part of FDA.
 125th Anniversary in 2009!
 best known in food and agriculture
 chemistry and microbiology methods for Official Methods of
AnalysisSM status (OMA) “Big Book of Methods” & online
 OMA methods are recognized in the U.S. Code of Federal
Regulations and are legally defensible in court worldwide
 Subsidiary - AOAC Research Institute,

established 1991
Methods validation:
AOAC Perspective
AOAC is Staying Relevant
15 Active
AOAC
Sections
Worldwide
Globally
recognized,
non-profit
Association
Over 3000
members
from
government,
industry, and
academia
Method Consensus with Global Stakeholders
Food Manufacturers
CFIA, HPB,
FDA BAM,
USDA MLG,
AOAC OMA,
AOAC RI
Harmonized
Performance
&
Acceptance
Criteria
Test Kit
Manufacturers
ISO
AFNOR
CEN
MicroVal
AOAC INTERNATIONAL Current
Projects
Stakeholder Panel on Infant Formula
and Adult Nutritionals (SPIFAN)
Stakeholder Panel on Strategic
Food Analytical Methods (SPSFAM)
International Stakeholder Panel
on Alternative Methods (ISPAM)
ISO harmonization, Produce, STEC
DHS - Stakeholder Panel
on Select Agent Detection Assays (SPADA)
 Microbiological methods are certified by
official validation bodies to meet accepted
performance standards set by official
validation organizations.
 The guidelines and criteria of AOAC in the U.S.
and ISO in Europe are accepted in most of the
rest of the world.
 AOAC in the U.S. and AFNOR in Europe are the
most widely recognized validation bodies.
AOAC International/ISO16140 Guidelines
Method Validation and Approval Organizations
AOAC
PTM
AOAC
PTM
MicroVal
NordVal
AOAC OMA
AFNOR
Health
Canada
NMKL
Acceptance of rapid method test kits across global
markets often depends on multiple validations
Regulatory Guidelines, Reference
Methods and Standards
AOAC
PTM
CFIA
FDA BAM
ISO
Health
Canada
USDA FSIS
AOAC Intl
CEN
Global Companies may need to use different validations
with unique protocols at various production sites worldwide
The Weary Food Microbiologist
CFIA
Health
AOAC OMA
AOAC
Canada
PTM
USDA FSIS
FDA BAM
Health
Canada
AOAC PTM
AFNOR
Help!!
AOAC Intl
ISO
MicroVal
CEN
NMKL
NordVAl
Insanity?
Insanity: doing the same thing over and over again and expecting
different results.”
Albert Einstein
BENEFITS OF METHOD HARMONIZATION
• No Scientifically Valid Reason For Validating
Food Microbiological Methods Differently
• Need For Method Harmonization (ISO/AOAC)
Has Been Recognized For Many Years with
Limited Progress
• Benefits of Harmonization:
- Broad Acceptance Of Microbiological
Methods And Associated Results
- Speeds Up Implementation Of New
Methods/Technologies Across Appropriate
Food Matrices
• ISPAM Harmonization Efforts Ongoing
IMPACT OF GLOBAL TRADE ON THE WORLDWIDE FOOD SUPPLY
With increasing global trade there is an increasing
concern about the safety and security of foods in the
global supply chain
IMPACT OF GLOBAL TRADE ON FOOD SUPPLY
Consumer warnings:
Recalls and Outbreaks:
Company liability,
economic stability
Can often be prevented by
implementation of HACCP
and other food safety programs
and validating/verifying the
effectiveness of these programs
with AOAC Official (OMA)
methods/AOAC RI PTM methods
How to Better Safeguard
the Global Food Supply
“”Trust but Verify”
Level of Confidence: Fit for purpose
 Validated methods must be chosen according to
the intended “fit for purpose for use” of the
method.
 AOAC validation requirements and lab verification
should be appropriate for the intended “purpose for
use” e.g.
Industrial process monitoring
Regulatory screening of foods
Industrial process verification
Regulatory compliance
Crisis management – outbreaks…traceback
Greater
Precision
required
Test Kit Manufacturer Perspective
bioMerieux Goals for Food Safety
Provide Innovation to Food Industry
Shorter Turnaround Time is needed :
Same Day solution
bioMerieux develop new technologies to
reach this goal
Provide Performance to Industry
Sensivity & specificity enhancements
Phage detection
Provide Useful Tool to Industry
Integrated solution, with high
throughput and limited hands-on-time
bioMerieux has a long history in
automation of microbiology: VITEK2,
VIDAS, TEMPO, BacT/Alert, AES
Test Kit Manufacturer Perspective
Methods validation:
bioMerieux Development Process:
Confidence in Performance Claims
1. Feasibility
2. Optimization
Design Control Phases (controlled by FDA)
3. Internal Verification
4. External Verification
5. Industrialization
6. External Validation
7. Launch authorization
8. AOAC/AFNOR Validation
Test kit Co. Development & Validation
Requirements for Pathogen Detection
1. Recognition of target analyte (Pure culture
Inclusivity/exclusivity)
Cell surface receptors (affinity/avidity) –
a. Antibody/antigen
b. Phage/conformational sites
c. Hybridization/PCR
2.
Limit of detection of the target analyte
(spiked enrichment and detection)
growth dependent detection
a. 0 tolerance
b. 1 CFU/25g
c. 1 CFU/375g, 1 CFU/125g
AOAC (bMx) VALIDATION PROCESS
Inclusivity: Ability To Detect Target Analyte
From A Wide Range Of Strains
-Minimum of 100 different serotypes for
Salmonella (>2200 serovars)
-Minimum of 50 for all other organisms
Exclusivity: Lack of Interference from a Relevant
Range of Nontarget Strains
-Minimum of 30 isolates
Balance between sensitivity and specificity w/
near neighbors (priority: sensitivity)
Method Comparison
(Core of Validation)
Evaluating a commercial kit in comparison to a reference method
Test
sample
(Paired or
unpaired)
Alternative method
Reference method
AOAC Official Methods (OMA)
VALIDATION PROCESS
Phase 1: Method Comparison
(Pre-collaborative study)
-Performed by one laboratory (contract lab)
-All matrices claimed are evaluated
-AOAC Performance Tested SM (PTM)
-Independent laboratory
Phase 2: Interlaboratory Collaborative Study
Multiple laboratories (minimum of 10 labs)
 AOAC International
 Official Methods of AnalysisSM (OMA)
 12 months for approval
-------------------------------------------------------------------------------------------------------------------
AOAC Research Institute
Performance Tested MethodsSM (PTM)
6 months for approved certificate
----------------------------------------------------------------------------------------------------------------------------
Harmonized Program
PTM + OMA (12 approval for both)
+
bioMerieux AOAC Approvals
21 OMA approvals (all foods/variety of foods, env.)
14 VIDAS- rapid pathogen detection
2 TEMPO- automated QI testing
2 VITEK 2- rapid identification
1 API
2 VITEK (legacy) rapid identification
bioMerieux AOAC Approvals
27 AOAC RI – PTM approvals (variety of foods, env.)
11 VIDAS
6 TEMPO
3 VITEK
6 Adiafood- PCR
2 Plate methods ALOA, CFA
Food Safety Modernization Act (FSMA)
•Mandatory Food Facility Registration
•Issuing recalls: Direct FDA recalls of food
products.
•Conducting inspections: more frequent
inspections based on risk
•Importing food: Greater oversight
•Preventing problems: HACCP plan in place
•Focusing on science and risk
ISO Requirements for Verification
Importance of 3rd Party Validated Methods as it
Relates to ISO 17025 Laboratory Accreditation
ISO Requirements for Verification
Official or reference (validated) methods already
published and intended for specific matrices and
target organisms that will be incorporated “as is”
into the current method document format do not
need to be revalidated.
However, the ability to perform the analysis must
be verified to ensure that the method is fit for its
intended purpose
ISO Requirements for Verification
In Addition:
Any modification to an official/reference method,
including extension to a new matrix or target
organism requires revalidation/verification
(Note: The level and rigor of verification will
depend on nature/extent of the modifications)
Laboratory demonstrates the capability of
achieving specified method performance
characteristics before implementation and
on an on-going basis.
Provides data to illustrate that the method
is fit-for (it’s)-purpose i.e. meets customer
requirements
AOAC Validation Programs
How much confidence do you need in method
performance?
Answer: SHOW-ME Validated and
Verified “Fit-for-Purpose” Methods
Laboratory 17025 Accreditation
A procedure by which an authoritative body
such as A2LA gives formal recognition that a
laboratory fulfills specified requirements and is
competent to carry out specific tasks.
Technical Elements of ISO 17025
• Personnel-Training
• Separation of incompatible activities
• Test Methods Verification Including
Measurement Uncertainty
Calculations and Data Transfers
• Equipment
Computers or Automated Equipment
Protection of Data
• Measurement Traceability
• Sampling
• Handling of Test Materials
• Quality Control
• Reporting the Results
Guideline applied: procedure in place
Big Picture of assessment for ISO accreditation:
1. Quality system
a.
All components in place
b.
Documentation
c.
Traceability
ISO 17025 Requirements
1.
2.
3.
Verification
Trending for measurement of uncertainty
Proficiency testing (annually) AOAC, API
programs
- Verify in house (initial verification of method
for your scope)
20 data points 4, 5 analysts – 5 samples each
Stable Matrix/target (NFDM, peanut butter,
instant mash potatoes 1-5 CFU/25g)
ISO 17025 Requirements-continued
- Measuring uncertainty daily (daily control
trending)
-track daily performance (pure cultures:
pathogens, APC, Coliforms,
enterobacteriaceae)
-Proficiency testing - Approved PT supplier
(AOAC, API)
In the method scope: Verifying a similar matrix
RTE turkey validated, but verifying RTE ham
-spike (i.e., 5-10 reps 2-5 cfu/25g)
Caveat: Use outside of method scope may
require re-validaton!
ISO 17025 Method verification
AOAC 2008 Document
AOAC Analytical Laboratory Accreditation Criteria Committee (ALACC)
NOTE: Regulatory Guidance
http://www.fda.gov/ScienceResearch/FieldScience/
LaboratoryManual/ucm171877.htm#1
http://www.fsis.usda.gov/wps/wcm/connect/966638
c7-1931-471f-a79e4155ce461d65/Validation_Studies_Pathogen_Det
ection_Methods.pdf?MOD=AJPERES
http://www.fsis.usda.gov/wps/wcm/connect/d3382b
4f-27df-4e69-9744-ea8d73a2d603/At-Least-Equalto-Guide-MPI-Programs-LabMethods.pdf?MOD=AJPERES
Automated (bMx) Systems
Easier to Accredit
VIDAS®, TEMPO®, VITEK®2
– For test kits
– Methodology
Automated (bMx) Systems
Easier to Accredit
Summary:
Avoids transcription errors
Easy access to data
Data management for trend analysis/HACCP
Complete traceability of results
Simplifies audits
Connected to the LIMS
Historical data on process/products
Reproducible results
Compatible with ISO 17025:2005
Test Kit Manufacturer Perspective
Final Thought:
• The development of new testing technology
presents a considerable challenge but this is
not the end of the story.
No wine before its time!
• Extensive work to achieve validation and
laboratory implementation/verification is
necessary before the new method is
accepted.
Final Food for thought for ISO training
AOAC Final thought:
Take Advantage of the power of
the AOAC Association!
Networking
Opportunities
• AOAC Annual
Meeting
• Section
Meetings
• Technical
Divisions
• Communities
Standards
• Performance
Tested
MethodsSM
• Official Methods
of AnalysisSM
Publications
• The Journal of
AOAC
INTERNATIONAL
• Inside Laboratory
Management
Member
Magazine
Laboratory
Services
• Laboratory
Auditing
• Proficiency
Testing
AOAC International Expostion
September 7 – 10, 2014Boca Raton, Flo
•
SCIENTIFIC SESSIONS
•
CALL for POSTERS
•
AOAC Section meetings: China, Europe,
India, Japan, Latin American-Caribbean,
Lowlands, Taiwan, Thailand, Mid-Atlantic,
Mid-Canada, Midwest, Pacific Northwest,
Pacific Southwest, Southern, Southern
California
From the voice of bMx,
Thank you and good evening Anaheim!
For more info:
www.aoac.org