Transcript ISO / TS 16949 : clause 7.6 - Naamsa - National

The National Metrology Laboratory of South Africa
Meeting the requirements of clauses 7.6
and 8.1 of ISO/TS 16949
NAAMSA Metrology Interest Working
Group – 16 February 2005
Eddie Tarnow
Metrologist
Automotive
CSIR National Metrology Laboratory
Metrology Laboratory
Your measurement technology partner for global competitiveness
Discussion overview
Points for discussion
• ISO/TS 16949: clause 7.6; Control of monitoring and measuring
devices
• Selection of the correct test & measuring equipment
• Calibration of equipment
• Verification/validation of equipment
• Calibration intervals
• Measurement traceability
• Calibration status of equipment
• Calibration integrity
• Assessing the impact of “out of calibration” equipment
• Calibration records
• Validation of software used for testing measurement
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
Discussion overview
Points for discussion continued
• ISO/TS 16949: clause 7.6; Control of monitoring and measuring
devices
• Laboratory requirements
• Internal Laboratory
• Laboratory scope
• Competency of personnel
• Testing of the product
• Ability to test according to national/international
specifications/methods
• External Laboratory
• Acceptance of laboratory to the customer
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
Discussion overview
Points for discussion continued
• ISO/TS 16949: clause 8.1; Measurement, analysis and
improvement, General
• Demonstrating conformity of the product
• ISO/TS 16949: clause 8.2.4; Monitoring and measurement of
product
• Measuring the characteristics of the product
• Summary
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 16949: Clause 7.6
Control of devices
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
Clause 7.6
Selection of appropriate test & measuring equipment
• Have the required tests/measurements been clearly identified?
• Has an analysis been conducted to determine the measurement
accuracy required? What uncertainty is acceptable?
• What instruments have been identified that will perform the
measurement/test? (Are they fit for the intended purpose?)
• Have the instrument accuracy specifications been correctly
interpreted? (especially taking into account environmental
tolerance)
• Has cost vs accuracy been adequately evaluated? (TUR)
• high accuracy – increased calibration costs, fewer
calibration service providers, shorter calibration intervals,
longer downtimes due to calibration
• lower accuracy – increased risk of non-conforming product,
lower cal costs, many cal service providers, longer cal
intervals, quicker cal turnaround time
• Is local technical support available?
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 16949: Clause 7.6
Control of devices
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
Clause 7.6
Calibration of test & measuring equipment
• Definition: Calibration – comparison against a standard of known
higher accuracy, adjustment to agree or noting of the difference.
• Is a full calibration conducted, or just a partial?
• Is the instrument calibrated according to the manufacturer’s
recommended performance verification procedure to within
specification or merely to within a stated uncertainty?
• Has the instrument been adjusted to get it to within spec? Or
have correction factors simply been determined?
• Have the results been recorded/reported “as found” as well “as
left? (before adjustment & after adjustment)
• Has the instrument been calibrated at the same points as
previous calibration, thereby providing a useful history?
• Has the calibration service provider evaluated the results for
conformance to accuracy specification and reported a statement to
that effect?
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 16949: Clause 7.6
Control of devices
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
Clause 7.6
Verification/validation of test & measuring equipment
• Definition: Verification – check that nothing has changed since
the calibration
• Definition: Validation – check to see the instrument is indeed fit
for purpose
• Has the required accuracy been proven by the calibration?
(validation)
• Is there evidence that the instrument accuracy remains within the
specification? (If compared to last cal & no adjustment performed,
verification)
• Are measurement checks performed in between calibrations to
maintain confidence in the original calibration status? Are the
results of these checks recorded?
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 16949: Clause 7.6
Control of devices
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
Clause 7.6
Calibration intervals
• Definition: Calibration interval – the interval between consecutive
calibrations to ensure the required accuracy is maintained.
• Has a suitable calibration interval been scientifically determined
taking into account the following factors?
• desired accuracy,
• stability of the instrument,
• risk analysis of the impact of an instrument NOT within the
required accuracy,
• history of previous calibrations,
• how often adjustment is required and when last was it
performed, (previous cal history destroyed after adjustment),
• influence of the cal service provider –
• use of the same calibration method &
• calibration at the same points
• Does the calibration service provider determine the cal interval?
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 16949: Clause 7.6
Control of devices
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 16949: Clause 7.6
Measurement traceability
• Definition: Measurement traceability – unbroken chain of
comparisons between the UUT and the national measurement
standard, each step of which has an estimated uncertainty
associated with it.
• Can the last step of the traceability chain be identified/described?
(Is the identity of the standard used known and unique?)
• Was the uncertainty of the standard used for the calibration
acceptable? (Was the test uncertainty ratio valid?)
• Was the calibration service provider SANAS accredited & was the
calibration performed within the laboratory’s published
accreditation schedule best measurement capability (BMC)?
• Was the calibration procedure used validated and approved by
the manufacturer of the instrument? (Were there any deviations
from the manufacturer’s procedure and if so were the changes
validated?)
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 16949: Clause 7.6
Measurement traceability continued
•Is there evidence of the competence of the metrologist who
performed the calibration?
• Is there an uncertainty of measurement reported with the results?
•Does the instrument provide “direct” or “indirect” traceability for
the intended measurement/test?
CALIBRATION POINT
100 mm ± 1 mm
CALIBRATION POINT
200 mm ± 1 mm
IMPORTED
TRACEABILITY
MEASUREMENT POINT
150 mm ± ?mm
TRANSFERRED
TRACEABILITY
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 16949: Clause 7.6
Control of devices
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 16949: Clause 7.6
To adjust or not to adjust during calibration
• Is there record of adjustments having been made during the
calibration?
• If adjustment has been performed, are the results before AND
after the adjustment reported?
• Does the calibration service provider have the technical ability to
perform adjustments? (manufacturer’s technical procedure for
performing adjustments as sequence, & influences of one
adjustment on another, can be critical)
• Advantages of adjusting to within spec
• eliminates the need to apply corrections,
• facilitates ease of use for the operator,
• instrument can be used in other applications,
• can reduce the cost of calibration,
• simplifies the estimation of uncertainty of measurement
when using the instrument
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 16949: Clause 7.6
To adjust or not to adjust during calibration continued…
•Disadvantages of adjusting to within spec
• adjustment immediately invalidates (destroys) previous
history, (this can be managed)
• can increase the cost of calibration (some calibration service
providers refuse to perform adjustments or do not have the
technical ability – typically a problem if not using the
authorised agents as the calibration service provider)
• the natural drift of the instrument may be disturbed (the
instrument may become more stable if left undisturbed for a
long period of time)
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 16949: Clause 7.6
Control of devices
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 16949: Clause 7.6
Calibration status
• Can the calibration status, (valid or invalid calibration), easily be
determined by the user? (How easily can the user inadvertently use
an “out-of- calibration” instrument? Who’s responsibility?)
• On what basis is “Valid Calibration” status assigned to an
instrument?
• Is the calibration expiry date indicated on the calibration label?
• If partially calibrated, is there indication to the user to prevent
accidental usage for another application?
• Is there a unique link between the calibration status information
and the instrument to avoid accidental applicability to the wrong
instrument?
• How will the quality system prevent the use of an “un-calibrated”
instrument?
• If the calibration was conducted by an accredited calibration
service provider, does the SANAS logo appear on the associated
documentation and calibration label?
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 16949: Clause 7.6
SANAS accredited calibration laboratory logos
Fig 1: Old Logo
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
Fig 2: New Logo
ISO/TS 16949: Clause 7.6
Control of devices
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 16949: Clause 7.6
Calibration integrity
• Is the calibration validity protected in some way? (Can the user
accidentally or intentionally interfere with the calibration?)
• Who is responsible for ensuring calibration integrity? (Cannot
always be the calibration service provider as the user may have
access to adjustments)
• Which methods of calibration integrity protection are employed?
Integrity seals over adjustment access points or chassis
opening points? Password protection – who keeps the
password?
• If an entire system is calibrated, does the protection apply to the
entire system or only one instrument within the system?
• If in-house integrity seals are used, can they be accessed and
applied by the instrument user?
• How is integrity sealing applied when more than one calibration
service provider is used to calibrate the full capability of an
instrument and one’s adjustment may affect the other’s
calibration?
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 16949: Clause 7.6 cont.
Control of devices cont.
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 16949: Clause 7.6
Evaluating the impact of an “out-of-calibration” instrument
• Is there communication from the calibration service provider
regarding an “out-of-calibration” instrument and any subsequent
adjustment?
• Was the error larger than the required uncertainty of the
measurement or test?
• Was the instrument out of cal at a point directly applicable to the
measurement function being used or was it on another range,
function?
• Has a risk profile of the measurement been drawn up? Will an
erroneous measurement have major consequences which cannot
be rectified?
• What actions have been implemented to reduce the risk such as:-
• reducing the calibration interval,
• performing in-between-calibration verification checks,
• repeating the measurements using more than one instrument
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 16949: Clause 7.6 cont.
Control of devices cont.
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 16949: Clause 7.6
Calibration records
• Have the records of calibration been reported on a calibration
certificate?
• If only an in-house document is generated are at least the
following recorded:• measurement results,
• identity of the metrologist & authorised signatory conferring
validity to the calibration,
• estimated uncertainty of the measurements,
• before and after adjustment results,
• unique identity of the instrument,
• description of the calibration procedure/method followed,
• date & location of the calibration,
• measurement standards used,
• conformance statement if required
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 16949: Clause 7.6
Verification records
• Does a verification policy exist defining verification intervals and
measurement points?
• Is verification performed straight after calibration to establish a
“signature”?
• Are the results of verification measurements recorded and
analysed? Do they demonstrate the ongoing ability of the
instrument to meet the requirements of its intended purpose?
• What action is implemented when the results obtained after
verification differ markedly from the previous results?
• How and by whom are the verification measurement points
chosen?
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 16949: Clause 7.6
General instrument records
• Does each instrument have a history file?
• Are details of maintenance work recorded in the file? (In
particular any possible invalidation of calibration resulting from
maintenance)
• Are costs of maintenance and calibrations recorded to track
lifecycle costs?
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 16949: Clause 7.6 cont.
Control of devices cont.
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 16949: Clause 7.6
Software validation
• Is there evidence that the software is fit for its intended purpose?
(proof that the software produces the correct measurement/test
results)
• Has this been “validated” by comparing software generated
results with results obtained by means of manual measurements?
• How is the integrity of the software protected?
• can a different version be inadvertently installed over the
original version?
• is a copy of the original version kept in a safe place?
• is a particular version easily uniquely identifiable?
(traceability)
• if changes are made, are these adequately documented and
controlled? (can the user of the software make changes?)
• Is the integrity of the entire measurement process controlled
including sampling, measurement and result storage?
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 16949: Clause 7.6.1
Measurement system analysis
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 16949: Clause 7.6.1
Statistical analysis of measurement result variation
• Is the measurement result variability statistically analysed using a
method agreed to by the customer
• Analysis of “repeatability” and “reproducibility”
Definition of Repeatability
“Closeness of agreement between successive measurement
results obtained using the same system under the same
conditions”
Definition of Reproducibility
“Closeness of agreement between successive measurement
results obtained using the same system under changing
conditions”
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 16949: Clause 7.6.1
Estimation of uncertainty of measurement
• Is an uncertainty of measurement estimated for each
measurement/test performed and is it recorded with the result?
• Is this uncertainty estimated according to the requirements of the
ISO document entitled, “Guide to the expression of uncertainty in
measurement” or ISO 14253-2, “Guide to the estimation of
uncertainty in GPS measurement, in calibration of measuring
equipment and in product verification”?
• Are decisions of product conformance based on ISO 14253-1,
“Decision rules for proving conformance or non-conformance with
specifications”?
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 14253-1
Result of a measurement, y, and result of measurement,
complete statement y’
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 14253-1
Example of conformance proved
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 14253-1
Example of non - conformance proved
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 14253-1
Example of neither conformance nor non - conformance
proved
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 14253-1
Uncertainty of measurement reduces conformance range
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 16949: Clause 7.6.2
Calibration/verification records
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 16949: Clause 7.6.2
Calibration/verification records
• Highlighted points from above:-
• Do the calibration records reflect any engineering changes
made to the instrument? (engineering changes can take place
without the knowledge of the customer)
• Does the contract review, between a calibration service
provider and the customer, record that special mention must
be made in the reporting that an instrument’s performance is
suspect for whatever reason?
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 16949: Clause 7.6.3
Laboratory requirements.
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 16949: Clause 7.6.3
Internal Laboratory requirements
Laboratory scope
• Is there evidence of a laboratory scope listing the methods or
tests/measurements which can be performed by the laboratory?
• Are the uncertainties, (Best Measurement Capabilities), for these
methods or tests/measurements quoted?
• Is the technically responsible person, who confers validity to the
tests/measurement reports, identified?
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 16949: Clause 7.6.3
Laboratory requirements.
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 16949: Clause 7.6.3
Internal Laboratory requirements
Adequacy of laboratory procedures/methods
• Are documented test/measurement methods available?
• Have they been authorised for use in the laboratory by a person
with appropriate signing power?
• Have they been declared technically capable of delivering the
desired technical results, i.e. validated, by an appropriately
technically competent person? (changes made since validation?)
• Do they reflect sufficient detail to ensure that the correct results
will still be obtained by junior personnel if a senior personnel
member is off sick or injured?
• Do the methods address the entire measurement process
including sampling, performing of the measurements, capturing
and storing of results, estimation of uncertainty of measurement
as well as reporting the result?
• Are methods reviewed for ongoing relevance at planned
intervals?
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 16949: Clause 7.6.3
Laboratory requirements.
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 16949: Clause 7.6.3
Internal Laboratory requirements
Competence of laboratory personnel
• Since the responsibility of assessing whether or not the
metrologist is competent is an internal one:• Have criteria for the required competence been drawn up?
• Is there evidence that the metrologist’s competence has
been assessed against these criteria?
• Was the metrologist’s competence assessed by means of
physically witnessing a measurement/test AND comparing the
results obtained with results of known accuracy?
• To what extent have audit sample measurements, or
participation in proficiency testing schemes, provided proof of
acceptable measurement capability?
• Does the metrologist have a training file containing records
of relevant training and experience?
• Is there sufficient depth of staff and is a designated “stand-in”
identified in the quality system?
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 16949: Clause 7.6.3
Laboratory requirements.
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 16949: Clause 7.6.3
Internal Laboratory requirements
Testing of the product
• Are suitable test methods available for testing of the product?
• Does the laboratory have access to the required technical
specifications in order to declare conformance or not?
• Are the uncertainties of measurement in the test adequately
estimated?
• Do decisions of conformance or non-conformance take the
estimated uncertainty of measurement into account?
• Has an impact analysis been conducted to evaluate the risk of
rejecting product which actually passes spec, and accepting
product which fails spec?
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 16949: Clause 7.6
Laboratory requirements.
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 16949: Clause 7.6.3
Internal Laboratory requirements
Capability to perform the services correctly to the relevant
process standard
• Are the latest revision applicable process standards available?
• Are revisions of process standards controlled?
• Do the relevant process standards apply in their totality, and if
not, are deviations clearly identified and authorised by an
appropriately competent person?
• Have all deviations from the process standard been validated
where applicable? (e.g. lab using different equipment to perform
the same test/measurement)
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 16949: Clause 7.6
Laboratory requirements.
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 16949: Clause 7.6.3
Internal Laboratory requirements
Review of the related records
• Is there evidence of ongoing planned review of all relevant
laboratory documentation?
• Is there evidence of methods being re-validated after a
modification?
• Are instruments routinely calibrated according to a calibration
schedule on which the calibration intervals have been scientifically
determined?
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 16949: Clause 7.6
Laboratory requirements.
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 16949: Clause 7.6.3
Laboratory requirements.
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 16949: Clause 7.6.3
External Laboratory requirements
Acceptability to the customer
• If the external laboratory is NOT SANAS accredited, does it meet
all the requirements of ISO/IEC 17025?
• Is there evidence of the customer having audited the external lab
to confirm this to be the case?
• If it is SANAS accredited and, or, meets the requirements of
ISO/IEC 17025, does it meet the technical requirements of the
customer? (Does the scope of the laboratory cover the
measurements/tests required by the customer) SANAS
accreditation does not by default mean the laboratory technically
meets the requirement of the customer!!
• Is there evidence that although the laboratory is SANAS
accredited, it has performed tests/measurements outside of its
accredited scope unbeknown to the customer?
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 16949: Clause 8
Measurement to prove conformity of product.
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness
ISO/TS 16949: Clause 8
Measurement to prove conformity of product continued.
CSIR National Metrology Laboratory
Your measurement technology partner for global competitiveness