Software Validation in Accredited Laboratories

A Practical Guide

Greg Gogates Fasor Inc.

[email protected]

A copy of this paper will be maintained on ftp.fasor.com/pub/iso25/validation/ 26 Sept 2001

Preamble

  ISO guide 25 “Adequate for Use” ISO 17025 “Suitably Validated”  Validation - Reasonable Assurance  Maintain Design Control  Know what's where!

 Keep it Simple  EA/GA(98)95 Draft “EA Guidelines for the use of Computers and Computer Systems in Accredited Laboratories” 26 Sept 2001 NCSL Presentation 2

Software Classifications

 COTS - Commercial Off the Shelf  MOTS - Modified Off the Shelf  CUSTOM - Written from scratch 26 Sept 2001 NCSL Presentation 3

COTS

 Use as delivered  No modifications/customizations  Verify functionality  Verify menu contents  Verify parameter files  Lock down changes  Installation checkout 26 Sept 2001 NCSL Presentation 4

MOTS

 Needs configuration/customization prior to use.

 Customizations require validation.

 Non-modified portion treated as COTS  Installation checkout.

26 Sept 2001 NCSL Presentation 5

CUSTOM

 Home Brew or Consultant code  Functional Requirements  Software Design  Traceability Matrix  Structural / Functional Tests  Installation Checkout 26 Sept 2001 NCSL Presentation 6

Software Life Cycle

26 Sept 2001 Requirements Design Construction

Traceabiliity Matrix

Testing Installation/ Checkout Configuration management NCSL Presentation 7

26 Sept 2001

Products / Systems

Software Product A Computer System 1 Software Product B Computer System 2 Software Product C

NCSL Presentation 8

Software Configuration Management

 What software is on what machine  Store originals on Network Share, Tape, or CD-R  Maintain all published versions  Control Access  Ensure user access 26 Sept 2001 NCSL Presentation 9

Documentation Configuration Management

 Evidence of validation  Evidence of adequate for use  Owners / User manuals  Software Lifecycle documents  Test Evidence 26 Sept 2001 NCSL Presentation 10

Good Practices - 1

 Treat each software product a piece of calibration equipment.

 Do checks to ensure nothing changes.

 Place software product masters in a read only directory.

 Network computers that access a shared program on a server.

26 Sept 2001 NCSL Presentation 11

Good Practices - 2

 Lock spreadsheet cells that contain math.

 Password protect configuration files or setup screens.

 Backup, back-up, and backup off site!

 Plan for hardware/software disaster recovery.

 Good information maintained @ ftp://ftp.fasor.com/pub/iso25/validation 26 Sept 2001 NCSL Presentation 12

Software Checklist - 1

1.

2.

3.

4.

5.

6.

7.

Has the Firmware been validated via the calibration by a cal lab who has the capability to thoroughly check the software (i.e. OEM or Authorized partner) (Note: for non-calibratable device Firmware, treat as Software) Have all of the "used" firmware parameters been documented and confirmed that they are correct? Does a Software Requirements document exist for the Software?

Does a Software Design document exist detailing either the full design or details of the configuration?

Does software testing documents exists describing completed, unique, test cases that exercise the design both +/- and confirms the requirements?

Is there a test log showing test failures and corresponding retests/dispositions?

Does evidence exist confirming correct software deployment at each target installation? [5.5.11] 26 Sept 2001 NCSL Presentation 13

Software Checklist - 2

8.

Has configuration control been applied to all of their Software/Firmware to ensure that: [4.12.1.4] a) b) c) The Software source code location is access controlled. [5.4.7.2.b] Firmware/Software formulas & parameters are "locked" to prevent inadvertent changes. [5.5.12] Equipment lists identify Software as a separate line item showing correct version and location. [5.5.5] 9.

Does evidence exist showing that personnel involved in Custom Software development have adequate training? 10.

Do Databases and spreadsheets include "audit trails" to not allow previously data to be obscured? [4.12.2.3] 11.

Do adequate instructions exist for the operation & maintenance of the Software? [5.4.1] 12.

Does the accuracy of the Firmware/Software meet or exceed the accuracy required by the test method? [5.5.2] 26 Sept 2001 NCSL Presentation 14

26 Sept 2001

S U M M A R Y Software Desired Define Requirements

COTS

What Type Confirm Purchased Product Meets Requirements

MOTS

Confirm Purchased Product Meets Requirements

CUSTOM

Define Design Verify User Parameters Build Customizations/ Modifications Build Code Test Product Install Product onto System(s)

NCSL Presentation 15

QUESTIONS?

26 Sept 2001 http://www.sfgate.com/sf/zippy/ NCSL Presentation 16