Transcript Agile 2009

Moving to Agile in an FDA Environment
An Experience Report
August 27, 2009
Introduction
• Available for download at http://agile2009.agilealliance.org/
• Agile Resources
– Agile Manifesto http://www.agilemanifesto.org/
– Agile Alliance http://www.agilealliance.org
• Agile Books
– http://www.agiletek.com/thoughtleadership/books
“Agile Software Development” by Alistair Cockburn
“Agile Project Management” by Jim Highsmith
An Experience Report
Slide 2
Outline
• The Abbott Experience
• Lessons Learned
1. The least burdensome approach?
2. A Risk Based Approach
3. Control what you don’t know, don’t let it
control you
4. Dispense with ceremony
• Results
An Experience Report
Slide 3
The Abbott Experience
• Comparing two FDA-regulated
medical device projects
– One not agile, one agile
– Class III devices (most stringent)
– Results of agile adoption
•
•
•
•
•
Lower cost
Shorter duration
Better, less prescriptive test cases
Accommodated change
Higher quality
• See paper “Adopting Agile in an
FDA Regulated Environment”
An Experience Report
Slide 4
Software Development Lifecycle
An Experience Report
Slide 6
Iteration Model
Iteration Model
Software Characterization
Product Definition
Feature Summary
Feature Descriptions
Software Requirements
...
Planning
Software Development Plan
Software Process Definition
Software Engineering Development Practice
Software Test Plan
Software Configuration Management Plan
Software Quality Assurance Plan
Software Measurement Plan
Capability Allocation Matrix
Software Schedule
...
High Level Design
High Level Design Description
Software Architecture Description
User Interface Design Document
User Interface Framework
...
Software
Construction
Software Confirmation
System Test
Design
Feature Detailed Design
Presentation
Business Rule
Data Access
Persistence
Regression Analysis
Integration Test
Acceptance Test
Functional Testing
...
Test Design
...
Stakeholder Review
Implementation
Application
Requirements
Feature Descriptions
Scenario Descriptions
...
Presentation
Business
Data Access
Database
Test Cases
...
Verification
Unit Test
Integration Test
Code Inspection
Design Review
...
An Experience Report
Slide 7
Software Release
Version Description
(Release Notes)
Traceability Matrix
Inputs -> Outputs
Validation / Verification
Data
...
Sample Design Control Documents
1.
QSR Ref.
820.30(b)
ISO Ref.
7.3.1
2.
820.30(c)
7.3.2
Design Control Element
Design and development
planning
Design input
3.
820.30(d)
7.3.3
Design output
4.
820.30(e)
7.3.4
Design review
5.
6.
820.30(f)
820.30(g)
7.3.5
7.3.6
Design verification
Design validation
7.
8.
9.
820.30(h)
820.30(i)
820.30(j)
NA
7.3.7
4.2.3-4
Design transfer
Design changes
Design history file
Documents
Design and Development Plan
System Requirements Specification
Safety Risk Analysis
System Design Description
Source code
Drawings/Schematics
System review reports
Design review reports
Verification Test Procedures
Validation Test Procedures
Test Summary Report
Requirements traceability matrix
Design transfer procedure
Change control procedures
Document control procedures
Courtesy of: Certified Compliance Solutions, Inc.
An Experience Report
Slide 8
Lessons Learned
Least Burdensome
We [FDA] are defining the term
“least burdensome” as a
successful means of
addressing a premarket issue
that involves the most
appropriate investment of
time, effort, and resources on
the part of industry and FDA.
--The Least Burdensome Provisions of the FDA
Modernization Act of 1997: Concept and
Principles; Final Guidance for FDA and Industry
An Experience Report
Slide 10
A Risk Based Approach
Courtesy of: Certified Compliance Solutions, Inc.
An Experience Report
Slide 11
Risk Based Approaches
• IEC 62304 section 5.1.1, page 31:
Note 1: The software model can identify different
activities for different software items according to
the safety classification
• FDA General Principles of Software
Validation, page 7, section 3.1.2:
The level of confidence, and therefore the level of
software validation, verification, and testing effort
needed, will vary depending on the safety risk
(hazard) posed by the automated functions of the
device.
Courtesy of: Certified Compliance Solutions, Inc.
An Experience Report
Slide 12
FDA Reviewer Guidance 2005
SOFTWARE
MINOR
MODERATE
DOCUMENTATION CONCERN CONCERN
Verification and
Validation
Documentation
Software
functional test
plan, pass /
fail criteria,
and results.
Description of
V&V
activities at
the unit,
integration,
and system
level. System
level test
protocol,
including
pass/fail
criteria, and
tests results.
MAJOR
CONCERN
Description of
V&V activities
at the unit,
integration, and
system level.
Unit,
integration and
system level
test protocols,
including
pass/fail
criteria, test
report,
summary, and
tests results.
Courtesy of: Certified Compliance Solutions, Inc.
An Experience Report
Slide 13
Control What You Don’t Know, Don’t Let It Control You
• What you do know:
– A typical medical device is developed over a 3-5 year
time horizon
– It is a myth that you can predict in detail your end
product requirements up-front
– Start with a core set of features that you must implement
– Implement the core features first
– Defer the most volatile features as long as possible
• Iterative approach allows the team to:
– Manage scope and limit feature creep
– Negotiate scope and tradeoffs with key stakeholders
• “At time of commercial launch, a number of features,
once thought to be essential, were not included. Some
were deferred as long as three years. Nonetheless,
the product was considered highly successful and
trading off nice to have features for three years of
sales is an easy choice.”
An Experience Report
Slide 14
Dispense With Ceremony
• If it is not adding value, and it is not required,
do not do it
• The design history files should contain the
minimum set of documentation that satisfies
the regulatory requirements
• There will be other activities that you will
want to document, no need to include in
design history file, make sure they add value
and do it in a least burdensome way
• Avoid doing things because “that’s the way
we’ve always done it”
• If it feels like you are wasting your time you
probably are
An Experience Report
Slide 15
Results
Results
• High visibility
– Easier to manage and control
– Far fewer surprises
• Lower cost and shorter duration
– Estimated schedule and team size reduction of 20%30%
– Estimated cost savings of 35%-50%
• Higher quality
– Availability of working software facilitated continuous
testing instead of back loaded V&V
– Resulted in fewer overall defects, especially at the end
of the project
• Better work life balance and team morale
– Project death marches are rarer because the issues
are surfaced as you go and are managed accordingly,
not all saved up for the end of the project
An Experience Report
Slide 17
Q&A
Thank You
Tim Hughes
Managing Partner
[email protected]
847-699-2260
J.R. Jenks
Managing Partner
[email protected]
847-699-2250
John Skach
Rod Rasmussen
Senior Technology Architect
[email protected]
847-699-2264
Director, Informatics & Software Systems
[email protected]
847-938-3633
An Experience Report
Slide 19