Transcript Title

VV40: Committee on Verification & Validation for
Modeling & Simulation of Medical Devices
Technical Symposium, Subgroup: Orthopaedics
A Case Study: Examination of RAM/CAM Application for
Evaluation of a Coupled Musculoskeletal-FEA Model
Anthony Petrella, PhD
Colorado School of Mines, Golden, CO
AnyBody Technology Inc, Cambridge, MA
Ruxi Marinescu, PhD
Brian McKinnon
Smith & Nephew, Inc, Memphis, TN
Jeff Bischoff, PhD
Zimmer, Inc, Warsaw, IN
SwRI, San Antonio TX
January 22, 2014
Aims of this Case Study
 Explore application of RAM/CAM to realistic modeling scenario
used in orthopaedic implant development
 Attempt to consider a more complex modeling workflow
comprised of multiple scales and simulation methods
 Consider a common modeling context in device new product
development – comparison to predicate device reference
 Specifically, we sought to examine the question...
How well do the RAM/CAM evaluation criteria work in their current
form for a coupled musculoskeletal-FE analysis for purposes of
device evaluation?
Literature Used

Kim et al., “Evaluation of Predicted Knee-Joint Muscle Forces
during Gait Using an Instrumented Knee Implant,” JOR, pp.13261331, Oct 2009.

Lin et al., “Simultaneous prediction of muscle and contact forces in
the knee during gait,” J Biomech, 43, pp.945-952, 2010.

Pegg et al., “Evaluation of Factors Affecting Tibial Bone Strain after
Unicompartmental Knee Replacement,” JOR, pp.821-828, May 2013.

Disclaimers
The authors did not organize the details of their articles with the intention of being
“scrutinized” in the context of the CAM
Evaluators have limited experience with RAM/CAM
Summary of Modeling Workflow
(Lin et al., 2013)
(Kim et al., 2009)
(Pegg et al., 2013)
COU
 Decision/Question… Have we satisfied design
verification requirements for this UKR design?
 Some patient needs:
a) Avoid subsidence of device
b) Avoid chronic pain
 Related design inputs:
a) Good coverage, modest resection, no excessive rise in
periprosthetic bone strain relative to successful predicate(s)
b) Modest resection, no excessive rise in periprosthetic bone
strain relative to successful predicate(s)
 Model serves as sole source of info to test whether
design inputs RE: bone strain have been met
V&V Workflow
 COU not explicitly defined, Draft v1.1
 One definition of COU is:
1.
2.
3.
Decision/question to be addressed
Influence of model on decision
RAM
Risk to patient

Any of the three elements can change
independently and affect COU

How can risk be separate from COU?
RAM – VV40 Guide Draft v1.1
Moderate: M&S is considered to
address only a part of the decision...
There are ample data from similar
sources...
Major: M&S is not the sole source of
information... Data are available from
similar sources to support the decision
but no data are available from the
actual environment...
Controlling: no data are available
from other sources for essential
aspects of the system and the M&S
plays a key role in the decision.
A. No adverse health consequences
B. Limited (transient, minor impairment
or complaints)
C. Temporary or reversible (without
medical intervention)
D. Necessitates medical or surgical
intervention
E. Results in permanent impairment of
body function or permanent damage to
a body structure
F. Life-threatening (death could occur)
G. Hazard cannot be assessed
COU Summary
 Decision/Question… Have we satisfied design verification
requirements (inputs a,b) for this UKR design?
 Related design inputs:
a)
b)
Good coverage, modest resection, no excessive rise in
periprosthetic bone strain relative to successful predicate(s)
Modest resection, no excessive rise in periprosthetic bone
strain relative to successful predicate(s)
Summary of Modeling Workflow
1
(Lin et al., 2013)
2
(Kim et al., 2009)
(Pegg et al., 2013)
CAM
A. Software Verification
Verification of the Software Code & Solution:
0.
1.
2.
3.
4.
Insufficient
Minimal
Some testing conducted
Some peer review conducted
All algorithms tested, independent peer
review conducted
Matlab model, geometry registration Geomagics. Based on the
information provided we don’t know if the author reviewed the
verification activities to determine if those were relevant to this
application. Based on prior analysis.
CAM A = 1
Note: element types, adequate mesh size, not applicable.
CAM
B. Validation computational model
B1. System configuration:
0. Insufficient
1. Minimal – abstraction of geometry
2. Simplified – single patient-specific case, captures major features
3. Minor/major features captured, ranges of possible geometry, multiple
cases
4. All features captured, multiple cases, statistically relevant

Skeletal model –
o
o


Implant geometric model – patient-specific post-surgery CT data, CAD
models of the patient’s implant components (Patient 1)
Bone geometric model – MRI-derived bone models from another
patient (Patient 2)
Muscle model – 11 muscles (values from literature), MRI-derived
models from Patient 2 with muscle and patellar ligament
origin/insertion locations
Articular contact model – TF and PF, elastic model
CAM B1 = 3?
CAM
B. Validation computational model
B2. Governing equations:
0. Insufficient
1. Substantially simplified
2. Model forms are based and tuned on data from related
systems
3. Representation of all important processes, tuning needed
4. Key physics captured, minimal need for tuning
Patient specific inverse dynamic model. The equations of
motion were derived using Autolev symbolic manipulation
software. The complete knee model was implemented in
Matlab (OpenSIM for Pegg et al. 2012)
CAM B2 = 3?
CAM
B. Validation computational model
B3. System properties (biological, physical
properties)
0. Insufficient
1. Simplified properties, sensitivities not addressed
2. Nominal properties, uncertainties
3. Distribution of properties, uncertainties identified
4. Key properties captured, sensitivity analysis

Skeletal model –
o
o


Implant geometric model – linear elastic isotropic materials (Pegg
et al. 2012)
Bone geometric model – Calculated from HU in CT scans? Pegg et
al. 2012)
Muscle model – muscles (strength values from literature),
patellar ligament (data from literature); mapping of the muscle
attachment sites (Matlab, Pegg et al. 2012)
Articular contact model – TF and PF, elastic model
CAM B3 = 3?
CAM
B. Validation computational model
B4. Boundary conditions (e.g., applied loading)
0. Insufficient
1. Significant simplification
2. Some simplification of BCs
3. Representative BCs, uncertainties identified
4. No simplifications, appropriate distribution of
variation, comprehensive sensitivity analysis
The femur was fixed to ground, where the tibia and
patella were allowed to move relative to it; Twolevel optimization approach (Matlab – min of the
sum of 3 compressive contact forces).
CAM B4 = 3?
CAM
C. Validation: Evidence-based comparator
C1. System configuration:
0.
1.
2.
3.
4.
Insufficient
Locations for data collection are roughly measured; geometry of parts is assumed
Locations for data collection are prescribed and measured; geometry of parts is
coarsely measured (?); calibrated system or signal/noise ratio>1
Locations for data collection are prescribed and error collected; geometry of parts
is measured to machine tolerance; signal/noise ratio is high
All dimensions known to greater than machine precision; high precision
Pegg et al. 2012
Force-measuring tibial prosthesis
Over ground walking trials (normal and medial-lateral trunk sway)
Experimentally measured contact forces (medial and lateral sides
of the tibial tray – loads magnitude, direction, position and contact
area recorded; custom python script)
CAM C1 = 3
CAM
C. Validation: Evidence-based comparator
C2. System Properties:
0. Insufficient
1.
2.
3.
4.
Material properties are average, homogeneous non-specific;
environment conditions unknown
Material properties are average, homogeneous specific to the system;
environment conditions known
Key material properties are measured and heterogeneity captured
All material properties are measured, environmental effects accounted
for.
Pegg et al. 2012
Adult male subject
Force-measuring tibial prosthesis
Gait analysis
Experimentally measured contact forces
CAM C2 = 4
CAM
C. Validation: Evidence-based comparator
C3. Boundary Conditions:
0. Insufficient
1.
2.
3.
4.
System states are not specifically measured
System states are specifically measured or perturbations are measured
System states are specifically measured, affected degrees of freedom
are known and perturbations are measured
System states are specifically measured and degrees of freedom are
known and perturbations are measured; variability known
Pegg et al. 2012
Adult male subject
Force-measuring tibial prosthesis
Gait analysis, over ground walking trials (normal and mediallateral trunk sway)
Experimentally measured contact forces
CAM C3 = 4?
CAM
C. Validation: Evidence-based comparator
C4. Sample Size:
0. Insufficient
1. Single case or few cases
2. Several cases or statistically relevant sample size
3. Several cases and statistically relevant sample size
4. Comprehensive parameter variability and statistically relevant
sample size for all parameters
Pegg et al. 2012
Single adult male subject
CAM C4 = 1
CAM
D. Determine model credibility

Discrepancy between the model and the comparator
“Model was implemented with the equivalent comparator conditions” (3 or 4?
Variability?)

Comparison: qualitative or quantitative
“Quantitative comparison of single achievable case” (2)

Applicability of V&V activities to Context of Use
“Embodies key CoU features and captures key system properties” (3)


Single adult male subject (Patient 1)
Musculoskeletal model
Skeletal model –
o
Implant model – patient-specific post-surgery CT (Patient 1)
o
Bone geometric model – MRI-derived bone models (Patient 2)
Muscle model – 11 muscles, MRI-derived models (Patient 2)
Articular contact model – TF and PF, elastic model
CAM D = 2.666666667 or 66.67% (8/12)
Credibility assessment matrix: MS Model
1
1
2
2
3
3
4 4
3
CAM. Verification
 A. Code






Mimics, CT segmentation
MATLAB, ICP & muscle force sites
SolidWorks, bone resection
Mimics + custom, material calcs & mapping
Abaqus, FE solution
Python scripting for Abaqus
 Application region for native contact load
 Analytical field for implant load to bone
 von Mises strain values
 Probabilistic variation of loading
 PASW Statistics, statistical analysis
 Score = 1
CAM. Verification
 B. Solution
 FE mesh convergence study performed
 FE simplifications, no sig affect on results
 Direct load vs. using actual implant
 Implant interface, tie vs. rough/friction
 Full length tibia vs. truncated
 Probabilistic variation in load magnitudes
based on errors reported in MS model
 Material properties from CT mapping
reported consistent with previous pub
 Score = 2
CAM. Model Validation
 A. Configuration
 Tibia bone geometry from single patient
extracted from CT using “previously
validated method”
 No information about implant model
 Loads applied directly to the bone surfaces
 Bone cuts made in accordance with surgical
technique published by implant company
 Score = 2
CAM. Model Validation
 B. Governing Equations
 Structural FE methods well established for
stress / strain calculation
 Physics…
 Static FE simulation
 Bone modeled as linear elastic and
isotropic – no rate effects
 Non-homogeneous material property
mapping from CT
 Muscle and joint loading derived from
gait simulation and instrumented TKR
 Score = 3
CAM. Model Validation
 C. System Properties
 Non-homogeneous material properties
mapped from CT based on published
equations
 Properties not compared to human subject,
but “consistent with previous” published data
 No sensitivity analysis reported in relation to
properties
 Score = 2
CAM. Model Validation
 D. Boundary Conditions
 Contact loading applied directly to bone surface;
compared to case with implant to confirm no
significant effect on outcomes
 Muscle forces from MS model applied to bone
 Load BC’s associated with level gait only
 Uncertainty in load magnitudes due to upstream
errors (MS model) incorporated using Monte
Carlo simulation with (only) 40 random cases
 Several custom Python scripts employed with no
direct verification – creates doubt
 Score = 2
CAM. Comparator Validation
 Comparator A. Configuration
 Comparator B. Governing Equations
 Comparator C. Properties
 Comparator D. Sample Size
 There was NO COMPARATOR, model
only used to assess relative change in
outcome metric (bone strain)
 Scores = 0, 0, 0, 0
CAM. Validation – Model/Comparator
 A. Discrepancy
 B. Comparison of Outputs
 There was NO COMPARATOR, model
only used to assess relative change in
outcome metric (bone strain)
 Scores = 0, 0
CAM. Validation – Model/Comparator
 C. Applicability of V&V to COU…
 Decision/Question… Have we satisfied design verification
requirements (design inputs a,b) for this UKR design?
a) Good coverage, modest resection, no excessive rise in periprosthetic bone
strain relative to successful predicate(s)
b) Modest resection, no excessive rise in periprosthetic bone strain relative to
successful predicate(s)
 Score = 4
CAM Summary
0
0
0
0
0
0
1
2
2
2
2
3
4
CAM Summary – Complete Workflow
 Is the multiscale workflow acceptable?
 Not obvious how to create composite score
 Does precursor (MS) model even need to be
evaluated, or simply captured in BC’s eval for
the FE model?
Comments

RAM/CAM application for a coupled MS-FEA modeling workflow





A model is rarely based on a single piece of code



How best to apply guidelines? CAM separate or combined?
Is CAM even needed explicitly for both (all) models? When?
An explicit definition of COU and how to identify it will probably be
needed for general users



Value in looking at published models?
Probably will be common in practice, and info often lacking in literature
Perhaps publication standards need to evolve
Library of Models, MS model repositories will help
We defined COU as: Question/Decision + RAM (influence, risk)
Any of three elements can independently change COU
Utility of RAM for orthopedic applications may be limited



Always “Medium”?
Same influence, same risk for any COU? Spine, joints, same risk for all?
Will some standard simplifications evolve for specific industries?
Comments

Should CAM be a measuring tool or a checklist?







Model Validation, BC’s



“Credibility = 2.6” could be misleading
Acceptance criteria? What is good enough? What does a “4” look like?
Perhaps some “grandfathering” of accepted modeling paradigms will occur
What impact does an incremental shift in COU have on acceptance?
Removing numbers could make a subtle but positive psychological difference, not just
post-hoc scoring but planning for specific CAM levels before model development
Moving “applicability” considerations to beginning of CAM may be more effective
Level 4 = “no simplifications”
Does this make sense for a “model”, which is inherently simplified?
Comparator evaluation difficult for human subjects



Especially for subject-specific modeling efforts
If model compares well to single subject, does that mean it is extensible to others?
Are all comparators equal? Is a weighting factor appropriate for human subjects?
Comments

510(k) pathway with comparison to predicate device extremely
common in orthopaedics

Predicate device comparison (relative analysis)




Strictly speaking, no direct comparator for outcome metric, but…
Predicate will have controlling influence on “decision”
Should predicate model be evaluated separately from primary model?
Should predicate evidence be critically considered, where/how?
Clinical
History

Predicate
Model
COU
Model
Incremental increase in value (CAM score) vs. increased cost to
improve model is a consideration