Transcript web.cs.uh.edu
The Reference Model: Improvement in Treatment Through Time in Diabetic Populations
Jacob Barhak Computational Surgery The Joseph B. Martin Conference Center Harvard Medical School Boston, MA, USA 9-10-11 Dec 2012
Modeling Scope Courtesy of C. Anthony Hunt - UCSF
The Reference Model Attempts to reduce uncertainty by competition and knowledge accumulation Jacob Barhak
The Reference Model
No CHD MI Survive MI CHD Death Process CHD Stroke Death Process Stroke Death No Stroke Stroke Survive Stroke Alive Other Death Process Competing Mortality • • • • Prototype demonstrating Technology Built from literature references and hence the name: The Reference Model Designed to serve as a reference for new equations and populations A League / Consumers Report for disease models Jacob Barhak
Key Features
•
Simple - Low Resolution
•
Based on secondary data:
– Published Risk Equations – Published Clinical Trials , i.e. no real individual data – Other publications •
Relies on computing power/techniques
– Parallel processing / High Performance Computing – Competition among alternative equation/hypothesis combinations – Cross validation One click validation – Ranks results according to fitness Fitness Engine – Accumulates knowledge Jacob Barhak
Risk Equations from the Literature Blinded
A1c, ACR, AF, Age, AgeAtDiagnosisOfDiabetes, BMI, DBP, DiabetesType2, FamilyHistoryCHD, HDL, LDL, LipidRatio, MacroAlbuminuria, Male, MicroAlbuminuria, Race, RheumatoidArthritis , SBP, Smoke, Survive_MI, Survive_Stroke, TC, Townsend, TreatedHypertension
1 2 3 4 5 6
Bio-Markers / Parameters
7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Risk Equations
3 4 1
*
2
* * * * * * * * * *
5
*
6
*
7
* *
8
*
9
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * Smoke Jacob Barhak
How Does it Work?
Pop 3 Pop 2 No CHD
1 4 2 3
MI Survive MI CHD Death Process CHD Pop 1 • • •
1 2
Process Stroke Death No Stroke
4 3
Stroke Survive Stroke Stroke Death
Eq 1 1 1 1 Eq 2 Pop 1 1
4
2
6
3
2 Alive Other Death Process Competing Mortality
Pop 2
2 2 4 3 6 9
Pop 3
… … … …
Loop through Risk Equation/Hypothesis combinations Loop through populations
Fitness Matrix
Calculate fitness of simulation outcomes to observed phenomena
… 1 2
4
1
1
… … … … … … Jacob Barhak
Hypotheses Tested
1. Biomarkers change in the first year
–
Risk equations use end of trial inputs, if available
–
Treatment effect through biomarker values 2. Treatment has improved with time
–
Assumes risk equations become outdated
–
Transition probabilities are multiplied
• –
Hypotheses were tested Separately
–
Together
–
With all equation combinations
Jacob Barhak
Biomarker Hypothesis
•
Example: ADVANCE standard control:
A1c BMI DBP SBP HDL LDL Trig Baseline 7.48
28 80.5
145 1.25
3.11
1.64
End of Trial 7.24
28 74.3
137.9
1.25
2.65
1.59
From Table 1 in: Intensive Blood Glucose Control and Vascular Outcomes in Patients with Type 2 Diabetes, NEJM Volume 358:2560-2572 June 12, 2008 Number 24 By default use these average numbers in the risk equation With biomarker hypothesis, adjust year 1 values to these averages in risk equation Jacob Barhak
Treatment Hypothesis
•
Healthcare quality improved through the years.
1.
• • M.L. Weisfeldt, S.J. Zieman. Advances In The Prevention And Treatment Of Cardiovascular Disease. Health Affairs, 26, no.1 (2007):25-37. DOI: 10.1377/hlthaff.26.1.25
Shows improvement trend Lists procedures and drugs on a timeline since 1970 2.
• • E.W. Gregg , Y.J. Cheng , S. Saydah , C. Cowie , S. Garfield , L. Geiss , L. Barker, Trends in death rates among U.S. adults with and without diabetes between 1997 and 2006: findings from the National Health Interview Survey. Diabetes Care. 2012 Jun;35(6):1252-7. DOI: 10.2337/dc11-1162 Quantifies improvement between NHANES cohorts Diabetic CVD death 2003–2004 vs. 1997–1998, Hazard Rate Ratio: 0.60
Jacob Barhak
Treatment Hypothesis
• • • – – – – The 0.6 HRR was translated to yearly probability decrease for: MI Stroke MI Death Stroke Death – – This probability was adjusted for time passed between: Model year = average of Model Data Time Interval Simulated Time Stamp = simulated study year/s Model Data Time Interval Simulated Time Stamp Time Adjustment Time This correction term accounts for model outdate Jacob Barhak
Simulation Overview
•
Measures fitness of Equations/Hypotheses/Cohort combinations
•
Calibration requires computing power :
–
64=4x4x2x2 equation/hypothesis variations
–
22 cohorts with known outcomes from 4 populations
–
10 repetitions of 1000 individuals for 10 years
–
14080 processes = 64x22x10
–
4 days of computation on a single 8 core desktop PC
Jacob Barhak
Without Treatment Hypothesis A1c changes BMI changes BP changes Lipid change Models Stroke Equation # CHD Death Equation # Stroke Death Equation # Treatment Improvement Correction
FITNESS: LOW SCORE = GOOD FITNESS
UKPDS33 Conventional UKPDS33 Intensive UKPDS33 Full ASPEN All Placebo ASPEN All Atorvastatin ASPEN Primary Placebo ASPEN Primary Atorvastatin ASPEN Secondary Placebo FITNESS ADVANCE Standard ADVANCE Intensive ADVANCE Asia Standard SCORE ADVANCE EME Intensive ADVANCE Eastern Europe Standard ADVANCE Eastern Europe Intensive ADVANCE Full ACCORD BP Standard Therapy ACCORD BP Intensive Therapy ACCORD BP Full
RANK POPULATIONS FOR EACH MODEL
UKPDS33 Conventional UKPDS33 Intensive UKPDS33 Full ASPEN All Placebo ASPEN Full RANK ASPEN Primary Atorvastatin ASPEN Secondary Placebo ASPEN Secondary Atorvastatin ADVANCE Standard ADVANCE Intensive FOR EACH ADVANCE EME Standard ADVANCE EME Intensive ADVANCE Eastern Europe Standard MODEL ADVANCE Full ACCORD BP Standard Therapy ACCORD BP Intensive Therapy ACCORD BP Full
RANK MODELS FOR EACH POPULATION
UKPDS33 Conventional UKPDS33 Intensive UKPDS33 Full ASPEN Full RANK ASPEN Primary Placebo ASPEN Primary Atorvastatin MODELS FOR ADVANCE Standard ADVANCE Intensive EACH ADVANCE EME Standard ADVANCE EME Intensive POPULATION ADVANCE Full ACCORD BP Standard Therapy ACCORD BP Intensive Therapy ACCORD BP Full
OVERALL MODEL RANKING RESULTS
Method_A1c Method_BMI Method_BP Method_Lipids Method_Smoke Method_MI Method_Stroke RANK Method_DeathCHD Method_DeathStroke Method_TimeImprove Weighted Mean 61 61 61 59 37 42 38 21 10 62 60 60 59 61 13 11 1 27 38 36 30 44 21 18 19 13 12 10 11 0 16 15 14 17 20 4 1 9 8 3 7 5 6 2 57 59 56 54 41 49 42 6 5 55 56 56 56 57 4 0 0 25 40 35 43 39 21 19 18 14 12 11 10 0 13 16 15 17 20 5 2 9 6 3 7 8 4 1 14 10 4 7 20 18 19 17 15 16 3 5 2 6 12 13 9 0 1 8 21 11 12 0 0 4 10 5 5 25 25 3 9 9 16 1 3 6 7 4 4 10 14 50
0 0 0 1 0 0 1 1 1 0
27.0
24.6
21.5
14.3
18.5
23.3
21.7
23.2
16.2
8.3
67.3
67.3
66.7
72.6
84.8
91.2
75.9
78.5
57.3
46.8
42.5
43.9
0 0 0 1 0 0 1 1 1 1
Treatment Hypothesis
0 0 0 1 0 0 2 1 1 0
With
Results
Best Model Overall
2 1 1 1 0 0 0 0 1 0
Without Biomarker Hypothesis
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 1 1 0 0 0 0 0 1 3 1 1 1 0 0 0 0 1 4 1 1 0 0 0 0 0 1 4 1 1 1 0 0 0 0 1 1 1 1 0 0 0 0 0 2 1 1 1 1 0 0 0 0 2 2 1 1 0 0 0 0 0 2 2 1 1 1 0 0 0 0 2 3 1 1 0 0 0 0 0 2 3 1 1 1 0 0 0 0 2 4 1 1 0 0 0 0 0 2 4 1 1 1 0 0 0 0 2 1 1 1 0 0 0 0 0 3 1 1 1 1 0 0 0 0 3 2 1 1 0 0 0 0 0 3 2 1 1 1 0 0 0 0 3 3 1 1 0 0 0 0 0 3 3 1 1 1 0 0 0 0 3 4 1 1 0 0 0 0 0 3 4 1 1 1 0 0 0 0 3 1 1 1 0 0 0 0 0 4 1 1 1 1 0 0 0 0 4 2 1 1 0 0 0 0 0 4 2 1 1 1 0 0 0 0 4 0 0 0 4 0 0 3 1 1 0 0 0 0 4 0 0 3 1 1 1 0 0 0 4 0 0 4 1 1 0 0 0 0 4 0 0 4 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 1 1 0 1 1 1 1 1 1 2 1 1 1 1 1 1 1 1 1 3 1 1 0 3 1 1 1 1 1 1 1 1 1
With Biomarker Hypothesis
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 4 1 1 0 1 1 1 1 1 4 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 1 2 1 1 1 1 1 1 1 1 2 2 1 1 0 1 1 1 1 2 2 1 1 1 1 1 1 1 2 3 1 1 0 1 1 1 1 2 3 1 1 1 1 1 1 1 2 4 1 1 0 1 1 1 1 2 4 1 1 1 1 1 1 1 2 1 1 1 0 1 1 1 1 3 1 1 1 1 1 1 1 1 3 2 1 1 0 1 1 1 1 3 2 1 1 1 1 1 1 1 3 3 1 1 0 1 1 1 1 3 3 1 1 1 1 1 1 1 3 4 1 1 0 1 1 1 1 3 4 1 1 1 1 1 1 1 3 1 1 1 0 1 1 1 1 4 1 1 1 1 1 1 1 1 4 2 1 1 0 1 1 1 1 4
24.1 24.6
22.5 21.4
23.0 21.3
10.2 13.8
9.4 20.5
9.5 23.1
13.1 24.3
37.6 21.2
18.8 10.3
11.8
7.9
9.9 54.0
12.1 59.3
18.9 58.4
21.2 64.4
14.0 80.4
21.6 83.0
15.7 71.0
10.8 69.2
12.7 54.9
30.4 48.0
27.9 46.7
27.9 45.4
31.7
26.7
26.3
9.7
10.2
11.1
11.4
39.7
20.9
13.4
10.4
12.8
17.0
17.5
20.4
27.4
19.5
17.5
14.6
27.9
28.6
29.2
24.6
24.6
22.4
10.2
17.6
22.3
21.7
21.9
12.2
7.6
60.1
62.7
64.5
71.2
79.7
82.4
69.1
73.6
55.4
54.3
49.8
53.0
33.1
30.1
29.9
10.2
11.3
9.0
10.3
40.4
29.0
13.1
16.0
14.8
12.2
18.0
26.4
30.3
21.5
19.6
18.3
35.1
32.7
34.0
26.9
24.2
23.6
15.1
15.8
20.6
22.9
21.0
11.6
8.2
62.7
60.3
70.9
79.4
81.1
87.1
70.8
75.8
57.0
53.0
48.9
51.8
31.2
26.5
29.5
10.6
8.0
9.0
11.9
37.3
20.7
9.7
12.8
14.5
13.2
17.3
22.4
28.3
17.1
16.2
12.7
32.0
29.2
29.3
43.0
37.0
35.1
15.9
22.7
19.7
25.0
22.7
14.8
9.8
36.6
42.7
35.2
32.8
75.6
82.3
65.6
66.3
52.7
41.5
43.5
44.3
24.6
31.9
29.5
21.9
28.4
31.5
32.1
6.2
18.5
17.3
24.1
23.6
21.5
24.5
49.2
45.7
41.2
44.2
38.8
21.5
15.1
17.7
42.5
34.1
34.5
13.5
22.6
25.4
23.0
19.0
13.1
10.3
27.5
34.5
26.8
32.7
67.9
74.9
59.7
61.1
52.5
44.7
41.6
43.2
32.7
35.2
39.4
22.0
28.5
26.7
32.7
11.6
20.7
17.9
22.8
27.2
20.5
23.2
51.7
57.0
48.0
47.7
44.8
24.1
24.2
25.3
36.3
35.6
35.1
17.8
18.7
24.6
24.2
20.5
15.3
9.4
31.0
36.4
29.2
37.8
65.9
70.5
58.1
56.1
53.9
50.6
48.4
49.1
35.8
38.2
37.3
18.1
30.0
31.3
31.2
17.7
27.5
19.5
27.2
27.2
25.3
23.9
52.9
58.5
48.7
49.2
48.1
34.7
31.7
34.8
39.2
34.2
34.3
19.0
21.6
23.0
25.5
18.1
13.2
9.0
32.9
37.6
32.6
37.3
65.1
75.3
55.9
59.3
53.3
49.5
43.4
50.1
33.4
34.0
33.8
22.1
29.9
27.8
31.6
11.9
23.6
20.1
25.0
28.9
19.9
26.7
49.7
56.0
42.8
45.1
46.6
29.0
29.0
28.2
39.7
32.2
31.5
12.5
16.8
15.5
17.2
18.4
25.9
9.4
54.1
59.1
53.0
58.9
79.4
79.0
62.9
69.5
53.1
62.6
54.4
58.9
29.4
40.8
37.1
23.5
26.0
23.8
25.2
15.7
40.1
16.6
39.4
47.0
43.5
49.5
55.1
57.2
44.9
51.9
50.2
32.3
35.3
33.6
39.7
32.5
32.0
12.6
15.4
14.3
15.6
13.6
23.0
5.6
48.0
51.3
48.3
53.9
66.9
76.3
57.1
57.8
56.4
63.3
55.9
60.4
36.9
43.3
42.4
18.7
25.2
22.6
25.7
15.4
39.2
15.7
42.2
48.9
43.6
52.4
57.7
66.0
56.7
53.2
51.7
39.1
36.5
41.2
39.6
34.3
33.6
10.9
16.6
16.0
17.4
14.6
24.3
7.6
50.1
53.4
52.4
56.5
70.0
72.9
51.4
55.0
52.5
66.3
63.6
64.3
46.2
45.7
45.9
18.2
29.5
21.7
26.2
24.4
46.6
13.7
52.4
53.5
47.4
54.1
59.0
63.8
51.2
57.0
54.9
49.3
47.7
47.7
35.9
29.6
34.2
12.1
13.9
15.8
16.5
12.8
24.2
8.0
49.9
56.2
53.1
57.5
72.4
73.9
51.6
54.4
54.3
63.5
60.3
63.3
38.1
41.8
42.5
19.3
28.2
21.7
25.7
16.2
42.7
17.4
43.4
34.6
42.4
43.8
23.2
28.8
33.3
34.7
20.9
20.8
18.9
65.4
50.8
47.0
52.9
61.4
73.2
67.2
79.7
89.0
65.4 100.6
50.0
81.8
49.3
55.1
42.4
39.8
41.1
79.0
64.6
52.8
47.5
49.9
22.7
21.6
21.9
12.5
9.8
9.8
13.1
45.7
26.2
14.8
12.9
9.8
23.3
16.9
16.9
16.0
23.8
16.2
13.8
37.0
33.8
35.1
35.8
41.4
39.7
22.4
25.1
31.3
31.4
13.8
19.3
18.8
60.5
63.5
63.9
70.7
82.2
86.4
72.2
72.7
61.4
49.4
44.3
48.2
27.8
25.6
27.9
13.4
11.8
10.2
10.5
47.3
26.4
16.3
10.1
8.5
21.0
14.3
19.8
17.1
22.2
19.1
12.8
36.2
32.2
34.4
33.7
43.7
42.6
22.9
28.9
29.9
32.2
19.4
19.3
17.3
67.6
67.9
67.1
74.4
87.3
92.6
76.2
78.8
64.5
54.4
56.8
51.8
30.4
30.6
32.3
13.5
12.1
9.0
10.0
50.7
33.1
16.2
12.3
11.6
12.2
7.6
24.5
20.6
20.8
21.0
13.1
38.3
37.6
37.6
34.4
41.4
41.7
25.1
29.0
32.7
32.7
16.6
20.6
19.7
60.2
70.2
66.7
75.2
81.5
89.3
70.2
68.4
66.2
52.5
52.0
56.2
29.8
29.5
28.7
13.4
10.1
9.3
10.4
48.1
26.0
15.3
11.0
11.6
11.7
7.7
20.8
18.1
22.5
16.7
13.6
36.2
34.9
36.0
23.6
22.8
23.7
15.5
15.0
23.4
17.0
22.7
12.0
8.2
45.9
40.4
54.4
45.3
64.3
61.4
54.0
43.5
39.6
46.5
36.7
43.5
24.7
22.7
21.6
10.3
9.4
8.7
11.3
37.9
21.9
11.9
10.6
9.8
21.2
18.7
11.1
11.9
20.4
19.3
15.8
30.0
28.7
28.4
25.0
22.4
22.5
14.5
12.8
21.3
15.7
21.1
7.6
10.1
40.1
37.6
42.2
42.6
58.7
54.7
47.9
39.9
35.5
47.7
37.2
42.2
29.2
29.0
28.1
9.0
10.6
9.8
10.7
36.3
24.7
13.7
7.2
7.1
19.3
15.3
17.7
19.5
17.6
16.8
11.6
28.5
29.2
30.1
25.5
22.4
21.9
12.3
12.8
20.8
16.5
19.5
12.5
10.4
43.5
39.2
47.3
42.2
64.6
57.3
48.5
40.9
38.2
46.9
43.6
48.0
33.0
28.6
29.6
9.3
11.3
9.0
9.8
39.7
28.3
13.7
9.1
8.4
12.4
10.7
21.5
18.2
19.0
15.1
10.9
34.8
31.4
34.2
26.7
22.0
23.0
14.5
9.1
22.4
16.0
20.2
14.2
9.3
44.0
41.8
48.7
41.8
62.0
56.0
56.1
38.0
37.6
50.9
41.3
46.2
33.1
29.0
27.6
9.9
8.7
9.1
12.0
38.7
25.9
12.4
9.6
8.8
11.1
9.7
19.1
19.5
16.6
14.2
7.2
31.9
32.3
30.3
40.4
34.6
35.6
13.7
17.1
23.4
21.6
24.8
10.3
11.0
27.7
30.3
22.3
29.9
64.2
70.2
52.6
52.9
45.3
42.2
36.1
39.6
24.7
33.7
30.0
21.1
22.3
30.1
24.9
7.9
13.0
19.8
18.1
20.2
22.7
21.9
41.8
40.1
34.9
36.1
36.6
20.0
13.9
16.9
39.9
31.6
33.8
17.5
17.8
24.3
21.4
19.2
7.3
10.2
24.6
25.7
23.8
25.8
60.7
66.8
46.0
51.6
47.3
41.7
36.9
40.6
30.3
32.6
34.5
25.7
26.9
30.5
26.7
10.2
11.8
17.3
19.6
18.8
20.2
20.4
45.8
46.7
39.7
37.8
40.6
23.1
20.8
22.3
40.3
32.0
36.3
14.6
15.8
23.5
22.0
20.9
11.2
9.0
26.9
31.1
24.0
25.5
62.4
61.0
45.7
48.0
45.0
49.3
40.6
46.0
34.2
38.3
37.4
21.1
24.9
29.5
26.8
18.2
21.7
17.3
23.7
23.1
20.2
19.1
50.2
47.1
43.8
39.1
42.1
31.7
28.4
31.1
38.6
33.2
37.5
16.7
16.0
26.5
24.0
18.3
11.7
8.9
29.6
28.7
24.2
28.8
59.7
64.4
46.3
44.5
48.7
48.3
45.1
44.4
30.4
36.1
34.5
22.5
23.1
31.1
28.6
11.5
16.8
16.6
23.2
24.8
22.2
21.5
43.4
48.7
40.5
43.0
40.2
28.8
28.7
27.1
37.7
34.0
34.1
13.7
14.9
14.6
15.6
18.7
21.3
7.6
45.2
40.2
42.5
47.0
65.6
62.3
49.9
47.6
44.0
58.3
44.2
52.4
33.7
39.4
37.7
21.6
24.6
25.1
22.0
16.0
30.5
16.0
37.6
31.6
37.7
35.1
46.9
43.9
36.0
35.2
37.7
28.8
25.5
30.5
36.0
34.2
33.9
11.4
13.3
16.1
15.9
13.8
18.1
7.2
34.7
34.3
39.1
39.1
59.1
54.0
46.9
44.7
45.7
56.0
46.9
51.7
35.0
45.7
44.8
24.3
23.8
25.1
23.8
18.2
31.8
13.4
33.6
38.7
41.4
37.7
49.7
46.0
44.6
37.9
46.6
39.6
31.9
34.9
40.2
32.9
36.4
12.3
14.5
15.0
13.3
14.9
22.3
8.0
40.9
36.5
44.5
38.5
57.2
56.3
48.9
38.8
47.4
63.1
53.3
57.5
41.1
47.1
47.3
19.1
21.2
21.5
23.5
25.8
37.0
15.7
40.5
40.7
34.8
38.6
53.7
49.1
45.3
35.7
45.3
45.7
35.9
39.2
38.6
32.3
34.3
14.5
15.6
17.3
15.1
13.6
20.9
10.1
41.6
39.1
41.0
45.3
61.1
58.7
42.1
43.6
44.7
60.7
54.6
57.5
37.6
45.0
41.0
23.4
21.3
24.3
24.4
18.3
35.7
16.3
37.7
37.2
35.4
40.1
55.1
49.9
41.7
41.1
46.5
43.0
34.9
38.6
35.8
43.4
42.6
22.0
17.0
29.4
23.1
23.2
11.7
18.7
50.7
52.0
48.5
50.8
74.1
68.8
60.6
62.4
46.5
45.9
30.7
37.4
22.3
21.9
23.3
14.6
11.8
9.4
11.8
46.9
29.9
16.7
19.9
12.8
25.8
22.5
19.0
17.1
29.1
25.2
19.0
38.9
38.3
40.0
38.4
42.9
40.5
24.0
16.5
31.1
25.4
14.9
6.1
14.2
40.7
45.8
48.0
49.4
67.9
60.4
56.2
52.2
44.8
47.0
37.8
37.6
29.3
27.4
28.3
13.3
12.8
10.2
10.3
49.7
29.3
18.2
15.5
12.6
21.9
19.9
21.0
20.6
27.2
21.5
13.2
37.2
35.7
37.5
1 1 1 4 1 1 2 1 1 1
56 56 59 56 54 55 56 13 4 57 58 58 58 56 6 10 5 5 34 31 31 40 21 17 19 13 12 10 11 0 14 15 16 18 20 7 1 9 8 3 6 4 5 2 16 11 9 7 17 18 19 20 15 14 0 10 12 6 2 5 8 4 21 13 1 3 13 5 7 12 4 7 8 33 29 5 12 7 10 8 24 14 12 2 7 15 10 53 59 58 60 58 52 54 54 8 8 60 57 63 62 58 12 9 10 33 27 25 35 37 21 16 18 13 12 10 11 0 15 14 17 19 20 7 2 9 8 3 4 6 5 1 16 11 10 9 20 17 19 18 15 14 1 5 7 6 4 8 12 3 0 2 21 13 15 8 10 14 17 19 15 50 28 14 15 2 11 15 28 22 23 6 9 4 4 55 55 55 54 48 28 44 40 19 18 33 43 32 29 54 62 45 40 35 46 24 46 41 21 18 19 17 12 15 16 14 11 8 2 0 10 13 9 7 20 4 3 6 5 1 15 10 9 5 20 16 17 19 14 18 3 11 13 2 6 8 7 4 21 12 0 1 14 2 3 3 13 11 9 29 17 9 14 5 9 13 23 13 21 8 12 49 0 3 51 52 52 47 33 41 36 16 22 26 30 28 28 48 61 37 37 21 45 46 36 29 21 18 19 17 16 13 15 14 10 11 2 0 8 12 7 9 20 4 6 5 3 1 20 18 19 17 6 1 3 11 15 13 7 2 9 8 5 10 21 12 14 16 0 4 18 20 30 21 1 1 1 24 51 21 19 17 22 20 5 26 20 50 56 59 59 0 48 51 48 51 47 53 49 20 15 29 31 29 33 46 44 43 39 38 39 43 42 34 21 19 18 17 16 14 15 11 10 9 2 0 8 12 7 13 20 3 6 5 4 1 21 19 18 17 7 8 9 14 15 16 4 11 2 6 20 1 5 12 10 13 0 3 30 33 35 29 3 3 3 30 55 18 22 14 20 24 25 42 49 51 57 48 61 4 52 45 51 50 46 43 51 17 13 30 35 30 31 44 52 38 35 42 43 34 49 22 21 18 19 17 15 14 16 13 9 10 3 0 8 12 7 11 20 4 5 6 2 1 21 18 19 17 12 11 13 14 16 15 1 2 5 6 4 3 20 8 10 9 0 7 33 35 37 40 15 15 18 48 60 25 23 26 21 25 40 46 45 39 63 58 56 20 54 54 57 49 59 58 60 43 16 56 55 52 55 55 54 28 25 14 31 19 26 27 20 18 19 11 17 13 14 0 12 16 10 15 21 7 1 9 8 3 2 4 5 6 21 17 18 19 11 10 8 14 16 15 3 6 20 2 5 9 1 12 7 13 0 4 27 23 33 36 8 10 6 39 63 23 25 11 25 21 30 36 30 53 62 49 58 5 53 50 50 57 61 60 61 38 0 49 50 47 49 47 55 30 26 16 24 16 18 8 21 16 17 15 19 14 18 0 10 12 11 13 20 7 1 9 8 4 3 5 2 6 21 15 19 18 7 9 8 6 13 10 2 1 11 16 14 17 20 5 3 4 0 12 31 26 40 43 14 23 14 61 46 36 46 40 45 28 17 49 44 59 53 40 47 15 49 40 46 46 63 63 63 41 2 51 52 51 51 50 53 40 28 9 30 21 27 11 21 11 15 13 19 17 18 0 10 14 12 16 20 7 1 9 8 4 3 5 2 6 21 19 18 16 8 6 10 7 13 12 2 1 11 15 14 17 20 4 3 5 0 9 44 48 43 44 26 27 34 60 38 42 48 41 47 30 45 55 55 41 52 33 51 14 50 41 44 52 62 62 62 40 6 50 54 53 54 52 42 21 33 11 20 20 24 6 21 11 14 13 19 17 18 0 10 15 12 16 20 8 1 9 7 3 4 5 2 6 21 14 19 18 13 12 11 0 15 16 10 17 20 7 1 8 6 5 2 4 3 9 40 39 49 53 43 52 45 63 33 54 53 45 50 32 63 62 62 40 61 29 52 45 63 63 63 62 51 51 50 31 59 61 63 62 63 63 37 53 59 57 58 63 63 35 21 19 17 13 12 10 11 0 14 16 15 18 20 9 3 6 8 4 5 7 2 1 21 16 15 19 10 7 8 11 2 6 9 1 13 17 14 18 20 4 0 12 5 3 43 38 38 55 31 36 33 62 53 43 49 43 48 37 48 52 56 44 55 30 50 17 58 60 58 60 45 47 48 27 58 59 59 57 57 60 41 51 50 54 51 57 57 10 21 18 19 14 12 10 11 1 13 15 16 17 20 8 3 7 9 4 5 6 0 2 8 16 9 6 20 18 19 14 12 13 3 7 4 2 15 10 11 0 1 5 21 17 1 11 4 11 20 20 20 46 35 10 5 22 8 2 1 1 3 15 5 11 15 56 62 62 62 61 55 61 53 26 50 63 61 61 60 62 32 57 58 56 59 52 60 31 21 18 19 13 11 12 10 0 15 16 14 17 20 8 3 7 9 4 5 6 2 1 9 13 10 5 20 18 19 16 14 17 6 8 1 0 12 7 11 4 2 3 21 15 5 19 17 17 3 9 8 47 43 4 2 15 6 7 14 12 15 19 11 13 7 58 60 57 55 63 50 56 57 28 61 58 62 59 61 59 36 50 54 62 60 62 62 18 21 18 16 14 11 10 12 1 13 17 15 19 20 9 3 7 8 4 6 5 0 2 10 11 12 7 20 19 18 14 15 16 8 9 6 3 5 0 13 4 1 2 21 17 10 7 12 6 23 31 24 56 42 8 7 3 0 14 20 23 27 22 15 6 3 63 37 44 28 22 36 28 37 12 9 48 40 54 42 41 11 34 2 8 23 37 25 42 20 18 13 11 17 10 14 0 16 12 19 15 21 9 3 8 6 2 7 4 5 1 11 13 10 8 20 18 19 17 16 15 7 9 4 5 6 0 12 2 1 3 21 14 6 10 5 9 18 22 21 45 36 7 8 1 1 10 18 20 19 20 6 8 6 61 26 32 24 16 40 30 35 2 20 37 34 38 40 31 9 4 6 30 16 27 20 38 20 19 13 10 18 11 16 1 14 12 15 17 21 8 3 9 7 2 6 4 5 0 8 12 11 9 20 19 18 17 16 14 3 7 4 2 13 10 5 1 0 6 21 15 13 9 9 7 0 6 9 36 26 6 4 16 12 0 7 7 2 7 3 1 9 51 32 34 25 20 38 45 47 14 23 44 38 44 39 43 10 5 4 12 18 26 23 32 20 19 12 10 16 15 18 0 14 11 17 13 21 7 1 9 8 3 6 4 5 2 13 10 9 6 16 19 20 18 17 15 2 7 1 0 12 8 11 4 3 5 21 14 2 5 12 10 8 3 6 42 32 0 0 10 7 3 15 19 16 0 8 12 8 48 28 46 21 18 48 40 44 18 14 45 42 49 38 38 11 3 8 28 2 32 22 33 19 20 11 10 18 12 16 1 15 13 17 14 21 8 3 9 6 0 7 4 5 2 11 12 10 6 20 17 19 18 15 16 3 9 2 0 8 5 13 7 1 4 21 14 1 16 14 16 7 4 2 49 31 1 1 4 4 12 26 17 22 1 10 5 2 54 47 42 42 32 30 26 29 5 24 27 26 20 27 40 59 41 41 23 33 38 29 46 21 18 19 17 16 13 14 15 11 12 2 10 5 9 20 1 8 3 6 4 7 0 13 11 10 0 18 19 17 20 16 15 6 9 4 2 7 5 12 1 3 8 21 14 0 12 18 11 9 1 1 44 27 2 6 2 3 0 27 18 14 3 1 7 13 52 45 29 39 38 29 29 32 1 21 22 21 23 24 35 56 25 29 37 35 42 28 30 21 17 19 18 16 13 15 14 11 12 2 1 8 9 6 10 20 3 7 5 4 0 20 17 18 19 6 2 3 12 16 14 8 5 4 7 11 9 21 10 15 13 0 1 16 16 18 17 0 0 0 15 62 15 17 21 18 16 8 34 24 47 44 53 45 1 36 28 36 30 44 37 43 7 12 24 27 24 23 39 58 27 42 31 26 39 34 36 20 16 18 15 19 14 17 13 11 12 2 10 7 8 21 0 9 3 6 5 4 1 21 18 17 19 9 7 8 13 15 16 10 2 4 3 5 6 20 12 14 11 0 1 20 18 19 24 2 2 2 11 49 16 16 12 15 18 19 31 36 63 54 54 53 2 41 30 31 42 42 48 41 9 11 28 24 25 26 34 50 33 47 36 28 47 41 26 21 17 15 19 18 16 14 13 11 12 3 0 10 8 6 9 20 2 7 5 4 1 20 19 17 18 13 10 12 14 16 15 4 0 7 6 3 2 21 8 11 9 1 5 21 24 23 25 11 6 13 35 52 20 18 13 13 23 33 47 46 48 50 51 54 24 38 37 34 26 57 46 55 34 3 47 39 39 43 45 47 35 32 24 22 17 19 28 20 17 16 12 19 13 18 0 14 10 11 15 21 8 1 9 7 3 2 4 5 6 21 18 19 17 12 11 9 13 16 15 5 1 7 8 4 3 20 6 14 10 0 2 22 19 27 23 7 8 4 22 45 19 20 19 17 17 21 44 38 55 47 55 55 3 25 31 32 33 56 50 52 23 1 32 29 35 36 33 43 39 31 10 19 22 21 9 19 16 14 15 20 17 18 11 8 7 1 0 10 9 13 12 21 2 5 4 3 6 20 14 13 17 7 6 8 11 19 16 2 1 15 10 18 12 21 4 5 3 0 9 17 17 16 19 6 4 12 53 40 34 28 34 30 19 31 48 48 49 49 44 32 16 29 36 22 39 60 57 59 37 7 40 32 42 34 29 57 32 43 13 21 18 16 12 18 16 11 15 21 17 20 12 7 8 1 0 13 9 14 10 19 3 5 2 4 6 19 16 12 20 14 7 9 10 18 17 4 0 8 13 15 11 21 2 5 3 1 6 19 25 20 37 27 16 19 54 30 31 36 37 32 22 38 61 60 61 48 45 40 23 34 22 29 27 58 59 58 32 19 41 37 36 41 36 51 29 34 29 25 23 17 7 19 13 14 15 20 17 18 0 12 10 11 16 21 8 2 9 7 4 5 3 1 6 20 16 7 15 17 8 11 14 18 19 1 0 12 13 6 10 21 2 3 4 5 9 23 27 17 31 34 25 27 59 37 38 41 31 35 26 60 63 63 43 41 28 39 47 46 53 53 34 35 13 22 10 57 53 51 48 46 53 39 56 57 52 32 50 37 43 20 18 19 13 12 7 9 8 11 10 3 2 15 17 14 16 21 1 6 4 5 0 20 16 15 19 17 6 12 10 18 14 3 0 11 9 7 13 21 2 4 5 1 8 24 21 26 35 32 21 26 58 44 35 33 33 37 27 46 60 52 58 42 41 44 25 20 19 18 13 15 8 7 9 12 10 4 1 11 14 16 17 21 3 6 5 2 0 6 16 14 7 19 18 20 11 10 13 4 3 15 12 8 5 9 1 21 17 2 0 4 15 15 15 25 33 30 52 47 17 13 27 19 4 9 32 0 2 12 9 11 57 9 13 11 4 19 18 20 16 14 15 5 7 6 2 12 8 10 3 0 1 21 17 35 47 41 28 39 32 25 0 34 39 44 46 44 49 49 54 51 60 29 56 48 13 20 19 17 13 16 8 10 7 9 11 4 2 14 12 15 18 21 3 6 5 1 0 39 49 47 41 49 39 39 3 54 52 47 50 53 51 35 59 61 45 37 61 38 19 11 14 13 7 21 24 23 51 56 13 11 18 14 11 16 15 17 18 17 14 5 62 2 12 11 8 24 35 31 57 41 12 6 8 3 9 29 24 18 17 14 2 1 60 9 13 12 6 18 19 20 16 15 14 5 3 10 2 11 8 7 0 21 17 4 1
1 1 1 4 1 1 3 1 1 0
34.3
44.1
45.1
20.8
18.4
32.7
23.2
17.3
9.2
17.9
50.2
47.7
51.4
56.6
70.0
63.5
57.1
55.8
48.3
52.5
41.2
44.2
1 1 1 4 1 1 3 1 1 1
33.2
31.2
28.7
12.8
11.9
8.8
8.7
48.1
34.8
16.0
14.4
11.5
17.5
9.7
20.5
16.8
26.0
20.7
13.3
38.6
39.2
40.3
1 1 1 4 1 1 4 1 1 0
34.3
43.7
41.2
20.9
17.9
32.2
22.0
18.0
10.2
16.8
45.1
46.6
55.2
56.6
64.4
64.7
53.9
54.5
51.9
54.0
40.8
47.9
31.0
27.5
27.4
13.9
11.8
7.9
8.5
47.4
32.9
15.7
13.7
12.4
14.7
11.9
19.0
17.9
26.1
24.1
13.6
38.1
39.0
39.3
1 1 1 4 1 1 4 1 1 1
10 13 12 5 18 19 20 16 15 14 7 9 6 4 8 3 11 2 0 1 21 17 21 15 17 14 16 8 13 7 10 9 4 1 11 12 18 19 20 2 6 5 3 0 42 43 45 45 53 38 46 4 48 46 45 55 52 42 34 58 53 46 36 60 33 21
1 1 1 1 1 1 1 0 0 0 0 0 1 1 1 1 1 1 1 1 2 0 0 0 0 0 4 1 1 1 1 1 1 1 4 0 0 0 0 0 4 2 1 1 1 1 1 1 3 0 0 0 0 0 1 2 0 0 0 0 0 1 4 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
18.51 18.92 19.78 20.04 20.51 20.72 20.73 20.92 21.48 21.66
4 4 0 0 0 0 0 4 1 1 1 1 1 1 1 1 1 1 1 4 4 1 1 1 1 1 1 1 1 1
22.5 22.63 22.88
4 3 0 0 0 0 0 4 2 1 1 1 1 1 1 1 1 1 1 4 3 0 0 0 0 0 1 3 1 1 1 1 1 2 1 1 1 1 1 1 2 2 0 0 0 0 0 2 1 1 1 1 1 1 2 4 0 0 0 0 0 2 2 1 1 1 1 1 2 3 0 0 0 0 0 2 4 1 1 1 1 1 2 2 1 1 1 1 1 3 1 1 1 1 1 1 1 2 0 0 0 0 0 2 3 1 1 1 1 1 2 3 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 0 1 1 1 1 1 0
23.2 23.57 23.77 25.67 27.65 27.91 30.56 31.15 31.41 32.09 33.88 33.89 34.14 34.43 35.32 35.69
1 1 0 2 1 1 1 1 1 1 1 1 0
35.7
1 3 1 1 1 1 1 2 4 1 1 1 1 1 1 1 1 1 1 1 4 1 1 0 1 1 0 1
36.1 36.12 36.34
1 0 1 1 1 1 1 1 1 1 1 0
37.7
3 2 1 1 1 1 1 1 1 1
37.9
3 2 1 1 1 1 1 2 2 0 0 0 0 0 1 1 1 1 1 3 4 0 0 0 0 0 3 1 0 0 0 0 0 2 3 0 0 0 0 0 2 4 1 1 1 1 1 3 3 1 1 1 1 1 3 3 1 1 1 1 1 3 4 1 1 1 1 1 3 1 0 0 0 0 0 2 1 0 0 0 0 0 3 2 1 1 1 1 1 4 2 0 0 0 0 0 3 4 0 0 0 0 0 1 2 1 1 1 1 1 4 4 1 1 1 1 1 4 1 0 0 0 0 0 3 2 1 1 1 1 1 4 3 1 1 0 1 1 1 1 1 1 1 1 0 1 1 0 1 1 0 1 1 1 1 1 0 1 1 0 1 1 0 1 1 1 1 1 0 1 1 1 1 1 0 1 1 0 1 1 0 1 1 0 1 1 0 1
38.1 38.96 39.15 39.45 39.65 39.92 40.18 40.52 40.76 41.61 42.15 42.85 43.02 43.71 44.55 44.57 44.99 45.38 45.46
1 0 3 4 0 0 0 0 0 1 3 0 0 0 0 0 0 0 0 0 0 3 3 0 0 0 0 0 3 3 0 0 0 0 0 1 4 1 1 0 1 1 0 1 1 1 1 1 0 1 1 0
46.3 46.87 47.07 47.24 47.62
1 1 0 0 0 0 0 3 1 0 0 0 0 0 0 0 0 0 0 4 2 0 0 0 0 0 4 4 0 0 0 0 0 4 3 1 1 0 1 1 0 1 1 0 1 1 0 1
47.7 51.37 53.41 55.47 55.58
1 0 4 1 0 0 0 0 0
5 13 14 10 22 34 28 55 39 11 10 6 5 5 22 16 13 26 13 0 0 59 Best Model Overall Jacob Barhak
Analysis
• Comparing overall model ranking for all results: – Out of 32 model variations with Treatment Improvement 29 (91%) performed better – Out of 32 model variations with Biomarker Change 28 (87%) performed better • Comparing model and population combinations: – Out of 512 model and population combinations with Treatment Improvement 409 (80%) performed better – Out of 512 model and population combinations with Biomarker Change 385 (75%) performed better – Out of 256 model and population combinations , in 133(52%) the scenario with both Treatment Improvement and Biomarker Change performed best compared to the three other variations.
•
Treatment Improvement was more effective than Biomarker Change
Jacob Barhak
Summary Points to Remember
•
Models become outdated
–
Models should include a temporal correction term for treatment improvement
–
Models should be periodically updated
•
The Reference Model provides a birds eye view of clinical trials
–
Phenomena observed in different trials can be compared
–
Does not require access to proprietary data
–
Accumulates knowledge for competition
Jacob Barhak
Disclaimer
• •
This is not the best This is the best within current environment
•
Limitations include:
–
Current technology
– –
Human nature Information used/available
•
Improvement is welcome
Jacob Barhak
Acknowledgments
• • • • • • • The very capable cluster builder & sys admin: Chris Scheller CAC / Cyber Infrastructure Team at U of M Michigan Python Users Group Wikipedia All the people who developed Linux and cluster OS and made them freely available The GPL modeling framework was supported by the Biostatistics and Economic Modeling Core of the MDRTC (P60DK020572) and by the Methods and Measurement Core of the MCDTR (P30DK092926), both funded by the National Institute of Diabetes and Digestive and Kidney Diseases. The modeling framework was initially defined as GPL and was funded by Chronic Disease Modeling for Clinical Research Innovations grant (R21DK075077) from the same institute. The Reference Model was developed independently without financial support.
Jacob Barhak