Considerations Regarding Choice of the Primary Analysis in Longitudinal Trials With Dropouts: An FDA Perspective Robert T.
Download ReportTranscript Considerations Regarding Choice of the Primary Analysis in Longitudinal Trials With Dropouts: An FDA Perspective Robert T.
Considerations Regarding Choice of the Primary Analysis in Longitudinal Trials With Dropouts: An FDA Perspective Robert T. O’Neill , Ph.D. Director , Office of Biostatistics CDER, FDA Presented at the FDA /Industry Workshop; Bethesda, Maryland; September 17-19, 2003 Disclaimer The opinions expressed are mine and do not represent CDER policy on this issue. Ongoing research within CDER is directed toward more specific guidance which will supplement that contained in the ICH E9 Guidance “Statistical Principles for Clinical Trials Outline Issues about the dropout process in longitudinal clinical trials : focus on informative, treatment related missingness Terminology used to express missing data The literature: Problem formulation and approaches - Different research approaches to the issues Some general conclusions from this research Concluding remarks on choosing a primary analysis and its pre-specification when the analysis may depend upon the data pattern Issues Regulatory setting places emphasis on confirmatory studies, pre-specification of objectives, hypotheses, analyses, documentation How to specify in the protocol the primary strategy for dealing with missing data - if you presume that it will be informative - and you have not observed the data yet Considerable literature on the matter, unclear as to which approach to choose, when and why, and the considerable computational efforts involved Documentation and reporting of the criteria, choice of approach (model), and certainty of the conclusions Patient withdrawals from treatment exposure Why do subjects stay in clinical trials, why do they withdraw from assigned therapy, when do they withdraw, and how do they differ from completers ? Efficacy (lack of or benefit from) Safety (toxicity, tolerability, neither) Both Aggravation of the trial When in time do they leave therapy in trial (short duration, long duration) Symptomatic relief vs unperceived benefit What is the shape of response time pattern Efficacy and Toxicity If toxicity is time dependent, and efficacy is time dependent, withdrawal due to either side effects or perceived early efficacy, censors the efficacy outcome whose time dependency may follow different pattern Bivariate outcome (Efficacy (t), Safety(t)) Four situations (E,S) at time T E S(T) Patient perceives effective response AND no toxicity occurs E S(T) Patient perceives effective response AND experiences toxicity may or may not leave depending upon tolerance and preferences E S(T) Patient perceives No effective response AND no toxicity occurs E S(T) Patient perceives No effective response AND experiences toxicity may or may not leave but is MORE likely to leave than patient above who perceives effective response The problem formulation in the literature ? Which to choose as primary ? Likelihood based methods mixed model repeated measures (MMRM) Pattern -mixture models Selection models Ad hoc methods (LOCF, worst case imputation) Single and multiple imputation approaches The terminology Terminology for missing data MCAR - missing completely at random MAR - missing at random MNAR - Missing not at random NIM - Non - Ignorable Missingness OC observed cases AAD - all available data CC - complete cases CAR - coarsened at random LVCF (last value carrying forward) - LOCF Dependent dropouts - DD More Terminology for missing data Classification of drop-out mechanism independent of data - IDA dependent on observed data - ODA dependent on missing data - MIDA dependent on observed and missing data - OMI dependent on observed data and covariates - ODACO MAD Missing and Deferred (or Delayed): Another new term MAD - missing and deferred at random completely at random completely uninterpretable What is unique about missing data in clinical trials ? Monotonically Missing data is potentially an outcome by itself Why ? - It can be a surrogate for patient preference, acceptability with therapy, and can potentially be unpredictive of where the subject would be in the future (where no observations are taken or available) With monotone missing data, the ‘dropout mechanism’ is very likely informative Possible to plan to collect information during study prior to treatment withdrawal and prior to study completion but post treatment withdrawal Example of longitudinal Efficacy response/score by visit - missing for toxicity/safety reasons not considered Test Control Higher is bad 1 2 Visit 3 4 5 Are slope and baseline predictive of how long a patient stays in trial ? No treatment effect Test Control Higher is bad 1 2 3 Visit 4 5 Is baseline predictive of how long a subject stays in trial Treatment relationship is less clear Test Control Higher is bad 1 2 3 Visit 4 5 Maximum likelihood approaches Factoring the likelihood Information is in the conditioning The observed data and the missing data Which do you wish to condition on and why ? Shared parameter models not considered here, eg. same parameter in observed and missing data models Selection Model Factors the joint distribution of the observed data Y and the missing data M into the marginal distribution of the observed response times M the conditional distribution of missingness given Y = y Heitjan, Ignorability and Bias in clinical trials; Stat Med 18, 2421-2434 (1999) Selection Model When the missing data are NOT IGNORABLE, one has to , an explicit model for the missing-data mechanism M specify to make an appropriate inference for theta Selection Model -What to do if non-ignorable occurs Specify an explicit model for the missing data mechanism Wu and Carroll for slope analysis in longitudinal studies Do sensitivity and robustness analysis, under different plausible models for the missing data mechanism Pattern-mixture models Factors the joint distribution of Y and M as the marginal distribution of M (the missing pattern) times the conditional distribution of observed data Y given M = m (the response given missing pattern ) The data are stratified by missing data patterns and inference is on the conditional model parameter Pattern - mixture models The first factor says that the data are stratified by missing data patterns and the inference is given to the conditional model parameter Choice of missingness models Pattern-mixture model: patterns of response, distribution of effects within patterns - need for a lot of data in each pattern - information in number of subjects in each pattern Selection model : response and dropout distribution - very dependent upon assumptions of models Shared parameter models How much missingness (% of total N) can be tolerated ? Overall Between treatment groups Early, middle, later in the study Reasonable imputation strategies, including LOCF Step 1: Model Selection ? MMRM Selection modeling - the second factor corresponds to the self selection of individuals into “observed” and “missing” groups. Pattern- mixture models - a mixture of different populations, characterized by the observed pattern of missingness e.g. 4 times measured; subjects with 1,2,3,4 measurements form the four patterns estimate treatment effects within patterns and then combine in some way Step 2 : Sensitivity Analysis Which strategy, when , why Determining evidence for MNAR Model correctness (selection, pattern-mixture) Local Influence of patients on power and detection ability Documenting the strategies and what was done Supporting the range of possible conclusions consistent with the data Some selected literature comparing different strategies Three papers on comparing performance of competing analysis strategies 1. 2. 3. Analysis of Longitudinal Clinical Trial Incomplete Data ; O. Siddiqui and J. Hung Compares impact of fixed value imputation (FVI) like LOCF with m.l. general linear models of the observed data Uses a Pattern - mixture model to make inference on the unconditional, hypothetical complete-data mean Non zero values to have dropouts have outcomes that are worse Miller, Morgan, Espeland, Emerson Message Variability of the measurements needs to be addressed The direction of bias, by not accounting for it, is not predictable Under several non-ignorable non-response scenarios, m.l. based analyses can yield equivalent hypothesis tests to those obtained when analyzing only the observed data. Consideration jointly of: 2 arm trial, with change in spine deformity index (SDI) over a 4 year duration with measurements at each of 4 years Linear progression of disease Disease progression mechanisms (early, middle , late) Dropout mechanisms Considers each subject’s last observation is dependent on prior repeated measurement 14 separate methodological strategies for dealing at the analysis with missing data Large black dot is unacceptable power reduction Large black square is inflation of type 1 above 7.5% Message The adequacy of a strategy for dealing with missing values strongly depends on whether the courses of disease are similar or very different in the study groups. Therefore knowledge about the courses of longitudinal data is important besides information on drop- out rates for planning an adequate ITT analysis. If the information about the courses of disease is not available at the planning stage of a clinical trial, the ICH E9 guideline suggests correcting the strategy for dealing with missing values in a blind review stage before analysis of the trial starts Pre- definition of methods [ of dealing with missing values ] may be facilitated by updating this aspect of the analysis plan during the blind review . Thus, the blind review is a possibility to get an idea about the patterns of courses of the endpoint, thus making the choice of an adequate strategy easier. However, a blind review including the main endpoint might induce problems if obvious treatment effects show up at this stage, giving away treatment groups. For judging the adequacy of an approach for dealing with missing values, information about rates and times of drop- outs as well as courses of disease must be provided in the publication of the results. Messages For drop-out rates less than 20% AND similar courses of disease in the treatment groups, missing values might be replaced by mean of other groups For larger drop-out rates OR less similar course of disease, no adequate recommendations can be given Type 1 error increases drastically for the different strategies, especially if the course of disease vary between treatment groups There is is NO strategy which is adequate for all different combinations of dropout mechanisms, drop-out rates or less similar courses of disease and no adequate recommendations can be given. Presuming informative missingness - what to do ? Computational burden is the issue Adjusting for Non-ignorable Drop-out Using Semiparametric Nonresponse Models Sharfstein, Rotnitzky and Robins, JASA,V 94; 1096-1120 (1999) See Commentaries pages 1121-1146 Comparison of estimation methods Wei, L and Shih, WJ Partial imputation approach to analysis of repeated measurements with dependent dropouts Statist. Med. 2001; 20: 1197-1214 3 Conditions 1. The drop-out rates are the same in both treatment groups 2. Dropout process depends on the outcome variable in the same manner in both treatment groups 3. Common variances for the outcome variable in both treatment groups The Wei and Shih approach is to control C1, so that the dropout rates become the same (or nearly similar) after partially imputing those needed to made the rates the same. One can condition only on what was observed and measured - Its effectiveness depends on what you know in advance to condition on Dropout rates in each treatment group Same or different Time pattern same or different How many identifiable cause specific reasons for dropouts , and are they the same or different in each treatment group Example: ES, E S, E S, E S Two papers on joint analysis of dropout as a response and observed repeated data Worst-rank score analysis with informatively missing observations Follow-up measurement is missing for some subjects because a disease-related event occurred prior to the time of the follow-up observation Example: Study of congestive heart failure, patients undergo exercise testing, but the measurement is missing for those who die of heart disease during the study Measurements are informatively missing because mortality from heart disease and a decline in exercise BOTH indicate progression of the underlying disease Considers two separate populations, completers and dropouts and tests a joint hypothesis (binomial for dropouts) regarding outcomes both in the same direction for the test treatment group Siddiqui and Hung What is the null hypothesis ? No difference between treatments at all time points No difference between treatments at the last time point Concluding Remarks In protocol planning, assume that monotonic missing data, if it occurs, is likely informative Non- ignorable non- response occurs when the probability of response depends on the unobserved outcome. In this situation, assumptions regarding the missing data process, which often are not directly verifiable, typically are necessary to provide valid estimates and inference Decide what data will be collected that will allow for conditioning on factors that matter to address bias adjustments Concluding Remarks Choose the primary strategy to be used, including approaches, criteria for model selection, model fits, sensitivity analysis, robustness, etc. Consider the justification for model selection and the sequence of analytical steps to assure that assumptions of the pre-specified analysis are met conditional on what is observed Consider, in advance, the joint distribution of efficacy outcome, the side effect outcomes and other response variables needed to satisfy the MAR requirement - and measure what you can Concluding Remarks Reporting and documentation advice Plots, graphs, model fits, comparisons How to convince others that your model choice and primary strategy was the most appropriate for the conclusions drawn extent of sensitivity analyses If possible, collect data on all subjects until the trial is completed, even if withdrawn from trial