Methods All patients were suffering from nonerosive reflux disease or functional ulcer-like dyspepsia and were prescribed a proton pump inhibitor to be taken as needed for relief of epigastric pain. The primary endpoint was a score variable that expresses the magnitude of gastro-intestinal symptoms at 8 weeks after randomization. We developed linear and loglinear versions of the structural mean models to derive an unbiased estimator of the reduction in symptom score as a function of exposure to the test drug. Semi-parametric models based on splines and corresponding simultaneous confidence bands identified the presence of potential interactions between drug exposure and baseline covariates.
Results The on-demand dosing regimen generated a wide range of drug exposure. Application of SMM showed that the potential treatment-induced reduction in symptom score was much greater than the average treatment reduction observed in this population of patients. Our diagnostic tool was useful for detecting the interaction between drug exposure and baseline covariates. Limitations Analysis could only be performed over the first 2 months after randomization because, afterwards, many patients dropped out from the placebo group. Conclusions The structural mean model approach allows one to estimate treatment efficacy in the presence of variable drug exposure. Similar results were obtained using linear and loglinear structural mean model. Clinical Trials 2009; 6: 403—415. http://ctj.sagepub.com
]]>Purpose As a case study, to examine the overlap of study cohorts in two large randomized controlled clinical trials that assess interventions to reduce risk of major cardiovascular disease events in adults with type 2 diabetes in order to explore the feasibility of cross-trial comparisons
Methods The Action for Health in Diabetes (Look AHEAD) and The Action to Control Cardiovascular Risk in Diabetes (ACCORD) trials enrolled 5145 and 10,251 adults with type 2 diabetes, respectively. Look AHEAD assesses the efficacy of an intensive lifestyle intervention designed to produce weight loss; ACCORD tests pharmacological therapies for control of glycemia, hyperlipidemia, and hypertension. Incidence of major cardiovascular disease events is the primary outcome for both trials. A sample was constructed to include participants from each trial who appeared to meet eligibility criteria and be appropriate candidates for the other trial’s interventions. Demographic characteristics, health status, and outcomes of members and nonmembers of this constructed sample were compared.
Results Nearly 80% of Look AHEAD participants were projected to be ineligible for ACCORD; ineligibility was primarily due to better glycemic control or no early history of cardiovascular disease. Approximately 30% of ACCORD participants were projected to be ineligible for Look AHEAD, often for reasons linked to poorer health. The characteristics of participants projected to be jointly eligible for both trials continued to reflect differences between trials according to factors likely linked to retention, adherence, and study outcomes.
Limitations Accurate ascertainment of cross-trial eligibility was hampered by differences between protocols.
Conclusions Despite several similarities, the Look AHEAD and ACCORD cohorts represent distinct populations. Even within the subsets of participants who appear to be eligible and appropriate candidates for trials of both modes of intervention, differences remained. Direct comparisons of results from separate trials of lifestyle and pharmacologic interventions are compromised by marked differences in enrolled cohorts. Clinical Trials 2009; 6: 416—429. http://ctj.sagepub.com
]]>Purpose SPERT’s goal was to propose a pharmaceutical industry standard for safety planning, data collection, evaluation, and reporting, beginning with planning first-in-human studies and continuing through the planning of the post-product-approval period.
Methods SPERT’s recommendations are based on our review of relevant literature and on consensus reached in our discussions.
Results An important recommendation is that sponsors create a Program Safety Analysis Plan early in development. We also give recommendations for the planning of repeated, cumulative meta-analyses of the safety data obtained from the studies conducted within the development program. These include clear definitions of adverse events of special interest and standardization of many aspects of data collection and study design. We describe a 3-tier system for signal detection and analysis of adverse events and highlight proposals for reducing "false positive" safety findings. We recommend that sponsors review the aggregated safety data on a regular and ongoing basis throughout the development program, rather than waiting until the time of submission.
Limitations We recognize that there may be other valid approaches.
Conclusions The proactive approach we advocate has the potential to benefit patients and health care providers by providing more comprehensive safety information at the time of new product marketing and beyond. Clinical Trials 2009; 6: 430—440. http://ctj.sagepub.com
]]>Purpose The purpose of this project was to automate the adverse event reporting process, replacing paper-based processes with computer-based processes, so that personnel effort and time required for serious adverse event reporting was reduced, and the monitoring of reporting performance and adverse event characteristics was facilitated.
Methods Use case analysis was employed to understand the reporting workflow and generate software requirements. The automation of the workflow was then implemented, employing computer databases, web-based forms, electronic signatures, and email communication.
Results In the initial year (2007) of full deployment, 588 SAE reports were processed by the automated system, eSAEyTM. The median time from initiation to Principal Investigator electronic signature was <2 days (mean 7 ± 0.7 days). This was a significant reduction from the prior paper-based system, which had a median time for signature of 24 days (mean of 45 ± 5.7 days). With eSAEyTM, reports on adverse event characteristics (type, grade, etc.) were easily obtained and had consistent values based on standard terminologies.
Limitation The automated system described was designed specifically for the workflow at Thomas Jefferson University. While the methodology for system design, and the system requirements derived from common clinical trials adverse reporting procedures are applicable in general, specific workflow details may not be relevant at other institutions.
Conclusion The system facilitated analysis of individual investigator reporting performance, as well as the aggregation and analysis of the nature of reported adverse events. Clinical Trials 2009; 6: 446—454. http://ctj.sagepub.com
]]>Purpose (1) Evaluate the efficacy of PD stenting to ameliorate abdominal pain in patients with CP and ductal strictures; (2) evaluate the placebo response rate from sham endoscopic therapy; (3) compare pain medication usage, healthcare utilization, psychological distress, and quality of life before and after endoscopic stenting; (4) prospectively evaluate the durability of the response.
Methods Patients with typical abdominal pain, imaging confirmation of CP and endoscopic retrograde cholangiopancreatography (ERCP) confirmation of PD stricture will complete questionnaires to assess quality of life, psychological distress, pain intensity/unpleasantness, pain medication usage, and healthcare utilization. Enrolled patients will be randomized to ERCP with sphincterotomy and PD stenting versus sham procedure. Pain level and medication usage will be assessed weekly with telephone interviews. At 6—8 weeks, patients treated with stents will undergo stent removal; those randomized to the sham procedure without significant improvement (<50% reduction in pain score) will cross over to the treatment group; and those randomized to sham procedure who experienced improvement (>50% reduction) will be followed clinically. Patients will be followed in clinic or by phone biannually (up to 3 years). The primary endpoint is improvement in abdominal pain. The secondary endpoints are reduction in narcotic use, healthcare utilization, and work days missed; return to employment; improvement in quality of life and weight gain.
Results Proposed study.
Limitations Strict inclusion criteria may limit enrollment.
Conclusion The proposed study represents the first trial of endoscopic stenting for symptomatic CP and ductal strictures with a credible sham procedure, assessment of multiple dimensions of pain, and psychosocial factors. Clinical Trials 2009; 6: 455—463. http://ctj.sagepub.com
]]>Purpose To report on the implementation of an international trial — CONTROL — intended to assess the efficacy and safety of rFVIIa in trauma, and discuss trauma research study design in light of this experience.
Methods The CONTROL trial international steering committee confronted a number of barriers in the design of the CONTROL trial. They addressed methodologies for (1) standardizing entry criteria for trauma patients suffering inherently heterogeneous injuries, (2) obtaining informed consent in an acutely injured population with altered levels of consciousness, (3) avoiding futile care, while recruiting subjects with incompletely diagnosed injuries, (4) standardizing trauma intensive care across different investigating sites and countries, and (5) establishing study endpoints that were both clinically relevant and convincing to regulatory authorities. The resulting study methodology is reported.
Results The CONTROL trial began active recruitment in October 2005, and was halted on June 11, 2008 because the observed mortality in the 576 enrolled patients was so far below expectations that the study would lack sufficient statistical power at the planned number of subjects to demonstrate a benefit. The utility of the endpoints selected for study will not be known until completion of data analysis. Limitations Any clinical trial in trauma patients must cope with the urgency of care required, issues of patient heterogeneity, standardization of care across multiple centers, and the difficulties of obtaining informed consent.
Conclusion Research in acutely hemorrhaging trauma patients presents numerous scientific and ethical challenges. The methodology of the CONTROL study is presented as an example of how some of these challenges can be approached and managed, and of the pitfalls that may arise. Clinical Trials 2009; 6: 467—479. http://ctj.sagepub.com
]]>Purpose Our goal is to propose a design for utilizing a sequential, multiple assignment, randomized trial design for patients with malignant melanoma to test the relative efficacy of drugs that target serotonin versus dopamine metabolism during 4 weeks of intravenous, then 8 weeks of subcutaneous, interferon-alpha therapy.
Methods Patients will be offered participation in a double-blinded, randomized, controlled, 14-week trial involving two treatment phases. During the first month of intravenous interferon-alpha therapy, we will test the hypotheses that escitalopram will be more effective in reducing depressed mood, anxiety, and irritability, whereas methylphenidate will be more effective in diminishing interferon-alpha-induced neurovegetative symptoms, such as fatigue and psychomotor slowing. During the next 8 weeks of subcutaneous interferon therapy, participants whose symptoms do not improve significantly will be randomized to the alternate agent alone versus escitalopram and methylphenidate together.
Results We present a prototype for a single-center, sequential, multiple assignment, randomized trial, which seeks to determine the efficacy of sequenced and targeted treatment for the two distinct symptom complexes suffered by patients treated with interferon-alpha.
Limitations Because we cannot completely control for external factors, a relevant question is whether or not ‘short-term’ neuropsychiatric interventions can increase the number of interferon-alpha doses tolerated and improve long-term survival. Conclusions This sequential, multiple assignment, randomized trial proposes a framework for developing optimal treatment strategies; however, additional studies are needed to determine the best strategy for treating or preventing neurobehavioral symptoms induced by the immunotherapy interferon-alpha. Clinical Trials 2009; 6: 480—490. http://ctj.sagepub.com
]]>Purpose The goal of this article is to propose a Bayesian approach to the design and analysis of pediatric trials to allow borrowing strength from previous or simultaneous adult trials.
Methods We apply a hierarchical model for which the efficacy parameter from the adult trial and that of the pediatric trail are considered to be draws from a normal distribution. The choice of (the variance of) this distribution is guided by discussion with medical experts. We show that with this information, one can calculate the sample size required for the pediatric trial. We discuss how inference of these studies in pediatric populations depends on the parameter that captures the similarity of the treatment efficacy in adults compared to children.
Results The Bayesian approach can substantially increase the power of a pediatric clinical trial (or equivalently decrease the number of subjects required) by formally leveraging the data from the adult trial.
Limitations Our method relies on obtaining a value for the inter-study variability, , which may be difficult to describe to a clinical investigator.
Conclusions The Bayesian approach has the potential of making pediatric clinical trials feasible because it has the effect of borrowing strength from adult trials, thus requiring a smaller pediatric trial to show efficacy of a drug in children. Clinical Trials 2009; 6: 297—304. http://ctj.sagepub.com
]]>Methods Published data from six contemporary cIMT studies (ASAP, ARBITER 2, RADIANCE 1, RADIANCE 2, ENHANCE, and METEOR) including data from a total of 3563 patients were examined. Bayesian and frequentist methods were used to assess the impact of between study variability on the likelihood of detecting true treatment effects on 1-year cIMT progression/regression and to provide a sample size estimate that would specifically compensate for the effect of between-study variability.
Results In addition to the well-described within-study variability, there is considerable between-study variability associated with the measurement of annualized change in cIMT. Accounting for the additional between-study variability decreases the power for existing study designs. In order to account for the added between-study variability, it is likely that future cIMT studies would require a large increase in sample size in order to provide substantial probability (≥90%) to have 90% power of detecting a true treatment effect.
Limitation Analyses are based on study level data. Future meta-analyses incorporating patient-level data would be useful for confirmation.
Conclusion Due to substantial within- and between-study variability in the measure of 1-year change of cIMT, as well as uncertainty about progression rates in contemporary populations, future study designs evaluating the effect of new lipid-modifying therapies on atherosclerotic disease progression are likely to be challenged by large sample sizes in order to demonstrate a true treatment effect.
]]>PurposeWe describe a simple area-based nonparametric estimator of RT and illustrate its use in the Acyclovir Prevention Trial.
Methods Let Q x(p) be the quantile function for the xth treatment group, defined as the time by which p% of the treatment group experience the event, and p x be the maximum event proportion observed. Our consistent estimator is defined as the ratio of the integrals of Q1(p) and Q0 (p) with integration over 0 to p, where p =min(p1, p 0). Confidence limits (CL) are provided by bootstrap.
Results A total of 703 immunocompetent adult men and women (54% male, 79% Caucasian, median age 49 years) with a history of ocular herpes simplex virus (HSV) were enrolled in 1992—1996, randomized to acyclovir or placebo, followed for up to 1 year for the 1st episode of ocular HSV, and 170 events were confirmed by a study-certified ophthalmologist using slit-lamp biomicroscopy. The nonparametric RT comparing acyclovir use with nonuse was 2.6 (bootstrap 95% CL: 1.6, 4.2). For comparison, the best-fitting parametric model was the lognormal (RT = 2.5; 95% CL: 1.5, 3.9). In limited simulations, the average proposed estimate of RT was similar to the true RT with a relative root mean squared error of 1.13 compared to a correctly specified parametric (lognormal) model.
Limitations An analytical variance estimator for the proposed RT is lacking. Also, more examples and more extensive simulations are warranted.
Conclusions Similar to Cox’s relative hazard estimator, the proposed RT does not assume the data are generated from a particular distribution. RTs should be more widely used as a measure of association in clinical trials. Clinical Trials 2009; 6: 320—328. http://ctj.sagepub.com
]]>Purpose To report the response from the Antihypertensive and Lipid-Lowering Treatment to Prevent Heart Attack Trial to the National Heart, Lung, and Blood Institute’s requirement for dissemination and evaluation of trials with potential immediate public health applicability.
Methods ALLHAT’s dissemination plan had two-components: (1) a traditional approach of media coverage, scientific presentation, and publication; and (2) a theory-based approach targeting determinants of clinician behavior. Strategies included: (1) academic detailing, in which physicians approach colleagues regarding blood pressure management, (2) direct patient messages to stimulate communication with physicians regarding blood pressure control, (3) approaches to formulary systems to use educational and economic incentives for evidence-based prescription, and (4) direct professional organization appeals to clinicians.
Results One hundred and forty-seven Investigator Educators reported 1698 presentations to 18,524 clinicians in 41 states and the District of Columbia. The pre- and post-test responses of 1709 clinicians in the face-to-face meetings indicated significant changes in expectations for positive patient outcomes and intention to prescribe diuretics. Information was mailed to 55 individuals representing 20 professional organizations and to eight formulary systems. Direct-to-patient messages were provided to 14 sites that host patient newsletters and Web sites such as health plans and insurance companies, 62 print mass media outlets, and 12 broadcast media sites.
Limitations It was not within the scope of the project to conduct a randomized trial of the impact of the dissemination. However, impact evaluation using quasi-experimental designs is ongoing.
Conclusion A large multi-method dissemination of clinical trial results is feasible. Planning for dissemination efforts, including evaluation research, should be considered as a part of the funding and design of the clinical trial and should begin early in trial planning. Clinical Trials 2009; 6: 329—343. http://ctj.sagepub.com
]]>Purpose The primary goal of CPP was to develop a community-based infrastructure for studying bipolar disorder that would enhance the ethnic/racial and socioeconomic diversity of participants.
Methods Selected academic sites partnered with local clinics (n = 6 partnerships in five cities). This report describes the conceptualization, implementation, and qualitative evaluation of CPP, as well as quantitative analysis of clinical and sociodemographic differences between the samples recruited at academic versus community sites.
Results Quantitative analysis of the 155 participants from the six partnerships revealed enrollment of 45% from minority populations (vs. 15% in academic sites). Significant sociodemographic differences were evident not only between academic and community sites, but within minority and non-minority groups across site types. Notably, clinical differences were not evident between participants from academic and community sites. Review of qualitative data suggests that certain factors around implementation of research protocols may enhance community participation.
Conclusions Moving research recruitment and participation into community sites was more successful in increasing minority enrollment than efforts to attract such individuals to academic sites. Recommendations for creating and maintaining academic/community partnerships are given.
Limitations Several important variables were not considered including mood severity, hospitalization, or treatment differences. Minority participants were grouped by combining African American and Hispanics, which may have obscured subgroup differences. A derivation of standard qualitative methods was used in this study. Clinical Trials 2009; 6: 344—354. http://ctj.sagepub.com
]]>Purpose The aim of this manuscript is to describe the challenges in planning and implementing this randomized clinical trial.
Methods The mixed incontinence: medical or surgical approach (MIMOSA) trial was designed as a practical or pragmatic clinical trial to establish the relative efficacy of two specific treatment approaches. The design presented many challenging decisions including: (1) selection of practical paradigm; (2) refining inclusion/ exclusion criteria to offer equipoise; (3) selection of feasibility sample size; (4) recruitment challenges for two divergent treatment approaches (medical vs. surgical), and (5) resolution of ethical and methodological issues.
Results MIMOSA recruitment was planned in two phases, starting with a 5-month pilot and feasibility phase followed by a full trial contingent on the outcome of the first phase. The feasibility portion of the MIMOSA trial started in November 2008. 1198 subjects were screened and approached forstudy enrollment, but only 27 consented to randomization. The feasibility study was halted due to lack of enrollment in March 2009.
Limitations The challenges of this trial included a lack of information from subjects who did not enroll, increasing the difficulty of interpreting the feasibility phase.
Conclusions Successful recruitment to a randomized trial that compares significantly different treatment approaches poses a challenge. Clinical Trials 2009; 6: 355—364. http://ctj.sagepub.com
]]>Purpose To describe and analyze recruitment strategies, ethical considerations, and recruitment outcomes from a study to evaluate the efficacy the Human Papilloma Virus vaccine in young men who have sex with men (MSM).
Methods The recruitment settings were university and community sites in the state of Morelos, Mexico. Eligibility requirement were men between 18 and 23 years old, who were free of anal-genital lesions as confirmed by clinical exploration, HIV negative, with no history of sexual relations with female partners and with fewer than five male lifetime sexual partners. Recruitment goals were 25 study participants in a four and a half month period. In addition to traditional recruitment strategies (flyers and media advertising, specific training of the recruitment team and adequate choice of recruitment sites)—engagement of local leaders in the MSM community formed a crucial part of the strategy. Special consideration was given to confidentiality and respect for study participants and a Bill of Participant Rights was developed as an explicit commitment to respect and acceptance.
Results In total 723 MSM were initially contacted, 243 filled out the recruitment questionnaire, of which 151 met the criteria to be invited to the clinical examination. After clinical examination and interviews with the recruitment team, 131 fulfilled the inclusion criteria, of whom 73 were enrolled in the study — nearly triple the recruitment goal. Among the initial recruitment strategies (application of the screening questionnaire) attending meetings with MSM activist organizations was the most successful (326), followed by recruitment at bars and dance clubs (107).
Limitations The recruitment strategies should be formally evaluated for their effectiveness to identify those which are most successful. In addition, future studies should consider the evaluation of study participants’ perceptions of the recruitment strategies. Conclusions Recruiting MSM in a developing country such as Mexico presented multiple challenges. We recommend that future studies actively engage the local MSM community and pay special attention to designing recruitment strategies that guarantee the confidentiality of and respect for participants.
]]>Purpose We present the use of process maps as a tool to visualize and to monitor the correctness of trial work flows. We show that process maps can be used to assure that trial processes are conducted according to the SOP.
Methods We describe how a process map is made. We then derive process maps from two sources: the SOP and trial procedures as currently implemented. We compare these maps to each other, using the SOP maps as the gold standard, to check that work is done according to the written procedures.
Results Eight process maps were produced from each source. 172 differences were found between the SOP maps and the walkthrough maps. Differences included the addition of extra steps, order errors, step mistakes, and ambiguities.
Limitations These process maps focused only on clinic procedures, so interactions with other trial components were not considered. The maps were made after the trial started, which may have biased their content and use.
Conclusion Process maps are a simple tool to check if clinical trial processes are operating as designed and offer an effective means to identify and correct such divergences. Further research should focus on using process maps in the design phase of trials, analyzing the cost to benefit ratio for process maps, and linking the analysis of the process map to monitor queries to quantify the improvement gained from using this technique. Clinical Trials 2009; 6: 373—377. http://ctj.sagepub.com
]]>Purpose The objective of this study was to test the possibility of centralizing various databases and to demonstrate ways of offering an alternative to a large-scale comprehensive and costly commercial product, especially for simple phase I and II trials, with reasonable convenience and security.
Methods We report a working procedure to transform and develop a standalone AccessTM database into a secure Web-based secure information system.
Results For data collection and reporting purposes, we centralized several individual databases; developed, and tested a web-based secure server using self-issued digital certificates.
Limitations The system lacks audit trails. The cost of development and maintenance may hinder its wide application.
Conclusions The clinical trial databases scattered in various departments of an institution could be centralized into a web-enabled secure information system. The limitations such as the lack of a calendar and audit trail can be partially addressed with additional programming. The centralized Web system may provide an alternative to a comprehensive clinical trial management system.
]]>Purpose The National Institute of Allergy and Infectious Diseases (NIAID) developed templates for investigators to serve as the foundation for protocol development. These protocol templates are designed as tools to support investigators in developing clinical protocols.
Methods NIAID established a series of working groups to determine how to improve its capacity to conduct clinical research more efficiently and effectively. The Protocol Template Working Group was convened to determine what protocol templates currently existed within NIAID and whether standard NIAID protocol templates should be produced. After review and assessment of existing protocol documents and requirements, the group reached consensus about required and optional content, determined the format and identified methods for distribution as well as education of investigators in the use of these templates.
Results The templates were approved by the NIAID Executive Committee in 2006 and posted as part of the NIAID Clinical Research Toolkit [1] website for broad access. These documents require scheduled revisions to stay current with regulatory and policy changes.
Limitations The structure of any clinical protocol template, whether comprehensive or specific to a particular study phase, setting or design, affects how it is used by investigators. Each structure presents its own set of advantages and disadvantages. While useful, protocol templates are not stand-alone tools for creating an optimal protocol document, but must be complemented by institutional resources and support. Education and guidance of investigators in the appropriate use of templates is necessary to ensure a complete yet concise protocol document. Due to changing regulatory requirements, clinical protocol templates cannot become static, but require frequent revisions.
]]>Purpose/Methods To address the extent and nature of Bayesian trials conducted at M. D. Anderson, we reviewed the protocols registered in the Protocol Document Online System between 2000 and early 2005. We summarize our findings and give details for three innovative trials that typify those in which a Bayesian approach has played a major role at the center.
Results Of 964 protocols reviewed, 59% were conducted solely at M. D. Anderson and the rest were multicenter trials. Bayesian designs and analyses were used in about 20% (195/964) of the protocols that we reviewed. Of the 520 protocols identified as phase I or II drug trials, about 34% were Bayesian. Most of the 195 Bayesian trials were designed by M. D. Anderson statisticians. The Bayesian design features most commonly used were the continuous reassessment method in phase I (toxicity) trials, adaptive randomization in phase II trials, and designs to monitor efficacy and toxicity simultaneously. We also provide an insider's view regarding some practical considerations that have made the design and implementation of so many Bayesian trials possible.
Limitations We reviewed only a subset of all M. D. Anderson protocols, but did not exclude any available in electronic form.
Conclusions The large number of Bayesian trials conducted at M. D. Anderson testifies to the receptivity to the Bayesian approach within the center, including principal investigators, regulatory review committees, and patients. Statisticians who take a Bayesian perspective can successfully work to establish a culture of innovation in clinical trial design.
]]>Methods We derive a new class of Bayesian designs based on formal hypothesis tests. The prior densities used to define the alternative hypotheses in these tests assign no mass to parameter values that are consistent with the null hypotheses and are called nonlocal alternative prior densities.
Results We show that Bayesian designs based on hypothesis tests and nonlocal alternative prior densities are more efficient than common Bayesian designs based on posterior credible intervals and common frequentist designs. In contrast to trial designs based on Bayesian credible intervals, we demonstrate that the mis-specification of the prior densities used to describe the anticipated effect of the experimental treatment in designs based on hypothesis tests cannot increase the expected weight of evidence in favor of the trial agent.
Limitations Extension of test-based designs to phase I—II designs and randomized phase II designs remains an open research question.
Conclusions Phase II single-arm trials designed using Bayesian hypothesis tests with nonlocal alternatives provide better operating characteristics, use fewer patients per correct decision, and provide more directly interpretable results than other commonly used Bayesian and frequentist designs. Because the mis-specification of the prior density on the effect of the experimental agent decreases the expected weight of evidence that is collected in favor of the experimental treatment, the use of Bayesian hypothesis tests to design clinical trials also eliminates a potential source of bias often associated with trials designed using posterior credible intervals.
]]>Methods By using indifference intervals, the initial guesses used in the CRM can be selected by specifying a range of acceptable toxicity probabilities in addition to the target probability of toxicity. An algorithm is proposed for obtaining the indifference interval that maximizes the average percentage of correct selection across a set of scenarios of true probabilities of toxicity and providing a systematic approach for selecting initial guesses in a much less time-consuming manner than the trial-and-error method. The methods are compared in the context of two real CRM trials. Results For both trials, the initial guesses selected by the proposed algorithm had similar operating characteristics as measured by percentage of correct selection, average absolute difference between the true probability of the dose selected and the target probability of toxicity, percentage treated at each dose and overall percentage of toxicity compared to the initial guesses used during the conduct of the trials which were obtained by trial and error through a time-consuming calibration process. The average percentage of correct selection for the scenarios considered were 61.5 and 62.0% in the lymphoma trial, and 62.9 and 64.0% in the stroke trial for the trial-and-error method versus the proposed approach. Limitations We only present detailed results for the empiric dose toxicity curve, although the proposed methods are applicable for other dose—toxicity models such as the logistic.
Conclusions The proposed method provides a fast and systematic approach for selecting initial guesses of probabilities of toxicity used in the CRM that are competitive to those obtained by trial and error through a time-consuming process, thus, simplifying the model calibration process for the CRM.
]]>Methods Data were retrieved from all cardiovascular outcomes trials conducted at the Population Health Research Institute (PHRI) between 1993 and 2006. These data included 10 trials with over 9000 events from 95 038 individuals. Differences in the log odds ratios between adjudicated and reported outcomes were analyzed and summarized using a ratio of odds ratios. Both masked and unmasked trials were included in this analysis.
Results There were no effects of event adjudication on the estimates of treatment effect for the primary outcomes, myocardial infarction (MI), stroke, or cardiovascular/vascular death. For the trial primary outcomes, the effect of adjudication vs. reported events was OR ratio = 1.00 [95% confidence interval (CI): 0.97—1.02]. There were also no significant differences in the number of outcomes included in the trials. Results were the same for masked and unmasked trials.
Limitations The number of unmasked trials were small, and this analysis was restricted to cardiovascular endpoints reported from trials managed by a single coordinating center, with similar sets of procedures. Individual patient data were not used for the analysis.
Conclusions This systematic meta-analysis failed to detect any effect of event adjudication on study conclusions and the numbers of events included in the final analyses were minimally changed. Given the considerable effort required to perform adjudication, there is a need to demonstrate that this process does indeed increase the sensitivity of trials. There is a need to conduct more systematic analyses of the effect of event adjudication in other trials to determine if this process is truly worthwhile.
]]>Purpose The specific aims of this secondary data analysis were to: (1) determine if there are differences in study retention rates by race/ethnicity and age, and (2) explore other client characteristics, as well as protocol and treatment program factors, that could account for differential retention rates.
Methods We conducted a secondary analysis using data from 1737 participants in the first six clinical trials whose databases were locked in the NIDA Clinical Trials Network. Protocol level characteristics were also abstracted from these studies, and we used data from a study which assessed characteristics of community treatment programs that participated in these studies. Logistic regression was used to study the effect on retention of: client, protocol, and program characteristics.
Results In the model of client characteristics, a significant age by race/ethnicity interaction term was detected based on a threshold of 0.1, with younger African Americans having the lowest odds of retention. Primary drug of abuse was also a significant factor in determining study retention, with heroin, methadone, and opiate users having the greatest odds of retention and polydrug users the lowest. Similar analyses testing treatment program characteristics found that only the presence of HIV risk screening and decreasing levels of female admissions (as a percent of total admissions) were related to study retention. In our final model, there was an effect of age, but not race/ethnicity, with younger participants having lower odds of retention. A multivariable model including protocol variables could not be developed due to the high correlation among protocol variables. Limitations We excluded those of multi-race/ethnicity and those from minority groups other than Hispanic or African American due to small numbers. Additionally, only three therapy types were represented among the six studies. Some potential variables that would influence retention, such as client housing, and client comorbidities, the race/ethnicity and gender of the staff who conducted study follow-up assessments, and reasons for loss to follow-up, were not collected by the CTN.
Conclusions Although in our client model older African Americans and Caucasians had the greatest odds of retention and younger African Americans the lowest, in our final model, only age was significantly related to study retention. Additionally, primary drug of abuse, having HIV risk screening as a program benefit, and lower percentages of female admissions were significantly related to study retention. Efforts should be made to increase the study retention of younger participants to improve the validity and generalizability of drug abuse treatment study results.
]]>Purpose/Methods The study aimed at randomizing 253 participants to two intervention arms: (1) MMT alone or (2) injectable opioids (DAM or hydromorphone) plus adjunctive MMT if deemed appropriate. The planned study duration was 3 years, with a 1-year intake period, 1 year of treatment, and an additional year of follow-up. The NAOMI trial was initiated in March 2005 at two Canadian sites (Vancouver and Montreal). This was the first multicenter RCT in North America to compare the relative efficacy of these different therapeutic strategies. We discuss the rationale behind the NAOMI study design, as well as the scientific and political issues and methodological challenges arising from the conduct of a trial that involves the prescription of a controlled substance to individuals with dependence on that substance.
Limitations Restrictive entry criteria led to the exclusion of many otherwise eligible participants, slowing recruitment into the study. Inability to offer DAM treatment beyond 12 months led to artificial boundary effects in the trial. Conclusions Addiction treatment research navigates between science and politics, and evidence-based medicine is many times confronted by moral beliefs. Political considerations influence study design to a further degree than in RCTs treating less-stigmatized disorders with more-reputable medications.
]]>Purpose To present the design, history, and analytic challenges of the Breast International Group (BIG) 1-98 trial: an international, multicenter, randomized, double-blind, phase-III study comparing the aromatase inhibitor letrozole with tamoxifen in this clinical setting.
Methods From 1998—2003, BIG 1-98 enrolled 8028 women to receive monotherapy with either tamoxifen or letrozole for 5 years, or sequential therapy of 2 years of one agent followed by 3 years of the other. Randomization to one of four treatment groups permitted two complementary analyses to be conducted several years apart. The first, reported in 2005, provided a head-to-head comparison of letrozole versus tamoxifen. Statistical power was increased by an enriched design, which included patients who were assigned sequential treatments until the time of the treatment switch. The second, reported in late 2008, used a conditional landmark approach to test the hypothesis that switching endocrine agents at approximately 2 years from randomization for patients who are disease-free is superior to continuing with the original agent. Results The 2005 analysis showed the superiority of letrozole compared with tamoxifen. The patients who were assigned tamoxifen alone were unblinded and offered the opportunity to switch to letrozole. Results from other trials increased the clinical relevance about whether or not to start treatment with letrozole or tamoxifen, and analysis plans were expanded to evaluate sequential versus single-agent strategies from randomization.
Limitations Due to the unblinding of patients assigned tamoxifen alone, analysis of updated data will require ascertainment of the influence of selective crossover from tamoxifen to letrozole.
Conclusions BIG 1-98 is an example of an enriched design, involving complementary analyses addressing different questions several years apart, and subject to evolving analytic plans influenced by new data that emerge over time.
]]>Purpose We demonstrate how selection of patients for an RCT can bias the results when the treatment effect varies across individuals. Indeed, not only the magnitude, but even the direction of the causal effect found in an RCT can differ from the causal effect in the target population.
Methods A counterfactual model is developed to represent the selection process explicitly. This simple extension of the standard counterfactual model is used to explore the implications of restrictive exclusion criteria intended to eliminate high-risk individuals. The counterintuitive findings of a recent FDA meta-analysis of suicidality in pediatric populations treated with antidepressant medications are interpreted in the light of this counterfactual model.
Results When the causal effect of an intervention can vary across individuals, the potential for selection bias (in the sense of a threat to external validity) can be serious. In particular, we demonstrate that the stricter the inclusion/exclusion criteria the greater the potential inflation of relative risk. A critical factor in determining bias is the extent to which individuals with differing types of causal effects can be distinguished prior to sampling. Furthermore, we propose methods that can sometimes be useful to identify the existence of bias in an actual study. When applied to the FDA meta-analysis of pediatric suicidality in RCTs of modern antidepressant medications, these methods suggest that the elevated risk observed may be an artifact of selection bias.
Limitations Real-life scenarios are generally more complex than the counterfactual model presented here. Future modeling efforts are needed to refine and extend our approach.
Conclusions When variation of treatment effects across individuals is plausible, lack of generalizability should be a serious concern. Therefore, external validity of RCTs needs to be carefully considered in the design of an RCT and the interpretation of its results, especially when the study can influence regulatory decisions about drug safety. RCTs should not automatically be considered definitive, especially when their results conflict with those of observational studies. Whenever possible, empirical evidence of bias resulting from sample selection should be obtained and taken into account. Clinical Trials 2009; 6: 109—118. http://ctj.sagepub.com
]]>Purpose To discuss approaches one might use to compare the inflation of the treatment-effect estimator when the trial is stopped early for positive results with the inflation that would be seen in a comparable set of positive trials that used fixed sample sizes with no interim monitoring, and to quantify the relative inflation of monitored trials relative to that of the corresponding subset of positive fixed sample-size trials.
Methods Via simulation for some O'Brien—Fleming and Haybittle—Peto monitoring boundaries, the inflation of the treatment-effect estimator when the trial crossed an interim-monitoring boundary for positive results is compared with the preferred approach to estimate the inflation from a comparable set of positive fixed sample-size trials.
Results Although the inflation of the treatment-effect estimator when a trial is stopped early can be considerable, only at very early interim analyses (≤25% of information) is this inflation much larger than the inflation that would be seen for an appropriate subset of similar positive fixed sample-size trials. The treatment-effect inflation from stopping at second or later interim analyses that are not so early is relatively small and similar to that seen in the corresponding subset of fixed sample-size trials. Limitations The results apply to adequately powered trials with well-designed prospectively specified interim-monitoring plans.
Conclusions For trials with a well-designed interim-monitoring plan, stopping at 50% or greater information has a negligible impact on estimation. Except for very early interim analyses (≤25% of the information), concerns about the inflation of the treatment effect should be minor. Clinical Trials 2009; 6: 119—125. http:// ctj.sagepub.com
]]>Purpose To describe the potential net effect of alternative allocation ratios on recruitment time and trial cost.
Methods Models of recruitment time and trial cost were developed and used to compare trials with 1:1 allocation to trials with alternative allocation ratios under a range of per subject costs, per day costs, and enrollment rates.
Results In regard to time required to complete recruitment, alternative allocation ratios are net beneficial if the recruitment rate improves by more than about 4% for trials with a 1.5:1 allocation ratio and 12% for trials with a 2:1 allocation ratio. More substantial improvements in recruitment rate, 13 and 47% respectively for scenarios we considered, are required for alternative allocation to be net beneficial in terms of tangible monetary cost.
Limitations The cost models were developed expressly for trials comparing proportions or means across treatment groups.
Conclusions Using alternative allocation ratio designs to improve recruitment may or may not be time and cost-effective. Using alternative allocation for this purpose should only be considered for trial contexts where there is both clear evidence that the alternative design does improve recruitment rates and the attained time or cost efficiency justifies the added study subject burden implied by a larger sample size. Clinical Trials 2009; 6: 126—132. http://ctj.sagepub.com
]]>Purpose We present findings from a review of a decade of internal data quality audits performed at the Duke Clinical Research Institute, a large academic research organization that conducts data management for a diverse array of clinical studies, both academic and industry-sponsored. In so doing, we hope to stimulate discussions that could benefit the wider clinical research enterprise by providing insight into methods of optimizing data collection and cleaning, ultimately helping patients and furthering essential research.
Methods We present our audit methodologies, including sampling methods, audit logistics, sample sizes, counting rules used for error rate calculations, and characteristics of audited trials. We also present database error rates as computed according to two analytical methods, which we address in detail, and discuss the advantages and drawbacks of two auditing methods used during this 10-year period.
Results Our review of the DCRI audit program indicates that higher data quality may be achieved from a series of small audits throughout the trial rather than through a single large database audit at database lock. We found that error rates trended upward from year to year in the period characterized by traditional audits performed at database lock (1997—2000), but consistently trended downward after periodic statistical process control type audits were instituted (2001—2006). These increases in data quality were also associated with cost savings in auditing, estimated at 1000 h per year, or the efforts of one-half of a full time equivalent (FTE). Limitations Our findings are drawn from retrospective analyses and are not the result of controlled experiments, and may therefore be subject to unanticipated confounding. In addition, the scope and type of audits we examine here are specific to our institution, and our results may not be broadly generalizable.
Conclusions Use of statistical process control methodologies may afford advantages over more traditional auditing methods, and further research will be necessary to confirm the reliability and usability of such techniques. We believe that open and candid discussion of data quality assurance issues among academic and clinical research organizations will ultimately benefit the entire research community in the coming era of increased data sharing and re-use. Clinical Trials 2009; 6: 141—150. http://ctj.sagepub.com
]]>Purpose: This article describes the experience of the National Institute on Drug Abuse's (NIDA) National Drug Abuse Treatment Clinical Trials Network (CTN) in devising and implementing a three-tiered QA model for rigorous multi-site randomized clinical trials implemented in community-based substance abuse treatment programs. The CTN QA model combined local and national resources and was developed to address the unique needs of clinical trial sites with limited research experience.
Methods: The authors reviewed internal records maintained by the sponsor, a coordinating site (Lead Nodes), and a local site detailing procedural development, training sessions, protocol violation monitoring, and site visit reporting.
Results: Between January 2001 and September 2005, the CTN implemented 21 protocols, of which 18 were randomized clinical trials, one was a quality improvement study and two were surveys. Approximately 160 community-based treatment programs participated in the 19 studies that were monitored, with a total of 6560 participants randomized across the sites. During this time 1937 QA site visits were reported across the three tiers of monitoring and the cost depended on the location of the sites and the salaries of the staff involved. One study reported 109 protocol violations (M = 15.6). Examples are presented to highlight training, protocol violation monitoring, site visit frequency and intensity and cost considerations.
Limitations: QA data from the entire network were not easily available for review as much of the data were not electronically accessible. The authors reviewed and discussed a representative sample of internal data from the studies and participating sites.
Conclusions: The lessons learned from the CTN's experience include the need for balancing thoroughness with efficiency, monitoring early, assessing research staff abilities in order to judge the need for proactive, focused attention, providing targeted training sessions, and developing flexible tools. The CTN model can work for sponsors overseeing studies at sites with limited research experience that require more frequent, in-depth monitoring. We recommend that sponsors not develop a rigid monitoring approach, but work with the study principal investigators to determine the intensity of monitoring needed depending on trial complexity, the risks of the intervention(s), and the experience of the staff with clinical research. After careful evaluation, sponsors should then determine the best approach to site monitoring and what resources will be needed. Clinical Trials 2009; 6: 151—161. http://ctj.sagepub.com
]]>Purpose To present the rationale and design of a randomized controlled surgical trial (RCT), the Outcomes following vaginal Prolapse repair and mid Urethral Sling (OPUS) Trial highlighting the challenges in the design and implementation.
Methods The challenges of implementing this surgical trial combined with a cost-effectiveness study and patient preference group are discussed including the study design, ethical issues regarding use of sham incision, maintaining the masking of study staff, and pragmatic difficulties encountered in the collection of cost data. The trial is conducted by the NICHD-funded Pelvic Floor Disorders Network.
Results The ongoing OPUS trial started enrollment in May 2007 with a planned accrual of 350. The use of sham incision was generally well accepted but the collection of cost data using conventional billing forms was found to potentially unmask key study personnel. This necessitated changes in the study forms and planned timing for collection of cost data. To date, the enrollment to the patient preference group has been lower than the limit established by the protocol suggesting a willingness on the part of women to participate in the randomization. Limitations Given the invasive nature of surgical intervention trials, potential participants may be reluctant to accept random assignment, potentially impacting generalizability.
Conclusion Findings from the OPUS trial will provide important information that will help surgeons to better counsel women on the benefits and risks of concomitant prophylactic anti-incontinence procedure at the time of vaginal surgery for prolapse. The implementation of the OPUS trial has necessitated that investigators consider ethical issues up front, remain flexible with regards to data collection and be constantly aware of unanticipated opportunities for unmasking. Future surgical trials should be aware of potential challenges in maintaining masking and collection of cost-related information. Clinical Trials 2009; 6: 162—171. http://ctj.sagepub.com
]]>Purpose INSPIRE II is a randomized clinical trial (RCT) funded by the NHLBI to evaluate the effects of telephone-based enhanced CST for patients with COPD and their caregivers compared to standardized medical care (SMC) including COPD education and symptom monitoring on medical outcomes, physical functioning, and quality of life.
Methods Six hundred COPD patients and their respective caregivers recruited from Duke University and Ohio State University will be evaluated and randomized (in a 1:1 ratio) to enhanced CST (including sessions promoting physical activity, relaxation, cognitive restructuring, communication skills, and problem solving) or to SMC. The primary outcomes include all-cause mortality, COPD-related hospitalizations/ physician visits, and quality of life. These endpoints will be measured through self-report questionnaires, behavioral measures of functional capacity (i.e., accelerometer and six minute walk test) and pulmonary function tests (e.g., FEV1).
Results This article reviews prior studies in the area and describes the design of INSPIRE-II. Several key methodological issues are discussed including the delivery of CST over the telephone, encouraging physical activity, and inclusion of caregivers as patient coaches to enhance the effectiveness of the intervention.
Limitations We recognize that SMC does not adequately control for attention, support, and non-specific factors, and that, in theory, non-specific effects of the intervention could account for some, or all, of the observed benefits. However, our fundamental question is whether the telephone intervention produces benefits over-and-above the usual care that patients typically receive. The SMC condition will provide education and additional weekly telephone contact, albeit less than the attention received by the CST group. We recognize that this attention control condition may not provide equivalent patient contact, but it will minimize group differences due to attention. We considered several alternative designs including adding a third usual care only arm as well as an education only control arm. However, these alternatives would require more patients, reduce the power to detect significant effects of our primary medical endpoints, and add a significant additional expense to the cost of the study that would make such an undertaking neither scientifically or financially viable.
Conclusions We believe that this novel approach to patient care in which caregivers are used to assist in the delivery of coping skills training to patients with COPD has the potential to change the way in which COPD patients are routinely managed in order to reduce distress, enhance quality of life, and potentially improve medical outcomes. Clinical Trials 2009; 6: 172—184. http:// ctj.sagepub.com
]]>Purpose The objective of the present study was to implement the multigroup confirmatory factor analysis (CFA) method for invariance testing across mode of administration for children's self-reported HRQOL.
Methods Multigroup CFA was performed specifying a five-factor model across three modes of administration groups (in-person, mail, and telephone survey). The 23-item PedsQLTM 4.0 Generic Core Scales was utilized as the measure of children's self-reported HRQOL in a sample of 3741 children ages 5—18.
Results Strong factorial invariance across the mode of administration groups was demonstrated based on stability of the Comparative Fit Index (CFI) between the models, and several additional indices of practical fit including the Root Mean Squared Error of Approximation (RMSEA), the Non-Normed Fit Index (NNFI), and the Parsimony Normed Fit Index (PNFI).
Limitations The mode of administration subpopulations in our study differed with regard to health status.
Conclusion The multigroup CFA statistical methods utilized in the present study have important implications for clinical trials in which mixed modes of administration are used. The present findings support an equivalent five-factor structure of the PedsQL TM 4.0 Generic Core Scales across the three modes of administration studied. Based on these data, it can be concluded that children across the three modes of administration groups interpreted items on the PedsQL TM 4.0 Generic Core Scales in a similar manner. Clinical Trials 2009; 6: 185—195. http://ctj.sagepub.com
]]>