Clinical Trials

 

Advanced Search

Journal Navigation

Journal Home

Subscriptions

Archive

Contact Us

Table of Contents

Register here to gain access to SAGE's 500+ Journals Online

Sign In to gain access to subscriptions and/or personal tools.
This Article
Right arrow Abstract Freely available
Right arrow Free Full Text (Free PDF) Free
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to Saved Citations
Right arrow Download to citation manager
Right arrowRequest Permissions
Right arrow Request Reprints
Right arrow Add to My Marked Citations
Google Scholar
Right arrow Articles by Abrahantes, J. C.
Right arrow Articles by Molenberghs, G.
PubMed
Right arrow PubMed Citation
Right arrow Articles by Abrahantes, J. C.
Right arrow Articles by Molenberghs, G.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us   Add to Digg   Add to Reddit   Add to Technorati  
What's this?
Clinical Trials, Vol. 5, No. 3, 194-208 (2008)
DOI: 10.1177/1740774508091677

Alternative methods to evaluate trial level surrogacy

Josè Cortiñas Abrahantes

Center for Statistics, Hasselt University, Campus Diepenbeek, B3590 Diepenbeek, Belgium, jose.cortinas{at}uhasselt.be

Ziv Shkedy

Center for Statistics, Hasselt University, Campus Diepenbeek, B3590 Diepenbeek, Belgium

Geert Molenberghs

Center for Statistics, Hasselt University, Campus Diepenbeek, B3590 Diepenbeek, Belgium

Background: The evaluation and validation of surrogate endpoints have been extensively studied in the last decade. Prentice [1] and Freedman, Graubard and Schatzkin [2] laid the foundations for the evaluation of surrogate endpoints in randomized clinical trials. Later, Buyse et al. [5] proposed a meta-analytic methodology, producing different methods for different settings, which was further studied by Alonso and Molenberghs [9], in their unifying approach based on information theory.

Purpose: In this article, we focus our attention on the trial-level surrogacy and propose alternative procedures to evaluate such surrogacy measure, which do not pre-specify the type of association. A promising correction based on cross-validation is investigated. As well as the construction of confidence intervals for this measure.

Methods: In order to avoid making assumption about the type of relationship between the treatment effects and its distribution, a collection of alternative methods, based on regression trees, bagging, random forests, and support vector machines, combined with bootstrap-based confidence interval and, should one wish, in conjunction with a cross-validation based correction, will be proposed and applied. We apply the various strategies to data from three clinical studies: in opthalmology, in advanced colorectal cancer, and in schizophrenia.

Results: The results obtained for the three case studies are compared; they indicate that using random forest or bagging models produces larger estimated values for the surrogacy measure, which are in general stabler and the confidence interval narrower than linear regression and support vector regression. For the advanced colorectal cancer studies, we even found the trial-level surrogacy is considerably different from what has been reported.

Limitations: In general the alternative methods are more computationally demanding, and specially the calculation of the confidence intervals, require more computational time that the delta-method counterpart.

Conclusions: First, more flexible modeling techniques can be used, allowing for other type of association. Second, when no cross-validation-based correction is applied, overly optimistic trial-level surrogacy estimates will be found, thus cross-validation is highly recommendable. Third, the use of the delta method to calculate confidence intervals is not recommendable since it makes assumptions valid only in very large samples. It may also produce range-violating limits. We therefore recommend alternatives: bootstrap methods in general. Also, the information-theoretic approach produces comparable results with the bagging and random forest approaches, when cross-validation correction is applied. It is also important to observe that, even for the case in which the linear model might be a good option too, bagging methods perform well too, and their confidence intervals were more narrow. Clinical Trials 2008; 5: 194—208. http://ctj.sagepub.com

References

  • Prentice RL Surrogate endpoints in clinical trials: definitions and operational criteria . Statistics in Medicine 1989; 8: 431-40.[ISI][Medline] [Order article via Infotrieve]
  • Freedman LS, Graubard BI, Schatzkin A. Statistical validation of intermediate endpoints for chronic diseases . Statistics in Medicine 1992; 11: 167-78.[ISI][Medline] [Order article via Infotrieve]
  • Buyse M., Molenberghs G. The validation of surrogate endpoints in randomized experiments . Biometrics 1998; 54: 186-201.
  • Daniels MJ, Hughes MD Meta-analysis for the evaluation of potential surrogate markers . Statistics in Medicine 1997; 16: 1965-82.[CrossRef][ISI][Medline] [Order article via Infotrieve]
  • Buyse M., Molenberghs G., Burzykowski T., Renard D., Geys H. The validation of surrogate endpoints in meta-analyses of randomized experiments . Biostatistics 2000; 1: 1-19.[Abstract]
  • Gail MH, Pfeiffer R., Van Houwelingen HC, Carroll R. On meta-analytic assessment of surrogate outcomes . Biostatistics 2000; 1: 231-46.[Abstract]
  • Burzykowski T., Molenberghs G., Buyse M. The Evaluation of Surrogate Endpoints. Springer, New York, 2005.
  • Tibaldi FS, Cortiñas Abrahantes J., Molenberghs G., Renard D., Burzykowski T., Buyse M., Parmar M., Stijnen T., Wolfinger R. Simplified hierarchical linear models for the evaluation of surrogate endpoints . Journal of Statistical Computation and Simulation 2003; 73: 643-58.[CrossRef][ISI]
  • Alonso A., Molenberghs G. Surrogate marker evaluation from an information theory perspective . Biometrics 2007; 63: 180-86.[CrossRef][ISI][Medline] [Order article via Infotrieve]
  • Oosterlinck W., Mattelaer J., Casselman J., Van Velthoven R., Derde MP, Kaufman L. PSA evolution: a prognostic factor during treatment of advanced prostatic carcinoma with total androgen blockade. Data from a Belgian multicentric study of 546 patients . Acta Urol Belg 1997; 65: 63-71.[Medline] [Order article via Infotrieve]
  • Verbeke G., Molenberghs G. Linear Mixed Models for Longitudinal Data. Springer, New York, 2000.
  • Corfu-A Study Group. Phase III randomized study of two fluorouracil combinations with either interferon alfa-2a or leucovorin for advanced colorectal cancer . Journal of Clinical Oncology 1995; 13: 921-28.[Abstract]
  • Greco FA, Figlin R., York M., Einhorn L., Schilsky R., Marshall EM, et al. Phase III randomized study to compare interferon alfa-2a in combination with fluorouracil versus fluorouracil alone in patients with advanced colorectal cancer . Journal of Clinical Oncology 1996; 14: 2674-81.[Abstract/Free Full Text]
  • Burzykowski T., Molenberghs G., Buyse M. The validation of surrogate endpoints by using data from randomized clinical trials: A case study in advanced colorectal cancer . Journal of the Royal Statistical Society, Series A 2004; 167: 103-24.
  • Alonso A., Geys H., Molenberghs G., Vangeneugden T. Investigating the criterion validity of psychiatric symptom scales using surrogate marker validation methodology . Journal of Biopharmaceutical Statistics 2002; 12: 161-79.[CrossRef][Medline] [Order article via Infotrieve]
  • Cortiñas Abrahantes J., Molenberghs G., Burzykowski T., Shkedy Z., Renard D. Choice of units of analysis and modeling strategies in multilevel hierarchical models . Computational Statistics and Data Analysis 2004; 47: 537-63.[CrossRef]
  • Kay SR, Opler LA, Lindenmayer JP Reliability and validity of the positive and negative syndrome scale of shizophrenics . Psychiatry Research 1988; 23: 99-110.[CrossRef][ISI][Medline] [Order article via Infotrieve]
  • Alonso A., Geys H., Kenward MG, Molenberghs G., Vangeneugden T. Validation of surrogate markers in multiple randomized clinical trials with repeated measurements . Biometrical Journal 2003; 45: 1-15.
  • Galecki AT General class of covariance structures for two or more repeated factors in longitudinal data analysis . Communications in Statistics 1994; 23: 3105-20.
  • Molenberghs G. Verbeke G. Models for Discrete Longitudinal Data. Springer, New York, 2005.
  • Ding CG On the computation of the distribution of the square of the sample multiple correlation coefficient . Computational Statistics and Data Analysis 1996; 22: 345-50.[CrossRef]
  • Verbyla AP, Cullis BR, Kenward MG, Welham SJ The analysis of designed experiments and longitudinal data by using smoothing splines . Applied Statistics 1999; 48: 269-311.
  • Alonso Abad A. Investigating validity of psychiatric symptom scales and surrogate markers. Doctoral Thesis, 2004; 48-86, Center for statistics, Hasselt University, Belgium.
  • Kraskov A., Stögbauer H., Grassberger P. Estimating Mutual Information . Physical Review E 2004; 69: 066138.
  • Efron B., Tibshirani R. Improvements on cross-validation: The.632+ bootstrap method . Journal of the American Statistical Association 1997; 92: 548-560.[CrossRef][ISI]
  • Lendasse A., Wertz V., Verleysen M. Icann 2003, Joint International Conference on Artificial Neural Networks, June 26-29, 2003, Istanbul (Turkey) . Artificial Neural Networks and Neural Information Processing - ICANN/ ICONIP 2003, Kaynak O, Alpaydin E, Oja E, Xu L, (eds)., Lecture Notes in Computer Science 2714 , Springer-Verlag 2003, pp. 573-80.
  • Baker SG A simple meta-analytic approach for binary surrogate and true endpoints . Biostatistics 2006; 7: 57-70.
  • Breiman L., Friedman JH, Olshen RA, Stone CJ Classification and Regression Trees. Chapman & Hall / CRC, New York, 1984.
  • Breiman L. Bagging predictors . Machine Learning 1996a; 26: 123-40.
  • Breiman L. Heuristics of instability and stabilization in model selection . Annals of Statistics 1996b; 24: 2350-83.[CrossRef][ISI]
  • Breiman L. Random forests . Machine Learning 2001; 45: 5-32.[CrossRef][ISI]
  • Vapnik V. The Nature of Statistical Learning Theory. Springer, New York, 1995.
  • Meyer D., Leisch F., Hornik K. Benchmarking support vector machines. Technical Report bfseries 78, SFB Adaptive Information Systems and Modeling in Economics and Management Science. 2002.
  • Ihaka R., Gentleman R. R: A Language for data analysis and graphics . Journal of Computional and Graphical Statistics 1996; 5: 299-314.[CrossRef]
  • Therneau TM, Atkinson EJ An introduction to recursive partitioning using the part routines. Technical Report 61, Department of Health Science Research, Mayo Clinic, Rochester, New York, 1997.
  • Liaw A., Wiener M. Classification and regression by random forest . The Newsletter of the R Project 2002; 2/3: 18-22.
  • Meyer D. Support vector machines, the interface to libsvm in package e1071 . The Newsletter of the R Project 2001; 1/3: 23-26.
  • Verbyla DL Classification trees: a new discrimination tool . Canadian Journal of Forestry Research 1987; 17: 1150-52.[CrossRef]
  • Svetnik V., Liaw A., Tong C., Culberson JC, Sheridan RP, Feuston BP Random forest: a classification and regression tool for compound classification and QSAR modeling . J. Chem. Inform. Comput. Sci 2003; 43: 1947-58.[CrossRef][ISI][Medline] [Order article via Infotrieve]
  • Moore DE, Lees BG, Davey SM A new method for predicting vegetation distributions using decision tree analysis in a geographic information system . Journal of Environmental Management 1991; 15: 59-71.[CrossRef]
  • Meyer D., Leisch F., Hornik K. The support vector machine under test . Neurocomputing 2003; 55: 169-86.[CrossRef][ISI]
  • Drucker H., Burges CJC, Kaufman L., Smola A., Vapnik V. Support vector regression machines. In: Mozer MC, Jordan MI, Petsche T. (eds), Advances in Neural Information Processing Systems 9, MIT Press, Cambridge, MA, 1997, pp. 155-61.
  • Müller KR, Smola A., Rätsch G., Schölkopf B., Kohlmorgen J., Vapnik V. Predicting time series with support vector machines. In: Gerstner W, Germond A, Hasler M, and Nicoud J.-D. (eds), Artificial Neural Networks ICANN 97, Springer Lecture Notes in Computer Science, Berlin . 1327, 1997, 999-1004.
  • Stitson M., Gammerman A., Vapnik V., Vovk V., Watkins C., Weston J. Support vector regression with ANOVA decomposition kernels. In: Schölkopf B, Burges CJC, Smola AJ. (eds), Advances in Kernel Methods-Support Vector Learning, MIT Press Cambridge, MA , 1999, pp. 285-92.
  • Mattera D., Haykin S. Support vector machines for dynamic reconstruction of a chaotic system. In: Schölkopf B, Burges CJC, Smola AJ. (eds), Advances in Kernel Methods-Support Vector Learning, MIT Press, Cambridge, MA, 1999, pp. 211-42.
  • Vapnik V., Lerner A. Pattern recognition using generalized portrait method . Automation and Remote Control 1963; 24: 774-80.
  • Vapnik V., Chervonenkis A. A note on one class of perceptrons . Automation and Remote Control, 1964; 25: 821-837.[ISI]
  • Smola A. Regression estimation with support vector learning machines. Master thesis. Technische Universität München, Munich, Germany, 1996.
  • Fletcher R. Practical Methods of Optimization. John Wiley, New York, 1989.
  • Hsu CW, Chang CC, Lin CJ A Practical Guide to Support Vector Classification. 2001, http://www.csie. ntu.edu.tw/cjlin/libsvm/.
  • Anderson TW An Introduction to Multivariate Statistical Analysis. Wiley, New York, 1958.

Add to CiteULike CiteULike   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us   Add to Digg Digg   Add to Reddit Reddit   Add to Technorati Technorati    What's this?



This Article
Right arrow Abstract Freely available
Right arrow Free Full Text (Free PDF) Free
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to Saved Citations
Right arrow Download to citation manager
Right arrowRequest Permissions
Right arrow Request Reprints
Right arrow Add to My Marked Citations
Google Scholar
Right arrow Articles by Abrahantes, J. C.
Right arrow Articles by Molenberghs, G.
PubMed
Right arrow PubMed Citation
Right arrow Articles by Abrahantes, J. C.
Right arrow Articles by Molenberghs, G.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us   Add to Digg   Add to Reddit   Add to Technorati  
What's this?