Article
Licensed
Unlicensed
Requires Authentication
Calculating Confidence Intervals for Prediction Error in Microarray Classification Using Resampling
-
Wenyu Jiang
Published/Copyright:
March 1, 2008
Cross-validation based point estimates of prediction accuracy are frequently reported in microarray class prediction problems. However these point estimates can be highly variable, particularly for small sample numbers, and it would be useful to provide confidence intervals of prediction accuracy.We performed an extensive study of existing confidence interval methods and compared their performance in terms of empirical coverage and width. We developed a bootstrap case cross-validation (BCCV) resampling scheme and defined several confidence interval methods using BCCV with and without bias-correction.The widely used approach of basing confidence intervals on an independent binomial assumption of the leave-one-out cross-validation errors results in serious under-coverage of the true prediction error. Two split-sample based methods previously proposed in the literature tend to give overly conservative confidence intervals. Using BCCV resampling, the percentile confidence interval method was also found to be overly conservative without bias-correction, while the bias corrected accelerated (BCa) interval method of Efron returns substantially anti-conservative confidence intervals. We propose a simple bias reduction on the BCCV percentile interval. The method provides mildly conservative inference under all circumstances studied and outperforms the other methods in microarray applications with small to moderate sample sizes.
Keywords: microarray study; class prediction; prediction error; confidence interval; resampling; bootstrap; cross-validation
Published Online: 2008-3-1
©2011 Walter de Gruyter GmbH & Co. KG, Berlin/Boston
You are currently not able to access this content.
You are currently not able to access this content.
Articles in the same Issue
- Article
- Self-Organizing Maps with Statistical Phase Synchronization (SOMPS) for Analyzing Cell Cycle-Specific Gene Expression Data
- Coalescent Time Distributions in Trees of Arbitrary Size
- Quantifying the Association between Gene Expressions and DNA-Markers by Penalized Canonical Correlation Analysis
- Nonparametric Functional Mapping of Quantitative Trait Loci Underlying Programmed Cell Death
- Accommodating Uncertainty in a Tree Set for Function Estimation
- Drifting Markov Models with Polynomial Drift and Applications to DNA Sequences
- Comparing the Characteristics of Gene Expression Profiles Derived by Univariate and Multivariate Classification Methods
- Calculating Confidence Intervals for Prediction Error in Microarray Classification Using Resampling
- Structure Learning in Nested Effects Models
- Correcting the Estimated Level of Differential Expression for Gene Selection Bias: Application to a Microarray Study
- Adapting Prediction Error Estimates for Biased Complexity Selection in High-Dimensional Bootstrap Samples
- Adaptive Choice of the Number of Bootstrap Samples in Large Scale Multiple Testing
- Re-Cracking the Nucleosome Positioning Code
- Semi-Parametric Differential Expression Analysis via Partial Mixture Estimation
- A SNP Streak Model for the Identification of Genetic Regions Identical-by-descent
- Detecting Two-Locus Gene-Gene Effects Using Monotonisation of the Penetrance Matrix
- Modeling DNA Methylation in a Population of Cancer Cells
- Phenotyping Genetic Diseases Using an Extension of µ-Scores for Multivariate Data
- The Estimator of the Optimal Measure of Allelic Association: Mean, Variance and Probability Distribution When the Sample Size Tends to Infinity
- Predicting Protein Concentrations with ELISA Microarray Assays, Monotonic Splines and Monte Carlo Simulation
- A Comparison of Normalization Techniques for MicroRNA Microarray Data
- Collapsing SNP Genotypes in Case-Control Genome-Wide Association Studies Increases the Type I Error Rate and Power
- Estimating Number of Clusters Based on a General Similarity Matrix with Application to Microarray Data
- Data Distribution of Short Oligonucleotide Expression Arrays and Its Application to the Construction of a Generalized Intellectual Framework
- Approximately Sufficient Statistics and Bayesian Computation
- A Composite-Conditional-Likelihood Approach for Gene Mapping Based on Linkage Disequilibrium in Windows of Marker Loci
- Statistical Methods in Integrative Analysis for Gene Regulatory Modules
- Reducing Spatial Flaws in Oligonucleotide Arrays by Using Neighborhood Information
- Pattern Classification of Phylogeny Signals
- A Unification of Multivariate Methods for Meta-Analysis of Genetic Association Studies
- Importance Sampling for the Infinite Sites Model
- Supervised Distance Matrices
- Addressing the Shortcomings of Three Recent Bayesian Methods for Detecting Interspecific Recombination in DNA Sequence Alignments
- A Sparse PLS for Variable Selection when Integrating Omics Data
- Software Communication
- TRAB: Testing Whether Mutation Frequencies Are Above an Unknown Background
Keywords for this article
microarray study;
class prediction;
prediction error;
confidence interval;
resampling;
bootstrap;
cross-validation
Articles in the same Issue
- Article
- Self-Organizing Maps with Statistical Phase Synchronization (SOMPS) for Analyzing Cell Cycle-Specific Gene Expression Data
- Coalescent Time Distributions in Trees of Arbitrary Size
- Quantifying the Association between Gene Expressions and DNA-Markers by Penalized Canonical Correlation Analysis
- Nonparametric Functional Mapping of Quantitative Trait Loci Underlying Programmed Cell Death
- Accommodating Uncertainty in a Tree Set for Function Estimation
- Drifting Markov Models with Polynomial Drift and Applications to DNA Sequences
- Comparing the Characteristics of Gene Expression Profiles Derived by Univariate and Multivariate Classification Methods
- Calculating Confidence Intervals for Prediction Error in Microarray Classification Using Resampling
- Structure Learning in Nested Effects Models
- Correcting the Estimated Level of Differential Expression for Gene Selection Bias: Application to a Microarray Study
- Adapting Prediction Error Estimates for Biased Complexity Selection in High-Dimensional Bootstrap Samples
- Adaptive Choice of the Number of Bootstrap Samples in Large Scale Multiple Testing
- Re-Cracking the Nucleosome Positioning Code
- Semi-Parametric Differential Expression Analysis via Partial Mixture Estimation
- A SNP Streak Model for the Identification of Genetic Regions Identical-by-descent
- Detecting Two-Locus Gene-Gene Effects Using Monotonisation of the Penetrance Matrix
- Modeling DNA Methylation in a Population of Cancer Cells
- Phenotyping Genetic Diseases Using an Extension of µ-Scores for Multivariate Data
- The Estimator of the Optimal Measure of Allelic Association: Mean, Variance and Probability Distribution When the Sample Size Tends to Infinity
- Predicting Protein Concentrations with ELISA Microarray Assays, Monotonic Splines and Monte Carlo Simulation
- A Comparison of Normalization Techniques for MicroRNA Microarray Data
- Collapsing SNP Genotypes in Case-Control Genome-Wide Association Studies Increases the Type I Error Rate and Power
- Estimating Number of Clusters Based on a General Similarity Matrix with Application to Microarray Data
- Data Distribution of Short Oligonucleotide Expression Arrays and Its Application to the Construction of a Generalized Intellectual Framework
- Approximately Sufficient Statistics and Bayesian Computation
- A Composite-Conditional-Likelihood Approach for Gene Mapping Based on Linkage Disequilibrium in Windows of Marker Loci
- Statistical Methods in Integrative Analysis for Gene Regulatory Modules
- Reducing Spatial Flaws in Oligonucleotide Arrays by Using Neighborhood Information
- Pattern Classification of Phylogeny Signals
- A Unification of Multivariate Methods for Meta-Analysis of Genetic Association Studies
- Importance Sampling for the Infinite Sites Model
- Supervised Distance Matrices
- Addressing the Shortcomings of Three Recent Bayesian Methods for Detecting Interspecific Recombination in DNA Sequence Alignments
- A Sparse PLS for Variable Selection when Integrating Omics Data
- Software Communication
- TRAB: Testing Whether Mutation Frequencies Are Above an Unknown Background
