Volume 24, Issue 3

In this paper we deal with the problem of fitting an autoregression of order p to given data coming from a stationary autoregressive process with infinite order. The paper is mainly concerned with the selection of an appropriate order of the autoregressive model. Based on the so–called final prediction error ( FPE ) a bootstrap order selection can be proposed, because it turns out that one relevant expression occurring in the FPE is ready for the application of the bootstrap principle. Some asymptotic properties of the bootstrap order selection are proved. To carry through the bootstrap procedure an autoregression with increasing but non–stochastic order is fitted to the given data. The paper is concluded by some simulations.

We are considering the problem of efficient inference on the shape matrix of an elliptic distribution with unspecified location and either (a) fully specified radial density, (b) radial density specified up to a scale parameter, or (c) completely unspecified radial density. Bickel in [1] has shown that efficiencies under (b) and (c), while being strictly less than under (a), coincide: the efficiency loss caused by an unspecified radial density thus is entirely due to the non-specification of its scale (scale here is not necessarily measured by standard error, as second-order moments may be infinite). Defining scale however requires the choice of a particular scale functional , a choice which has an impact on efficiency bounds. We provide a closed form expression for this efficiency loss, both in hypothesis testing and in point estimation, as a function of the standardized radial density and the scale functional. We show that this loss is maximum at arbitrarily light tails whereas, under arbitrarily heavy tails, it is arbitrarily close to zero: hence, under very heavy tails, ignoring the scale (ignoring the exact density) asymptotically does not harm inference on shape. However, the same loss is nil, irrespective of the standardized radial density, when the scale functional (in dimension k ) is the k -th root of the scatter determinant.

We consider choosing an estimator or model from a given class by cross validation consisting of holding a nonneglible fraction of the observations out as a test set. We derive bounds that show that the risk of the resulting procedure is (up to a constant) smaller than the risk of an oracle plus an error which typically grows logarithmically with the number of estimators in the class. We extend the results to penalized cross validation in order to control unbounded loss functions. Applications include regression with squared and absolute deviation loss and classification under Tsybakov’s condition.

Suppose that we observe a sample of independent and identically distributed realizations of a random variable, and a parameter of interest can be defined as the minimizer, over a suitably defined parameter set, of the expectation of a (loss) function of a candidate parameter value and the random variable. For example, squared error loss in regression or the negative log-density loss in density estimation. Minimizing the empirical risk (i.e., the empirical mean of the loss function) over the entire parameter set may result in ill-defined or too variable estimators of the parameter of interest. In this article, we propose a cross-validated ε-net estimation method, which uses a collection of submodels and a collection of ε-nets over each submodel. For each submodel  s and each resolution level ε, the minimizer of the empirical risk over the corresponding ε-net is a candidate estimator. Next we select from these estimators (i.e. select the pair ( s ,ε)) by multi-fold cross-validation. We derive a finite sample inequality that shows that the resulting estimator is as good as an oracle estimator that uses the best submodel and resolution level for the unknown true parameter. We also address the implementation of the estimation procedure, and in the context of a linear regression model we present results of a preliminary simulation study comparing the cross-validated ε-net estimator to the cross-validated L 1 -penalized least squares estimator (LASSO) and the least angle regression estimator (LARS).