A Note on Cross-Validation for Lasso Under Measurement Errors

Abhirup Datta, Hui Zou

Research output: Contribution to journalComment/debate

Abstract

Variants of the Lasso or l1-penalized regression have been proposed to accommodate for presence of measurement errors in the covariates. Theoretical guarantees of these estimates have been established for some oracle values of the regularization parameters which are not known in practice. Data-driven tuning such as cross-validation has not been studied when covariates contain measurement errors. We demonstrate that in the presence of error-in-covariates, even when using a Lasso-variant that adjusts for measurement error, application of naive leave-one-out cross-validation to select the tuning parameter can be problematic. We provide an example where such a practice leads to estimation inconsistency. We also prove that a simple correction to cross-validation procedure restores consistency. We also study the risk consistency of the two cross-validation procedures and offer guideline on the choice of cross-validation based on the measurement error distributions of the training and the prediction data. The theoretical findings are validated using simulated data. Supplementary materials for this article are available online.

Original languageEnglish (US)
JournalTechnometrics
DOIs
StateAccepted/In press - Jan 1 2019

Fingerprint

Lasso
Measurement errors
Cross-validation
Measurement Error
Covariates
Tuning
Penalized Regression
Parameter Tuning
Regularization Parameter
Data-driven
Inconsistency
Prediction
Estimate
Demonstrate

Keywords

  • Cross-validation
  • Inconsistency
  • Lasso
  • Measurement errors

ASJC Scopus subject areas

  • Statistics and Probability
  • Modeling and Simulation
  • Applied Mathematics

Cite this

A Note on Cross-Validation for Lasso Under Measurement Errors. / Datta, Abhirup; Zou, Hui.

In: Technometrics, 01.01.2019.

Research output: Contribution to journalComment/debate

@article{32bdb9452c0e4c6da0b76a81585f0d32,
title = "A Note on Cross-Validation for Lasso Under Measurement Errors",
abstract = "Variants of the Lasso or l1-penalized regression have been proposed to accommodate for presence of measurement errors in the covariates. Theoretical guarantees of these estimates have been established for some oracle values of the regularization parameters which are not known in practice. Data-driven tuning such as cross-validation has not been studied when covariates contain measurement errors. We demonstrate that in the presence of error-in-covariates, even when using a Lasso-variant that adjusts for measurement error, application of naive leave-one-out cross-validation to select the tuning parameter can be problematic. We provide an example where such a practice leads to estimation inconsistency. We also prove that a simple correction to cross-validation procedure restores consistency. We also study the risk consistency of the two cross-validation procedures and offer guideline on the choice of cross-validation based on the measurement error distributions of the training and the prediction data. The theoretical findings are validated using simulated data. Supplementary materials for this article are available online.",
keywords = "Cross-validation, Inconsistency, Lasso, Measurement errors",
author = "Abhirup Datta and Hui Zou",
year = "2019",
month = "1",
day = "1",
doi = "10.1080/00401706.2019.1668856",
language = "English (US)",
journal = "Technometrics",
issn = "0040-1706",
publisher = "American Statistical Association",

}

TY - JOUR

T1 - A Note on Cross-Validation for Lasso Under Measurement Errors

AU - Datta, Abhirup

AU - Zou, Hui

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Variants of the Lasso or l1-penalized regression have been proposed to accommodate for presence of measurement errors in the covariates. Theoretical guarantees of these estimates have been established for some oracle values of the regularization parameters which are not known in practice. Data-driven tuning such as cross-validation has not been studied when covariates contain measurement errors. We demonstrate that in the presence of error-in-covariates, even when using a Lasso-variant that adjusts for measurement error, application of naive leave-one-out cross-validation to select the tuning parameter can be problematic. We provide an example where such a practice leads to estimation inconsistency. We also prove that a simple correction to cross-validation procedure restores consistency. We also study the risk consistency of the two cross-validation procedures and offer guideline on the choice of cross-validation based on the measurement error distributions of the training and the prediction data. The theoretical findings are validated using simulated data. Supplementary materials for this article are available online.

AB - Variants of the Lasso or l1-penalized regression have been proposed to accommodate for presence of measurement errors in the covariates. Theoretical guarantees of these estimates have been established for some oracle values of the regularization parameters which are not known in practice. Data-driven tuning such as cross-validation has not been studied when covariates contain measurement errors. We demonstrate that in the presence of error-in-covariates, even when using a Lasso-variant that adjusts for measurement error, application of naive leave-one-out cross-validation to select the tuning parameter can be problematic. We provide an example where such a practice leads to estimation inconsistency. We also prove that a simple correction to cross-validation procedure restores consistency. We also study the risk consistency of the two cross-validation procedures and offer guideline on the choice of cross-validation based on the measurement error distributions of the training and the prediction data. The theoretical findings are validated using simulated data. Supplementary materials for this article are available online.

KW - Cross-validation

KW - Inconsistency

KW - Lasso

KW - Measurement errors

UR - http://www.scopus.com/inward/record.url?scp=85074557648&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85074557648&partnerID=8YFLogxK

U2 - 10.1080/00401706.2019.1668856

DO - 10.1080/00401706.2019.1668856

M3 - Comment/debate

AN - SCOPUS:85074557648

JO - Technometrics

JF - Technometrics

SN - 0040-1706

ER -