A flexible and nearly optimal sequential testing approach to randomized testing: QUICK-STOP

Julian Hecker, Ingo Ruczinski, Michael H. Cho, Edwin K. Silverman, Brent Coull, Christoph Lange

Research output: Contribution to journalArticle

Abstract

In the analysis of current life science datasets, we often encounter scenarios in which the application of asymptotic theory to hypothesis testing can be problematic. Besides improved asymptotic results, permutation/simulation-based tests are a general approach to address this issue. However, these randomized tests can impose a massive computational burden, for example, in scenarios in which large numbers of statistical tests are computed, and the specified significance level is very small. Stopping rules aim to assess significance with the smallest possible number of draws while controlling the probabilities of errors due to statistical uncertainty. In this communication, we derive a general stopping rule, QUICK-STOP, based on the sequential testing theory that is easy to implement, controls the error probabilities rigorously, and is nearly optimal in terms of expected draws. In a simulation study, we show that our approach outperforms current stopping approaches for general randomized tests by factor 10 and does not impose an additional computational burden. We illustrate our approach by applying our stopping rule to a single-variant analysis of a whole-genome sequencing study for lung function.

Original languageEnglish (US)
JournalGenetic epidemiology
DOIs
StateAccepted/In press - Jan 1 2019

Fingerprint

Biological Science Disciplines
Uncertainty
Genome
Lung
Datasets

Keywords

  • association p-value
  • next-generation sequencing
  • permutation
  • randomized test
  • sequential testing

ASJC Scopus subject areas

  • Epidemiology
  • Genetics(clinical)

Cite this

A flexible and nearly optimal sequential testing approach to randomized testing : QUICK-STOP. / Hecker, Julian; Ruczinski, Ingo; Cho, Michael H.; Silverman, Edwin K.; Coull, Brent; Lange, Christoph.

In: Genetic epidemiology, 01.01.2019.

Research output: Contribution to journalArticle

Hecker, Julian ; Ruczinski, Ingo ; Cho, Michael H. ; Silverman, Edwin K. ; Coull, Brent ; Lange, Christoph. / A flexible and nearly optimal sequential testing approach to randomized testing : QUICK-STOP. In: Genetic epidemiology. 2019.
@article{8754a296e69e41c3a6faef1d50b99b17,
title = "A flexible and nearly optimal sequential testing approach to randomized testing: QUICK-STOP",
abstract = "In the analysis of current life science datasets, we often encounter scenarios in which the application of asymptotic theory to hypothesis testing can be problematic. Besides improved asymptotic results, permutation/simulation-based tests are a general approach to address this issue. However, these randomized tests can impose a massive computational burden, for example, in scenarios in which large numbers of statistical tests are computed, and the specified significance level is very small. Stopping rules aim to assess significance with the smallest possible number of draws while controlling the probabilities of errors due to statistical uncertainty. In this communication, we derive a general stopping rule, QUICK-STOP, based on the sequential testing theory that is easy to implement, controls the error probabilities rigorously, and is nearly optimal in terms of expected draws. In a simulation study, we show that our approach outperforms current stopping approaches for general randomized tests by factor 10 and does not impose an additional computational burden. We illustrate our approach by applying our stopping rule to a single-variant analysis of a whole-genome sequencing study for lung function.",
keywords = "association p-value, next-generation sequencing, permutation, randomized test, sequential testing",
author = "Julian Hecker and Ingo Ruczinski and Cho, {Michael H.} and Silverman, {Edwin K.} and Brent Coull and Christoph Lange",
year = "2019",
month = "1",
day = "1",
doi = "10.1002/gepi.22268",
language = "English (US)",
journal = "Genetic Epidemiology",
issn = "0741-0395",
publisher = "Wiley-Liss Inc.",

}

TY - JOUR

T1 - A flexible and nearly optimal sequential testing approach to randomized testing

T2 - QUICK-STOP

AU - Hecker, Julian

AU - Ruczinski, Ingo

AU - Cho, Michael H.

AU - Silverman, Edwin K.

AU - Coull, Brent

AU - Lange, Christoph

PY - 2019/1/1

Y1 - 2019/1/1

N2 - In the analysis of current life science datasets, we often encounter scenarios in which the application of asymptotic theory to hypothesis testing can be problematic. Besides improved asymptotic results, permutation/simulation-based tests are a general approach to address this issue. However, these randomized tests can impose a massive computational burden, for example, in scenarios in which large numbers of statistical tests are computed, and the specified significance level is very small. Stopping rules aim to assess significance with the smallest possible number of draws while controlling the probabilities of errors due to statistical uncertainty. In this communication, we derive a general stopping rule, QUICK-STOP, based on the sequential testing theory that is easy to implement, controls the error probabilities rigorously, and is nearly optimal in terms of expected draws. In a simulation study, we show that our approach outperforms current stopping approaches for general randomized tests by factor 10 and does not impose an additional computational burden. We illustrate our approach by applying our stopping rule to a single-variant analysis of a whole-genome sequencing study for lung function.

AB - In the analysis of current life science datasets, we often encounter scenarios in which the application of asymptotic theory to hypothesis testing can be problematic. Besides improved asymptotic results, permutation/simulation-based tests are a general approach to address this issue. However, these randomized tests can impose a massive computational burden, for example, in scenarios in which large numbers of statistical tests are computed, and the specified significance level is very small. Stopping rules aim to assess significance with the smallest possible number of draws while controlling the probabilities of errors due to statistical uncertainty. In this communication, we derive a general stopping rule, QUICK-STOP, based on the sequential testing theory that is easy to implement, controls the error probabilities rigorously, and is nearly optimal in terms of expected draws. In a simulation study, we show that our approach outperforms current stopping approaches for general randomized tests by factor 10 and does not impose an additional computational burden. We illustrate our approach by applying our stopping rule to a single-variant analysis of a whole-genome sequencing study for lung function.

KW - association p-value

KW - next-generation sequencing

KW - permutation

KW - randomized test

KW - sequential testing

UR - http://www.scopus.com/inward/record.url?scp=85075003225&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85075003225&partnerID=8YFLogxK

U2 - 10.1002/gepi.22268

DO - 10.1002/gepi.22268

M3 - Article

C2 - 31713269

AN - SCOPUS:85075003225

JO - Genetic Epidemiology

JF - Genetic Epidemiology

SN - 0741-0395

ER -