Simple and globally convergent methods for accelerating the convergence of any em algorithm

Ravi Varadhan, Christophe Roland

Research output: Contribution to journalArticlepeer-review

Abstract

The expectation-maximization (EM) algorithm is a popular approach for obtaining maximum likelihood estimates in incomplete data problems because of its simplicity and stability (e.g. monotonic increase of likelihood). However, in many applications the stability of EM is attained at the expense of slow, linear convergence. We have developed a new class of iterative schemes, called squared iterative methods (SQUAREM), to accelerate EM, without compromising on simplicity and stability. SQUAREM generally achieves superlinear convergence in problems with a large fraction of missing information. Globally convergent schemes are easily obtained by viewing SQUAREM as a continuation of EM. SQUAREM is especially attractive in high-dimensional problems, and in problems where model-specific analytic insights are not available. SQUAREM can be readily implemented as an 'off-the-shelf' accelerator of any EM-type algorithm, as it only requires the EM parameter updating. We present four examples to demonstrate the effectiveness of SQUAREM. A general-purpose implementation (written in R) is available.

Original languageEnglish (US)
Pages (from-to)335-353
Number of pages19
JournalScandinavian Journal of Statistics
Volume35
Issue number2
DOIs
StatePublished - Jun 2008

Keywords

  • Causal inference
  • Conjugate gradient
  • EM acceleration
  • Finite mixtures
  • Fixed point iteration
  • Quasi-Newton
  • Squared iterative method

ASJC Scopus subject areas

  • Statistics and Probability
  • Statistics, Probability and Uncertainty

Fingerprint Dive into the research topics of 'Simple and globally convergent methods for accelerating the convergence of any em algorithm'. Together they form a unique fingerprint.

Cite this