A Direct Algorithm For Optimization Problems With The Huber Penalty

Jingyan Xu, Frederic Noo, Benjamin Tsui

Research output: Contribution to journalArticle

Abstract

We present a direct (noniterative) algorithm for one dimensional (1-D) quadratic data fitting with neighboring intensity differences penalized by the Huber function. Applications of such an algorithm include 1-D processing of medical signals, such as smoothing of tissue time concentration curves in kinetic data analysis or sinogram preprocessing, and using it as a subproblem solver for 2-D or 3-D image restoration and reconstruction. Dynamic programming (DP) was used to develop the direct algorithm. The problem was reformulated as a sequence of univariate optimization problems, for k = 1,…,N, where N is the number of data points. The solution to the univariate problem at index k is parameterized by the solution at k+1, except at k = N. Solving the univariate optimization problem at k = N yields the solution to each problem in the sequence using backtracking. Computational issues and memory cost are discussed in detail. Two numerical studies, tissue concentration curve smoothing and sinogram preprocessing for image reconstruction, are used to validate the direct algorithm and illustrate its practical applications. In the example of 1-D curve smoothing, the efficiency of the direct algorithm is compared with four iterative methods: the iterative coordinate descent, Nesterov’s accelerated gradient descent algorithm, FISTA, and an off-the-shelf second order method. The first two methods were applied to the primal problem, the others to the dual problem. The comparisons show that the direct algorithm outperforms all other methods by a significant factor, which rapidly grows with the curvature of the Huber function. The second example, sinogram preprocessing, showed that robustness and speed of the direct algorithm are maintained over a wide range of signal variations, and that noise and streaking artifacts could be reduced with almost no increase in computation time. We also outline the application of the proposed 1-D solver for imaging applications.

Original languageEnglish (US)
JournalIEEE Transactions on Medical Imaging
DOIs
StateAccepted/In press - Oct 4 2017

Fingerprint

Image reconstruction
Computer-Assisted Image Processing
Tissue
Three-Dimensional Imaging
Iterative methods
Dynamic programming
Artifacts
Noise
Efficiency
Imaging techniques
Data storage equipment
Costs and Cost Analysis
Kinetics
Processing
Costs

Keywords

  • Algorithm design and analysis
  • denoising
  • dynamic programming
  • Heuristic algorithms
  • Huber penalty
  • Imaging
  • Minimization
  • robust estimation
  • Signal processing algorithms
  • smoothing
  • Smoothing methods
  • total variation
  • TV

ASJC Scopus subject areas

  • Software
  • Radiological and Ultrasound Technology
  • Computer Science Applications
  • Electrical and Electronic Engineering

Cite this

A Direct Algorithm For Optimization Problems With The Huber Penalty. / Xu, Jingyan; Noo, Frederic; Tsui, Benjamin.

In: IEEE Transactions on Medical Imaging, 04.10.2017.

Research output: Contribution to journalArticle

@article{8163b255137b4fcd84ad1386381661fb,
title = "A Direct Algorithm For Optimization Problems With The Huber Penalty",
abstract = "We present a direct (noniterative) algorithm for one dimensional (1-D) quadratic data fitting with neighboring intensity differences penalized by the Huber function. Applications of such an algorithm include 1-D processing of medical signals, such as smoothing of tissue time concentration curves in kinetic data analysis or sinogram preprocessing, and using it as a subproblem solver for 2-D or 3-D image restoration and reconstruction. Dynamic programming (DP) was used to develop the direct algorithm. The problem was reformulated as a sequence of univariate optimization problems, for k = 1,…,N, where N is the number of data points. The solution to the univariate problem at index k is parameterized by the solution at k+1, except at k = N. Solving the univariate optimization problem at k = N yields the solution to each problem in the sequence using backtracking. Computational issues and memory cost are discussed in detail. Two numerical studies, tissue concentration curve smoothing and sinogram preprocessing for image reconstruction, are used to validate the direct algorithm and illustrate its practical applications. In the example of 1-D curve smoothing, the efficiency of the direct algorithm is compared with four iterative methods: the iterative coordinate descent, Nesterov’s accelerated gradient descent algorithm, FISTA, and an off-the-shelf second order method. The first two methods were applied to the primal problem, the others to the dual problem. The comparisons show that the direct algorithm outperforms all other methods by a significant factor, which rapidly grows with the curvature of the Huber function. The second example, sinogram preprocessing, showed that robustness and speed of the direct algorithm are maintained over a wide range of signal variations, and that noise and streaking artifacts could be reduced with almost no increase in computation time. We also outline the application of the proposed 1-D solver for imaging applications.",
keywords = "Algorithm design and analysis, denoising, dynamic programming, Heuristic algorithms, Huber penalty, Imaging, Minimization, robust estimation, Signal processing algorithms, smoothing, Smoothing methods, total variation, TV",
author = "Jingyan Xu and Frederic Noo and Benjamin Tsui",
year = "2017",
month = "10",
day = "4",
doi = "10.1109/TMI.2017.2760104",
language = "English (US)",
journal = "IEEE Transactions on Medical Imaging",
issn = "0278-0062",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - JOUR

T1 - A Direct Algorithm For Optimization Problems With The Huber Penalty

AU - Xu, Jingyan

AU - Noo, Frederic

AU - Tsui, Benjamin

PY - 2017/10/4

Y1 - 2017/10/4

N2 - We present a direct (noniterative) algorithm for one dimensional (1-D) quadratic data fitting with neighboring intensity differences penalized by the Huber function. Applications of such an algorithm include 1-D processing of medical signals, such as smoothing of tissue time concentration curves in kinetic data analysis or sinogram preprocessing, and using it as a subproblem solver for 2-D or 3-D image restoration and reconstruction. Dynamic programming (DP) was used to develop the direct algorithm. The problem was reformulated as a sequence of univariate optimization problems, for k = 1,…,N, where N is the number of data points. The solution to the univariate problem at index k is parameterized by the solution at k+1, except at k = N. Solving the univariate optimization problem at k = N yields the solution to each problem in the sequence using backtracking. Computational issues and memory cost are discussed in detail. Two numerical studies, tissue concentration curve smoothing and sinogram preprocessing for image reconstruction, are used to validate the direct algorithm and illustrate its practical applications. In the example of 1-D curve smoothing, the efficiency of the direct algorithm is compared with four iterative methods: the iterative coordinate descent, Nesterov’s accelerated gradient descent algorithm, FISTA, and an off-the-shelf second order method. The first two methods were applied to the primal problem, the others to the dual problem. The comparisons show that the direct algorithm outperforms all other methods by a significant factor, which rapidly grows with the curvature of the Huber function. The second example, sinogram preprocessing, showed that robustness and speed of the direct algorithm are maintained over a wide range of signal variations, and that noise and streaking artifacts could be reduced with almost no increase in computation time. We also outline the application of the proposed 1-D solver for imaging applications.

AB - We present a direct (noniterative) algorithm for one dimensional (1-D) quadratic data fitting with neighboring intensity differences penalized by the Huber function. Applications of such an algorithm include 1-D processing of medical signals, such as smoothing of tissue time concentration curves in kinetic data analysis or sinogram preprocessing, and using it as a subproblem solver for 2-D or 3-D image restoration and reconstruction. Dynamic programming (DP) was used to develop the direct algorithm. The problem was reformulated as a sequence of univariate optimization problems, for k = 1,…,N, where N is the number of data points. The solution to the univariate problem at index k is parameterized by the solution at k+1, except at k = N. Solving the univariate optimization problem at k = N yields the solution to each problem in the sequence using backtracking. Computational issues and memory cost are discussed in detail. Two numerical studies, tissue concentration curve smoothing and sinogram preprocessing for image reconstruction, are used to validate the direct algorithm and illustrate its practical applications. In the example of 1-D curve smoothing, the efficiency of the direct algorithm is compared with four iterative methods: the iterative coordinate descent, Nesterov’s accelerated gradient descent algorithm, FISTA, and an off-the-shelf second order method. The first two methods were applied to the primal problem, the others to the dual problem. The comparisons show that the direct algorithm outperforms all other methods by a significant factor, which rapidly grows with the curvature of the Huber function. The second example, sinogram preprocessing, showed that robustness and speed of the direct algorithm are maintained over a wide range of signal variations, and that noise and streaking artifacts could be reduced with almost no increase in computation time. We also outline the application of the proposed 1-D solver for imaging applications.

KW - Algorithm design and analysis

KW - denoising

KW - dynamic programming

KW - Heuristic algorithms

KW - Huber penalty

KW - Imaging

KW - Minimization

KW - robust estimation

KW - Signal processing algorithms

KW - smoothing

KW - Smoothing methods

KW - total variation

KW - TV

UR - http://www.scopus.com/inward/record.url?scp=85031768683&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85031768683&partnerID=8YFLogxK

U2 - 10.1109/TMI.2017.2760104

DO - 10.1109/TMI.2017.2760104

M3 - Article

C2 - 28981412

AN - SCOPUS:85031768683

JO - IEEE Transactions on Medical Imaging

JF - IEEE Transactions on Medical Imaging

SN - 0278-0062

ER -