Comparison of kernel based PDF estimation methods

David E. Freund, Philippe Burlina, Amit Banerjee, Erik Justen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

There are a number of challenging estimation, tracking, and decision theoretic problems that require the estimation of Probability Density Functions (PDFs). When using a traditional parametric approach, the functional model of the PDF is assumed to be known. However, these models often do not capture the complexity of the underlying distribution. Furthermore, the problems of validating the model and estimating its parameters are often complicated by the sparsity of prior examples. The need for exemplars grows exponentially with the dimension of the feature space. These methods may yield PDFs that do not generalize well to unseen data because these tend to overfit or underfit the training exemplars. We investigate and compare alternate approaches for estimating a PDF and consider instead kernel based estimation methods which generalize the Parzen estimator and use a Linear Mixture of Kernels (LMK) model. The methods reported here are derived from machine learning methods such as the Support Vector Machines and the Relevance Vector Machines. These PDF estimators provide the following benefits: (a) they are data driven; (b) they do not overfit the data and consequently have good generalization properties; (c) they can accommodate highly irregular and multi-modal data distributions; (d) they provide a sparse and succinct description of the underlying data which leads to efficient computation and communication. Comparative experimental results are provided illustrating these properties using simulated Mixture of Gaussian-distributed data. ^

Original languageEnglish (US)
Title of host publicationAutomatic Target Recognition XIX
PublisherSPIE
Volume7335
ISBN (Print)9780819476012
DOIs
StatePublished - Jan 1 2009
EventAutomatic Target Recognition XIX - Orlando, FL, United States
Duration: Apr 13 2009Apr 14 2009

Other

OtherAutomatic Target Recognition XIX
CountryUnited States
CityOrlando, FL
Period4/13/094/14/09

Fingerprint

Function Estimation
Density Estimation
probability density functions
Probability density function
kernel
estimators
estimating
Relevance Vector Machine
Estimator
machine learning
Generalise
Functional Model
Data Distribution
Feature Space
Sparsity
Data-driven
Alternate
Support vector machines
Learning systems
Irregular

Keywords

  • Anomaly detection
  • Compression
  • Kernel methods
  • Parzen
  • PDF estimation
  • RVM
  • SVDD
  • SVM

ASJC Scopus subject areas

  • Electronic, Optical and Magnetic Materials
  • Condensed Matter Physics
  • Computer Science Applications
  • Applied Mathematics
  • Electrical and Electronic Engineering

Cite this

Freund, D. E., Burlina, P., Banerjee, A., & Justen, E. (2009). Comparison of kernel based PDF estimation methods. In Automatic Target Recognition XIX (Vol. 7335). [733508] SPIE. https://doi.org/10.1117/12.819115

Comparison of kernel based PDF estimation methods. / Freund, David E.; Burlina, Philippe; Banerjee, Amit; Justen, Erik.

Automatic Target Recognition XIX. Vol. 7335 SPIE, 2009. 733508.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Freund, DE, Burlina, P, Banerjee, A & Justen, E 2009, Comparison of kernel based PDF estimation methods. in Automatic Target Recognition XIX. vol. 7335, 733508, SPIE, Automatic Target Recognition XIX, Orlando, FL, United States, 4/13/09. https://doi.org/10.1117/12.819115
Freund DE, Burlina P, Banerjee A, Justen E. Comparison of kernel based PDF estimation methods. In Automatic Target Recognition XIX. Vol. 7335. SPIE. 2009. 733508 https://doi.org/10.1117/12.819115
Freund, David E. ; Burlina, Philippe ; Banerjee, Amit ; Justen, Erik. / Comparison of kernel based PDF estimation methods. Automatic Target Recognition XIX. Vol. 7335 SPIE, 2009.
@inproceedings{5a167bd242a14c1b902d034f5c89c9f2,
title = "Comparison of kernel based PDF estimation methods",
abstract = "There are a number of challenging estimation, tracking, and decision theoretic problems that require the estimation of Probability Density Functions (PDFs). When using a traditional parametric approach, the functional model of the PDF is assumed to be known. However, these models often do not capture the complexity of the underlying distribution. Furthermore, the problems of validating the model and estimating its parameters are often complicated by the sparsity of prior examples. The need for exemplars grows exponentially with the dimension of the feature space. These methods may yield PDFs that do not generalize well to unseen data because these tend to overfit or underfit the training exemplars. We investigate and compare alternate approaches for estimating a PDF and consider instead kernel based estimation methods which generalize the Parzen estimator and use a Linear Mixture of Kernels (LMK) model. The methods reported here are derived from machine learning methods such as the Support Vector Machines and the Relevance Vector Machines. These PDF estimators provide the following benefits: (a) they are data driven; (b) they do not overfit the data and consequently have good generalization properties; (c) they can accommodate highly irregular and multi-modal data distributions; (d) they provide a sparse and succinct description of the underlying data which leads to efficient computation and communication. Comparative experimental results are provided illustrating these properties using simulated Mixture of Gaussian-distributed data. ^",
keywords = "Anomaly detection, Compression, Kernel methods, Parzen, PDF estimation, RVM, SVDD, SVM",
author = "Freund, {David E.} and Philippe Burlina and Amit Banerjee and Erik Justen",
year = "2009",
month = "1",
day = "1",
doi = "10.1117/12.819115",
language = "English (US)",
isbn = "9780819476012",
volume = "7335",
booktitle = "Automatic Target Recognition XIX",
publisher = "SPIE",

}

TY - GEN

T1 - Comparison of kernel based PDF estimation methods

AU - Freund, David E.

AU - Burlina, Philippe

AU - Banerjee, Amit

AU - Justen, Erik

PY - 2009/1/1

Y1 - 2009/1/1

N2 - There are a number of challenging estimation, tracking, and decision theoretic problems that require the estimation of Probability Density Functions (PDFs). When using a traditional parametric approach, the functional model of the PDF is assumed to be known. However, these models often do not capture the complexity of the underlying distribution. Furthermore, the problems of validating the model and estimating its parameters are often complicated by the sparsity of prior examples. The need for exemplars grows exponentially with the dimension of the feature space. These methods may yield PDFs that do not generalize well to unseen data because these tend to overfit or underfit the training exemplars. We investigate and compare alternate approaches for estimating a PDF and consider instead kernel based estimation methods which generalize the Parzen estimator and use a Linear Mixture of Kernels (LMK) model. The methods reported here are derived from machine learning methods such as the Support Vector Machines and the Relevance Vector Machines. These PDF estimators provide the following benefits: (a) they are data driven; (b) they do not overfit the data and consequently have good generalization properties; (c) they can accommodate highly irregular and multi-modal data distributions; (d) they provide a sparse and succinct description of the underlying data which leads to efficient computation and communication. Comparative experimental results are provided illustrating these properties using simulated Mixture of Gaussian-distributed data. ^

AB - There are a number of challenging estimation, tracking, and decision theoretic problems that require the estimation of Probability Density Functions (PDFs). When using a traditional parametric approach, the functional model of the PDF is assumed to be known. However, these models often do not capture the complexity of the underlying distribution. Furthermore, the problems of validating the model and estimating its parameters are often complicated by the sparsity of prior examples. The need for exemplars grows exponentially with the dimension of the feature space. These methods may yield PDFs that do not generalize well to unseen data because these tend to overfit or underfit the training exemplars. We investigate and compare alternate approaches for estimating a PDF and consider instead kernel based estimation methods which generalize the Parzen estimator and use a Linear Mixture of Kernels (LMK) model. The methods reported here are derived from machine learning methods such as the Support Vector Machines and the Relevance Vector Machines. These PDF estimators provide the following benefits: (a) they are data driven; (b) they do not overfit the data and consequently have good generalization properties; (c) they can accommodate highly irregular and multi-modal data distributions; (d) they provide a sparse and succinct description of the underlying data which leads to efficient computation and communication. Comparative experimental results are provided illustrating these properties using simulated Mixture of Gaussian-distributed data. ^

KW - Anomaly detection

KW - Compression

KW - Kernel methods

KW - Parzen

KW - PDF estimation

KW - RVM

KW - SVDD

KW - SVM

UR - http://www.scopus.com/inward/record.url?scp=69649095329&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=69649095329&partnerID=8YFLogxK

U2 - 10.1117/12.819115

DO - 10.1117/12.819115

M3 - Conference contribution

AN - SCOPUS:69649095329

SN - 9780819476012

VL - 7335

BT - Automatic Target Recognition XIX

PB - SPIE

ER -