Abstract
Often it is necessary to estimate the parameters of a model or unknown system. Various techniques exist to accomplish this task, including Kalman and Wiener filtering, least-mean-square (LMS) algorithms, and the Levenberg-Marquardt(L-M) algorithm. These techniques require an analytic form of the gradient of the function of the parameters to be estimated. A key feature of the simultaneous perturbation stochastic approximation (SPSA) method is that it is a gradient-free optimization technique (Spall; 1992,1998a,b, 1999). In the current problem, the function of parameters to be identified is highly non-linear and of sufficient difficulty that obtaining an analytic form of the gradient is impractical. Therefore, in this paper the performance of the SPSA algorithm will be examined in terms of parameter selection, data requirements, and convergence performance on this non-linear problem. Results will be reported on both a first-order "standard" implementation of SPSA and on a second-order version of SPSA that tends to enhance convergence.
| Original language | English (US) |
|---|---|
| Pages (from-to) | 1247-1256 |
| Number of pages | 10 |
| Journal | Communications in Statistics - Theory and Methods |
| Volume | 29 |
| Issue number | 5-6 |
| State | Published - 2000 |
Fingerprint
Keywords
- Optimization
- SPSA
ASJC Scopus subject areas
- Safety, Risk, Reliability and Quality
- Statistics and Probability
Cite this
Parameter estimation in a highly non-linear model using simultaneous perturbation stochastic approximation. / Whitney, James E.; Duncan, Kerron; Richardson, Maria; Bankman, Isaac.
In: Communications in Statistics - Theory and Methods, Vol. 29, No. 5-6, 2000, p. 1247-1256.Research output: Contribution to journal › Article
}
TY - JOUR
T1 - Parameter estimation in a highly non-linear model using simultaneous perturbation stochastic approximation
AU - Whitney, James E.
AU - Duncan, Kerron
AU - Richardson, Maria
AU - Bankman, Isaac
PY - 2000
Y1 - 2000
N2 - Often it is necessary to estimate the parameters of a model or unknown system. Various techniques exist to accomplish this task, including Kalman and Wiener filtering, least-mean-square (LMS) algorithms, and the Levenberg-Marquardt(L-M) algorithm. These techniques require an analytic form of the gradient of the function of the parameters to be estimated. A key feature of the simultaneous perturbation stochastic approximation (SPSA) method is that it is a gradient-free optimization technique (Spall; 1992,1998a,b, 1999). In the current problem, the function of parameters to be identified is highly non-linear and of sufficient difficulty that obtaining an analytic form of the gradient is impractical. Therefore, in this paper the performance of the SPSA algorithm will be examined in terms of parameter selection, data requirements, and convergence performance on this non-linear problem. Results will be reported on both a first-order "standard" implementation of SPSA and on a second-order version of SPSA that tends to enhance convergence.
AB - Often it is necessary to estimate the parameters of a model or unknown system. Various techniques exist to accomplish this task, including Kalman and Wiener filtering, least-mean-square (LMS) algorithms, and the Levenberg-Marquardt(L-M) algorithm. These techniques require an analytic form of the gradient of the function of the parameters to be estimated. A key feature of the simultaneous perturbation stochastic approximation (SPSA) method is that it is a gradient-free optimization technique (Spall; 1992,1998a,b, 1999). In the current problem, the function of parameters to be identified is highly non-linear and of sufficient difficulty that obtaining an analytic form of the gradient is impractical. Therefore, in this paper the performance of the SPSA algorithm will be examined in terms of parameter selection, data requirements, and convergence performance on this non-linear problem. Results will be reported on both a first-order "standard" implementation of SPSA and on a second-order version of SPSA that tends to enhance convergence.
KW - Optimization
KW - SPSA
UR - http://www.scopus.com/inward/record.url?scp=18844369048&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=18844369048&partnerID=8YFLogxK
M3 - Article
AN - SCOPUS:18844369048
VL - 29
SP - 1247
EP - 1256
JO - Communications in Statistics - Theory and Methods
JF - Communications in Statistics - Theory and Methods
SN - 0361-0926
IS - 5-6
ER -