Stochastic Approximation

Also found in: Acronyms, Wikipedia.

Stochastic Approximation


a method of solving a broad class of problems in statistical estimation. In this method, each successive estimate is obtained in the form of a correction of the preceding estimate, the correction being based solely on new observations. The principal features responsible for the popularity of stochastic approximation in both theoretical and applied work are the method’s nonparametric character (its applicability when the information available on the object of observation is scanty) and its recursive nature (simplicity of recalculation of the estimate when new observation results are obtained). Stochastic approximation is used in many applied problems in control theory, instruction theory, and in problems of engineering, biology, and medicine.

Stochastic approximation was introduced in 1951 by the American statisticians H. Robbins and S. Monro. They set forth a recursion scheme for finding the root of a regression equation—that is, the root α of the equation R(x) = α where each measured value yk of the function R(x) at a point xk contains a random error. The Robbins-Monro procedure is given by the formula xk+ y = × k + a k (y k – α). Under certain conditions on R(x), on the sequence ak approaching zero, and on the character of the random errors, it has been proved that xk → θ as k increases.

The method of stochastic approximation was subsequently applied to other problems, such as the finding of the maximum of a regression function and the estimation of the unknown parameters of a distribution on the basis of observations. Investigation of the limiting distribution of the normalized difference xk – 8 has led to the construction of asymptotically optimal stochastic approximation procedures, in which the sequence ak must be chosen independence on the observations.


Wasan, M. Stokhasiicheskaia approksimatsiia. Moscow, 1972. (Translated from English.)
Nevel’son, M. B., and R. Z. Khas’minskii. Stokhasiicheskaia approksimatsiia i rekurrentnoe otsenivanie. Moscow, 1972.


References in periodicals archive ?
For comparison, the results using the stochastic approximation method (10) are also included.
Unlike the stochastic approximation approach (10), which relies on determining the assigned values by an iterative process involving many tests and calculations, the described approach is simpler and provides for at least as good calibrator uncertainty.
A Stochastic Approximation Method for Assigning Values to Calibrators.
00 (a) Prealbumin level 8 (highest-level) Internal Working Calibrator assigned-value estimates by instrument for the BCI value-assignment method and the stochastic approximation method (10).
In this paper, there are two distinct but related ways of using the stochastic approximation method.
Because there are two aforementioned ways of using the stochastic approximation method of value assignment, the standard which is iteratively adjusted (whether it be prepared from the reference or from calibrator bulk material) will be denoted as the "adjusted standard".
The following assumptions are made with the stochastic approximation method.
The stochastic approximation algorithm for updating the analyte concentration correction factor in the preparation of the adjusted standard is given by (3, 4):
A Stopping Rule for the Stochastic Approximation Algorithm
For each iteration i of the stochastic approximation algorithm, the following limits of a two-sided 95% confidence interval are first constructed (6):
The following is one possible stopping rule for the stochastic approximation algorithm:
Obtaining Initial Estimates of the Value Assignment and Calibration Curve Slope for the Stochastic Approximation Algorithm