# Limit Theorems

## Limit Theorems

in probability theory, a group of theorems that give the conditions governing the appearance of specific regularities as a result of the action of a large number of random factors.

Historically, the first limit theorems were Bernoulli’s theorem, which was set forth in 1713, and the Laplace theorem, which was published in 1812. These theorems are concerned with the distribution of the deviations of the frequency of occurrence of an event *E* in *n* independent trials from the event’s probability *p*, where 0 < *p* < 1. The frequency is defined as the ratio *m/n*, where *m* is the number of occurrences of *E* in *n* trials. In 1837, *S*. Poisson extended the theorems to the case where the probability *pk* of the occurrence of *E* in the *k*th trial may depend on *k*; he described the limiting behavior, as *n* → ∞, of the distribution of the deviations of the frequency *m/n* from the arithmetic mean *p̄* of the probabilities *pk* (1 ≤ *k* ≤ *n*):

If *X*_{k} denotes a random variable whose value is equal to unity when *E* occurs in the *k*th trial and is equal to zero when *E* does not occur, then *m* can be represented as the sum

*m* = *X*_{1} + *X*_{2} + ··· + *X _{n}*

which permits the above theorems to be treated as special cases of general limit theorems dealing with sums of independent random variables—the law of large numbers and the central limit theorem.

*Law of large numbers*. Suppose

(*) *X*_{1}, *X*_{2}, …, *X _{n}*, …

is a sequence of independent random variables and *s _{n}* is the sum of the first

*n*of the variables:

*s _{n}* =

*X*

_{1}+

*X*

_{2}+ · · · +

*X*

_{n}Let *A _{n}* and

*B*

_{n}^{2}be respectively the mathematical expectation

*A _{n}* = E

*s*= E

_{n}*X*

_{1}+ E

*X*

_{2}+ · · · + E

*X*

_{n}

and the variance

of the sum *s _{n}*. The sequence (*) is said to obey the law of large numbers if for any ∊ > 0 the probability of the inequality

approaches zero as *n* → ∞.

Broad conditions for the applicability of the law of large numbers were first obtained by P. L. Chebyshev in 1867. The conditions were subsequently generalized by A. A. Markov (the elder). The problem of the necessary and sufficient conditions for the applicability of the law of large numbers was definitively solved by A. N. Kolmogorov in 1928. When the *X*_{n} have a common distribution function, these conditions, as shown by A. Ia. Khinchin in 1929, reduce to a single condition: the *X*_{n} must have finite mathematical expectations.

*Central limit theorem*. The central limit theorem is applicable to the sequence (*) if for any *z*_{1} and *z*_{2} the limit of the probability of the inequality

*Z*_{1}*B*_{n} < *s*_{n} - *A*_{n} < *z*_{2}*B*_{n}

as *n* → ∞ is the quantity

Fairly general sufficient conditions for the applicability of the central limit theorem were given by Chebyshev in 1887. Gaps, however, were found in his proof that were filled only later by Markov, in 1898. An almost definitive solution of the problem was obtained by A. M. Liapunov in 1901. A precise formulation of the Liapunov theorem is as follows. Let

*c*_{k} = E ǀ*X _{k}* – E

*X*ǀ

_{k}^{2+δ}δ > 0

*C*_{n} = *c*_{1} + *c*_{2} + ··· + *c _{n}*

If the ratio *L _{n}* =

*C*/

_{n}*B*

_{n}

^{2+δ}approaches zero as

*n*→ ∞, then the central limit theorem is applicable to the sequence (*). A definite solution of the problem of the conditions of applicability of the central limit theorem was obtained in its general outlines by S. N. Bernshtein in 1926 and completed by W. Feller in 1935.

Other trends in the study of limit theorems include the following.

(1) Investigations were begun by Markov and continued by Bernshtein and others on the conditions governing the applicability of the law of large numbers and the central limit theorem to sums of dependent variables.

(2) Even in the case of a sequence of identically distributed random variables simple examples can be given where the sums have a limiting distribution different from a normal distribution (nondegenerate distributions are meant here—that is, distributions that are not concentrated entirely at one point). In the papers of such mathematicians as the Soviets A. Ia. Khinchin and B. V. Gnedenko and the Frenchmen P. Lévy and W. Dublin, a complete study has been made of both the class of possible limiting distributions for sums of independent random variables and the conditions for convergence of the distributions of the sums to specific limiting distributions.

(3) Considerable attention has been devoted to local limit theorems. For example, let the variables *X _{n}* take on only integral values. Then the sums

*s*also assume only integral values, and the question naturally arises of the limiting behavior of the probabilities

_{n}*P*(

_{n}*m*) that

*s*=

_{n}*m*, where

*m*is an integer. A simple example of a local limit theorem is the Laplace theorem.

(4) Limit theorems in their classical formulation describe the behavior of individual sums *s _{n}* as

*n*increases. Sufficiently general limit theorems for the probabilities of events that depend simultaneously on several sums were obtained for the first time by Kolmogorov in 1931. It follows from his results, for example, that under very broad conditions the limit of the probability of the inequality

is the quantity

(5) The above limit theorems are concerned with sums of random variables. Limit theorems of a different kind are exemplified by limit theorems for the terms of an ordered series. Such theorems have been studied in detail by the Soviet mathematicians B. V. Gnedenko and N. V. Smirnov.

(6) Finally, also classed with limit theorems are theorems that establish for sequences of random variables properties occurring with a probability equal to unity. An example is the law of iterated logarithm.

### REFERENCES

Gnedenko, B. V., and A. N. Kolmogorov.*Predel’nye raspredeleniia dlia summ nezavisimykh sluchainykh velichin*. Moscow-Leningrad, 1949.

Ibragimov, I. A., and Iu. V. Linnik.

*Nezavisimye i statsionarno sviazannye velichiny*. Moscow, 1965.

Prokhorov, Iu. V., and Iu. A. Rozanov.

*Teoriia veroiatnostei: Osnovnye poniatiia: Predel’nye teoremy: Sluchainye protsessy*, 2nd ed. Moscow, 1973.

IU. V. PROKHOROV