Law of large numbers

In probability theory, the weak law of large numbers states that if X₁, X₂, X₃, ... is an infinite sequence of random variables, all of which have the same expected value μ and the same finite variance σ², and they are uncorrelated (i.e., the correlation between any two of them is zero), then the sample average

converges in probability to μ. Somewhat less tersely: For any positive number ε, no matter how small, we have

Chebyshev's inequality is used to prove this result.

A consequence of the weak law of large numbers is the asymptotic equipartition property.

The strong law of large numbers states that if X₁, X₂, X₃, ... is an infinite sequence of random variables that are independent and identically distributed, and have a common expected value μ then

i.e., the sample average converges almost surely to μ.

This law justifies the intuitive interpretation of the expected value of a random variable as the "long-term average when sampling repeatedly".