• ↑
  • Limiting Distributions

Weak Law of Large Numbers

STA256

• $\{X_n\}$ is a sequence of iid random variables.

• Common $\mu$ and ${\sigma }^2<\mathrm{\infty }$

• Let $\overline{X}=\frac{1}{n}\sum _{i=1}^n{X}_i$ (sample mean)

• Then $X\stackrel{P}{\to }\mu$

  • $\mu$ is the average of everything (population mean)

• Example:

  • Average time taken for students to commute to UTM
  • $\sim 30000$ students
  • What's $\mu =?$
  • Asking all 30k students, won't really work.
  • Find a subset (sample) which is representative of the whole
  • $n=100$
  • For our subset
  • We can use $\overline{X}$ to determine for the main set.
  • As the sample size increases, the sample mean $\overline{X}$ becomes a reliable estimate of the population mean.

• Proof

  • Sample mean converges to the population mean
  • Chebyshev's Inequality
  • $P(|X-\mu |\ge k\sigma )\le \frac{1}{k^2}$
  • We have $X_1,X_2,\dots ,X_n$ be iid with $\mu$ and finite variance ${\sigma }^2<\mathrm{\infty }$.
  • Let $\overline{X}=\frac{1}{n}\sum _{i=1}^n{X}_i$
    • Mean: $\mathrm{E}[\overline{X}]=\mu ={\mu }_{\overline{X}}$
    • Variance: $\mathrm{Var}(\overline{X})=\frac{{\sigma }^2}{n}={\sigma }_{\overline{X}}^2$
    • Was proven
  • Want to show $\overline{X_n}\stackrel{P}{\to }\mu$
    • $\underset{n\to \mathrm{\infty }}{lim}P[|X_n-X|\ge ϵ]=0$
    • $\underset{n\to \mathrm{\infty }}{lim}P[|X_n-X|<ϵ]=1$
  • We want $\overline{X_n}$ and $\mu$
    • $\underset{n\to \mathrm{\infty }}{lim}P(|\overline{X_n}-\mu |\ge ϵ)=0$
    • $\underset{n\to \mathrm{\infty }}{lim}P(|\overline{X_n}-\mu |<ϵ)=1$
    • $P(|X-\mu |\ge k\sigma )\le \frac{1}{k^2}$ chebyshev's inequality for $\overline{X_n}$ and $k>0$
    • $P(|\overline{X_n}-{\mu }_{\overline{X_n}}|>k{\sigma }_{\overline{X_n}})\le \frac{1}{k^2}$
    • Let $ϵ=k{\sigma }_{\overline{X_n}}$ and $k=\frac{ϵ}{{\sigma }_{\overline{X_n}}}$
    • $P(|\overline{X_n}-{\mu }_{\overline{X_n}}|>ϵ)\le \frac{1}{{\left({\displaystyle \frac{ϵ}{{\sigma }_{\overline{X_n}}}}\right)}^2}$
    • $P(|\overline{X_n}-{\mu }_{\overline{X_n}}|>ϵ)\le {\left(\frac{{\sigma }_{\overline{X_n}}}{ϵ}\right)}^2$
    • Resub back in the originals
    • $P(|\overline{X_n}-\mu |>ϵ)\le {\left(\frac{\frac{{\sigma }^2}{n}}{ϵ}\right)}^2$
    • $P(|\overline{X_n}-\mu |>ϵ)\le {\left(\frac{{\sigma }^2}{nϵ}\right)}^2$
    • $P(|\overline{X_n}-\mu |>ϵ)\le \frac{{\sigma }^2}{n{ϵ}^2}$
    • $\underset{n\to \mathrm{\infty }}{lim}P(|\overline{X_n}-\mu |>ϵ)\le \underset{n\to \mathrm{\infty }}{lim}\frac{{\sigma }^2}{n{ϵ}^2}$
    • $\underset{n\to \mathrm{\infty }}{lim}P(|\overline{X_n}-\mu |>ϵ)\le 0$
    • Since we can't have a negative probability then we have that $\underset{n\to \mathrm{\infty }}{lim}P(|\overline{X_n}-\mu |>ϵ)=0$
    • We wanted to show this, so $\overline{X_n}\stackrel{P}{\to }\mu$

STA258

• You have $X_1,\dots ,X_n$ be a sequence of iid RVs with $\mathrm{E}[X_i]=\mu$ and $\mathrm{Var}(X_i)={\sigma }^2<\mathrm{\infty }$
• $\overline{X}\stackrel{P}{\to }\mu$
• The Sample Mean converges in probability to the population mean.