• ↑
  • Limiting Distributions

Convergence in Probability

• Not as strong as Convergence Almost Surely
• $X_n\stackrel{\text{P}}{\to }X$
• Weak Law of Large Numbers
• Theorem
  • $X_n\stackrel{P}{\to }X$ and $Y_n\stackrel{P}{\to }Y$ then $X_n+Y_n\stackrel{P}{\to }X+Y$
  • $X_n\stackrel{P}{\to }X$ and $a$ is a constant then $a{X}_n\stackrel{P}{\to }aX$
  • $X_n\stackrel{P}{\to }X$ and $Y_n\stackrel{P}{\to }Y$ then $X_n{Y}_n\stackrel{P}{\to }XY$
  • $X_n\stackrel{P}{\to }a$. $a$ is a constant, if you have a continuous function $g$ then $g(X_n)\stackrel{P}{\to }g(a)$
    • Continuous Mapping Theorem
• Example:
  • Random variable $X_n$ has a distribution $\mathrm{Bin}(n,p)$
  • Show that $\frac{X_n}{n}\stackrel{P}{\to }p$
  • $X\sim \mathrm{Bin}(n,p)$
  • See that the mean is ${\mu }_{X_n}=np$ and $\mathrm{Var}(X)=np(1-p)$
  • We want $\frac{X_n}{n}$
  • So ${\mu }_{\frac{X_n}{n}}=\mathrm{E}\left[\frac{X_n}{n}\right]$
  • ${\mu }_{\frac{X_n}{n}}=\frac{1}{n}\mathrm{E}[X_n]$
  • ${\mu }_{\frac{X_n}{n}}=\frac{1}{n}np=p$
  • $\mathrm{Var}(X_n)=np(1-p)$
  • $\mathrm{Var}\left(\frac{X_n}{n}\right)=\frac{1}{n^2}\mathrm{Var}(X_n)$
  • $\mathrm{Var}\left(\frac{X_n}{n}\right)=\frac{1}{n^2}(np(1-p))=\frac{np(1-p)}{n^2}=\frac{p(1-p)}{n}$
  • Chebyshev's Inequality
  • $P(|X-\mu |\ge k\sigma )\le \frac{1}{k^2}$
  • Here we have $P(|\frac{X_n}{n}-{\mu }_{\frac{X_n}{n}}|>k{\sigma }_{\frac{X_n}{n}})\le \frac{1}{k^2}$ for $k>0$
  • $P(|\frac{X_n}{n}-{\mu }_{\frac{X_n}{n}}|>k{\sigma }_{\frac{X_n}{n}})\le \frac{1}{k^2}$
  • $ϵ=k{\sigma }_{\frac{X_n}{n}}$
  • $P(|\frac{X_n}{n}-{\mu }_{\frac{X_n}{n}}|>ϵ)\le \frac{1}{k^2}$
  • $k=\frac{ϵ}{{\sigma }_{\frac{X_n}{n}}}$
  • $P(|\frac{X_n}{n}-{\mu }_{\frac{X_n}{n}}|>ϵ)\le \frac{1}{{\left(\frac{ϵ}{{\sigma }_{\frac{X_n}{n}}}\right)}^2}$
  • $P(|\frac{X_n}{n}-{\mu }_{\frac{X_n}{n}}|>ϵ)\le \frac{1}{\frac{{ϵ}^2}{{\sigma }_{\frac{X_n}{n}}^2}}$
  • $P(|\frac{X_n}{n}-{\mu }_{\frac{X_n}{n}}|>ϵ)\le \frac{{\sigma }_{\frac{X_n}{n}}^2}{{ϵ}^2}$
  • Back subtitute
  • $P(|\frac{X_n}{n}-p|>ϵ)\le \frac{\frac{p(1-p)}{n}}{{ϵ}^2}$
  • $P(|\frac{X_n}{n}-p|>ϵ)\le \frac{p(1-p)}{n{ϵ}^2}$
  • $\underset{n\to \mathrm{\infty }}{lim}P(|\frac{X_n}{n}-p|>ϵ)\le \underset{n\to \mathrm{\infty }}{lim}\frac{p(1-p)}{n{ϵ}^2}$
  • $\underset{n\to \mathrm{\infty }}{lim}P(|\frac{X_n}{n}-p|>ϵ)\le 0$
  • probability cannot be negative
  • $\underset{n\to \mathrm{\infty }}{lim}P(|\frac{X_n}{n}-p|>ϵ)=0$
  • So this shows us that $\frac{X_n}{n}\stackrel{P}{\to }p$
• Example:
  • #tk
  • Slide 8/22
• Example:
  • Was on final exam
  • Weak Law of Large Numbers
    • $\overline{X}\stackrel{P}{\to }\mu$
    • Sample mean converges to the population mean
  • Show $S^2=\frac{1}{n-1}\sum _{i=1}^n(X_i-\overline{X}{)}^2$ converges to ${\sigma }^2$
  • Want to show that $S^2\stackrel{P}{\to }{\sigma }^2$
  • Since mean is
    • Sample: $\overline{X}=\frac{1}{n}\sum _{i=1}^n{X}_i$
    • Population: $\mu =\frac{1}{N}\sum _{i=1}^n{X}_i$
  • Variance is
    • Sample: $S^2=\frac{1}{n-1}\sum _{i=1}^n(X_i-\overline{X}{)}^2$
      • We learn why we have $n-1$ in STA260
    • Population: ${\sigma }^2=\frac{1}{N}\sum _{i=1}^n(X_i-\mu {)}^2$
  • Use WLLN, that $\overline{X_n}\stackrel{P}{\to }\mu$
  • Use Continuous Mapping Theorem, that $X_n\stackrel{P}{\to }a$ and $g$ is continuous, then $g(X_n)\stackrel{P}{\to }g(a)$
  • Use definition of variance, that $\mathrm{Var}(X)=\mathrm{E}[X^2]-\mathrm{E}[X{]}^2$
  • Use that $\mathrm{E}[S^2]={\sigma }^2$
  • $S^2=\frac{1}{n-1}\sum _{i=1}^n(X_i-\overline{X}{)}^2$
  • Expand
  • $S^2=\frac{1}{n-1}\sum _{i=1}^n(X_i^2-2\overline{X}{X}_i+{\overline{X}}^2)$
  • $S^2=\frac{1}{n-1}\sum _{i=1}^n{X}_i^2-\sum _{i=1}^{n}2\overline{X}{X}_i+\sum _{i=1}^n{\overline{X}}^2$
  • $\overline{X}$ is a constant
  • $S^2=\frac{1}{n-1}\sum _{i=1}^n{X}_i^2-2\overline{X}\sum _{i=1}^n{X}_i+\sum _{i=1}^n{\overline{X}}^2$
  • $S^2=\frac{1}{n-1}\sum _{i=1}^n{X}_i^2-2\overline{X}\sum _{i=1}^n{X}_i+(\underset{n\text{ times}}{\underbrace{{\overline{X}}^2+{\overline{X}}^2+\cdots +{\overline{X}}^2}})$
  • $S^2=\frac{1}{n-1}\sum _{i=1}^n{X}_i^2-2\overline{X}\sum _{i=1}^n{X}_i+n{\overline{X}}^2$
    • $\overline{X}=\frac{1}{n}\sum _{i=1}^n{X}_i$
    • $n\overline{X}=\sum _{i=1}^n{X}_i$
  • $S^2=\frac{1}{n-1}[\sum _{i=1}^n{X}_i^2-2\overline{X}(n\overline{X})+n{\overline{X}}^2]$
  • $S^2=\frac{1}{n-1}\left[\sum _{i=1}^n{X}_i^2\underset{-2n{\overline{X}}^2}{\underbrace{-2\overline{X}(n\overline{X})}}\underset{n{\overline{X}}^2}{\underbrace{+n{\overline{X}}^2}}\right]$
  • $S^2=\frac{1}{n-1}[\sum _{i=1}^n{X}_i^2-n{\overline{X}}^2]$
  • $S^2=\frac{1}{n-1}[\left(\sum _{i=1}^n{X}_i^2\right)-n{\overline{X}}^2]=\frac{1}{n-1}\sum _{i=1}^n(X_i-\overline{X}{)}^2$
  • $S^2=\frac{1}{n-1}[\left(\sum _{i=1}^n{X}_i^2\right)-n{\overline{X}}^2]$
  • $S^2=\frac{1}{n-1}[\frac{n}{n}\left(\sum _{i=1}^n{X}_i^2\right)-n{\overline{X}}^2]$
  • $S^2=\frac{n}{n-1}[\frac{1}{n}\left(\sum _{i=1}^n{X}_i^2\right)-{\overline{X}}^2]$
  • By WLLN
  • $\overline{X_n}\stackrel{P}{\to }\mu$
  • $\frac{1}{n}\sum _{i=1}^n{X}_i\stackrel{P}{\to }\mu$
  • $\mu =\mathrm{E}[X]$
  • $\frac{1}{n}\sum _{i=1}^n{X}_i\stackrel{P}{\to }\mathrm{E}[X]$
  • $\boxed{{\displaystyle \frac{1}{n}\sum _{i=1}^n{X}_i^2\stackrel{P}{\to }\mathrm{E}[X^2]}}\boxed{{\displaystyle \mathbb{1}}}$
  • Continuous Mapping Theorem
  • $g(x)=x^2$
  • $\overline{X_n}\stackrel{P}{\to }\mu$
  • $g(\overline{X_n})\stackrel{P}{\to }g(\mu )$
  • $\boxed{{\displaystyle {\overline{X_n}}^2\stackrel{P}{\to }{\mu }^2}}\boxed{{\displaystyle \mathbb{2}}}$
  • By result $\mathbb{1}$, this quantity converges to $\mathrm{E}[X^2]$
  • By result $\mathbb{2}$, this quantity converges to ${\mu }^2$
  • By both results we have $S^2\stackrel{P}{\to }\frac{n}{n-1}[\mathrm{E}[X^2]-{\mu }^2]$
  • See that $\mathrm{Var}(X)=\mathrm{E}[X^2]-{\mu }^2$
  • So $S^2\stackrel{P}{\to }\frac{n}{n-1}{\sigma }^2$
  • $\underset{n\to \mathrm{\infty }}{lim}{S}^2\stackrel{P}{\to }\underset{n\to \mathrm{\infty }}{lim}\frac{n}{n-1}{\sigma }^2$
  • $\underset{n\to \mathrm{\infty }}{lim}{S}^2\stackrel{P}{\to }(1){\sigma }^2$
  • $\underset{n\to \mathrm{\infty }}{lim}{S}^2\stackrel{P}{\to }{\sigma }^2$
  • So $S^2\stackrel{P}{\to }{\sigma }^2$