• ↑
  • STA256 Lecture 17

Linear Combinations of Random Variables

STA256

• See theorem 23 #tk
  • Let $X_1,X_2,X_3$ be Random Variables
  • Let $a_1,a_2,a_3\in \mathbb{R}$
  • $E[a_1{X}_2+a_2{X}_2]=a_{1}E[X_1]+a_{2}E[X_2]$
    • Linear combination of two rvs
  • $var(a_1{X}_1+a_2{X}_2)=a_1^{2}var(X_1)+a_2^{2}var(X_2)+2a_1{a}_{2}cov(X_1,X_2)$
    • Example: $E[aX+b]=aE[X]+b$
    • $var(aX+b)=a^{2}var(x)$
    • $b$ doesn't mean anything, as $var(x)$ measures spread side to side.
    • If $X_1$ and $X_2$ are independent, we can simplify:
      • $var(a_1{X}_1+a_2{X}_2)=a_1^{2}var(X_1)+a_2^{2}var(X_2)$
  • $cov(a_1{X}_1,a_2{X}_2)=a_1{a}_{2}cov(X_1,X_2)$
  • $cov(a_1{X}_1,a_2{X}_2+a_3{X}_3)=a_1{a}_{2}cov(X_1{X}_2)+a_1{a}_{3}cov(X_1,X_3)$
• Example:
  • $cov(U,W)$
  • $U=X_1-2X_2-X_3$
  • $W=3X_1+X_2$
  • $cov(X_1-2X_2-X_3,3X_1+X_2)+cov(-X_3,3X_1+X2)$
  • $cov(X_1,3X_1+X_2)+cov(-2X_2,3X_1+X_2)+cov(-X_3,3X_1+X_2)$
  • $(3)(1)cov(X_1,X_1)+(1)(1)cov(X_1,X_2)+(-2)(3)cov(X_2,X_1)+(-2)(1)cov(X_2,X_2)+(-1)(3)cov(X_3,X_1)+(-1)(1)cov(X_3,X_2)$
  • We know that $cov(X_a,X_a)=var(X_a)$
  • #tk
  • Substitutions from slides
• Example:
  • $X_1\sim f(x_1)$
  • $X_2\sim f(x_2)$
  • $\dots$
  • $X_n\sim f(x_n)$
  • Same exactly distribution
  • They're also independent
  • This is called a random sample
  • IID: Independent and Identically Distributed
  • So $X_1,X_2,\dots ,X_n\underset{iid}{\underbrace{\sim }}f(x)$
• Example:
  • Sample Mean:
  • $X_n$ iid random variables
  • Sample mean is:
    • $\overline{X}=\frac{1}{n}\sum _{i=1}^n{X}_i=\frac{X_1+X_2+\cdots +X_n}{n}$
  • Population mean is:
    • $\mu =\int xf(x)dx$
  • $E(\overline{X})=E\left(\frac{1}{n}\sum _{i=1}^n{X}_i\right)=\mu$
    • $E(\overline{X})=E\left(\frac{1}{n}\sum _{i=1}^n{X}_i\right)$
    • $E(\overline{X})=\frac{1}{n}E\left(\sum _{i=1}^n{X}_i\right)$
      • By Linear Combinations of Random Variables
    • $E(\overline{X})=\frac{1}{n}[E(X_1+X_2+\cdots +X_n)]$
    • $E(\overline{X})=\frac{1}{n}[E(X_1)+E(X_2)+\cdots +E(X_n)]$
      • Once you calculate the mean, it won't change. We have the exact same distribution
    • $E(\overline{X})=\frac{1}{n}\underset{n\text{ times}}{\underbrace{[\mu +\mu +\cdots +\mu ]}}$
    • $E(\overline{X})=\frac{n\mu }{n}$
    • $E(\overline{X})=\mu$
    • The expected value of the sample mean, is the population mean.
  • $var(\overline{X})=var\left(\frac{1}{n}\sum _{i=1}^n{X}_i\right)$
    • $var(\overline{X})=var\left(\frac{1}{n}\sum _{i=1}^n{X}_i\right)$
    • $var(\overline{X})=\frac{1}{n^2}var\left(\sum _{i=1}^n{X}_i\right)$
    • $var(\overline{X})=\frac{1}{n^2}[var(X_1+X_2+\cdots +X_n)]$
    • $var(\overline{X})=\frac{1}{n^2}[var(X_1)+var(X_2)+\cdots +var(X_n)+cov(X_1,X_2)+cov(X_1,X_3)+\cdots +cov(X_{n-1},X_n)]$
    • Because these are independent variables, the covariance is $0$. They'e iid
    • $var(\overline{X})=\frac{1}{n^2}[var(X_1)+var(X_2)+\cdots +var(X_n)+0+0+\cdots +0]$
    • $var(\overline{X})=\frac{1}{n^2}[var(X_1)+var(X_2)+\cdots +var(X_n)]$
    • $var(\overline{X})=\frac{1}{n^2}[\underset{n\text{ times}}{\underbrace{{\sigma }^2+{\sigma }^2+\cdots +{\sigma }^2}}]$
    • $var(\overline{X})=\frac{n{\sigma }^2}{n^2}$
    • $var(\overline{X})=\frac{{\sigma }^2}{n}$
• Example:
  • Sample Variance:
    • $var(X)={\sigma }_x^2=E(X^2)-(E(X){)}^2$
    • ${\sigma }_x^2=E(X^2)-{\mu }_x^2$
    • $E[X^2]={\sigma }_x^2+{\mu }_x^2$
    • $E[S^2]=E[\frac{1}{n-1}\sum _{i=1}^n(X_i-\overline{X}{)}^2]$
      • $E[S^2]=\frac{1}{n-1}E[\sum _{i=1}^n(X_i-\overline{X}{)}^2]$
        • Show this summation is $=\sum X_i^2-n(\overline{X}{)}^2$
        • $\overline{X}=\frac{\sum X_i}{n}$
        • $n\overline{X}=\sum X_i$
      • $E[S^2]=\frac{1}{n-1}\sum _{i=1}^{n}E[(X_i-\overline{X}{)}^2]$
        • #tk
      • $E[S^2]=\frac{1}{n-1}E[\sum _{i=1}^n(X_i^2)-(n\overline{X}{)}^2]$
      • $E[S^2]=\frac{1}{n-1}[\sum _{i=1}^{n}E(X_i^2)-nE({\overline{X}}^2)]$
      • $E[S^2]=\frac{1}{n-1}[\sum _{i=1}^n({\sigma }^2+{\mu }^2)-nE({\overline{X}}^2)]$
    • $var(x)=E[X^2]-[E(X){]}^2$
    • $\underset{{\displaystyle \frac{{\sigma }^2}{n}}}{\underbrace{var(\overline{x})}}=E[{\overline{X}}^2]-[E(\overline{X}){]}^2$
    • $var(\overline{x})=E[{\overline{X}}^2]-{\mu }^2$
    • $E[{\overline{X}}^2]=\frac{{\sigma }^2}{n}+{\mu }^2$
    • $S^2=\frac{1}{n-1}[\sum _{i=1}^n({\sigma }^2+{\mu }^2)-nE({\overline{X}}^2)]$
    • $S^2=\frac{1}{n-1}[\sum _{i=1}^n({\sigma }^2+{\mu }^2)-n(\frac{{\sigma }^2}{n}+{\mu }^2)]$
    • $S^2=\frac{1}{n-1}[n({\sigma }^2+{\mu }^2)-n(\frac{{\sigma }^2}{n}+{\mu }^2)]$
    • $S^2=\frac{1}{n-1}[n{\sigma }^2\cancel{+n{\mu }^2}-{\sigma }^2\cancel{-n\mu s^2}]$
    • $S^2=\frac{1}{n-1}[n{\sigma }^2-{\sigma }^2]$
    • $S^2=\frac{1}{n-1}[(n-1){\sigma }^2]$
    • $S^2=\frac{1}{\cancel{n-1}}\left[\cancel{(n-1)}{\sigma }^2\right]$
    • $S^2={\sigma }^2$
• Theorem 25:
  • $X_1,X_2,\dots ,X_n$ are independent but can have different distributions
  • Moment Generating Function of the sum:
    • $M_{X_1+\cdots +X_n}(t)=M_{X_1}(t)\cdot M_{X_2}(t)\cdot \dots \cdot M_{X_n}(t)$

STA260

• We have RV $X_1,\dots ,X_n$
• Constants $a_1,\dots ,a_n$
• Random variable $Y=\sum _{i=1}^n{a}_i{X}_i$ is a linear combination of the $X_i$s
• When $a_i=\frac{1}{n}$ for $i=1,\dots ,n$, we have that $Y=\overline{X}$
• Let $\mathrm{E}[X_i]={\mu }_i$ for $i=1,\dots ,n$
• Let $X_1,\dots ,X_n$ have expected values of ${\mu }_1,\dots ,{\mu }_n$
  • See that $\mathrm{E}[Y]=\sum _{i=1}^n{a}_i{\mu }_i$
  • Given $X_1,\dots ,X_n$ are independent, $\mathrm{Var}(Y)=\sum _{i=1}^n{a}_i^2{\sigma }_i^2$
  • $\mathrm{E}[X_i]={\mu }_i$
  • $\mathrm{E}[Y]=\mathrm{E}\left[\sum _{i=1}^n{a}_i{X}_i\right]=\sum _{i=1}^n{a}_i\mathrm{E}[X_i]=\sum _{i=1}^n{a}_i{\mu }_i$
  • $\mathrm{Var}(Y)=\mathrm{Var}\left(\sum _{i=1}^n{a}_i{X}_i\right)=\sum _{i=1}^n{a}_i^2\mathrm{Var}(X_i)+2\sum _{\mathrm{\forall }i}^{}\sum _{\mathrm{\forall }j}^{}{a}_i{a}_j\mathrm{Cov}(X_i,X_j)$
  • Since they're independent, $\mathrm{Cov}(X_i,X_j)=0$
  • $=\sum _{i=1}^n{a}_i^2{\sigma }_i^2$
  • #tk why is it independent