• ↑
  • Special Distributions

Chi-squared Distribution

STA256

• Special case of the Gamma Distribution.
• $X$ has a chi-squared distribution with $\nu$ degrees of freedom (${\chi }_{\nu }^2$) if and only if $X$ is a random variable with a Gamma Distribution with $\alpha =\frac{\nu }{2}$ and $\beta =2$
• A ${\chi }_{\nu }^2$ random variable is obtained by square and summing $\nu$ standard normal variables $(z)$
  • $z_1^2+z_2^2+\cdots +z_{\nu }^2\sim {\chi }_{\nu }^2$
  • Where $z\sim N(0,1)$
  • Proven using MGF
  • $M_{X_1+X_2+\cdots +X_{\nu }}(t)=M_{X_1}(t)\dots$
• Properties:
  • Mean: $E[X]=\nu$
  • Variance: ${\sigma }^2=var(x)=2\nu$
  • MGF: $M_X(t)=\frac{1}{(1-2t{)}^{\frac{\nu }{2}}}$ for $t<\frac{1}{2}$

STA258

• $X\sim {\chi }_{(\nu )}^2$
• $f_X(x)=\frac{1}{2^{\frac{\nu }{2}}\mathrm{\Gamma }\left(\frac{\nu }{2}\right)}x{e}^{\frac{\nu }{2}-1}{e}^{-\frac{x}{2}}x>0$
• $\mathrm{E}[{\chi }_{(\nu )}^2]=\nu$
• $\mathrm{Var}({\chi }_{(\nu )}^2)=2\nu$

STA260

• ${\chi }_{\nu }^2$ is a case of Gamma Distribution with $\alpha =\frac{n}{2}$ and $\beta =2$.
• Very skewed for lower values of $n$
• $\mathrm{\Gamma }(1,2)$
• $\mathrm{\Gamma }(\frac{3}{2},2)$
• $\mathrm{\Gamma }(2,2)$
• $\mathrm{\Gamma }(3,2)$

\begin{document}
  \begin{tikzpicture}[xscale=0.8, yscale=10, domain=0:12, samples=100, every node/.style={font=\small}]
    % Draw axes
    \draw[->] (-0.2,0) -- (12.5,0) node[right] {$x$};
    \draw[->] (0,-0.02) -- (0,0.55) node[above] {$f(x; k, \theta)$};

    % Draw grid for better readability
    \draw[very thin, gray!30, step=1] (0,0) grid (12,0.5);

    % Gamma(1, 2) - Exponential
    \draw[thick, color=red] plot (\x, {0.5*exp(-0.5*\x)}) 
      node[right] at (1, 0.35) {$\Gamma(1,2)$};

    % Gamma(1.5, 2)
    \draw[thick, color=blue] plot (\x, {(sqrt(\x)*exp(-0.5*\x))/2.5066}) 
      node[right] at (2.5, 0.28) {$\Gamma(1.5,2)$};

    % Gamma(2, 2)
    \draw[thick, color=orange] plot (\x, {(\x*exp(-0.5*\x))/4}) 
      node[right] at (4.5, 0.2) {$\Gamma(2,2)$};

    % Gamma(3, 2)
    \draw[thick, color=teal] plot (\x, {(\x*\x*exp(-0.5*\x))/16}) 
      node[right] at (7, 0.12) {$\Gamma(3,2)$};

    % X-axis labels
    \foreach \x in {2,4,6,8,10,12}
      \draw (\x, 0.01) -- (\x, -0.01) node[below] {\x};

    % Y-axis labels
    \foreach \y in {0.1, 0.2, 0.3, 0.4, 0.5}
      \draw (0.1, \y) -- (-0.1, \y) node[left] {\y};

    % Legend or Title (optional)
    \node[draw, fill=white, anchor=north east] at (12, 0.5) {$\theta = 2$};
  \end{tikzpicture}
\end{document}

• Distribution of Sum of Squared Random Variables from a Standard Normal Distribution
  • Let $Z_1,\dots ,Z_n\stackrel{iid}{\sim }N(0,1)$
  • But $Z_1^2,\dots ,Z_n^2\sim {\chi }_{(\nu )}^2$
  • A chi-squared random variable is obtained from the sum of $n$ Standard Normal Distribution variables squared.
  • Moment Generating Function is standard for getting a random variable from another random variable.
  • MGF of the Normal Distribution:
    • Normal: $M_X(t)=e^{\mu t+\frac{{\sigma }^2{t}^2}{2}}$
    • ${\chi }_{(\nu )}^2$: $M_X(t)=(1-2t{)}^{-\frac{\nu }{2}}t<\frac{1}{2}$
  • Gaussian Integral
  • Proof:
    • $\sum _{i=1}^n{Z}_i^2\sim {\chi }_{(\nu )}^2,Z_i\stackrel{iid}{\sim }N(0,1)$
    • Let $Z\sim N(0,1)$
      • Normal:
        • $X\sim N(\mu ,{\sigma }^2)$
        • $f_X(x)=\frac{1}{\sigma \sqrt{2\pi }}{e}^{-\frac{(x-\mu {)}^2}{2{\sigma }^2}}$
    • $f_Z(z)=\frac{1}{\sqrt{2\pi }}{e}^{-\frac{z^2}{2}}$
    • The Moment Generating Function of $Z^2$ is:
      • $M_X(t)=\mathrm{E}[e^{tx}]=\int e^{tx}f(x)\text{differential}x$
      • $M_{Z^2}(t)=\mathrm{E}\left[e^{t{Z}^2}\right]=\int e^{t{Z}^2}{f}_Z(z)\text{differential}z$
        • $={\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{e}^{t{Z}^2}\frac{1}{\sqrt{2\pi }}{e}^{-\frac{z^2}{2}}\text{differential}z$
        • $=\frac{1}{\sqrt{2\pi }}{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{e}^{-\frac{z^2}{2}+t{z}^2}\text{differential}z$
        • $=\frac{1}{\sqrt{2\pi }}{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{e}^{-z^2(\frac{1}{2}+t)}\text{differential}z$
        • $=\frac{1}{\sqrt{2\pi }}{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{e}^{-(\frac{1}{2}+t)z^2}\text{differential}z$
          • This is taking form of the Gaussian Integral
          • Consider we didn't let $k=\frac{1}{2}-t$ and did U-Substitution.
            • We'd need to have $u=z^{}\sqrt{\frac{1}{2}-t}$ and $u^2=z^2(\frac{1}{2}-t)$
            • $\text{differential}u={(\frac{1}{2}-t)}^{\frac{1}{2}}\text{differential}z$
            • $\text{differential}z=\frac{\text{differential}u}{\sqrt{\frac{1}{2}-t}}$
            • We'd need to change the limits.
            • $\begin{array}{c}z\to \mathrm{\infty }⟹u\to \mathrm{\infty }\\ z\to -\mathrm{\infty }⟹u\to -\mathrm{\infty }\end{array}$
            • $=\frac{1}{\sqrt{2\pi }}{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{e}^{-u^2}\mathrm{d}\frac{u}{\sqrt{\frac{1}{2}-t}}$
            • $=\frac{1}{\sqrt{2\pi }}\frac{1}{\sqrt{\frac{1}{2}-t}}{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{e}^{-u^2}\text{differential}u$
        • Let $k=\frac{1}{2}-t$
        • $=\frac{1}{\sqrt{2\pi }}{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{e}^{-k{z}^2}\text{differential}z$
        • From the Gaussian Integral we know what this will be.
        • $=\frac{1}{\sqrt{2\pi }}\sqrt{\frac{\pi }{k}}$
        • $=\frac{1}{\sqrt{2\pi }}\sqrt{\frac{\pi }{\frac{1}{2}-t}}$
        • $=\frac{1}{\sqrt{2\pi }}\sqrt{\frac{\pi }{\frac{1}{2}-t}}\cdot \frac{2}{2}$
        • $=\frac{1}{\sqrt{2\pi }}\frac{\sqrt{2\pi }}{\sqrt{1-2t}}$
        • $=(1-2t{)}^{-\frac{1}{2}}t<\frac{1}{2}$
        • See this looks like the Moment Generating Function of Chi-squared Distribution with $\nu =1$
    • It's shown that $Z^2\sim {\chi }_{(1)}^2$
    • Now let's move on to the sum of $n$ of these random variables.
    • Let $Z_1,\dots ,Z_n\stackrel{iid}{\sim }N(0,1)$
    • $M_{Z_1^2+\cdots +Z_n^2}(t)=\mathrm{E}\left[e^{t(Z_1^2+\cdots +Z_n^2)}\right]$
      • $=\mathrm{E}\left[e^{(Z_1^{2}t+\cdots +Z_n^{2}t)}\right]$
      • $=\mathrm{E}[e^{Z_1^{2}t}{e}^{Z_2^{2}t}\dots e^{Z_n^{2}t}]$
      • Because of independence and linearity of expectation
      • $=\mathrm{E}[e^{Z_1^{2}t}]\mathrm{E}[e^{Z_2^{2}t}]\dots \mathrm{E}[e^{Z_n^{2}t}]$
      • From above we know $\mathrm{E}[e^{Z_1^{2}t}]$, or a single standard normal variable squared.
      • $=\underset{n\text{ times}}{\underbrace{(1-2t{)}^{-\frac{1}{2}}(1-2t{)}^{-\frac{1}{2}}\dots (1-2t{)}^{-\frac{1}{2}}}}$
      • $=(1-2t{)}^{-\frac{n}{2}}$
      • See this is the Moment Generating Function of a Chi-squared Distribution with $\nu =n$ degrees of freedom. ${\chi }_{(n)}^2$.
• Examples: STA260 Lecture 05