STA260 Makeup Lecture

• Relationship between a $T$ and $F$ random variable.
  • Let $T\sim t_{(n)}$ then $T^2\sim F_{(1,n)}$
  • Proof:
    • Since $T=\frac{Z}{\sqrt{\frac{{\chi }_{(n)}^2}{n}}}$ and $F=\frac{\frac{{\chi }_{(m)}^2}{m}}{\frac{{\chi }_{(n)}^2}{n}}$
    • Let $Z\sim N(0,1)$ and $W\sim {\chi }_{(n)}^2$
    • By definition of a $T$ random variable.
      • $T=\frac{Z}{\sqrt{\frac{{\chi }_{(n)}^2}{n}}}\sim t_{(n)}$
    • $T^2=\frac{Z^2}{\frac{{\chi }_{(n)}^2}{n}}$
    • Sum of Squares is Chi-squared!!!!
    • Chi-squared from Sum of Squared Standard Normals!!!!!
      • $\sum _{i=1}^n{Z}_i^2\sim {\chi }_{(n)}^2$
      • $Z^2\sim {\chi }_{(1)}^2$
      • $=\frac{{\chi }_{(1)}^2}{\frac{{\chi }_{(n)}^2}{n}}$
      • $=\frac{\frac{{\chi }_{(1)}^2}{1}}{\frac{{\chi }_{(n)}^2}{n}}\sim F_{(1,n)}$
  • $F_{(n,m)}=\frac{1}{F_{(m,n)}}$
• Chapter 8
  • Parameter estimation techniques
  • Parameter of Statistical Distribution
    • A numerical value in a Probability Mass Function or Probability Density Function which affects the characteristics of the function.
    • Like in Gamma Distribution, we have shape and scale.
    • Like in Normal Distribution, we have $\mu ,{\sigma }^2$
  • Point Estimate
    • A single numeric value, which often is the single best estimate of a parameter.
    • Distribution or population parameter.
    • Example:
      • $\overline{X}$ is a point estimate of the population mean $\mu$.
      • $S^2$ is a point estimate of the population variance ${\sigma }^2$
  • Interval Estimate
    • A range of values which we believe captures a parameter.
    • This is quantified with a level of confidence.
    • Example:
      • I'm $100\mathrm{\%}$ certain you will get between $0\to 100\mathrm{\%}$ is useless information.
      • I'm $80\mathrm{\%}$ certain you will get between $80\to 90\mathrm{\%}$ is very useful.
  • Notation:
    • Random Sample and a Numerical Realization
    • The Random Sample will be uppercase letters: $X_1,X_2,\dots ,X_n$
    • The Numerical Realization will be $x_1,x_2,\dots ,x_n$
  • Statistic
    • Numerical Measures calculated from a sample are Random Variables when expressed in uppercase.
    • Depends on data.
    • Example: $\overline{X}=\frac{1}{n}\sum _{i=1}^n{X}_i$, $\overline{X}$ is our random variable.
    • $\overline{x}$ is our numerical realization. This isn't random.
• Fitting the Poisson Distribution to Emission Alpha Particles
  • Used to model counts $\{0,1,2,\dots \}$ of data which occur in a unit interval of time, space, area, volume, etc.
  • $P(X=x)=\frac{e^{-\lambda }{\lambda }^x}{x!}\text{ for }x=0,1,2,3,\dots$
  • Distribution parameter $\lambda$ is the rate per unit time / space.
  • Example:
    • Model number of students visiting the registrar.
    • Model the number of olives per unit area on a pizza.
  • Assumptions
    • Assume occurance of events in disjoint intervals is independent.
    • The rate parameter $\lambda$ is fixed for an interval
    • Negligible probability in small intervals
    • No clustering
• Radioactive decay, alpha particles
  • Emission rate is $0.8392$ per second
  • Intervals of $10$ seconds
  • $$$$n & \text{Obs} & \text{Exp} \
    0-2 & 18 & 12.2 \
    3 & 28 & 27 \
    4 & 56 & 56.5 \
    5 & 105 & 94.9 \
    6 & 126 & 132.7 \
    7 & 146
    \end$$$$
    • Use information to determine $\lambda$ for an interval.
    • Interval is $10$ seconds.
    • We're informed that we have $0.8392$ emissions per second.
    • To adjust to $10$ seconds:
      • $0.8392:1$
      • $8.392:10$
    • $\lambda =8.392$
    • We have $1207$ intervals were examined.
    • $$\begin{array}{l}\underset{―}{x=0\to 2}\\ P(0\le X\le 2)\\ =P(X=0)+P(X=1)+P(X=2)=0.0101107323940626\\ P(X=0)=\frac{e^{-8.392}{8.392}^0}{0!}\\ P(X=1)=\frac{e^{-8.392}{8.392}^1}{1!}\\ P(X=2)=\frac{e^{-8.392}{8.392}^2}{2!}\\ \underset{―}{x=3}\\ P(X=3)=\frac{e^{-8.392}{8.392}^3}{3!}=0.0223277974145552\\ \underset{―}{x=4}\\ P(X=4)=\frac{e^{-8.392}{8.392}^4}{4!}=0.0468437189757368\\ \underset{―}{x\ge 17}\\ P(X\ge 17)=1-P(X<17)\\ =1-P(X=0)-P(X=1)-\cdots -P(X=16)\end{array}$$
    • Expected is $(1207)(0.01011)=12.20277$
    • Expected is $(1207)(0.0223)=26.9161$
    • Expected is $(1207)(0.0468)=56.4876$
• Parameter Estimation
  • If you see $f(x;\theta )$, $x$ is the value of the random variable, and $\theta$ is the parameter or parameters of the Probability Density Function.
    • Example: Normal Distribution has $\mu ,{\sigma }^2$ so it could be $f(x;{\theta }_1,{\theta }_2)$
  • A Statistic is a function of a sample, $T=g(X_1,\dots ,X_n)$ which does not depend on any unknown parameters.
    • $X_1,\dots ,X_n$ is a Random Sample
  • A statistic $T$ that is used to Estimate the value of $\theta$ is an Estimator of $\theta$.
    • An observed value of the Statistic is $t=g(x_1,\dots ,x_n)$ is an Estimate of $\theta$
  • $\overline{X}$ is one possible Point Estimator of the population mean $\mu$
  • Notation
    • $\theta$: parameter (generic)
    • $\hat{\theta }$: Estimator of $\theta$ (generic)
    • Example:
      • Let $X_1,\dots ,X_n\sim N(\mu ,{\sigma }^2)$
      • $\mu ,{\sigma }^2$ are the parameters of the distribution
      • For a question on $X_1,\dots ,X_n\sim N(\mu ,{\sigma }^2)$, parameters of interest are $\mu ,{\sigma }^2$
        • $\theta =\mu$
        • $\theta ={\sigma }^2$
        • $\theta =(\mu ,{\sigma }^2)⟹{\theta }_1=\mu ,{\theta }_2={\sigma }^2$
    • Estimators
      • Common ones we know.
      • $\hat{\theta }=\overline{X}$ or $\hat{\mu }=\overline{X}$ is an Estimator of the mean $\theta =\mu$
      • $\hat{\theta }=S^2$ or ${\hat{\sigma }}^2=S^2$ is an Estimator of the variance $\theta ={\sigma }^2$
  • Poisson Dist:
    • $f(x;\lambda )=\frac{e^{-\lambda }{\lambda }^x}{x!}$ for $x=0,1,2,\dots$
    • $\mathrm{E}[X]=\lambda$ and $\mathrm{Var}(X)=\lambda$
    • Suppose $\theta =\lambda$ is the parameter of interest.
    • $\hat{\theta }=\hat{\lambda }=\overline{X}$ is an Estimator of $\lambda$
      • Maximum Likelihood Estimation
  • Unbiased Estimators :
    • Let $\hat{\theta }$ be a Point Estimator for a parameter $\theta$ then $\hat{\theta }$ is an unbiased estimator of $\theta$ if $\mathrm{E}[\hat{\theta }]=\theta$. If $\mathrm{E}[\hat{\theta }]\ne \theta$, $\hat{\theta }$ is biased.
      • $\mathrm{E}[\text{Estimator}]=\text{Parameter}$
    • Bias of a Point Estimator is given by $b(\hat{\theta })=\mathrm{E}[\hat{\theta }]-\theta$
      • The difference between the expected value of an Estimator and the parameter it is estimating.
      • $\theta =\mathrm{E}[\hat{\theta }]-b(\hat{\theta })$
    • It's desired to have unbiasedness as a property of an Estimator
      • Because on average, the estimator will not overfit or underfit
    • Example:
      • In previous chapter we show that $\mathrm{E}[\overline{X}]=\mu$, so it is unbiased. Sample Mean
      • We also shows that $\mathrm{E}[S^2]={\sigma }^2$. Sample Variance
    • Example:
      • $S^2=\frac{1}{n-1}\sum _{i=1}^n(X_i-\overline{X}{)}^2$ is an unbiased Estimator of ${\sigma }^2$
      • Bessel's Correction
      • $S_n^2=\frac{1}{n}\sum _{i=1}^n(X_i-\overline{X})s^2$ is a biased Estimator.
      • Quantify the bias of $S_n^2$
        • $\underset{S_{n-1}^2}{\underbrace{\frac{1}{n-1}\sum _{i=1}^n(X_i-\overline{X}{)}^2}}$
        • $\frac{n-1}{n}\frac{1}{n-1}\sum _{i=1}^n(X_i-\overline{X}{)}^2=\underset{S_n^2}{\underbrace{\frac{1}{n}\sum _{i=1}^n(X_i-\overline{X}{)}^2}}$
        • So $\frac{n-1}{n}{S}_{n-1}^2=S_n^2$
        • The bias
        • $b(\hat{\theta })=\mathrm{E}[\hat{\theta }]-\theta$
        • $b(S_n^2)=\mathrm{E}[S_n^2]-{\sigma }^2$
        • $b(S_n^2)=\mathrm{E}\left[\frac{n-1}{n}{S}_{n-1}^2\right]-{\sigma }^2$
        • $b(S_n^2)=\frac{n-1}{n}\mathrm{E}\left[S_{n-1}^2\right]-{\sigma }^2$
          • We already know that $\mathrm{E}[S^2]={\sigma }^2$
        • $b(S_n^2)=\frac{n-1}{n}{\sigma }^2-{\sigma }^2$
        • $b(S_n^2)={\sigma }^2(\frac{n-1}{n}-1)=-\frac{{\sigma }^2}{n}$
        • It underestimates by $\frac{{\sigma }^2}{n}$
• Sampling Distribution#STA260