Hypothesis Testing

Given a baseline population mean ${\mu }_0,$ a sample mean $\overline{x},$ a sample count $n,$ and either a sample standard deviation $s$ or a population standard deviation $\sigma ,$ we might want to know if the sample data provides sufficient evidence to conclude that the true population mean $\mu$ differs significantly from ${\mu }_0.$

We'd then compute a test statistic. Here, we will assume we know the population's standard deviation $\sigma$

$$z=\frac{\overline{x}-{\mu }_0}{\sigma /\sqrt{n}}.$$

Then, since we want to know the p-value, we have a two-tailed test because the sample mean could either be lower than or higher than our actual mean. So, we do

$$\text{p-value}=P(|Z|\ge |z|)=2\times P(Z\ge |z|)=2(1-{\int }_{-\mathrm{\infty }}^{|z|}\frac{1}{\sqrt{2\pi }}{e}^{-\frac{t^2}{2}}dt)$$

or in Mathematica notation:

$$2\ast (1-CDF[NormalDistribution[0,1],Abs[z]]).$$

Low $p$-values (typically, or so) lead us to reject the null-hypothesis and adopt the alternative hypothesis. A very low $p$-value means that the probability of getting the sample we got if $\mu ={\mu }_0$ is very low.

Instead of hypothesizing that $\mu \ne {\mu }_0,$ we could hypothesize that or that These would be single-tailed tests and we'd use, for example:

$$H_0:\mu ={\mu }_0,$$

$$1-CDF[NormalDistribution[0,1],z]$$

instead.

Additionally, if we don't know $\sigma$ but only know $s,$ the standard deviation of the sample, we would use a Student T Distribution. We'd calculate a $t$ value instead of a $z$ value:

$$t=\frac{\overline{x}-{\mu }_0}{s/\sqrt{n}}.$$

$$1-CDF[StudentTDistribution[n-1],t].$$

Note that if we wanted to test the alternative hypothesis we'd use

$$CDF[StudentTDistribution[n-1],t],$$

and similarly for when $\sigma$ is known, except we'd use the standard normal distribution and a $z$ value.