Two random variables $X$ and $Y$ can be paired as a single random vector or bivariate random variable. If it's discrete (i.e. both $X$ and $Y$ are discrete) it has a joint probability mass function and if it's continuous ($X$ and $Y$ are both continuous) it has a joint probability density function.

In the discrete case, $f(x,y)=P(X=x,Y=y).$

We can marginalize over $Y$ to find solely the distribution of $X,$ we denote this $f_X(x)$ and give it as

$$f_X(x)=\sum _{y}f(x,y).$$

We can similarly find $f_Y(y)$ by marginalizing over $X$:

$$f_Y(y)=\sum _{x}f(x,y).$$

TODO: define $f_{Y|X}(y|x)$ for the discrete and continuous cases; note it is a probability distribution in $y$ for each fixed $x$ with Relate to conditional probability, which states the event-level version.

A probability distribution $\mu$ on ${\mathcal{X}}_1\times \cdots \times {\mathcal{X}}_n$ is a product distribution if there are distributions ${\mu }_1,\dots ,{\mu }_n$ on ${\mathcal{X}}_1,\dots ,{\mathcal{X}}_n$ respectively such that $$\mu (x_1,\dots ,x_n)=\prod _{i=1}^n{\mu }_i(x_i)$$ for every tuple $(x_1,\dots ,x_n)$.

Such a distribution is written ${\mu }_1\otimes \cdots \otimes {\mu }_n$, or ${\mu }^{\otimes n}$ when every factor is equal to a common distribution $\mu$.

For a sequence of $n$ independent, identical experiments with each one of the experiments resulting in $r$ outcomes with probabilities $p_1,p_2,\dots ,p_r,$ respectively, where $\sum _{i=1}^r{p}_i=1,$ we let $X_i$ count the number of the $n$ experiments that result in the $i$th of the $r$ outcomes. Then

$$P(X_1=n_1,X_2=n_2,\dots ,X_r=n_r)=(\genfrac{}{}{0ex}{}{n}{n_1,n_2,\dots ,n_r})p_1^{n_1}{p}_2^{n_2}\cdots p_r^{n_r},$$

where $\sum _{i=1}^r{n}_i=n.$

The multinomial distribution uses the multinomial coefficient, which is defined as

$$(\genfrac{}{}{0ex}{}{n}{n_1,n_2,\dots ,n_r})=\frac{n!}{n_1!n_2!\cdots n_r!}.$$

Sums of independent random variables are called convolutions.

If $X$ and $Y$ are continuous, the probability density function of $X+Y$ is given by

$$f_{X+Y}(x)={\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{f}_X(t)f_Y(x-t)dt.$$

If they are discrete, the probability mass function of $X+Y$ is given by

$$f_{X+Y}(n)=\sum _k{f}_X(k)f_Y(n-k).$$

Covariance is a measure of the joint variability of two random variables.

$$\mathrm{Cov}(X,Y)=E[(X-E[X])(Y-E[Y])].$$

If large $X$ values go with large $Y$ values and small $X$ goes with small $Y$, covariance will be positive. The covariance will be negative if large $X$ values go with small $Y$ values and vice-versa. An alternative, equivalent form of covariance is:

$$\mathrm{Cov}(X,Y)=E[XY]-E[X]E[Y].$$

The correlation coefficient of $X$ and $Y$, ${\rho }_{x,y},$ is given by

$${\rho }_{x,y}=\frac{\mathrm{Cov}(X,Y)}{{\sigma }_X{\sigma }_Y}.$$