Random Variables and Probability Distributions
Random Variables
A random variable is a function that associates a real number with each element in the sample space. For sample space , we have and for , we have where is a value of the random variable.
Referenced by (12 direct, 42 transitive)
Direct references:
Transitive (depth 1):
- Cross-Entropy
- Entropy Rate
- Gibbs' Inequality
- Chain rule for joint entropy
- joint-entropy-is-less-than-or-equal-to-entropy-of-parts
- KL Divergence
- note-33
- note-8
- proof-of-joint-entropy-is-less-than-or-equal-to-entropy-of-parts
- proof-of-theorem-18
- proof-of-theorem-29
- theorem-12
- Weak Asymptotic Equipartition Property
- theorem-29-intuition
- Noiseless channel transmitting discrete symbols
- markov-inequality-fish
- proof-of-markovs-inequality
- proof-of-theorem-2
- Sum of Independent Normal Random Variables
- note-23
- remark-8
- Binomial Distribution
- chebyshevs-inequality-note
- Confidence Interval for a Mean, Known Variance
- Confidence Interval for a Mean, Unknown Variance
- Geometric Distribution
- Hypergeometric Distribution
- Poisson Distribution
- Standard Deviation
- Uniform Distribution
- variance-interpretation
Transitive (depth 2):
- Normal Approximation to the Binomial
- Poisson Approximation to the Binomial
- Model Training
- Strong Asymptotic Equipartition Property (Shannon--McMillan--Breiman)
- Negative Binomial Distribution
- proof-of-theorem-36
- remark-26
- Sum of Independent Poisson Random Variables
- Chebyshev's Inequality
Transitive (depth 3):
Random variables don't have to map to all reals; for example, Bernoulli random variables map to either 0 or 1.
Returning to sample spaces briefly:
If a sample space contains a finite number of possibilities, or an unending sequence with as many elements as there are whole numbers (countably infinite), it is called a discrete sample space.
If a sample space contains an infinite number of possibilities equal to the number of points on a line segment, it is called a continuous sample space.
A random variable is called a discrete random variable if its set of possible outcomes is countable.
A random variable is called a continuous random variable if it takes values on a continuous scale.
Discrete random variables tend to represent counted data, while continuous random variables tend to represent measured data.
Probability Distributions
Discrete Probability Distributions
A discrete random variable takes each of its values with a certain probability. The function that gives the probability of each value of a random variable occurring is called the probability function, probability mass function, or probability distribution. More formally, the set of ordered pairs is a probability distribution of the discrete random variable if, for each possible outcome :
Sometimes we want to know the probability that a random variable will be less than or equal to some real number If we let for all real , we define to be the cumulative distribution function of the random variable More formally, the cumulative distribution function of a discrete random variable with a probability distribution is:
Continuous Probability Distributions
A continuous random variable has a probability of of exactly assuming any particular value. That's because, in any interval of real numbers, there are infinitely many points, and so the denominator for the proportion is infinite. For example, there are infinitely many points between and , and again, infinitely many points between and and so on, regardless of the level of precision.
So, instead of focusing on probabilities of particular values, we focus on probabilities of values falling in various intervals. For example, instead of asking what the probability that someone weighs pounds is, we can ask what is the probability that they weigh between and pounds.
The function is a probability density function (pdf) for the continuous random variable , defined on the reals, if
- for all
In other words, if we integrate from to we will get the probability of taking on a value between and .
From here, the cumulative distribution function of a continuous random variable with density function is attained by simply integrating from to :