Logarithms
For the unique real such that is called the logarithm of to the base and is denoted as
Referenced by (1 direct, 35 transitive)
Direct references:
Transitive (depth 1):
Transitive (depth 2):
Transitive (depth 3):
- Cross-Entropy
- doubly-stochastic-maps-increase-entropy-intuition
- Entropy Rate
- Gibbs' Inequality
- Chain rule for joint entropy
- joint-entropy-is-less-than-or-equal-to-entropy-of-parts
- KL Divergence
- Noiseless channel transmitting discrete symbols
- note-25
- note-8
- proof-of-concavity-of-entropy
- proof-of-doubly-stochastic-maps-increase-entropy
- proof-of-joint-entropy-is-less-than-or-equal-to-entropy-of-parts
- proof-of-uniform-distribution-maximizes-entropy
- remark-14
- theorem-12
- Uniform distribution maximizes entropy
- Weak Asymptotic Equipartition Property
Transitive (depth 4):
- Model Training
- Strong Asymptotic Equipartition Property (Shannon--McMillan--Breiman)
- proof-of-nonnegativity-of-conditional-mutual-information
- proof-of-nonnegativity-of-conditional-relative-entropy
- proof-of-nonnegativity-of-mutual-information
- proof-of-nonnegativity-of-relative-entropy
- proof-of-chain-rule-for-entropy
- proof-of-mutual-information-and-entropy
- proof-of-theorem-54
- remark-32
- concavity-of-entropy-intuition
Note
\@{note-2}
The logarithm extends to complex arguments; see the complex logarithm.
TODO: the log of a product is the sum of the logs.
Referenced by (1 direct)
Direct references:
TODO: the log of a quotient is the difference of the logs.
TODO: the log of a power moves the exponent out front.
Proof
\@{proof-of-logarithm-change-of-base}
By definition,
Referenced by (1 direct)
Direct references:
The function is concave, i.e. is convex.
Proof
\@{proof-of-log-is-concave}
The second derivative of natural is which is always non-positive. By Second Derivative Test for Convexity, is therefore concave.