Logarithms
For the unique real such that is called the logarithm of to the base and is denoted as
Referenced by (2 direct, 47 transitive)
Direct references:
Transitive (depth 1):
Transitive (depth 2):
Transitive (depth 3):
- Gibbs' Inequality
- KL Divergence
- Model Training
- proof-of-concavity-of-entropy
- Conditional Mutual Information
- Conditioning Reduces Entropy
- doubly-stochastic-maps-increase-entropy-intuition
- Entropy Rate
- fanos-inequality-remark
- Chain rule for joint entropy
- joint-entropy-is-less-than-or-equal-to-entropy-of-parts
- lemma-93
- Noiseless channel transmitting discrete symbols
- note-25
- note-8
- proof-of-doubly-stochastic-maps-increase-entropy
- proof-of-joint-entropy-is-less-than-or-equal-to-entropy-of-parts
- proof-of-uniform-distribution-maximizes-entropy
- remark-14
- theorem-12
- Uniform distribution maximizes entropy
- Weak Asymptotic Equipartition Property
Transitive (depth 4):
- proof-of-chain-rule-for-mutual-information
- proof-of-fanos-inequality
- Strong Asymptotic Equipartition Property (Shannon--McMillan--Breiman)
- proof-of-nonnegativity-of-conditional-mutual-information
- proof-of-nonnegativity-of-conditional-relative-entropy
- proof-of-nonnegativity-of-mutual-information
- proof-of-nonnegativity-of-relative-entropy
- proof-of-chain-rule-for-entropy
- proof-of-conditioning-reduces-entropy
- proof-of-mutual-information-and-entropy
- Conditional Relative Entropy
- convexity-of-relative-entropy-intuition
- Mutual Information
- proof-of-chain-rule-for-relative-entropy
- proof-of-concavity-and-convexity-of-mutual-information
- remark-32
- concavity-of-entropy-intuition
Transitive (depth 5):
Note
\@{note-2}
The logarithm extends to complex arguments; see the complex logarithm.
TODO: the log of a product is the sum of the logs.
Referenced by (1 direct)
Direct references:
TODO: the log of a quotient is the difference of the logs.
TODO: the log of a power moves the exponent out front.
Proof
\@{proof-of-logarithm-change-of-base}
By definition,
Referenced by (1 direct)
Direct references:
The function is concave, i.e. is convex.
Proof
\@{proof-of-log-is-concave}
The second derivative of natural is which is always non-positive. By Second Derivative Test for Convexity, is therefore concave.