Vector Space
A vector space over a field is a set together with two operations:
satisfying the following axioms:
- Scalar multiplication is associative:
- Distributive laws hold
- Identity: for all
Referenced by (11 direct, 158 transitive)
Direct references:
Transitive (depth 1):
- Weakly Nonlinear Oscillator
- Basis and Dimension for Column Space
- Basis and Dimension for Left Nullspace
- Basis and Dimension for Nullspace
- Basis and Dimension for Row Space
- Hessian Matrix
- note-3
- Parallel
- remark-32
- Scalar Function
- Vector Multiplication by a Scalar
- Bound vector
- Cauchy-Schwarz Inequality
- Cross Product
- Difference equation
- Direction
- Expected Value
- Four Fundamental Subspaces
- Free vector
- Gradient
- incompressible
- Inner product
- intuition-13
- invariance-of-curl
- Jacobian Matrix
- Layer
- Left Stochastic Matrix
- Linear Combination
- Neuron
- note-15
- note-5
- note-9
- Perpendicular
- Probability Vector
- projection
- proof-of-gradient-as-surface-normal-vector
- proof-of-theorem-9
- Pythagorean Theorem
- remark-16
- remark-36
- remark-45
- remark-9
- Stochastic Matrix
- Surface Normal Vector
- Tangent Space
- theorem-19
- Unit Vector
- Vector Addition
- Vector Equality
- Vector Function
Transitive (depth 2):
- proof-of-mean-value-inequality-for-vector-valued-functions
- remark-23
- Directional Derivative
- directional-derivative-is-inner-product-of-vector-and-grad
- Normal Derivative
- proof-of-theorem-19
- remark-30
- remark-46
- Zero Vector
- Cross-Entropy
- Entropy
- Entropy Rate
- Jensen's Inequality
- markov-inequality-fish
- Interpretation
- markovs-inequality-note
- proof-of-markovs-inequality
- proof-of-weak-asymptotic-equipartition-property
- Row space and nullspace are orthogonal complements
- grad-div-curl-related
- gradient-as-surface-normal-vector
- Laplacian
- Potential Function
- remark-12
- is an inner product space
- cos-characterization-of-dot-product
- proof-of-fourier-basis
- remark-5
- theorem-7-remark
- Vector Valued Continuously Differentiable implies Lipschitz Continuous
- Neural Network
- Doubly Stochastic Matrix
- Convex Combination
- The Four Ways to View Matrix Multiplication
- intuition-2
- proof-of-pythagorean-theorem
- Surface Normal
- Doubly stochastic maps increase entropy
- probability-vectors-form-a-convex-compact-set
- proof-of-convexity-of-relative-entropy
- proof-of-probability-vectors-form-a-convex-compact-set
- cauchy-schwarz-inequality-intuition
- proof-of-cauchy-schwarz-inequality
- remark-8
- Gradient System
- note-19
- proof-of-strong-asymptotic-equipartition-property-shannon-mcmillan-breiman
- remark-3
- Tangent Vector
- Divergence Theorem of Gauss
- Line Integral of Vector Function
- Stoke's Theorem
- Surface Integral over Vector Field
- Vector Field
Transitive (depth 3):
- doubly-stochastic-maps-increase-entropy-intuition
- note-21
- note-8
- proof-of-concavity-and-convexity-of-mutual-information
- proof-of-jensens-inequality
- Gibbs' Inequality
- KL Divergence
- Model Training
- proof-of-concavity-of-entropy
- Conditional Mutual Information
- Conditioning Reduces Entropy
- fanos-inequality-remark
- Chain rule for joint entropy
- joint-entropy-is-less-than-or-equal-to-entropy-of-parts
- lemma-93
- Noiseless channel transmitting discrete symbols
- note-25
- proof-of-doubly-stochastic-maps-increase-entropy
- proof-of-joint-entropy-is-less-than-or-equal-to-entropy-of-parts
- proof-of-uniform-distribution-maximizes-entropy
- remark-14
- theorem-12
- theorem-16
- Uniform distribution maximizes entropy
- Weak Asymptotic Equipartition Property
- Strong Asymptotic Equipartition Property (Shannon--McMillan--Breiman)
- closed-orbits-impossible-in-gradient-systems
- proof-of-corollary-95
- proof-of-gibbs-inequality
- proof-of-lemma-93
- proof-of-log-sum-inequality
- gravitational-potential-is-a-solution-to-laplaces-equation
- Tangent Plane
- Curl
- Divergence
- Divergence Theorem, or, Gauss's Theorem
- Irrotational
- poincare-bendixson
Transitive (depth 4):
- proof-of-chain-rule-for-mutual-information
- proof-of-fanos-inequality
- proof-of-nonnegativity-of-conditional-mutual-information
- proof-of-nonnegativity-of-conditional-relative-entropy
- proof-of-nonnegativity-of-mutual-information
- proof-of-nonnegativity-of-relative-entropy
- proof-of-chain-rule-for-entropy
- proof-of-conditioning-reduces-entropy
- proof-of-mutual-information-and-entropy
- Conditional Relative Entropy
- convexity-of-relative-entropy-intuition
- Mutual Information
- proof-of-chain-rule-for-relative-entropy
- find-tangent-plane
- concavity-of-entropy-intuition
Transitive (depth 5):
Scalars
A scalar is an element of a field used to define a vector space.
Referenced by (8 direct, 162 transitive)
Direct references:
Transitive (depth 1):
- proof-of-theorem-19
- grad-div-curl-related
- Gradient
- gradient-as-surface-normal-vector
- Gradient System
- Potential Function
- remark-46
- theorem-19
- Vector Space
Transitive (depth 2):
- Jacobian Matrix
- Laplacian
- remark-12
- remark-16
- closed-orbits-impossible-in-gradient-systems
- gravitational-potential-is-a-solution-to-laplaces-equation
- Linear Transformation
- Null Space
- Null Space and Range are Subspaces
- proof-of-complex-inner-product-space
- Range is span of transformation of domain basis
- rank
- Rank Nullity Theorem
- rn-vector-space-vs-metric-space
- Vector
Transitive (depth 3):
- remark-45
- Vector Valued Continuously Differentiable implies Lipschitz Continuous
- Weakly Nonlinear Oscillator
- Basis and Dimension for Column Space
- Basis and Dimension for Left Nullspace
- Basis and Dimension for Nullspace
- Basis and Dimension for Row Space
- Bound vector
- Cauchy-Schwarz Inequality
- Cross Product
- Difference equation
- Direction
- Expected Value
- Four Fundamental Subspaces
- Free vector
- incompressible
- Inner product
- intuition-13
- invariance-of-curl
- Layer
- Left Stochastic Matrix
- Linear Combination
- Neuron
- note-15
- note-5
- note-9
- Perpendicular
- Probability Vector
- projection
- proof-of-gradient-as-surface-normal-vector
- proof-of-theorem-9
- Pythagorean Theorem
- remark-36
- remark-9
- Stochastic Matrix
- Surface Normal Vector
- Tangent Space
- Unit Vector
- Vector Addition
- Vector Equality
- Vector Function
Transitive (depth 4):
- proof-of-mean-value-inequality-for-vector-valued-functions
- remark-23
- Directional Derivative
- directional-derivative-is-inner-product-of-vector-and-grad
- Normal Derivative
- remark-30
- Zero Vector
- Cross-Entropy
- Entropy
- Entropy Rate
- Jensen's Inequality
- markov-inequality-fish
- Interpretation
- markovs-inequality-note
- proof-of-markovs-inequality
- proof-of-weak-asymptotic-equipartition-property
- Row space and nullspace are orthogonal complements
- is an inner product space
- cos-characterization-of-dot-product
- proof-of-fourier-basis
- remark-5
- theorem-7-remark
- Neural Network
- Doubly Stochastic Matrix
- Convex Combination
- The Four Ways to View Matrix Multiplication
- intuition-2
- proof-of-pythagorean-theorem
- Surface Normal
- Doubly stochastic maps increase entropy
- probability-vectors-form-a-convex-compact-set
- proof-of-convexity-of-relative-entropy
- proof-of-probability-vectors-form-a-convex-compact-set
- cauchy-schwarz-inequality-intuition
- proof-of-cauchy-schwarz-inequality
- remark-8
- note-19
- proof-of-strong-asymptotic-equipartition-property-shannon-mcmillan-breiman
- remark-3
- Tangent Vector
- Divergence Theorem of Gauss
- Line Integral of Vector Function
- Stoke's Theorem
- Surface Integral over Vector Field
- Vector Field
Transitive (depth 5):
- doubly-stochastic-maps-increase-entropy-intuition
- note-21
- note-8
- proof-of-concavity-and-convexity-of-mutual-information
- proof-of-jensens-inequality
- Gibbs' Inequality
- KL Divergence
- Model Training
- proof-of-concavity-of-entropy
- Conditional Mutual Information
- Conditioning Reduces Entropy
- fanos-inequality-remark
- Chain rule for joint entropy
- joint-entropy-is-less-than-or-equal-to-entropy-of-parts
- lemma-93
- Noiseless channel transmitting discrete symbols
- note-25
- proof-of-doubly-stochastic-maps-increase-entropy
- proof-of-joint-entropy-is-less-than-or-equal-to-entropy-of-parts
- proof-of-uniform-distribution-maximizes-entropy
- remark-14
- theorem-12
- theorem-16
- Uniform distribution maximizes entropy
- Weak Asymptotic Equipartition Property
- Strong Asymptotic Equipartition Property (Shannon--McMillan--Breiman)
- proof-of-corollary-95
- proof-of-gibbs-inequality
- proof-of-lemma-93
- proof-of-log-sum-inequality
- Tangent Plane
- Curl
- Divergence
- Divergence Theorem, or, Gauss's Theorem
- Irrotational
- poincare-bendixson
Transitive (depth 6):
- proof-of-chain-rule-for-mutual-information
- proof-of-fanos-inequality
- proof-of-nonnegativity-of-conditional-mutual-information
- proof-of-nonnegativity-of-conditional-relative-entropy
- proof-of-nonnegativity-of-mutual-information
- proof-of-nonnegativity-of-relative-entropy
- proof-of-chain-rule-for-entropy
- proof-of-conditioning-reduces-entropy
- proof-of-mutual-information-and-entropy
- Conditional Relative Entropy
- convexity-of-relative-entropy-intuition
- Mutual Information
- proof-of-chain-rule-for-relative-entropy
- find-tangent-plane
- concavity-of-entropy-intuition
Transitive (depth 7):
Typically, especially in physics, a scalar is simply a number, especially a real number.
Vectors
A vector is an element in a vector space.
Referenced by (51 direct, 125 transitive)
Direct references:
- Dot Product
- Null Space
- Four Fundamental Subspaces
- projection
- Pythagorean Theorem
- Cauchy-Schwarz Inequality
- note-5
- Free vector
- Bound vector
- note-9
- Direction
- Vector Equality
- Unit Vector
- Vector Addition
- note-15
- Vector Multiplication by a Scalar
- Inner product
- Perpendicular
- Parallel
- Cross Product
- Linear Combination
- Polar Coordinates
- Vector Function
- remark-9
- Gradient
- remark-16
- theorem-19
- Surface Normal Vector
- proof-of-gradient-as-surface-normal-vector
- incompressible
- remark-36
- invariance-of-curl
- Hessian Matrix
- Jacobian Matrix
- remark-45
- Tangent Space
- intuition-13
- Geometric Problems
- Pursuit Curves
- Difference equation
- Machine Learning Basics
- Neuron
- Layer
- Discrete Fourier Transform
- Newtonian Motion
- Expected Value
- proof-of-theorem-9
- Probability Vector
- Stochastic Matrix
- Left Stochastic Matrix
- Projective Transformations
Transitive (depth 1):
- proof-of-mean-value-inequality-for-vector-valued-functions
- remark-23
- Directional Derivative
- directional-derivative-is-inner-product-of-vector-and-grad
- Normal Derivative
- proof-of-theorem-19
- remark-30
- remark-46
- Zero Vector
- Cross-Entropy
- Entropy
- Entropy Rate
- Jensen's Inequality
- markov-inequality-fish
- Interpretation
- markovs-inequality-note
- proof-of-markovs-inequality
- proof-of-weak-asymptotic-equipartition-property
- Basis and Dimension for Column Space
- Basis and Dimension for Left Nullspace
- Basis and Dimension for Nullspace
- Basis and Dimension for Row Space
- Row space and nullspace are orthogonal complements
- grad-div-curl-related
- gradient-as-surface-normal-vector
- Laplacian
- Potential Function
- remark-12
- is an inner product space
- cos-characterization-of-dot-product
- proof-of-complex-inner-product-space
- proof-of-fourier-basis
- remark-5
- theorem-7-remark
- Vector Valued Continuously Differentiable implies Lipschitz Continuous
- Neural Network
- Doubly Stochastic Matrix
- Convex Combination
- The Four Ways to View Matrix Multiplication
- Null Space and Range are Subspaces
- rank
- intuition-2
- proof-of-pythagorean-theorem
- Surface Normal
- Doubly stochastic maps increase entropy
- probability-vectors-form-a-convex-compact-set
- proof-of-convexity-of-relative-entropy
- proof-of-probability-vectors-form-a-convex-compact-set
- cauchy-schwarz-inequality-intuition
- proof-of-cauchy-schwarz-inequality
- remark-8
- note-19
- proof-of-strong-asymptotic-equipartition-property-shannon-mcmillan-breiman
- remark-3
- Tangent Vector
- Vector Space
- Divergence Theorem of Gauss
- Line Integral of Vector Function
- Stoke's Theorem
- Surface Integral over Vector Field
- Vector Field
Transitive (depth 2):
- doubly-stochastic-maps-increase-entropy-intuition
- note-21
- note-8
- proof-of-concavity-and-convexity-of-mutual-information
- proof-of-jensens-inequality
- Gibbs' Inequality
- KL Divergence
- Model Training
- proof-of-concavity-of-entropy
- Conditional Mutual Information
- Conditioning Reduces Entropy
- fanos-inequality-remark
- Chain rule for joint entropy
- joint-entropy-is-less-than-or-equal-to-entropy-of-parts
- lemma-93
- Noiseless channel transmitting discrete symbols
- note-25
- proof-of-doubly-stochastic-maps-increase-entropy
- proof-of-joint-entropy-is-less-than-or-equal-to-entropy-of-parts
- proof-of-uniform-distribution-maximizes-entropy
- remark-14
- theorem-12
- theorem-16
- Uniform distribution maximizes entropy
- Weak Asymptotic Equipartition Property
- Strong Asymptotic Equipartition Property (Shannon--McMillan--Breiman)
- proof-of-corollary-95
- proof-of-gibbs-inequality
- proof-of-lemma-93
- proof-of-log-sum-inequality
- gravitational-potential-is-a-solution-to-laplaces-equation
- Tangent Plane
- Curl
- Divergence
- Divergence Theorem, or, Gauss's Theorem
- Irrotational
- poincare-bendixson
- remark-32
- Linear Transformation
- Range is span of transformation of domain basis
- Rank Nullity Theorem
- rn-vector-space-vs-metric-space
- Scalar
Transitive (depth 3):
- proof-of-chain-rule-for-mutual-information
- proof-of-fanos-inequality
- proof-of-nonnegativity-of-conditional-mutual-information
- proof-of-nonnegativity-of-conditional-relative-entropy
- proof-of-nonnegativity-of-mutual-information
- proof-of-nonnegativity-of-relative-entropy
- proof-of-chain-rule-for-entropy
- proof-of-conditioning-reduces-entropy
- proof-of-mutual-information-and-entropy
- Conditional Relative Entropy
- convexity-of-relative-entropy-intuition
- Mutual Information
- proof-of-chain-rule-for-relative-entropy
- Weakly Nonlinear Oscillator
- note-3
- Scalar Function
- find-tangent-plane
- concavity-of-entropy-intuition
Transitive (depth 4):
Transitive (depth 5):
Typically, in physics and many other applications, when we say vector we mean a vector in which is an ordered tuple of real numbers, This is the algebraic representation of vector .
The geometric representation of vector is, in an dimensional space, an arrow or directed line segment. When starting from the origin, it would be a line segment from to the point .
A vector with a fixed endpoint is called a bound vector.
Properties of Vectors
Let . The magnitude, or length, is denoted as and is defined as:
Referenced by (25 direct, 36 transitive)
Direct references:
- Dot Product
- projection
- note-5
- note-9
- Vector Equality
- Unit Vector
- Cross Product
- proof-of-sequence-in-rk-converges-iff-its-components-converge
- Polar Coordinates
- Vector Differential Calculus
- Differentiable
- theorem-19
- proof-of-theorem-19
- remark-36
- invariance-of-curl
- intuition-13
- Outer Measure
- Vector Valued Continuously Differentiable implies Lipschitz Continuous
- Real Numbers
- Coulomb's Law
- Electric Fields
- Gauss's Law
- Gravitation
- Newtonian Motion
- rn-vector-space-vs-metric-space
Transitive (depth 1):
- remark-23
- Continuously Differentiable
- gradient-as-surface-normal-vector
- Laplacian
- Level Surface
- Mean Value Inequality for Vector Valued Functions
- Mean Value Theorem
- proof-of-mean-value-inequality-for-vector-valued-functions
- remark-9
- Second Derivative Test for Convexity
- Total Derivative
- cauchy-schwarz-inequality-intuition
- proof-of-cauchy-schwarz-inequality
- remark-8
- Directional Derivative
- directional-derivative-is-inner-product-of-vector-and-grad
- Tangent Vector
Transitive (depth 2):
- grad-div-curl-related
- Gradient System
- poincare-bendixson
- Real Valued Continuously Differentiable implies Lipschitz Continuous
- Normal Derivative
- remark-45
- gravitational-potential-is-a-solution-to-laplaces-equation
- Tangent Plane
- proof-of-vector-valued-continuously-differentiable-implies-lipschitz-continuous
- proof-of-real-valued-continuously-differentiable-implies-lipschitz-continuous
- proof-of-log-is-concave
- Stoke's Theorem
- Total derivatives are unique
Transitive (depth 3):
- closed-orbits-impossible-in-gradient-systems
- remark-30
- find-tangent-plane
- proof-of-gradient-as-surface-normal-vector
- Surface Normal
Transitive (depth 4):
This is essentially the Pythagorean theorem in dimensions; in the magnitude of a vector corresponds to the length of the hypotenuse of a right triangle whose other sides are of length and
While we define norm to be equivalent to magnitude here, this is actually a special case of the more general concept of a norm - there are other norms we could define on but I don't have need to explore that yet.
The direction of a vector can be specified by the angle between it and some fixed reference, such as the -axis.
Referenced by (19 direct, 7 transitive)
Direct references:
- Dot Product
- projection
- note-5
- Zero Vector
- Vector Equality
- Cross Product
- Vector Differential Calculus
- Directional Derivative
- directional-derivative-is-inner-product-of-vector-and-grad
- theorem-19
- proof-of-theorem-19
- Surface Normal Vector
- Normal Derivative
- remark-30
- remark-36
- invariance-of-curl
- remark-45
- remark-46
- Geometric Problems
The zero vector is denoted as and has no direction.
Referenced by (2 direct)
Direct references:
Referenced by (7 direct, 10 transitive)
Direct references:
Transitive (depth 1):
Transitive (depth 2):
Transitive (depth 3):
Transitive (depth 4):
We perform vector addition by adding two vectors and according to the following rule:
that is, by making a new vector where the coordinates are the sums of the respective coordinates in the vectors being summed.
Referenced by (3 direct, 166 transitive)
Direct references:
Transitive (depth 1):
- Linear Transformation
- Null Space
- Null Space and Range are Subspaces
- proof-of-complex-inner-product-space
- Range is span of transformation of domain basis
- rank
- Rank Nullity Theorem
- rn-vector-space-vs-metric-space
- Scalar
- Vector
Transitive (depth 2):
- Weakly Nonlinear Oscillator
- Basis and Dimension for Column Space
- Basis and Dimension for Left Nullspace
- Basis and Dimension for Nullspace
- Basis and Dimension for Row Space
- Hessian Matrix
- note-3
- Parallel
- remark-32
- Scalar Function
- Vector Multiplication by a Scalar
- Bound vector
- Cauchy-Schwarz Inequality
- Cross Product
- Difference equation
- Direction
- Expected Value
- Four Fundamental Subspaces
- Free vector
- Gradient
- incompressible
- Inner product
- intuition-13
- invariance-of-curl
- Jacobian Matrix
- Layer
- Left Stochastic Matrix
- Linear Combination
- Neuron
- note-5
- note-9
- Perpendicular
- Probability Vector
- projection
- proof-of-gradient-as-surface-normal-vector
- proof-of-theorem-9
- Pythagorean Theorem
- remark-16
- remark-36
- remark-45
- remark-9
- Stochastic Matrix
- Surface Normal Vector
- Tangent Space
- theorem-19
- Unit Vector
- Vector Equality
- Vector Function
Transitive (depth 3):
- proof-of-mean-value-inequality-for-vector-valued-functions
- remark-23
- Directional Derivative
- directional-derivative-is-inner-product-of-vector-and-grad
- Normal Derivative
- proof-of-theorem-19
- remark-30
- remark-46
- Zero Vector
- Cross-Entropy
- Entropy
- Entropy Rate
- Jensen's Inequality
- markov-inequality-fish
- Interpretation
- markovs-inequality-note
- proof-of-markovs-inequality
- proof-of-weak-asymptotic-equipartition-property
- Row space and nullspace are orthogonal complements
- grad-div-curl-related
- gradient-as-surface-normal-vector
- Laplacian
- Potential Function
- remark-12
- is an inner product space
- cos-characterization-of-dot-product
- proof-of-fourier-basis
- remark-5
- theorem-7-remark
- Vector Valued Continuously Differentiable implies Lipschitz Continuous
- Neural Network
- Doubly Stochastic Matrix
- Convex Combination
- The Four Ways to View Matrix Multiplication
- intuition-2
- proof-of-pythagorean-theorem
- Surface Normal
- Doubly stochastic maps increase entropy
- probability-vectors-form-a-convex-compact-set
- proof-of-convexity-of-relative-entropy
- proof-of-probability-vectors-form-a-convex-compact-set
- cauchy-schwarz-inequality-intuition
- proof-of-cauchy-schwarz-inequality
- remark-8
- Gradient System
- note-19
- proof-of-strong-asymptotic-equipartition-property-shannon-mcmillan-breiman
- remark-3
- Tangent Vector
- Divergence Theorem of Gauss
- Line Integral of Vector Function
- Stoke's Theorem
- Surface Integral over Vector Field
- Vector Field
Transitive (depth 4):
- doubly-stochastic-maps-increase-entropy-intuition
- note-21
- note-8
- proof-of-concavity-and-convexity-of-mutual-information
- proof-of-jensens-inequality
- Gibbs' Inequality
- KL Divergence
- Model Training
- proof-of-concavity-of-entropy
- Conditional Mutual Information
- Conditioning Reduces Entropy
- fanos-inequality-remark
- Chain rule for joint entropy
- joint-entropy-is-less-than-or-equal-to-entropy-of-parts
- lemma-93
- Noiseless channel transmitting discrete symbols
- note-25
- proof-of-doubly-stochastic-maps-increase-entropy
- proof-of-joint-entropy-is-less-than-or-equal-to-entropy-of-parts
- proof-of-uniform-distribution-maximizes-entropy
- remark-14
- theorem-12
- theorem-16
- Uniform distribution maximizes entropy
- Weak Asymptotic Equipartition Property
- Strong Asymptotic Equipartition Property (Shannon--McMillan--Breiman)
- closed-orbits-impossible-in-gradient-systems
- proof-of-corollary-95
- proof-of-gibbs-inequality
- proof-of-lemma-93
- proof-of-log-sum-inequality
- gravitational-potential-is-a-solution-to-laplaces-equation
- Tangent Plane
- Curl
- Divergence
- Divergence Theorem, or, Gauss's Theorem
- Irrotational
- poincare-bendixson
Transitive (depth 5):
- proof-of-chain-rule-for-mutual-information
- proof-of-fanos-inequality
- proof-of-nonnegativity-of-conditional-mutual-information
- proof-of-nonnegativity-of-conditional-relative-entropy
- proof-of-nonnegativity-of-mutual-information
- proof-of-nonnegativity-of-relative-entropy
- proof-of-chain-rule-for-entropy
- proof-of-conditioning-reduces-entropy
- proof-of-mutual-information-and-entropy
- Conditional Relative Entropy
- convexity-of-relative-entropy-intuition
- Mutual Information
- proof-of-chain-rule-for-relative-entropy
- find-tangent-plane
- concavity-of-entropy-intuition
Transitive (depth 6):
Geometrically, vector addition connects vectors head to tail.
Similarly, vector subtraction can be performed as:
Multiplication
Referenced by (1 direct, 167 transitive)
Direct references:
Transitive (depth 1):
- Linear Transformation
- Null Space
- Null Space and Range are Subspaces
- proof-of-complex-inner-product-space
- Range is span of transformation of domain basis
- rank
- Rank Nullity Theorem
- rn-vector-space-vs-metric-space
- Scalar
- Vector
Transitive (depth 2):
- Weakly Nonlinear Oscillator
- Basis and Dimension for Column Space
- Basis and Dimension for Left Nullspace
- Basis and Dimension for Nullspace
- Basis and Dimension for Row Space
- Hessian Matrix
- note-3
- Parallel
- remark-32
- Scalar Function
- Bound vector
- Cauchy-Schwarz Inequality
- Cross Product
- Difference equation
- Direction
- Expected Value
- Four Fundamental Subspaces
- Free vector
- Gradient
- incompressible
- Inner product
- intuition-13
- invariance-of-curl
- Jacobian Matrix
- Layer
- Left Stochastic Matrix
- Linear Combination
- Neuron
- note-15
- note-5
- note-9
- Perpendicular
- Probability Vector
- projection
- proof-of-gradient-as-surface-normal-vector
- proof-of-theorem-9
- Pythagorean Theorem
- remark-16
- remark-36
- remark-45
- remark-9
- Stochastic Matrix
- Surface Normal Vector
- Tangent Space
- theorem-19
- Unit Vector
- Vector Addition
- Vector Equality
- Vector Function
Transitive (depth 3):
- proof-of-mean-value-inequality-for-vector-valued-functions
- remark-23
- Directional Derivative
- directional-derivative-is-inner-product-of-vector-and-grad
- Normal Derivative
- proof-of-theorem-19
- remark-30
- remark-46
- Zero Vector
- Cross-Entropy
- Entropy
- Entropy Rate
- Jensen's Inequality
- markov-inequality-fish
- Interpretation
- markovs-inequality-note
- proof-of-markovs-inequality
- proof-of-weak-asymptotic-equipartition-property
- Row space and nullspace are orthogonal complements
- grad-div-curl-related
- gradient-as-surface-normal-vector
- Laplacian
- Potential Function
- remark-12
- is an inner product space
- cos-characterization-of-dot-product
- proof-of-fourier-basis
- remark-5
- theorem-7-remark
- Vector Valued Continuously Differentiable implies Lipschitz Continuous
- Neural Network
- Doubly Stochastic Matrix
- Convex Combination
- The Four Ways to View Matrix Multiplication
- intuition-2
- proof-of-pythagorean-theorem
- Surface Normal
- Doubly stochastic maps increase entropy
- probability-vectors-form-a-convex-compact-set
- proof-of-convexity-of-relative-entropy
- proof-of-probability-vectors-form-a-convex-compact-set
- cauchy-schwarz-inequality-intuition
- proof-of-cauchy-schwarz-inequality
- remark-8
- Gradient System
- note-19
- proof-of-strong-asymptotic-equipartition-property-shannon-mcmillan-breiman
- remark-3
- Tangent Vector
- Divergence Theorem of Gauss
- Line Integral of Vector Function
- Stoke's Theorem
- Surface Integral over Vector Field
- Vector Field
Transitive (depth 4):
- doubly-stochastic-maps-increase-entropy-intuition
- note-21
- note-8
- proof-of-concavity-and-convexity-of-mutual-information
- proof-of-jensens-inequality
- Gibbs' Inequality
- KL Divergence
- Model Training
- proof-of-concavity-of-entropy
- Conditional Mutual Information
- Conditioning Reduces Entropy
- fanos-inequality-remark
- Chain rule for joint entropy
- joint-entropy-is-less-than-or-equal-to-entropy-of-parts
- lemma-93
- Noiseless channel transmitting discrete symbols
- note-25
- proof-of-doubly-stochastic-maps-increase-entropy
- proof-of-joint-entropy-is-less-than-or-equal-to-entropy-of-parts
- proof-of-uniform-distribution-maximizes-entropy
- remark-14
- theorem-12
- theorem-16
- Uniform distribution maximizes entropy
- Weak Asymptotic Equipartition Property
- Strong Asymptotic Equipartition Property (Shannon--McMillan--Breiman)
- closed-orbits-impossible-in-gradient-systems
- proof-of-corollary-95
- proof-of-gibbs-inequality
- proof-of-lemma-93
- proof-of-log-sum-inequality
- gravitational-potential-is-a-solution-to-laplaces-equation
- Tangent Plane
- Curl
- Divergence
- Divergence Theorem, or, Gauss's Theorem
- Irrotational
- poincare-bendixson
Transitive (depth 5):
- proof-of-chain-rule-for-mutual-information
- proof-of-fanos-inequality
- proof-of-nonnegativity-of-conditional-mutual-information
- proof-of-nonnegativity-of-conditional-relative-entropy
- proof-of-nonnegativity-of-mutual-information
- proof-of-nonnegativity-of-relative-entropy
- proof-of-chain-rule-for-entropy
- proof-of-conditioning-reduces-entropy
- proof-of-mutual-information-and-entropy
- Conditional Relative Entropy
- convexity-of-relative-entropy-intuition
- Mutual Information
- proof-of-chain-rule-for-relative-entropy
- find-tangent-plane
- concavity-of-entropy-intuition
Transitive (depth 6):
If and are vectors in then their inner product is defined as
Referenced by (12 direct, 69 transitive)
Direct references:
Transitive (depth 1):
- proof-of-theorem-19
- Cross-Entropy
- Entropy
- Entropy Rate
- Jensen's Inequality
- markov-inequality-fish
- Interpretation
- markovs-inequality-note
- proof-of-markovs-inequality
- proof-of-weak-asymptotic-equipartition-property
- Cross Product
- intuition-13
- intuition-2
- projection
- proof-of-gradient-as-surface-normal-vector
- proof-of-pythagorean-theorem
- Pythagorean Theorem
- remark-30
- Surface Normal
Transitive (depth 2):
- Gibbs' Inequality
- KL Divergence
- Model Training
- proof-of-concavity-of-entropy
- remark-23
- Conditional Mutual Information
- Conditioning Reduces Entropy
- doubly-stochastic-maps-increase-entropy-intuition
- fanos-inequality-remark
- Chain rule for joint entropy
- joint-entropy-is-less-than-or-equal-to-entropy-of-parts
- lemma-93
- Noiseless channel transmitting discrete symbols
- note-25
- note-8
- proof-of-doubly-stochastic-maps-increase-entropy
- proof-of-joint-entropy-is-less-than-or-equal-to-entropy-of-parts
- proof-of-uniform-distribution-maximizes-entropy
- remark-14
- theorem-12
- theorem-16
- Uniform distribution maximizes entropy
- Weak Asymptotic Equipartition Property
- Strong Asymptotic Equipartition Property (Shannon--McMillan--Breiman)
- proof-of-corollary-95
- proof-of-gibbs-inequality
- proof-of-lemma-93
- proof-of-log-sum-inequality
- cauchy-schwarz-inequality-intuition
- proof-of-cauchy-schwarz-inequality
- remark-8
- Surface Normal Vector
Transitive (depth 3):
- proof-of-chain-rule-for-mutual-information
- proof-of-fanos-inequality
- proof-of-nonnegativity-of-conditional-mutual-information
- proof-of-nonnegativity-of-conditional-relative-entropy
- proof-of-nonnegativity-of-mutual-information
- proof-of-nonnegativity-of-relative-entropy
- proof-of-chain-rule-for-entropy
- proof-of-conditioning-reduces-entropy
- proof-of-mutual-information-and-entropy
- Conditional Relative Entropy
- convexity-of-relative-entropy-intuition
- Mutual Information
- proof-of-chain-rule-for-relative-entropy
- proof-of-concavity-and-convexity-of-mutual-information
- remark-32
- gradient-as-surface-normal-vector
- concavity-of-entropy-intuition
Transitive (depth 4):
The dot product of and is
where is the angle between and
Referenced by (1 direct)
Direct references:
Two vectors and are said to be perpendicular or orthogonal if the angle between them is radians, or, equivalently, if the inner product
Referenced by (18 direct, 6 transitive)
Direct references:
- Dot Product
- projection
- intuition-2
- Pythagorean Theorem
- proof-of-pythagorean-theorem
- Cross Product
- Polar Coordinates
- Surface Normal
- proof-of-gradient-as-surface-normal-vector
- remark-30
- intuition-13
- Harmonic Functions
- Discrete Fourier Transform
- proof-of-fourier-basis
- Interpolation and Polynomial Approximation
- Numerical Integration
- Electric Potential
- Newtonian Motion
Transitive (depth 1):
- remark-23
- cauchy-schwarz-inequality-intuition
- proof-of-cauchy-schwarz-inequality
- remark-8
- Surface Normal Vector
Transitive (depth 2):
Two vectors and are said to be parallel if is a scalar multiple of , i.e., if there exists some scalar where
Referenced by (1 direct)
Direct references:
The cross product (read "a cross b") of two vectors and is the vector denoted by
I. If or then we define
II. If both vectors are nonzero vectors, then vector has the length
where is the angle between and Furthermore, and form the sides of a parallelogram on a plane in space. The area of this parallelogram is precisely given by (1), such that the length of the vector is equal to the area of the parallelogram.
III. If and lie in the same straight line, i.e. and have the same or opposite directions, then is or so that In that case so that
IV. If cases I and III do not occur, then is a nonzero vector. The direction of is perpendicular to both and such that precisely in this order, form a right-handed triple
Referenced by (2 direct)
Direct references:
If and then we use the symbolic determinant to find the cross product, in this fashion:
Let and Then, the vector
is called a linear combination of
Let The set of all linear combinations of is called their span, denoted
That is:
Referenced by (6 direct, 10 transitive)
Direct references:
Transitive (depth 1):
- Basis and Dimension for Left Nullspace
- doubly-stochastic-maps-increase-entropy-intuition
- note-21
- note-8
- proof-of-concavity-and-convexity-of-mutual-information
- proof-of-jensens-inequality
- proof-of-probability-vectors-form-a-convex-compact-set
- Basis and Dimension for Column Space
- Basis and Dimension for Nullspace
- Row space and nullspace are orthogonal complements
TODO: define this.
Referenced by (2 direct)
Direct references:
TODO: define this.
Referenced by (8 direct, 4 transitive)
Direct references:
Matrices
An matrix with entries from a field is a rectangular array of the form
where each entry is an element of .
Referenced by (14 direct, 21 transitive)
Direct references:
- matrix product
- Four Fundamental Subspaces
- Basis and Dimension for Column Space
- Basis and Dimension for Nullspace
- Basis and Dimension for Row Space
- Basis and Dimension for Left Nullspace
- Vector Differential Calculus
- Hessian Matrix
- Jacobian Matrix
- remark-46
- Doubly stochastic maps increase entropy
- Layer
- Markov Chain
- Doubly Stochastic Matrix
Transitive (depth 1):
- note-21
- Row space and nullspace are orthogonal complements
- remark-45
- Vector Valued Continuously Differentiable implies Lipschitz Continuous
- Neural Network
- Conditional Independence Given Intermediate Step
- Ergodic
- note-19
- proof-of-characterization-of-sufficiency
- proof-of-data-processing-inequality
- proof-of-fanos-inequality
- proof-of-processing-cannot-increase-information
- Reversibility of Markov Chains
- Stationary Distribution of an Ergodic Chain
- Strong Asymptotic Equipartition Property (Shannon--McMillan--Breiman)
- Sufficient Statistic
Transitive (depth 2):
Referenced by (1 direct, 72 transitive)
Direct references:
Transitive (depth 1):
- Curl
- del
- Directional Derivative
- Hessian Matrix
- Jacobian Matrix
- Left Stochastic Matrix
- Line Integral of Vector Function
- Markov Chain
- Matrix
- note-3
- Partial Derivative
- Probability Vector
- remark-36
- The Four Ways to View Matrix Multiplication
- remark-46
- stationary-distribution-note
- Stochastic Matrix
- Stoke's Theorem
Transitive (depth 2):
- grad-div-curl-related
- Irrotational
- directional-derivative-is-inner-product-of-vector-and-grad
- Normal Derivative
- proof-of-theorem-19
- remark-45
- Vector Valued Continuously Differentiable implies Lipschitz Continuous
- Doubly Stochastic Matrix
- Conditional Independence Given Intermediate Step
- Ergodic
- note-19
- proof-of-characterization-of-sufficiency
- proof-of-data-processing-inequality
- proof-of-fanos-inequality
- proof-of-processing-cannot-increase-information
- Reversibility of Markov Chains
- Stationary Distribution of an Ergodic Chain
- Strong Asymptotic Equipartition Property (Shannon--McMillan--Breiman)
- Sufficient Statistic
- Basis and Dimension for Column Space
- Basis and Dimension for Left Nullspace
- Basis and Dimension for Nullspace
- Basis and Dimension for Row Space
- Doubly stochastic maps increase entropy
- Four Fundamental Subspaces
- Layer
- matrix product
- analytic-implies-cr-equations
- Cauchy-Riemann Equations
- Divergence Theorem of Gauss
- Gradient
- Green's Theorem
- Laplacian
- proof-of-analytic-implies-cr-equations
- proof-of-cauchys-integral-theorem
- theorem-19
- probability-vectors-form-a-convex-compact-set
- proof-of-convexity-of-relative-entropy
- proof-of-probability-vectors-form-a-convex-compact-set
- proof-of-strong-asymptotic-equipartition-property-shannon-mcmillan-breiman
Transitive (depth 3):
- note-21
- Row space and nullspace are orthogonal complements
- gradient-as-surface-normal-vector
- Potential Function
- remark-12
- remark-16
- gravitational-potential-is-a-solution-to-laplaces-equation
- Neural Network
- remark-30
- proof-of-corollary-77
- Characterization of Sufficiency
- characterization-of-sufficiency-example
- Minimal Sufficient Statistic
Transitive (depth 4):
A specific element in an n-tuple is called an entry of the -tuple, and indicates its ordinal position, i.e. is the th entry in
Referenced by (24 direct, 54 transitive)
Direct references:
- matrix
- note-3
- The Four Ways to View Matrix Multiplication
- Matrix
- n-tuple
- Vector Differential Calculus
- Partial Derivative
- del
- Directional Derivative
- Curl
- remark-36
- Hessian Matrix
- Jacobian Matrix
- remark-46
- Line Integral of Vector Function
- Stoke's Theorem
- Discrete Fourier Transform
- Newtonian Motion
- Markov Chain
- stationary-distribution-note
- Probability Vector
- Stochastic Matrix
- Left Stochastic Matrix
- Projective Transformations
Transitive (depth 1):
- grad-div-curl-related
- Irrotational
- directional-derivative-is-inner-product-of-vector-and-grad
- Normal Derivative
- proof-of-theorem-19
- remark-45
- Vector Valued Continuously Differentiable implies Lipschitz Continuous
- Doubly Stochastic Matrix
- Conditional Independence Given Intermediate Step
- Ergodic
- note-19
- proof-of-characterization-of-sufficiency
- proof-of-data-processing-inequality
- proof-of-fanos-inequality
- proof-of-processing-cannot-increase-information
- Reversibility of Markov Chains
- Stationary Distribution of an Ergodic Chain
- Strong Asymptotic Equipartition Property (Shannon--McMillan--Breiman)
- Sufficient Statistic
- Basis and Dimension for Column Space
- Basis and Dimension for Left Nullspace
- Basis and Dimension for Nullspace
- Basis and Dimension for Row Space
- Doubly stochastic maps increase entropy
- Four Fundamental Subspaces
- Layer
- matrix product
- analytic-implies-cr-equations
- Cauchy-Riemann Equations
- Divergence Theorem of Gauss
- Gradient
- Green's Theorem
- Laplacian
- proof-of-analytic-implies-cr-equations
- proof-of-cauchys-integral-theorem
- theorem-19
- probability-vectors-form-a-convex-compact-set
- proof-of-convexity-of-relative-entropy
- proof-of-probability-vectors-form-a-convex-compact-set
- proof-of-strong-asymptotic-equipartition-property-shannon-mcmillan-breiman
Transitive (depth 2):
- note-21
- Row space and nullspace are orthogonal complements
- gradient-as-surface-normal-vector
- Potential Function
- remark-12
- remark-16
- gravitational-potential-is-a-solution-to-laplaces-equation
- Neural Network
- remark-30
- proof-of-corollary-77
- Characterization of Sufficiency
- characterization-of-sufficiency-example
- Minimal Sufficient Statistic