lacunary - Mathnotes

Multivariable Differential Operators

First, some preliminary definitions.

Definition: Vector Function \@{vector-function}

Let P be any point in a domain of definition. Then we define a vector function v whose values are vectors, that is,

v=v(P)=[v1(P),v2(P),v3(P)]

that depends on points P in space. A vector function depends only on the point P, not on the coordinate system chosen to represent its components.

Definition: Scalar Function (also: scalar field) \@{scalar-function}

Let P be any point in a domain of definition. Then we define a scalar function f, whose values are scalars, that is,

f=f(p)

that depends on P.

A scalar function depends only on the point P, not on the coordinate system chosen to represent it.

Definition: Component Function \@{component-function}

Let f be a function that maps an open set E⊂Rn into Rm. Let {e1,…,en} and {u1,…,un} be the standard bases of Rn and Rm. The component functions of f are the real functions f1,…,fm defined by

f(x)=∑i=1mfi(x)ui(x∈E),

or, equivalently, by fi(x)=f(x)⋅ui,1≤i≤m.

Definition: Partial Derivative \@{partial-derivative}

Using the setup from the definition of component, for x∈E,1≤i≤m,1≤j≤n, we define

(Djfi)(x)=limt→0fi(x+tej)−fi(x)t,

provided the limit exists. Writing fi(x1,…,xn) in place of fi(x), we see that Djfi is the derivative of fi with respect to xj, keeping the other variables fixed. The notation

∂fi∂xj

is therefore often used in place of Djfi, and Djfi is called a partial derivative.

Definition: Tangent Vector \@{tangent-vector}

For a space curve given parametrically by r(t), the tangent vector (or specifically, the unit tangent vector) at the point r(t) is the unit vector defined by:

T(t)=r′(t)‖r′(t)‖

Referenced by (2 direct, 5 transitive)

Differentiation

Remark \@{remark-7}

If f is a real function with domain of definition (a,b)⊂R1 and if x∈(a,b), then f′(x) is usually defined to be the real number

limh→0(x+h)−f(x)h,

provided this limit exists. Thus,

f(x+h)−f(x)=f′(x)h+r(h)(8)

where the "remainder" r(h) is small, that is,

limh→0=r(h)h=0.

Note that (8) expresses the difference f(x+h)−f(x) as the sum of the linear function that takes h to f′(x)h, plus a small remainder. We can therefore view the derivative of f at x as the linear operator on R1 that takes h to f′(x)h.

Note \@{note-8}

On linearity: every real number α gives rise to a linear operator on R1:

Lαx=αx.

Conversely, every linear function from R1 to R1 is multiplication by some real number. Thus, we have a 1−1 correspondence between R1 and L(R1) (the set of all linear transformations on R1.)

Remark \@{remark-9}

Now, consider a function f that maps (a,b)⊂R1 into Rm. Then f′(x) is defined to be the vector y∈Rm (if one exists) for which

limh→0(f(x+h)−f(x)h−y)=0.

This can be rewritten as

f(x+h)−f(x)=hy+r(h),

where r(h)/h→0 as h→0. Again, the right side is a linear function of h. Every y∈Rm gives a linear transformation of R1 into Rm:

Lyh=hy,

which gives us an identification of Rm with L(R1,Rm) (the set of all linear transformations from R1 to Rm,) allowing us to say f′(x)∈L(R1,Rm).

Therefore, if f is a differentiable function of (a,b)⊂R1 into Rm, and if x∈(a,b), then f′(x) is the linear transformation of R1 into Rm that satisfies

limh→0f(x+h)−f(x)−f′(x)hh=0,

or, equivalently,

limh→0|f(x+h)−f(x)−f′(x)h||h|=0.

We can now take on the general Rn→Rm case.

Definition: Differentiable \@{differentiable}

Suppose E is an open set in Rn, f:E→Rm, and x∈E. If there exists a linear transformation A:Rn→Rm such that

limh→0|f(x+h)−f(x)−Ah||h|=0,(14)

then we say that f is differentiable at x and we write

f′(x)=A.

If f is differentiable at every x∈E, we say that f is differentiable in E.

Note that in (14), h∈Rn. If |h| is small enough, then x+h∈E, because E is open. Therefore, f(x+h) is defined, f(x+h)∈Rm, and since A∈L(Rn,Rm), Ah∈Rm. Therefore,

f(x+h)−f(x)−Ah∈Rm.

The norm in the numerator of (14) is that of Rm, while the norm in the denominator is the Rn-norm.

We can also rewrite (14) as

f(x+h)−f(x)=f′(x)h+r(h),(17)

where the remainder r(h) satisfies

limh→0|r(h)||h|=0.

This means that for a fixed x and a small h, the left side of (17) is approximately equal to f′(x)h, that is, to the value of a linear transformation applied to h.

Definition: Total Derivative \@{total-derivative}

The derivative defined above in differentiable is called the total derivative of f at x, or the differential of f at x.

Theorem: Total derivatives are unique \@{total-derivatives-are-unique}

Suppose E is an open set in Rn, f:E→Rm, and x∈E, and that A1 and A2 are total derivatives of f at x. Then, A1=A2.

Definition: Continuously Differentiable \@{continuously-differentiable}

A differentiable function f of an open set E⊂Rn into Rm is said to be continuously differentiable in E if f′ is a continuous function of E into L(Rn,Rm). More explicitly, it is required that to every x∈E and to every ϵ>0 corresponds a δ>0 such that

‖f′(y)−f′(x)‖<ϵ

if y∈E and |x−y|<δ.

If this is the case, we also say that f is a C′-mapping or that f∈C′(E).

These are some differential operators that apply to multivariable and/or vector valued functions.

Vector Differential Operator

Definition: del (also: vector differential operator, nabla) \@{del}

In Euclidean space Rn with coordinates (x1,…,xn) and standard basis (e1,…,en), del is a vector operator whose x1,…,xn components are the partial derivative operators ∂∂x1,…,∂∂xn; that is

∇=∑i=1nei∂∂xi=(∂∂x1,…,∂∂xn).

Gradient

Definition: Gradient \@{gradient}

Given a scalar function f:Rn→R, the gradient of f, denoted as ∇f, is defined as the vector of its partial derivatives. Specifically, for a function f(x1,x2,⋯,xn), the gradient is given by

∇f=[∂f∂x1∂f∂x2 ⋮ ∂f∂xn]

Remark \@{remark-16}

We can write the gradient as either a row vector or a column vector. Different texts use different conventions. I'll show it here as a row vector, while we used a column vector above, just to mix it up.

In three-dimensional Euclidean space, the gradient of a function f, if it exists, is given by

gradf=∇f=∂f∂xi+∂f∂yj+∂f∂zk=[∂f∂x,∂f∂y,∂f∂z]

Directional Derivative

Definition: Directional Derivative \@{directional-derivative}

Using the setup from the definition of component, let us fix an x∈E (with E⊂Rn,) and let u∈Rn be a unit vector. Then,

limt→0f(x+tu)−f(x)t=(∇f)(x)⋅u

is called the directional derivative of f at x in the direction of the unit vector u and is denoted as Duf(x).

Theorem \@{directional-derivative-is-inner-product-of-vector-and-grad}

The directional derivative of f in the direction of a unit vector u is the inner product of u and gradf, that is,

Du=u⋅gradf.

Referenced by (1 direct)

Direct references:

Theorem \@{theorem-19}

Let f(P)=f(x,y,z) be a scalar function having continuous first partial derivatives in some domain B in space. Then, gradf exists in B and is a vector, that is, its length and direction are independent of the particular choice of Cartesian coordinates. If gradf(P)≠0 at some point P, it has the direction of maximum increase of f at P.

Proof \@{proof-of-theorem-19}

The directional derivative of f in the direction of some unit vector u is

Duf=u⋅gradf=|u||gradf|cos⁡θ(a)

where θ is the angle between u and gradf (see The directional derivative of f in the... and The dot product of u and v...). Note that f is a scalar function, as is the directional derivative of f. Now, cosθ has its maximum value of 1 whenever θ=0, and since u is a unit vector with magnitude of 1, (a) simplifies to

Duf=|gradf|,

which tells us that that direction and magnitude of gradf are independent of the coordinate system chosen. Now, since θ=0 if and only if b and gradf are parallel, gradf is the direction of maximum increase of f at P, assuming gradf≠0 at P.

◻
Definition: Level Surface \@{level-surface}

Let S be a surface represented by f(x,y,z)=c, where c is constant and f is differentiable. Such a surface is called a level surface of f, and for different c, we get different level surfaces.

Referenced by (1 direct, 5 transitive)

Direct references:

Definition: Tangent Plane \@{tangent-plane}

If S is a level surface of a function f, and P is a point of S, then the set of all tangent vectors of all curves passing through P will generally form a plane, called the tangent plane of S at P.

Referenced by (3 direct, 2 transitive)

Transitive (depth 1):

Transitive (depth 2):

Definition: Surface Normal \@{surface-normal}

Given a tangent plane to a surface S at P, the normal to this plane (the straight line through P perpendicular to the tangent plane) is called the surface normal to S at P.

Referenced by (1 direct, 2 transitive)

Direct references:

Definition: Surface Normal Vector \@{surface-normal-vector}

A vector in the direction of the surface normal of a surface S at point P is called a surface normal vector of S at P.

Theorem \@{gradient-as-surface-normal-vector}

Let f be a differentiable scalar function in space. Let f(x,y,z)=c (with c constant) represent a surface S. Then, if the gradient of f at a point P of S is not the zero vector, it is a surface normal vector of S at P.

Proof \@{proof-of-gradient-as-surface-normal-vector}

Any curve C lying in S can be parameterized as r=[x(t),y(t),z(t)] such that

f(x(t),y(t),z(t))=c.(a)

Now, if we differentiate (a) with respect to t, we get

dfdt=∂f∂xx′+∂f∂yy′+∂f∂zz′=(gradf)⋅r′=0.

Therefore, gradf is orthogonal to all the vectors r′ in the tangent plane of S at P, and is therefore a surface normal vector of S at P.

◻
Theorem \@{find-tangent-plane}

If S is given by F(x,y,z)=0, then at P=(x0,y0,z0), the tangent plane is

∇F(P)⋅⟨x−x0,y−y0,z−z0⟩=0.

Definition: Potential Function \@{potential-function}

A potential function is a scalar function whose gradient is a vector field. They allow representing certain vector fields in a simpler, more fundamental form.

In other words, given a vector field F, a potential function ϕ is a scalar function such that

F=−gradϕorF=gradϕ.

Normal Derivative

Definition: Normal Derivative \@{normal-derivative}

The normal derivative is the directional derivative in the direction of the @normal-vector.

Referenced by (1 direct)

Direct references:

Remark \@{remark-30}

The normal derivative tells us how the value of a function changes as we move in the direction normal (orthogonal) to a curve or surface.

Divergence

Definition: Divergence \@{divergence}

For a vector field F=(F1,F2,F3) defined in three-dimensional space R3, the divergence is defined as

divF=∇⋅F=∂F1∂x+∂F2∂y+∂F3∂z.

Remark \@{remark-32}

The divergence of a vector field quantifies the extent to which the vector field behaves as a source or a sink at a given point.

Divergence gives a scalar value for each point, i.e., it is a scalar field. A positive value indicates a net flow away from the point, while a negative value indicates a net flow towards a point.

Theorem: Divergence Theorem, or, Gauss's Theorem \@{divergence-theorem-or-gausss-theorem}

An important theorem related to divergence is the Divergence Theorem (also known as Gauss's theorem) which connects the flux of a vector field through a closed surface to the divergence of the field inside the volume bounded by the surface:

∭V(∇⋅F)dV=∮sF⋅dS

Definition: incompressible \@{incompressible}

Let v be the velocity vector of of the motion of particles in a fluid. If divv=0, then the fluid has constant density and is said to be incompressible.

Curl

Definition: Curl \@{curl}

For a vector field F=(F1,F2,F3) defined in three-dimensional space R3, with each component function Fi depending on the variables x, y, and z, the curl of F is defined as

curlF=∇×F=(∂F3∂y−∂F2∂z)i+(∂F1∂z−∂F3∂x)j+(∂F2∂x−∂F1∂y)k

Remark \@{remark-36}

The curl at a point in the field is represented by a vector whose length and direction denote the magnitude and axis of the maximum circulation. Circulation is the line integral of a vector field around a closed curve.

More intuitively, curl measures the rotation of the vector field at a given point.

Curl can also be expressed as the determinant of a 3x3 matrix involving the unit vectors, partial derivatives, and the components of the vector field:

∇×F=|ijk ∂∂x∂∂y∂∂z F1F2F3|

Definition: Irrotational \@{irrotational}

If the curl of a vector field is 0, i.e. if curlv=0, the field is said to be irrotational.

Referenced by (1 direct)

Direct references:

Theorem \@{invariance-of-curl}

curlv is a vector. It has a length and a direction that are independent of the particular choice of a Cartesian coordinate system in space.

Laplacian

Definition: Laplacian \@{laplacian}

The Laplace operator is a second-order differential operator in the n-dimensional Euclidean space, defined as the divergence (∇⋅) of the gradient (∇f). Thus, if f is a twice differentiable real-valued function, then the Laplacian of f is the real-valued function defined by

Δf=∇2f=∇⋅∇f.

The Laplacian of f is the sum of all the unmixed second partial derivatives in the Cartesian coordinates xi:

∇2f=∑i=1n∂2f∂xi2.

In two dimensions, using Cartesian coordinates, the Laplace operator is given by

∇2f=∂2f∂x2+∂2f∂y2

and in three dimensions by

∇2f=∂2f∂x2+∂2f∂y2+∂2f∂z2

Note \@{note-41}

Solutions to Laplace's equation ∇2f=0 are called harmonic functions.

Theorem \@{gravitational-potential-is-a-solution-to-laplaces-equation}

The force of attraction

p=−cr3r=−c[x−x0r3,y−y0r3,z−z0r3]

between two particles at points P0=(x0,y0,z0) and P=(x,y,z) (as given by Newton's law of gravitation) has the potential function f(x,y,z)=c/r, where r>0 is the distance between P0 and P.

Thus, p=gradf=grad(c/r). This potential function f is a solution of Laplace's Equation

∇2f=∂2f∂x2+∂2f∂y2+∂2f∂z2=0,

that is, f has a Laplacian of 0.

Hessian

Definition: Hessian Matrix (also: Hessian) \@{hessian-matrix}

Suppose f : Rn→R is a function taking as input a vector x∈Rn and outputting a scalar f(x)∈R. If all second-order partial derivatives of f exist, then the Hessian matrix H of f is a square n×n matrix, usually defined and arranged as

Hf=[∂2f∂x12∂2f∂x1∂x2⋯∂2f∂x1∂xn∂2f∂x2∂x1∂2f∂x22⋯∂2f∂x2∂xn⋮⋮⋱⋮∂2f∂xn∂x1∂2f∂xn∂x2⋯∂2f∂xn2].

That is, the entry of the ith row and the jth column is

(Hf)i,j=∂2f∂xi∂xj.

For a function f : R3→R, this is

H=[∂2f∂x2∂2f∂x∂y∂2f∂x∂z∂2f∂y∂x∂2f∂y2∂2f∂y∂z∂2f∂z∂x∂2f∂z∂y∂2f∂z2].

Referenced by (1 direct)

Note that the Laplacian of f is equal to the @trace of the Hessian of f. In this way, the Laplacian is to the Hessian what divergence is to the Jacobian.

Jacobian

Definition: Jacobian Matrix (also: Jacobian, Total Derivative) \@{jacobian-matrix}

Let f : Rn→Rm be a function such that each of its first-order partial derivatives exists on Rn. This function takes a point x=(x1,…,xn)∈Rn as input and produces the vector f(x)=(f1(x),…,fm(x))∈Rm as output. Then the Jacobian matrix of f, denoted Jf, is the m×n matrix whose (i,j) entry is ∂fi∂xj; explicitly

[∂f∂x1⋯∂f∂xn]=[∇Tf1⋮∇Tfm]=[∂f1∂x1⋯∂f1∂xn⋮⋱⋮∂fm∂x1⋯∂fm∂xn],

where ∇Tfi is the @transpose (@row-vector) of the gradient of the i-th component.

Remark \@{remark-45}

For a vector valued function,

F:Rn→Rm,

the Jacobian J(x) is the linear map that best @approximates F near x:

F(x0+Δx)≈F(x0)+J(x0)Δx.

At a point x0∈Rn, the directional derivative of F in the direction of v∈Rn is

DvF(x0)=J(x0)v.

Remark \@{remark-46}

Given a vector field F:

F=[F1(x,y,z)F2(x,y,z)F3(x,y,z)]:R3→R3,

each component Fi is itself a scalar field, and so we can take the gradient of each one:

∇F1=[∂F1∂x∂F1∂y∂F1∂z],∇F2=[∂F2∂x∂F2∂y∂F2∂z],∇F3=[∂F3∂x∂F3∂y∂F3∂z].

Now, the Jacobian of F is the matrix that contains all of these gradients:

J=∇F=F=[(∇F1)T(∇F2)T(∇F3)T]=[∂F1∂x∂F1∂y∂F1∂z∂F2∂x∂F2∂y∂F2∂z∂F3∂x∂F3∂y∂F3∂z].

Here:

Thus the Jacobian tells us how each component of F changes with every @coordinate. It can be viewed as the "gradient of a vector field."

Given a vector field F:

F=[F1(x,y,z)F2(x,y,z)F3(x,y,z)]:R3→R3,

with a Jacobian

J=∇F=[∂F1∂x∂F1∂y∂F1∂z∂F2∂x∂F2∂y∂F2∂z∂F3∂x∂F3∂y∂F3∂z],

divergence is the @sum of the @diagonal entries of this matrix:

∇⋅F=∂F1∂x+∂F@∂y+∂F3∂z=trJ.

So, algebraically, divergence is the @trace of the Jacobian.

The trace measures how much the linear transformation represented by J stretches space along its coordinate axes - it's the infinitesimal "net expansion rate."

Geometrically, if we have a very small cube of points centered at x0, with a side length of dx, then each corner of that cube corresponds to a slightly different x=x0+Δx. If we take all the points in the box and map them through F, the cube's image becomes a tiny, possibly deformed box in the F-space. The way the box stretches, shears and rotates is determined by J. The change in volume of that box is determined by det(J) (the determinant of the Jacobian.)

For an infinitesimally small cube, the @determinant can be expanded as

det(I+Jϵ)≈1+ϵtr(J),

for tiny ϵ. Here, the @trace of J,

trJ=∂F1∂x+∂F2∂y+∂F3∂z=∇⋅F

is exactly the divergence. Each of these terms tells us how much the points in an infinitesimally small box around x0 are stretched/compressed in each coordinate direction.

Physically, for example, if F represents a velocity field, ∂F1∂x says how much the x-component of the velocity changes as you move along x. Note that if F1 is constant with respect to x, this term is 0, which means there is no stretch/compression along that axis, and if we think of the velocity being that of a fluid, the same amount of fluid will flow into any region in the x direction as flows out in the x direction.

Given a vector field F:

F=[F1(x,y,z)F2(x,y,z)F3(x,y,z)]:R3→R3,

with a Jacobian

J=∇F=[∂F1∂x∂F1∂y∂F1∂z∂F2∂x∂F2∂y∂F2∂z∂F3∂x∂F3∂y∂F3∂z],

We can split J into its @symmetric and @antisymmetric parts:

J=S+A,S=12(J+JT),A=12(J−JT).

Now,

  • S encodes stretching/shearing.
  • A encodes local rotation.

The components of A are:

A=12[0∂F1∂y−∂F2∂x∂F1∂z−∂F3∂x∂F2∂x−∂F1∂y0∂F2∂z−∂F3∂y∂F3∂x−∂F1∂z∂F3∂y−∂F2∂z0].

If we let ω=(ωx,ωy,ωz), we can write A as

A=[0−ωzωyωz0−ωx−ωyωx0].

Then,

ω=12∇×F,

so curl is twice the @axial-vector of the @antisymmetric part of the Jacobian.

Geometrically, if we linearize F near some point x0,

F(x0+Δx)≈F(x0)+J(x0)Δx.

The matrix J is acting on small displacements. When we apply A to Δx, we get

AΔx=ω×Δx.

Then,

  • The direction of ω is the axis of rotation (via the right-hand rule).

More visually, if we take a small ball of points around x0,

  • The symmetric part S of J turns the ball into an @ellipsoid (by stretching/shearing it).
  • The antisymmetric part A spins the ball about the axis ω without changing its shape or volume.

Thus, divergence comes from the @trace of S and curl comes from A.

Physically, if F is a velocity field of a fluid, then:

∇×F=2ω

means that the fluid near that point is rotating like a tiny rigid body, with @angular-velocity ω.

So, if we dropped a tiny paddle wheel into the flow:

  • The wheel's axis aligns with ∇×F.
  • Its spin rate is 12‖∇×F‖.

This is mostly covered above, so just note that the Jacobian is a sort of machine for producing directional derivatives:

DvF=Jv,

where the multiplication on the right is matrix-vector multiplication, i.e. the image of J when applied to v.

Algebraically,

∇2f=∇⋅(∇f).

To unpack that, we're saying that the Laplacian is the divergence of the gradient of f. Now, when we take the gradient of f, we get a vector field that has the first partial derivatives of f. Now, if we take the Jacobian of that gradient, we get ∇(∇f)), which is the Hessian of f. Now, if we take the @trace of that Hessian, we get the Laplacian, which is the sum of the pure second @derivatives of f.

Geometrically, the Hessian is telling us how the gradient of f curves near a point, and taking the trace of it tells us the total or average @curvature at that point.

If ∇2f=0, the gradient's inflow and outflow balance in every direction: the field is @harmonic and locally curvature-neutral.

Exterior Derivative