lacunary - Mathnotes

Multivariable Differential Operators

First, some preliminary definitions.

Definition: Vector Function \@{vector-function}

Let P be any point in a domain of definition. Then we define a vector function v whose values are vectors, that is,

v=v(P)=[v1(P),v2(P),v3(P)]

that depends on points P in space. A vector function depends only on the point P, not on the coordinate system chosen to represent its components.

Definition: Scalar Function (also: scalar field) \@{scalar-function}

Let P be any point in a domain of definition. Then we define a scalar function f, whose values are scalars, that is,

f=f(p)

that depends on P.

A scalar function depends only on the point P, not on the coordinate system chosen to represent it.

Definition: Component \@{component}

Let f be a function that maps an open set ERn into Rm. Let {e1,,en} and {u1,,un} be the standard bases of Rn and Rm. The components of f are the real functions f1,,fm defined by

f(x)=i=1mfi(x)ui(xE),

or, equivalently, by fi(x)=f(x)ui,1im.

Definition: Partial Derivative \@{partial-derivative}

Using the setup from the definition of component, for xE,1im,1jn, we define

(Djfi)(x)=limt0fi(x+tej)fi(x)t,

provided the limit exists. Writing fi(x1,,xn) in place of fi(x), we see that Djfi is the derivative of fi with respect to xj, keeping the other variables fixed. The notation

fixj

is therefore often used in place of Djfi, and Djfi is called a partial derivative.

Definition: Tangent Vector \@{tangent-vector}

For a space curve given parametrically by r(t), the tangent vector (or specifically, the unit tangent vector) at the point r(t) is the unit vector defined by:

T(t)=r(t)||r(t)||

Referenced by (1 direct)

Direct references:

Differentiation

Remark \@{remark-7}

If f is a real function with domain of definition (a,b)R1 and if x(a,b), then f(x) is usually defined to be the real number

limh0(x+h)f(x)h,

provided this limit exists. Thus,

f(x+h)f(x)=f(x)h+r(h)(8)

where the "remainder" r(h) is small, that is,

limh0=r(h)h=0.

Note that (8) expresses the difference f(x+h)f(x) as the sum of the linear function that takes h to f(x)h, plus a small remainder. We can therefore view the derivative of f at x as the linear operator on R1 that takes h to f(x)h.

Note \@{note-8}

On linearity: every real number α gives rise to a linear operator on R1:

Lαx=αx.

Conversely, every linear function from R1 to R1 is multiplication by some real number. Thus, we have a 11 correspondence between R1 and L(R1) (the set of all linear transformations on R1.)

Remark \@{remark-9}

Now, consider a function f that maps (a,b)R1 into Rm. Then f(x) is defined to be the vector yRm (if one exists) for which

limh0(f(x+h)f(x)hy)=0.

This can be rewritten as

f(x+h)f(x)=hy+r(h),

where r(h)/h0 as h0. Again, the right side is a linear function of h. Every yRm gives a linear transformation of R1 into Rm:

Lyh=hy,

which gives us an identification of Rm with L(R1,Rm) (the set of all linear transformations from R1 to Rm,) allowing us to say f(x)L(R1,Rm).

Therefore, if f is a differentiable function of (a,b)R1 into Rm, and if x(a,b), then f(x) is the linear transformation of R1 into Rm that satisfies

limh0f(x+h)f(x)f(x)hh=0,

or, equivalently,

limh0|f(x+h)f(x)f(x)h||h|=0.

We can now take on the general RnRm case.

Definition: Differentiable \@{differentiable}

Suppose E is an open set in Rn, f:ERm, and xE. If there exists a linear transformation A:RnRm such that

limh0|f(x+h)f(x)Ah||h|=0,(14)

then we say that f is differentiable at x and we write

f(x)=A.

If f is differentiable at every xE, we say that f is differentiable in E.

Note that in (14), hRn. If |h| is small enough, then x+hE, because E is open. Therefore, f(x+h) is defined, f(x+h)Rm, and since AL(Rn,Rm), AhRm. Therefore,

f(x+h)f(x)AhRm.

The norm in the numerator of (14) is that of Rm, while the norm in the denominator is the Rn-norm.

We can also rewrite (14) as

f(x+h)f(x)=f(x)h+r(h),(17)

where the remainder r(h) satisfies

limh0|r(h)||h|=0.

This means that for a fixed x and a small h, the left side of (17) is approximately equal to f(x)h, that is, to the value of a linear transformation applied to h.

Definition: Total Derivative \@{total-derivative}

The derivative defined above in differentiable is called the total derivative of f at f, or the differential of f at x.

Theorem: Total derivatives are unique \@{total-derivatives-are-unique}

Suppose E is an open set in Rn, f:ERm, and xE, and that A1 and A2 are total derivatives of f at x. Then, A1=A2.

Definition: Continuously Differentiable \@{continuously-differentiable}

A differentiable function f of an open set ERn into Rm is said to be continuously differentiable in E if f is a continuous function of E into L(Rn,Rm). More explicitly, it is required that to every xE and to every ϵ>0 corresponds a δ>0 such that

||f(y)f(x)||<ϵ

if yE and |xy|<δ.

If this is the case, we also say that f is a C-mapping or that fC(E).

These are some differential operators that apply to multivariable and/or vector valued functions.

Vector Differential Operator

Definition: del (also: vector differential operator, nabla) \@{del}

In Euclidean space Rn with coordinates (x1,,xn) and standard basis (e1,,en), del is a vector operator whose x1,,xn components are the partial derivative operators x1,,xn; that is

=i=1neixi=(x1,,xn).

Gradient

Definition: Gradient \@{gradient}

Given a scalar function f:RnR, the gradient of f, denoted as f, is defined as the vector of its partial derivatives. Specifically, for a function f(x1,x2,,xn), the gradient is given by

f=[fx1fx2  fxn]

Remark \@{remark-16}

We can write the gradient as either a row vector or a column vector. Different texts use different conventions. I'll show it here as a row vector, while we used a column vector above, just to mix it up.

In three-dimensional Euclidean space, the gradient of a function f, if it exists, is given by

gradf=f=fxi+fyj+fzk=[fx,fy,fz]

Directional Derivative

Definition: Directional Derivative \@{directional-derivative}

Using the setup from the definition of component, let us fix an xE (with ERn,) and let uRn be a unit vector. Then,

limt0f(x+tu)f(x)t=(f)(x)u

is called the directional derivative of f at x in the direction of the unit vector u and is denoted as Duf(x).

Theorem \@{directional-derivative-is-inner-product-of-vector-and-grad}

The directional derivative of f in the direction of a unit vector u is the inner product of u and gradf, that is,

Du=ugradf.

Referenced by (1 direct)

Direct references:

Theorem \@{theorem-19}

Let f(P)=f(x,y,z) be a scalar function having continuous first partial derivatives in some domain B in space. Then, gradf exists in B and is a vector, that is, its length and direction are independent of the particular choice of Cartesian coordinates. If gradf(P)0 at some point P, it has the direction of maximum increase of f at P.

Proof \@{proof-of-theorem-19}

The directional derivative of f in the direction of some unit vector u is

Duf=ugradf=|u||gradf|cosθ(a)

where θ is the angle between u and gradf (see The directional derivative of f in the... and The dot product of u and v...). Note that f is a scalar function, as is the directional derivative of f. Now, cosθ has its maximum value of 1 whenever θ=0, and since u is a unit vector with magnitude of 1, (a) simplifies to

Duf=|gradf|,

which tells us that that direction and magnitude of gradf are independent of the coordinate system chosen. Now, since θ=0 if and only if b and gradf are parallel, gradf is the direction of maximum increase of f at P, assuming gradf0 at P.

Definition: Level Surface \@{level-surface}

Let S be a surface represented by f(x,y,z)=c, where c is constant and f is differentiable. Such a surface is called a level surface of f, and for different c, we get different level surfaces.

Referenced by (1 direct, 5 transitive)

Direct references:

Definition: Tangent Plane \@{tangent-plane}

If S is a level surface of a function f, and P is a point of S, then the set of all tangent vectors of all curves passing through P will generally form a plane, called the tangent plane of S at P.

Referenced by (3 direct, 2 transitive)

Transitive (depth 1):

Transitive (depth 2):

Definition: Surface Normal \@{surface-normal}

Given a tangent plane to a surface S at P, the normal to this plane (the straight line through P perpendicular to the tangent plane) is called the surface normal to S at P.

Referenced by (1 direct, 2 transitive)

Direct references:

Definition: Surface Normal Vector \@{surface-normal-vector}

A vector in the direction of the surface normal of a surface S at point P is called a surface normal vector of S at P.

Theorem \@{gradient-as-surface-normal-vector}

Let f be a differentiable scalar function in space. Let f(x,y,z)=c (with c constant) represent a surface S. Then, if the gradient of f at a point P of S is not the zero vector, it is a surface normal vector of S at P.

Proof \@{proof-of-gradient-as-surface-normal-vector}

Any curve C lying in S can be parameterized as r=[x(t),y(t),z(t)] such that

f(x(t),y(t),z(t))=c.(a)

Now, if we differentiate (a) with respect to t, we get

dfdt=fxx+fyy+fzz=(gradf)r=0.

Therefore, gradf is orthogonal to all the vectors r in the tangent plane of S at P, and is therefore a surface normal vector of S at P.

Theorem \@{find-tangent-plane}

If S is given by F(x,y,z)=0, then at P=(x0,y0,z0), the tangent plane is

F(P)xx0,yy0,zz0=0.

Definition: Potential Function \@{potential-function}

A potential function is a scalar function whose gradient is a vector field. They allow representing certain vector fields in a simpler, more fundamental form.

In other words, given a vector field F, a potential function ϕ is a scalar function such that

F=gradϕorF=gradϕ.

Normal Derivative

Definition: Normal Derivative \@{normal-derivative}

The normal derivative is the directional derivative in the direction of the @normal-vector.

Referenced by (1 direct)

Direct references:

Remark \@{remark-30}

The normal derivative tells us how the value of a function changes as we move in the direction normal (orthogonal) to a curve or surface.

Divergence

Definition: Divergence \@{divergence}

For a vector field F=(F1,F2,F3) defined in three-dimensional space R3, the divergence is defined as

divF=F=F1x+F2y+F3z.

Remark \@{remark-32}

The divergence of a vector field quantifies the extent to which the vector field behaves as a source or a sink at a given point.

Divergence gives a scalar value for each point, i.e., it is a scalar field. A positive value indicates a net flow away from the point, while a negative value indicates a net flow towards a point.

Theorem: Divergence Theorem, or, Gauss's Theorem \@{theorem-33}

An important theorem related to divergence is the Divergence Theorem (also known as Gauss's theorem) which connects the flux of a vector field through a closed surface to the divergence of the field inside the volume bounded by the surface:

V(F)dV=sFdS

Definition: incompressible \@{incompressible}

Let v be the velocity vector of of the motion of particles in a fluid. If divv=0, then the fluid has constant density and is said to be incompressible.

Curl

Definition: Curl \@{curl}

For a vector field F=(F1,F2,F3) defined in three-dimensional space R3, with each component function Fi depending on the variables x, y, and z, the curl of F is defined as

curlF=×F=(F3yF2z)i+(F1zF3x)j+(F2xF1y)k

Remark \@{remark-36}

The curl at a point in the field is represented by a vector whose length and direction denote the magnitude and axis of the maximum circulation. Circulation is the line integral of a vector field around a closed curve.

More intuitively, curl measures the rotation of the vector field at a given point.

Curl can also be expressed as the determinant of a 3x3 matrix involving the unit vectors, partial derivatives, and the components of the vector field:

×F=|ijk xyz F1F2F3|

Definition: Irrotational \@{irrotational}

If the curl of a vector field is 0, i.e. if curlv=0, the field is said to be irrotational.

Referenced by (1 direct)

Direct references:

Theorem \@{invariance-of-curl}

curlv is a vector. It has a length and a direction that are independent of the particular choice of a Cartesian coordinate system in space.

Laplacian

Definition: Laplacian \@{laplacian}

The Laplace operator is a second-order differential operator in the n-dimensional Euclidean space, defined as the divergence () of the gradient (f). Thus, if f is a twice differentiable real-valued function, then the Laplacian of f is the real-valued function defined by

Δf=2f=f.

The Laplacian of f is the sum of all the unmixed second partial derivatives in the Cartesian coordinates xi:

2f=i=1n2fxi2.

In two dimensions, using Cartesian coordinates, the Laplace operator is given by

2f=2fx2+2fy2

and in three dimensions by

2f=2fx2+2fy2+2fz2

Note \@{note-41}

Solutions to Laplace's equation 2f=0 are called harmonic functions.

Theorem \@{gravitational-potential-is-a-solution-to-laplaces-equation}

The force of attraction

p=cr3r=c[xx0r3,yy0r3,zz0r3]

between two particles at points P0=(x0,y0,z0) and P=(x,y,z) (as given by Newton's law of gravitation) has the potential function f(x,y,z)=c/r, where r>0 is the distance between P0 and P.

Thus, p=gradf=grad(c/r). This potential function f is a solution of Laplace's Equation

2f=2fx2+2fy2+2fz2=0,

that is, f has a Laplacian of 0.

Hessian

Definition: Hessian Matrix (also: Hessian) \@{hessian-matrix}

Suppose f : RnR is a function taking as input a vector xRn and outputting a scalar f(x)R. If all second-order partial derivatives of f exist, then the Hessian matrix H of f is a square n×n @matrix, usually defined and arranged as

Hf=[2fx122fx1x22fx1xn2fx2x12fx222fx2xn2fxnx12fxnx22fxn2].

That is, the @entry of the ith row and the jth column is

(Hf)i,j=2fxixj.

For a function f : R3R, this is

H=[2fx22fxy2fxz2fyx2fy22fyz2fzx2fzy2fz2].

Note that the Laplacian of f is equal to the @trace of the Hessian of f. In this way, the Laplacian is to the Hessian what divergence is to the Jacobian.

Jacobian

Definition: Jacobian Matrix (also: Jacobian, Total Derivative) \@{jacobian-matrix}

Let f : RnRm be a function such that each of its first-order partial derivatives exists on Rn. This function takes a point x=(x1,,xn)Rn as input and produces the vector f(x)=(f1(x),,fm(x))Rm as output. Then the Jacobian matrix of f, denoted Jf, is the m×n @matrix whose (i,j) @entry is fixj; explicitly

[fx1fxn]=[Tf1Tfm]=[f1x1f1xnfmx1fmxn],

where Tfi is the @transpose (@row-vector) of the gradient of the i-th component.

Referenced by (1 direct)
Remark \@{remark-45}

For a vector valued function,

F:RnRm,

the Jacobian J(x) is the @linear-map that best @approximates F near x:

F(x0+Δx)F(x0)+J(x0)Δx.

At a point x0Rn, the directional derivative of F in the direction of vRn is

DvF(x0)=J(x0)v.

Remark \@{remark-46}

Given a vector field F:

F=[F1(x,y,z)F2(x,y,z)F3(x,y,z)]:R3R3,

each component Fi is itself a scalar field, and so we can take the gradient of each one:

F1=[F1xF1yF1z],F2=[F2xF2yF2z],F3=[F3xF3yF3z].

Now, the Jacobian of F is the @matrix that contains all of these gradients:

J=F=F=[(F1)T(F2)T(F3)T]=[F1xF1yF1zF2xF2yF2zF3xF3yF3z].

Here:

Thus the Jacobian tells us how each component of F changes with every @coordinate. It can be viewed as the "gradient of a vector field."

Given a vector field F:

F=[F1(x,y,z)F2(x,y,z)F3(x,y,z)]:R3R3,

with a Jacobian

J=F=[F1xF1yF1zF2xF2yF2zF3xF3yF3z],

divergence is the @sum of the @diagonal @entries of this @matrix:

F=F1x+F@y+F3z=trJ.

So, algebraically, divergence is the @trace of the Jacobian.

The trace measures how much the @linear-transformation represented by J stretches space along its coordinate axes - it's the infinitesimal "net expansion rate."

Geometrically, if we have a very small cube of points centered at x0, with a side length of dx, then each corner of that cube corresponds to a slightly different x=x0+Δx. If we take all the points in the box and map them through F, the cube's image becomes a tiny, possibly deformed box in the F-space. The way the box stretches, shears and rotates is determined by J. The change in volume of that box is determined by det(J) (the determinant of the Jacobian.)

For an infinitesimally small cube, the @determinant can be expanded as

det(I+Jϵ)1+ϵtr(J),

for tiny ϵ. Here, the @trace of J,

trJ=F1x+F2y+F3z=F

is exactly the divergence. Each of these terms tells us how much the points in an infinitesimally small box around x0 are stretched/compressed in each coordinate direction.

Physically, for example, if F represents a velocity field, F1x says how much the x-component of the velocity changes as you move along x. Note that if F1 is constant with respect to x, this term is 0, which means there is no stretch/compression along that axis, and if we think of the velocity being that of a fluid, the same amount of fluid will flow into any region in the x direction as flows out in the x direction.

Given a vector field F:

F=[F1(x,y,z)F2(x,y,z)F3(x,y,z)]:R3R3,

with a Jacobian

J=F=[F1xF1yF1zF2xF2yF2zF3xF3yF3z],

We can split J into its @symmetric and @antisymmetric parts:

J=S+A,S=12(J+JT),A=12(JJT).

Now,

  • S encodes stretching/shearing.
  • A encodes local rotation.

The components of A are:

A=12[0F1yF2xF1zF3xF2xF1y0F2zF3yF3xF1zF3yF2z0].

If we let ω=(ωx,ωy,ωz), we can write A as

A=[0ωzωyωz0ωxωyωx0].

Then,

ω=12×F,

so curl is twice the @axial-vector of the @antisymmetric part of the Jacobian.

Geometrically, if we linearize F near some point x0,

F(x0+Δx)F(x0)+J(x0)Δx.

The matrix J is acting on small displacements. When we apply A to Δx, we get

AΔx=ω×Δx.

Then,

  • The direction of ω is the axis of rotation (via the right-hand rule).

More visually, if we take a small ball of points around x0,

  • The symmetric part S of J turns the ball into an @ellipsoid (by stretching/shearing it).
  • The antisymmetric part A spins the ball about the axis ω without changing its shape or volume.

Thus, divergence comes from the @trace of S and curl comes from A.

Physically, if F is a velocity field of a fluid, then:

×F=2ω

means that the fluid near that point is rotating like a tiny rigid body, with @angular-velocity ω.

So, if we dropped a tiny paddle wheel into the flow:

  • The wheel's axis aligns with ×F.
  • Its spin rate is 12||×F||.

This is mostly covered above, so just note that the Jacobian is a sort of machine for producing directional derivatives:

DvF=Jv,

where the multiplication on the right is matrix-vector multiplication, i.e. the image of J when applied to v.

Algebraically,

2f=(f).

To unpack that, we're saying that the Laplacian is the divergence of the gradient of f. Now, when we take the gradient of f, we get a vector field that has the first partial derivatives of f. Now, if we take the Jacobian of that gradient, we get (f)), which is the Hessian of f. Now, if we take the @trace of that Hessian, we get the Laplacian, which is the sum of the pure second @derivatives of f.

Geometrically, the Hessian is telling us how the gradient of f curves near a point, and taking the trace of it tells us the total or average @curvature at that point.

If 2f=0, the gradient's inflow and outflow balance in every direction: the field is @harmonic and locally curvature-neutral.

Exterior Derivative