Multivariable Calculus Vector Fields Gradient Divergence Curl Matrices Linear Algebra Eigenvectors

# Multivariable Calculus, Vector Fields, Matrices, and Eigenvalues

## 1. From One Input to Many

Single-variable calculus studies functions such as y = f(x). Many scientific models depend on several inputs:

or

A temperature field may depend on three spatial coordinates and time. A pressure surface may depend on latitude and longitude. A loss function may depend on thousands of model parameters.

Multivariable calculus asks how output changes when one or several inputs move. Linear algebra provides the language for combining components, changing coordinates, and discovering important directions.

## 2. Geometry of Functions of Two Variables

A scalar function f(x, y) assigns one number to each point in a plane. Its graph z = f(x, y) is a surface in three-dimensional space.

For

the graph is an upward-opening paraboloid. Curves where f has constant value are level curves:

These are circles. A contour map draws many level curves in the input plane. Closely spaced contours indicate rapid change; widely spaced contours indicate slow change.

The domain matters. The function sqrt(1 − x² − y²) is real only inside or on the unit disk.

## 3. Partial Derivatives

A partial derivative changes one variable while holding the others fixed:

Similarly,

For

treat y as constant to obtain

Treat x as constant to obtain

Partial derivatives are slopes of surface cross-sections. Units are output units per unit of the chosen input.

## 4. Higher Partials and Smoothness

Differentiate again to obtain second partials:

When mixed partials are continuous near a point,

The second derivatives form the Hessian matrix:

The Hessian describes local curvature. At a critical point where the gradient is zero, its eigenvalues help classify the point:

• both positive: local minimum;
• both negative: local maximum;
• opposite signs: saddle point;
• a zero eigenvalue: the simple test may be inconclusive.

## 5. Gradient and Directional Derivatives

The gradient collects first partial derivatives:

For a unit direction u, the directional derivative is

The gradient points in the direction of steepest increase. Its magnitude is the maximum rate of increase per unit distance. The direction of steepest decrease is −∇f.

The gradient is perpendicular to a level curve or level surface. If motion stays on f = constant, then the tangent direction dr satisfies

This fact links contour maps, optimization, and constraint geometry.

## 6. Tangent Planes and Linear Approximation

Near point (a, b), a differentiable function is approximately linear:

The right side is the tangent plane. It is the multivariable version of a tangent line.

For small input changes Δx and Δy,

This total differential supports sensitivity analysis and uncertainty propagation. If inputs have covariance matrix C, a first-order output variance is approximately

The approximation becomes less accurate farther from the expansion point or when curvature is large.

## 7. Multivariable Chain Rule

Suppose x and y depend on time and z = f(x, y). Then

In matrix form, derivatives compose through Jacobian matrices. If F maps R^n to R^m, its Jacobian is

For a composition F(G(x)),

Order matters because matrix multiplication generally does not commute.

## 8. Multiple Integrals

A double integral accumulates over an area:

If f is surface density, the integral gives mass. If f = 1, it gives area.

Over a rectangular region,

For nonrectangular regions, bounds may depend on the outer variable. Sketching the region before writing bounds prevents many errors.

Changing coordinates introduces a Jacobian determinant. In polar coordinates,

The extra factor r measures how coordinate grid cells stretch.

## 9. Vector Fields

A vector field assigns a vector to each point:

Examples include fluid velocity, electric field, gravitational field, and heat flux.

Field lines are curves tangent to F. Their density in a drawing is only a visualization choice unless explicitly tied to magnitude.

A scalar potential φ generates a conservative field F = ∇φ or F = −∇φ, depending on convention. In a simply connected region, a continuously differentiable conservative field has path-independent line integrals.

## 10. Divergence

Divergence measures local net outflow:

Positive divergence indicates a local source; negative divergence indicates a sink. Zero divergence means locally incompressible or source-free, though the field may still vary and circulate.

For F = (x, y, z),

The divergence theorem connects volume divergence to boundary flux:

It is a three-dimensional generalization of the Fundamental Theorem of Calculus.

## 11. Curl

Curl measures local rotational tendency:

For the planar rotation field F = (−y, x, 0),

Stokes’ theorem connects curl through a surface to circulation around its boundary:

The identity curl(grad f) = 0 holds for sufficiently smooth f. The identity div(curl F) = 0 also holds under suitable smoothness.

## 12. Matrices as Linear Transformations

An m-by-n matrix maps vectors in R^n to vectors in R^m. For

the product is

Matrix multiplication composes transformations. If B acts first and A acts second, the combined matrix is AB:

Usually AB is not equal to BA. Dimensions must match: the inner dimensions agree, and the outer dimensions determine the result.

## 13. Systems, Elimination, and Inverses

A linear system can be written

Gaussian elimination uses row operations to produce an equivalent simpler system. Pivot positions reveal whether the system has one solution, none, or infinitely many.

A square matrix is invertible if there is A^(−1) satisfying

Then x = A^(−1)b, but numerical computation usually solves the system through factorization instead of explicitly forming the inverse.

Ill-conditioned matrices amplify small data errors. The condition number measures this sensitivity. A residual Ax − b may be small even when the solution error is large if A is poorly conditioned.

## 14. Determinants

For a 2-by-2 matrix,

The determinant is a signed area or volume scaling factor. A zero determinant means the transformation collapses dimension and is not invertible.

Important properties are

Swapping two rows changes determinant sign. Multiplying one row by c multiplies the determinant by c. Adding a multiple of one row to another does not change it.

Large determinants should be computed through stable factorizations. Direct cofactor expansion becomes inefficient rapidly.

## 15. Eigenvalues and Eigenvectors

An eigenvector keeps its direction under a linear transformation:

The scalar λ is the eigenvalue. Rearranging gives

A nonzero solution exists only when the matrix is singular:

This is the characteristic polynomial. For a 2-by-2 matrix it is quadratic.

Eigenvectors identify natural modes: principal axes, vibration shapes, population-growth patterns, quantum states, and stable or unstable directions in dynamical systems.

A real matrix can have complex eigenvalues. A symmetric real matrix has real eigenvalues and orthogonal eigenvectors.

## 16. Diagonalization and Matrix Powers

If A has enough linearly independent eigenvectors, place them in columns of P and eigenvalues on diagonal of Λ:

Then

Powers of a diagonal matrix are easy: raise each diagonal element to the nth power. This helps solve discrete-time systems x_(k+1) = Ax_k.

Not every matrix is diagonalizable. Defective matrices require Jordan forms in theory, though numerical work often uses Schur decomposition because it is more stable.

## 17. Basis and Coordinate Transformations

A basis is an ordered set of independent vectors that spans a vector space. Coordinates depend on the chosen basis; the geometric vector does not.

If columns of P are new basis vectors written in old coordinates, then

Therefore,

A linear operator transforms under a basis change as

This similarity transformation preserves eigenvalues, determinant, and trace.

For an orthonormal basis, P^(−1) = P^T, which improves both interpretation and numerical stability.

## 18. Orthogonality, Projections, and Least Squares

Two vectors are orthogonal when their dot product is zero. An orthonormal matrix Q satisfies

Projection onto the column space of a full-rank matrix A solves a least-squares problem:

The normal equations are

In computation, QR or singular-value decomposition is usually more stable than forming A^T A because squaring the condition number worsens roundoff sensitivity.

Least squares appears in regression, calibration, inverse problems, and data fitting.

## 19. Matrix Decompositions

Useful decompositions reveal structure:

• LU: triangular factors for repeated linear solves.
• QR: orthogonal times triangular, useful for least squares.
• Eigen decomposition: natural directions for diagonalizable square matrices.
• SVD: UΣV^T, valid for every matrix.
• Cholesky: LL^T for symmetric positive-definite matrices.

The singular values in Σ measure stretching along orthogonal directions. Very small singular values reveal near-dependencies and ill-conditioned inverse problems.

A truncated SVD removes directions dominated by noise, but it also introduces bias. The truncation choice must be justified.

## 20. Python Linear Algebra and 3D Gradient Field Solver

import numpy as np
import matplotlib.pyplot as plt

A = np.array([[4.0, 1.0, 0.0],
              [1.0, 3.0, 1.0],
              [0.0, 1.0, 2.0]])
b = np.array([1.0, 2.0, 3.0])

x = np.linalg.solve(A, b)
eigenvalues, eigenvectors = np.linalg.eigh(A)
Q, R = np.linalg.qr(A)
U, singular_values, VT = np.linalg.svd(A)

print("Solution of A x = b:", x)
print("Residual norm:", np.linalg.norm(A@x-b))
print("Determinant:", np.linalg.det(A))
print("Eigenvalues:", eigenvalues)
print("Orthogonality error:", np.linalg.norm([email protected](3)))
print("QR reconstruction error:", np.linalg.norm(Q@R-A))
print("Singular values:", singular_values)

# Basis transformation.
P = eigenvectors
A_in_eigenbasis = P.T@A@P
print("A in eigenvector basis:
", A_in_eigenbasis)

# Scalar field f = x^2 + 2y^2 + 0.5z^2.
grid = np.linspace(-2.0, 2.0, 9)
X, Y, Z = np.meshgrid(grid, grid, grid, indexing="ij")
F = X**2+2*Y**2+0.5*Z**2
Fx, Fy, Fz = 2*X, 4*Y, Z
magnitude = np.sqrt(Fx**2+Fy**2+Fz**2)
mask = magnitude > 0
Fx_plot = np.divide(Fx, magnitude, out=np.zeros_like(Fx), where=mask)
Fy_plot = np.divide(Fy, magnitude, out=np.zeros_like(Fy), where=mask)
Fz_plot = np.divide(Fz, magnitude, out=np.zeros_like(Fz), where=mask)

# A 2D vector field for divergence and curl illustration:
# G(x,y) = (-y, x), divergence 0 and z-curl 2.
g2 = np.linspace(-2, 2, 21)
x2, y2 = np.meshgrid(g2, g2)
Gx, Gy = -y2, x2

fig = plt.figure(figsize=(13, 10))
ax1 = fig.add_subplot(2, 2, 1, projection="3d")
ax1.quiver(X, Y, Z, Fx_plot, Fy_plot, Fz_plot,
           length=0.28, normalize=False)
ax1.set(xlabel="x", ylabel="y", zlabel="z", title="Normalized gradient field")

ax2 = fig.add_subplot(2, 2, 2)
contours = ax2.contour(x2, y2, x2**2+2*y2**2, levels=12)
ax2.clabel(contours)
ax2.set(xlabel="x", ylabel="y", title="Level curves")

ax3 = fig.add_subplot(2, 2, 3)
ax3.quiver(x2, y2, Gx, Gy)
ax3.set_aspect("equal")
ax3.set(xlabel="x", ylabel="y", title="Rotational field: div=0, curl_z=2")

ax4 = fig.add_subplot(2, 2, 4)
ax4.bar(np.arange(1, 4), eigenvalues)
ax4.set(xlabel="Mode", ylabel="Eigenvalue", title="Eigenvalue spectrum")
ax4.grid(alpha=0.3)

plt.tight_layout()
plt.show()

## 21. Numerical Verification Habits

Never accept a matrix result only because software returned it. Check:

• Reconstruction: does Q R reproduce A?
• Residual: is Ax − b small relative to data size?
• Orthogonality: is Q^T Q close to I?
• Eigenpairs: is Av − λv small?
• Sensitivity: how large is the condition number?
• Grid convergence: do numerical gradients stabilize as spacing shrinks?

For a vector field, symbolic identities may fail numerically near boundaries because finite differences use one-sided stencils. Distinguish mathematical error from discretization error.

## 22. Common Conceptual Errors

Gradient confused with divergence: Gradient maps scalar to vector. Divergence maps vector to scalar.

Curl treated as ordinary rotation everywhere: Curl measures infinitesimal circulation, not whether drawn arrows look curved globally.

Matrix multiplication performed entry by entry: Products use row-by-column sums.

Determinant treated as matrix size: It measures signed volume scaling and invertibility.

Eigenvectors assumed unique: Scaling an eigenvector gives another eigenvector. Repeated eigenvalues may have multidimensional eigenspaces.

Basis coordinates confused with the geometric vector: Coordinates change when basis changes.

Explicit inverse used automatically: Solve or factor instead for better numerical stability.

## 23. A Unified Example

Consider a quadratic energy

with symmetric positive-definite A. Its gradient is

Setting the gradient to zero gives the linear system

The Hessian is A. Positive eigenvalues mean positive curvature in every eigenvector direction, so the critical point is a unique minimum.

This one example connects partial derivatives, gradients, matrices, linear systems, eigenvalues, basis directions, optimization, and numerical conditioning. That unity is the main reason multivariable calculus and linear algebra are usually learned together.