Gradient of a function
The gradient of a differentiable function contains the first derivatives of the function with respect to each variable. As seen here, the gradient is useful to find the linear approximation of the function near a point.
Definition
Composition rule
Examples
Geometric interpretation
Definition
The gradient of at , denoted , is the vector in given by
Examples:
The function is differentiable, provided , which we assume. Then
The gradient of at is
where , . More generally, the gradient of the function with values
is given by
where , and .
Composition rule with an affine function
If is a matrix, and is a vector, the function with values
is called the composition of the affine map with . Its gradient is given by (see here for a proof)
Geometric interpretation
Geometrically, the gradient can be read on the plot of the level set of the function. Specifically, at any point , the gradient is perpendicular to the level set, and points outwards from the sub-level set (that is, it points towards higher values of the function).
|
Level and sub-level sets of the function with values
The gradient at a point (shown in red) is perpendicular to the level set, and points outside the corresponding sub-level set. The length of the gradient determines how fast the function changes locally (The length of the gradient has been scaled up by a factor of .)
|
|