Schur Complement Lemma
Lemma: Schur Complement
Let be a symmetric matrix partitioned into blocks:
where both are symmetric and square. Assume that is positive definite. Then the following properties are equivalent:
Proof: Recall that the matrix is positive semi-definite if and only if for any vector . Partitioning the vector similarly to , as , we obtain that is positive semi-definite if and only if
This is equivalent to: for every ,
Since is positive semi-definite, the corresponding quadratic function is convex, jointly in its two arguments. Due to the partial minimization result, we obtain that the partial minimum is convex as well.
It is easy to obtain a closed-form expression for . We simply have to minimize the convex quadratic function with respect to its second argument. Since the problem of minimizing is not constrained, we just set the gradient of with respect to to zero (see here):
which leads to the (unique) optimizer . Plugging this value we obtain:
Since is convex, its Hessian must be positive semi-definite. Hence , as claimed.
|