Matrix NormsMatrices > Basics | Matrix products | Special matrices | QR | Matrix inverses | Linear maps | Matrix norms | Applications
Motivating example: effect of noise in a linear systemWe saw how a matrix (say, Now, assume that there is some noise in the vector One approach is to try to measure the norm of the error vector, Clearly, depending on the choice of the set, the norms we use to measure norm lengths, and how we choose to capture many numbers RMS gain: the Frobenius normLet us first assume that the noise vector ![]() where ![]() is the Frobenius norm of The function The Frobenius norm is useful to measure the RMS (root-mean-square) gain of the matrix, its average response along given mutually orthogonal directions in space. Clearly, this approach does not capture well the variance of the error, only the average effect of noise. The computation of the Frobenius norm is very easy: it requires about Matlab syntax
>> frob_norm = norm(A,'fro'); Peak gain: the largest singular value normTo try to capture the variance of the output noise, we may take a worst-case approach. Let us assume that the noise vector is bounded but otherwise unknown. Specifically, all we know about ![]() The quantity ![]() measures the peak gain of the mapping The quantity The computation of the largest singular value norm of a matrix is not as easy as with the Frobenius norm. Hovewer, it can be computed with linear algebra methods seen here, in about Matlab syntax
>> lsv_norm = norm(A); Other normsMany other matrix norms are possible, and sometimes useful. In particular, we can generalize the notion of peak norm by using different norms to measure vector size in the input and output spaces. For example, the quantity ![]() measures the peak gain with inputs bounded in maximum norm, and outputs measured with the The norms we have just introduced, the Frobenius and largest singular value norms, are the most popular ones, and are easy to compute. Many other norms are hard to compute. ApplicationsDistance between matricesMatrix norms are ways to measure the size of a matrix. This allows to quantify the difference between matrices. Assume for example that we are trying to estimate a matrix Another way is to look at the difference in the output: ![]() when ![]() which is the largest singular value norm of the difference Direction of maximal varianceConsider a data set described as a collection of vectors ![]() Let us try to visualize the data set by projecting it on a single line passing through the origin. The line is thus defined by a vector It can be argued that a good line to project data on is one which spreads the numbers We can find a direction in space which accomplishes this, as follows. The average of the numbers is ![]() while their variance is ![]() The direction of maximal variance is found by computing the LSV norm of ![]() (It turns out that this quantity is the same as the LSV norm of |