Matrix analysis (4) norm of vectors and matrices

My wechat account is MultiAgent1024. I mainly study deep learning, machine game, reinforcement learning and other related topics. Look forward to your attention, welcome to learn and exchange progress together!

We have used the inner product to define the length of an element in vector space. It is a generalization of geometric length. Using the concept of length, we can discuss the problem of limit and approximation. In the analysis and solution of these problems, the most important is to use the basic properties of length, non-negative, homogeneity and trigonometric expression.

The norm of vectors

Definition of norm

• Definition 4.1: If to anyThey all have a real numberCorresponding to, and satisfy:

1. The negative:When,whenWhen,;
2. homogeneity: To any.;
3. Triangle inequality: To any. There are:

saidforVector norm ofThe vector norm.

Several common norms

• 2 norm

Set:

Requirements:

It’s easy to show that this is the norm, called the 2 norm of vectors. The 2 norm is invariant under unitary transformations.

• 1 norm

Set:

Requirements:

theIt’s a norm, called a vector1 norm.

• The vectornorm

Set:

Requirements:

theIs the norm, which is calledThe vectornorm.

• The vectornorm

set, regulation,

theIt’s also the norm, and it’s calledThe vectornorm.

• Other:

Requirements:

  theIs the norm of the function

In the space of continuous functions, it is specified that:

  theIs also a norm.

Generate the norm

An infinite number of norms can be constructed in a vector space, and the ones described above are only the most common. A method for constructing new vector norms from known norms is given below.

• Example 4 set:

provisions

theIs the norm.

• Case 5 设.isA norm of phi, for any phi,,isThe norm of.

Because of the matrixThere could be an infinite number of them, so this way you canConstruct infinitely many norms.

Equivalence of norms

• Definition 4.2Given:The sequence of vectors on, including

If:

saidConvergence, denoted as:

A sequence that does not converge is called a divergent sequence.

• Theorem 4.1 Vector sequence inConverge toThe necessary and sufficient condition forThe norm.

Convergence is the property of vector sequences, which should not be affected by the metric, that is, if a sequence of vectors converges in the sense of one norm, it should also converge in the sense of another norm. If a sequence in a space converges in one norm, then it converges in another norm.

• Definition 4.3 设andisTwo vector norms of phi, if there are positive numbersandSuch that for anyThere are:

Is called the vector normandEquivalence.

• Theorem 4.2:All norms are equivalent in space.

wheninIs convergence, theninAlso converges in the sense.The convergence of vector sequences is not affected by norm selection.

The length of the same vector is generally different under different norms, such as:

Is:

It’s a big difference, but when we’re talking about convergence, it’s the same thing, but remember, we’re talking about finite dimensions, infinite dimensions can be different.

The norm of a matrix

Due to aThe matrix can be viewed asDimensional vectors, so you can define the matrix norm the same way you define the vector norm, butThere is also matrix multiplication between matrices, which should be considered when studying matrix norm.

The norm of a square matrix

• Definition 4.4: If for anyThey all have a real numberCorresponding to, and satisfy:

1. The negative:.;.;
2. homogeneity: To any:

3. Triangle inequality: To any.

4. compatibility: To anyThere are

saidforNorm of the upper matrix, abbreviatedMatrix norm.

• Since the first three in the definition are consistent with the vector norm, the matrix norm has similar properties to the vector norm, such as:

As well asThe two matrix norms of.

Common norm

• Matrix norm:

withIs similar,Rules:

theisThe matrix norm of, is calledfornorm.

• Matrix norm:

withBe similar toRules:

theisA matrix norm ofThe Frobenius norm, for shortnorm.

• Norm of matrix:

setRules:

theisThe matrix norm of.

Compatibility with vector norm

• Definition 4.5Set:isOn theMatrix norm.isOn theThe vector normFor any., there are:

saidMatrix normAnd the vector normIs compatible.

• On theNorm andThe 1 norm of.
• On theNorm andThe 2 norm of.

Vector norms are defined in terms of matrix norms

• setisIs a matrix norm ofA vector norm can be defined. Take a two-dimensional space as an example, suppose, takeSet,isThe norm of, can be arbitrarily:

Is:

Now take:

Is:

isThe matrix. Requirements:

In theAn operation is defined in.

• Such as pick up:

Is:

Pick up:

Is:

• Theorem 4.3Set:isIs a norm ofThere must be a vector norm compatible with it.

Affiliate norm

The method of defining vector norm by matrix norm was introduced earlier, and the method of defining matrix norm by vector norm will be introduced next.

We know that the identity matrix plays a role in matrix multiplication the same way that 1 plays a role in matrix multiplication. But for the matrix norm that we already know, e.g..Norm,Order identity matrixThe norm.

Is it possible to construct such thatWhat is the norm of phi?

• Theorem 4.4Known:Vector norm ofFor anyRules:

theisOn theMatrix norm, is called the dependent vector normThe derived matrix norm, abbreviatedExport the normorAffiliate norm.

Calculation of dependent norm

The calculation of the dependent norm is to find the maximum value of the multivariate function, which is not easy to calculate.The resulting matrix norms are respectively.., then:

1. .
2. .
3. .forThe maximum eigenvalue of.

  Is the matrixI take the modulo of the element, and I take theEach of these columns add up, take the maximum value of the column sum. whileIs theThe magnitude of each row adds up, and then take the maximum.

Examples of norm application

• Definition 4.6Set:.istheOf the eigenvalues, calledforIs the spectral radius of, i.etheSpectral radius isThe maximum value of the eigenvalue module of.

• Theorem 4.6Set:, theAny matrix norm of, there are:

• Theorem 4.7: Let, you can find a matrix norm if you take any positive numberThat: