Small knowledge, big challenge! This article is participating in the creation activity of “Essential Tips for Programmers”.

preface

Hello! Friend!!!

Thank you very much for reading haihong’s article, if there are any mistakes in the article, please point out ~

 

Self-introduction ଘ(੭, ᵕ)੭

Nickname: Haihong

Tag: programmer monkey | C++ contestant | student

Introduction: because of C language to get acquainted with programming, then transferred to the computer major, had the honor to get some national awards, provincial awards… Has been confirmed. Currently learning C++/Linux/Python

Learning experience: solid foundation + more notes + more code + more thinking + learn English well!

 

Machine learning little White stage

The article is only used as my own study notes for the establishment of knowledge system and review

Know what is, know why!

If the view mathematical formula is incomplete or display error

Linear dependence of vector groups in linear algebra

4.2 Linear correlation of vector groups

Define the four

Linearly dependent/independent

Given vector set: A: A1,a2,…. ,amA:a_1,a_2,…. ,a_mA:a1,a2,…. If there are not all zero numbers k1 k2… ,kmk_1,k_2,… ,k_mk1,k2,… ,km (at least one KKK is not 0) enables


k 1 a 1 + k 2 a 2 + . . . . + k m a m = 0 k_1a_1 + k_2a_2 + …. + k_ma_m = 0

The vector group A is linearly dependent

Otherwise it is called linearly independent (k1,k2… Km all 0 k_1, k_2,… K_m full of 0 k1, k2,… Km is all 0)

A special case

(1) When m = 1, vector group A contains only one vector, namely, A:aA:aA: A

  • A =0, linearly dependent (any k makes ka=0ka=0ka=0, there must be a KKK such that ka=0ka=0ka=0, so linearly dependent)
  • a ! =0, linearly independent (ka=0ka =0ka =0 is linearly independent only when k =0)

(2) When m = 2, A: A1,a2A:a_1,a_2A: A1,a2, the sufficient and necessary condition for linear correlation is that the components of A1, A2a_1, a_2A1 and A2 are proportional. The geometric meaning is that the two vectors are collinear

When A: A1,a2A:a_1,a_2A: A1,a2 are linearly correlated, k1A1 + K2A2 = 0K_1A_1 + K_2 A_2 = 0K1A1 + K2A2 =0 K1A1 =− K2A2K_1A_1 = -k_2 A_2k1A1 =− K2A2 A1a2 = – k2k1 (k1) indicates a 0 \ frac {a_1} {a_2} = \ frac {- k_2} {k_1} (k1 \ neq0) a2a1 = k1 – k2 (k1  = 0) get a1, a2a_1, a_2a1, a2 is proportional to the corresponding measure The reverse is also true

(3) When m = 3, A: A1, A2,a3A: A_1, A_2,a_3A: A1, A2, A3, the geometric meaning of linear correlation is three-vector coplanar

when
A : a 1 . a 2 . a 3 A:a_1,a_2,a_3 Linearly dependent
k 1 a 1 + k 2 a 2 + k 3 a 3 = 0 k_1a_1 + k_2a_2 + k_3a_3 = 0
k 3 a 3 = – k 1 a 1 – k 2 a 2 k_3 a_3 = – k_1 a_1 – k_2 a_2
a 3 = – k 1 k 3 a 1 – k 2 k 3 a 2 a_3 = -\frac{k_1}{k_3}a_1 – \frac{k_2}{k_3}a_2 Explain the vector
a 3 a_3 Can be defined by the vector
a 1 . a 2 a_1,a_2 Synthesis of (
k 1 . k 2 . k 3 Is constant K_1,k_2, and k_3 are constants )

They’re all in the same plane so all three vectors are coplanar

(4) Vector group A: A1, A2… , am (m 2 or higher) A: a_1 and a_2,… ,a_m(m \geq 2)A:a1,a2,… ,am(m≥2) is linearly correlated, that is, at least one vector in vector group AAA can be linearly represented by other m− 1m-1m-1 vectors

Proof:

Assuming that the vector group AAA is linearly dependent, then there are numbers k1,k2… ,kmk_1,k_2,… ,k_mk1,k2,… Km makes k1a1 + k2a2 +… +kmam=0k_1a_1 + k_2a_2+…. + k_ma_m= 0k1a1+k2a2+…. +kmam=0

k1,k2,…. ,kmk_1,k_2,…. ,k_mk1,k2,…. ,km is not all 0, then we can set k1≠0k_1 \neq 0k1=0


k 1 a 1 = – ( k 2 a 2 + . . . + k m a m ) k_1a_1 = -(k_2a_2+… +k_ma_m)


a 1 = – 1 k 1 ( k 2 a 2 + . . . + k m a m ) a_1= \frac{-1}{k_1}(k_2a_2+… +k_ma_m)

A1a_1a1 can be represented by a2,… ,ama_2,… ,a_ma2,… ,am linear representation

(5) Vector group A: A1, A2… ,amA:a_1,a_2,… ,a_mA:a1,a2,… ,am constitutes the matrix A=(a1,a2… ,am)A=(a_1,a_2,… ,a_m)A=(a1,a2,… ,am), the vector group AAA is linearly correlated, which is the odd linear equations x1A1 + x2A2 +…. +xmam=0x_1a_1 + x_2a_2 +…. + x_ma_m=0x1a1+x2a2+…. +xmam=0 Ax=0Ax =0Ax =0 has a non-zero solution

Theorem 4

From the vector group A1, A2… ,ama_1,a_2,… ,a_ma1,a2,… The necessary and sufficient condition for the linear dependence of am is that the matrix A=(a1,a2… ,am)A=(a_1,a_2,… ,a_m)A=(a1,a2,… ,am) is less than the number of vectors MMM; The necessary and sufficient condition for linear independence of the vector set is that R(A)=mR(A)=mR(A)=m

Linear correlation: R (A) (A) (A) < < mR mR < m linear independence: R (A) = mR = mR = m (A) (A)

For example,

Example 4

The linear correlation of NNN dimension unit coordinate vector group is discussed

Answer:

E=(e1,e2,… ,en)E=(e_1,e_2,… ,e_n)E=(e1,e2,… ,en) is the identity matrix of order NNN

Identity matrix: The diagonal from the upper left corner to the lower right corner (called the main diagonal) is all 1, otherwise all 0

clearly


∣ E ∣ = 1 indicates 0 |E| = 1 \neq 0

get


R ( E ) = n R(E)=n

According to theorem 4, NNN dimension unit coordinate vector set is linearly independent

Case 5

known


a 1 = [ 1 1 1 ] . a 2 = [ 0 2 5 ] . a 3 = [ 2 4 7 ] a_1=\begin{bmatrix} 1\\ 1\\ 1 \end{bmatrix},a_2=\begin{bmatrix} 0\\ 2\\ 5 \end{bmatrix},a_3=\begin{bmatrix} 2\\ 4\\ 7 \end{bmatrix}

The linear dependence of vector groups A1, A2, A3a_1, A_2, a_3A1, A2, A3 and a1,a2a_1, a_2A1, A2 are discussed

Answer:


( a 1 . a 2 . a 3 ) = [ 1 0 2 1 2 4 1 5 7 ] …… [ 1 0 2 0 2 2 0 5 5 ] …… [ 1 0 2 0 2 2 0 0 0 ] …… [ 1 0 2 0 1 1 0 0 0 ] (a_1,a_2,a_3)=\begin{bmatrix} 1 & 0 & 2\\ 1 & 2 & 4\\ 1 & 5 & 7 \end{bmatrix} \sim \begin{bmatrix} 1 & 0 & 2\\ 0 & 2 & 2\\ 0 & 5 & 5 \end{bmatrix} \sim \begin{bmatrix} 1 & 0 & 2\\ 0 & 2 & 2\\ 0 & 0 & 0 \end{bmatrix}\sim \begin{bmatrix} 1 & 0 & 2\\ 0 & 1 & 1\\ 0 & 0 & 0 \end{bmatrix}

You can get

  • R = (a1, a2, a3) 2 R (a_1, a_2, a_3) = 2 R (a1, a2, a3) = 2, a1, a2, a3a_1, a_2, a_3a1, a2, a3 linear correlation

  • R (a1, a2) = 2 R (a_1, a_2) = 2 R (a1, a2) = 2, a1, a2a_1, a_2a1, a2 linearly independent

Case 6

Given that the vector groups A1, A2, A3a_1, A_2, a_3A1, A2,a3 are linearly independent, B1 = a1 + a2, b2 = a2 + a3, b3 = a3 + a1b_1 = a_1 and a_2, b_2 = a_2 + a_3, b_3 = a_3 + a_1b1 = a1 + a2, b2 = a2 + a3, b3 = a3 + a1, Prove that vector groups B1, B2, b3B_1, B_2, B_3B1, B2,b3 are linearly independent

Prove a

set


B = ( b 1 . b 2 . b 3 ) . A = ( a 1 . a 2 . a 3 ) B=(b_1,b_2,b_3),A=(a_1,a_2,a_3)

by


{ b 1 = a 1 + a 2 b 2 = a 2 + a 3 b 3 = a 3 + a 1 \begin{cases} b_1=a_1 + a_2\\ b_2=a_2 + a_3\\ b_3=a_3+a_1 \end{cases}

launch


( b 1 . b 2 . b 3 ) . = ( a 1 . a 2 . a 3 ) [ 1 0 1 1 1 0 0 1 1 ] (b_1,b_2,b_3),=(a_1,a_2,a_3)\begin{bmatrix} 1& 0 & 1\\ 1 & 1 & 0\\ 0 & 1 & 1 \end{bmatrix}

Set K=[101110011]K =\begin{bmatrix} 1&0&1 \\ 1&1&0 \\ 0&1&1 \end{bmatrix}K=⎣⎢, 110011101 \ ⎥⎤


B = A K B=AK

set


x 1 b 1 + x 2 b 2 + x 3 b 3 = 0 x_1b_1 + x_2b_2 + x_3b_3=0


( b 1 . b 2 . b 3 ) [ x 1 x 2 x 3 ] = 0 (b_1,b_2,b_3)\begin{bmatrix} x_1\\ x_2\\ x_3 \end{bmatrix}=0

make


x = [ x 1 x 2 x 3 ] x=\begin{bmatrix} x_1\\ x_2\\ x_3 \end{bmatrix}

get


B x = 0 Bx=0

And since B is equal to AKB is equal to AKB is equal to AK


B x = ( A K ) x = A ( K x ) = 0 – A ( K x ) = 0 Bx=(AK)x=A(Kx)=0\rightarrow A(Kx)=0

Because the columns in AAA are linearly independent, so R of A is equal to 3R of A is equal to 3R of A is equal to 3

So the equation A(Kx)=0A(Kx)=0A(Kx)=0 has only one solution, which is zero

The Kx = 0 Kx = 0 Kx = 0

Because by ∣ ∣ K = 2 indicates zero \ | | K = 2 neq ∣ ∣ K = 2  = 0 0 R (K) (K) = 3 = 3 R R (K) = 3

If AAA is NNN sub

  • When ∣ A ∣ indicates 0 | | \ A neq 0 ∣ A ∣  = 0, R = nR (A) (A) = nR (A) = n
  • When ∣ A ∣ = 0 | A | = 0 ∣ A ∣ = 0, R (A) nR nR (A) (A) < < < n

So the equation Kx=0Kx=0Kx=0 has only one solution, which is zero


x = 0 x=0

namely


[ x 1 x 2 x 3 ] = [ 0 0 0 ] \begin{bmatrix} x_1\\ x_2\\ x_3 \end{bmatrix}=\begin{bmatrix} 0\\ 0\\ 0 \end{bmatrix}

X1b1 + x2B2 + x3B3 =0x_1b_1 + X_2b_2 + X_3B_3 = 0x1B1 + x2B2 + x3B3 =0. Note B1, B2, B3B_1, B_2, B_3B1, B2,b3 are linearly independent

Act 2


( b 1 . b 2 . b 3 ) . = ( a 1 . a 2 . a 3 ) [ 1 0 1 1 1 0 0 1 1 ] (b_1,b_2,b_3),=(a_1,a_2,a_3)\begin{bmatrix} 1& 0 & 1\\ 1 & 1 & 0\\ 0 & 1 & 1 \end{bmatrix}

Write for


B = A K B=AK

You can also write


B = E A K B=EAK

Because ∣ ∣ K = 2 indicates zero \ | | K = 2 ∣ ∣ neq 0 K = 2 get KKK reversible  = 0

And because E and K are invertible

so


A …… B A \sim B

The necessary and sufficient condition for A ~ BA \sim BA ~ B is the existence of m order invertible matrix P and n order invertible matrix Q such that PAQ=B

so


R ( A ) = R ( B ) R(A)=R(B)

Theorem in the previous chapter 2: if A ~ BA \ sim BA ~ B, there are R = R (A) (B) R = R (A) (B) R = R (A) (B)

because


R ( A ) = 3 R(A)=3

so


R ( B ) = 3 R(B)=3

According to theorem 4, BBB is linearly independent

Theorem 5

(1) If vector group A: A1… amA:a_1,… a_mA:a1,… Am is linearly correlated, then the vector group B: A1… am,am+1B:a_1,… a_m,a_{m+1}B:a1,… Am,am+1 is also linearly dependent. Conversely, if the vector set BBB is linearly independent, then the vector set AAA is linearly independent

Prove: Vector group A: A1… amA:a_1,… a_mA:a1,… Am is linearly correlated, then the vector group B: A1… am,am+1B:a_1,… a_m,a_{m+1}B:a1,… Am,am+1 is also linearly dependent

A = (a1,… , am), B = (a1,… ,am,am+1)A=(a_1,… , a_m), B = (a_1,… ,a_m,a_{m+1})A=(a1,… , am), B = (a1,… ,am,am+1)

It’s clear that


R ( B ) Or less R ( A ) + 1 R(B) \leq R(A)+1

Linearly dependent on AAA


R ( A ) < m R(A) < m

so


R ( B ) < m + 1 R(B) < m + 1

So BBB must be linearly dependent

Proof: vector groups
B B Linearly independent, then the vector group
A A Linearly independent

If BBB is linearly independent


R ( B ) = m + 1 R(B)=m+1

Because R(B)≤R(A)+1R(B) \leq R(A)+1R(B)≤R(A)+1


R ( A ) p R ( B ) – 1 = ( m + 1 ) – 1 = m R(A) \geq R(B) -1=(m+1)-1=m

namely


R ( A ) p m R(A)\geq m

Because A = a1,… ,amA=a_1,… ,a_mA=a1,… , am


R ( A ) Or less m R(A) \leq m

In conclusion there are


R ( A ) = m R(A)=m

So AAA is linearly independent

(2) the vector group composed of MMM NNN dimension vectors. When the dimension NNN is less than the number of vectors MMM, there is a certain linear correlation. In particular, n+1n+1n+1 NNN dimension vectors must be linearly dependent

Proof:

A = (a1, a2,… ,am)A=(a_1,a_2,… ,a_m)A=(a1,a2,… Am) apparently there is


R ( A ) Or less m R(A) \leq m

And because the ai (I ∈ (1, m]) a_i (I \ [m] 1,) in ai (I ∈ [m] 1,) for NNN d column


R ( A ) Or less n R(A) \leq n

Because n


R ( A ) < m R(A) < m

So AAA must be linearly dependent

When m=n+1m=n+1m=n+1 still satisfy the condition that N

(3) Set vector group A: A1,a2,…. ,amA:a_1,a_2,…. ,a_mA:a1,a2,…. ,am is linearly independent, and the vector group B: A1… ,amB:a_1,… ,a_mB:a1,… ,am, BBB are linearly correlated, then the vector BBB must be linearly represented by vector group AAA, and the representation is unique

Proof:

Remember A = (a1, a2,… , am), B = (a1, a2,… ,am,b)A=(a_1,a_2,… , a_m), B = (a_1, a_2,… ,a_m,b)A=(a1,a2,… , am), B = (a1, a2,… , am, b)


R ( A ) Or less R ( B ) R(A) \leq R(B)

Because AAA is linearly independent and BBB is linearly dependent


R ( A ) = m , R ( B ) < m + 1 R(A)=m, R(B) < m+1

In conclusion there are


m Or less R ( B ) < m + 1 m \leq R(B) < m+1

get


R ( B ) = m = R ( A ) R(B)=m=R(A)

In Ax=bAx=bAx=b

Because the R = R (A) (B) = R (A, B) R = R (A) (B) = R (A, B) R = R (A) (B) = R (A, B)

It shows that the equations have unique solutions

So vector B can be linearly represented by vector AAA, and it’s unique

conclusion

Description:

  • Refer to textbook “linear algebra” fifth edition tongji University mathematics department
  • With the book concept explanation combined with some of their own understanding and thinking

The essay is just a study note, recording a process from 0 to 1

Hope to help you, if there is a mistake welcome small partners to correct ~

I am haihong ଘ(੭, ᵕ)੭

If you think it’s ok, please give it a thumbs up

Thanks for your support ❤️