Small knowledge, big challenge! This article is participating in the creation activity of “Essential Tips for Programmers”.

preface

Hello! Friend!!!

Thank you very much for reading haihong’s article, if there are any mistakes in the article, please point out ~

 

Self-introduction ଘ(੭, ᵕ)੭

Nickname: Haihong

Tag: programmer monkey | C++ contestant | student

Introduction: because of C language to get acquainted with programming, then transferred to the computer major, had the honor to get some national awards, provincial awards… Has been confirmed. Currently learning C++/Linux/Python

Learning experience: solid foundation + more notes + more code + more thinking + learn English well!

 

Machine learning little White stage

The article is only used as my own study notes for the establishment of knowledge system and review

Know what is, know why!

If the view mathematical formula is incomplete or display error

Linear algebra for Machine Learning (6) : Operations on matrices

2.1 matrix

define

M ∗nm*nm∗n number aij(I =1,2… ,m; J = 1, 2,… , n) a_ {ij} (I = 1, 2,… ,m; J = 1, 2,… , n) aij (I = 1, 2,… ,m; J = 1, 2,… N) the table of numbers arranged in m rows and n columns is called the m row and n column matrix, denoted as

The number of M ∗nm*nm∗ N is called the element of matrix A, which is referred to as element, aija_{ij} Aij is located in row I and column J of matrix A, and is called the (I, j) element of matrix A.

Matrix type

Commonly used matrices are:

• N order matrix or square matrix
• Line matrix
• Column matrices
• Same type matrix
• The matrix zero
• Diagonal matrix

A matrix with n rows and columns is called an n-order matrix or square matrix

Square matrix/matrix of order n is also called AnA_nAn

Row matrices (or row vectors, matrices with only one row)

Column matrix (or column vector, a matrix with only one column)

Homotype matrix: two matrices have the same number of rows and columns.

If two matrices are of the same type and their corresponding elements are equal, they are said to be equal and are denoted as A=BA=BA=B

Zero matrix: a matrix in which all elements are zero

The identity matrix (or identity matrix) : The elements on the line (main diagonal) from the top left to the bottom right of the matrix are all 1’s and the rest of the elements are 0’s

1. Diagonal matrix (or diagonal matrix) : All elements not on the main diagonal are 0, also called A=diag(λ1,λ2… , lambda n) A = diag (\ lambda _1, \ lambda _2,… , \ lambda _n) A = diag (lambda 1, lambda. 2,… Lambda, n)

2.2 Operation of matrix

2.2.1 Matrix addition

define

Suppose two M ∗nm*nm∗ N matrices A=(AIj)A=\begin{pmatrix} a_{ij} \end{pmatrix}A=(aij) and B=(Bij)B=\begin{pmatrix} B_{ij} \end{pmatrix}B=(Bij), then the sum of the matrices A and B is A+B

Note: you can add only if two matrices are of the same type (with the same number of rows and columns).

Operation rule

The operation law that matrix addition satisfies (let A, B and C be m* N matrices)

• 
  $A+B=B+A$
• 
  $(A+B)+C=A+(B+C)$

supplement

A = A = A matrix (aij) \ begin {pmatrix} a_ A = {pmatrix} {the ij} \ end (aij), remember – A = (- aij) – A = \ begin {pmatrix} – a_ {pmatrix} {the ij} \ end – A = (- aij)

−A-A−A is called the negative matrix of the matrix AAA

A+(−A)=0A+(-a)=0A+(−A)=0 (where 0 represents the zero matrix)

So, the subtraction of the matrix is zero

$A – B= A + (-B)$

2.2.2 Multiply a number by a matrix

define

The product of the number lambda lambda with the matrix AAA is denoted as lambda A lambda A lambda A or A lambda A lambda A lambda, specified

Operation rule

The scalar multiplication matrix satisfies the following operation law (let A and B be m*n matrices, λ,μ\lambda, muλ,μ number)

• 
  $(\lambda \mu)A=\lambda(\mu A)$
• 
  $(\lambda+\mu)A=\lambda A + \mu A$
• 
  $\lambda (A+B) = \lambda A + \lambda B$

2.2.3 Multiplying matrices by matrices

define

Set A = A = (aij) a_ (ij) attach A = (aij) is A matrix, ms B = (bij) = B (b_ {ij}) B = (bij) is A matrix of sn

So the product of matrix AAA and matrix BBB is an M * N matrix C=(cij)C=(C_ {ij})C=(cij), where CIj = AI1b1j + AI2b2j +… + aisbsj = ∑ k = 1 saikbkj (I = 1, 2,… m; J = 1, 2,… ,n)c_{ij}=a_{i1}b_{1j}+a_{i2}b_{2j}+… B_ + a_ {is} {sj} = \ sum_ {k = 1} ^ s a_ b_ {ik} {kj} (I = 1, 2,… m; J = 1, 2,… ,n)cij=ai1b1j+ai2b2j+… + aisbsj = ∑ k = 1 saikbkj (I = 1, 2,… m; J = 1, 2,… N) and write this product as C=ABC=ABC=AB

Note: Two matrices can only be multiplied if the number of columns in the first matrix (left matrix) equals the number of rows in the second matrix (right matrix).

Operation rule

The multiplication of matrices does not satisfy the commutative law in general, that is, AB≠BAAB \neq BAAB=BA

For two square matrices of order N A, BA, BA, B, if AB=BAAB=BAAB=BA, the square matrix AAA and BBB are said to be commutative.

The matrix A≠0,B≠0A\neq 0,B \neq 0A=0,B=0, but ABABAB or BABABA may be 0. (0 is the value 0 matrix)

If AB is equal to 0, AB is equal to 0, AB is equal to 0, it doesn’t mean that A is equal to 0A is equal to 0 or B is equal to 0B is equal to 0B is equal to 0

Matrix multiplication satisfies both associative and distributive laws

• ​
  $(AB)C=A(BC)$
• ​
  $\lambda (AB) = (\lambda A) B=A(\lambda B)$
• ​
  $A(B+C)=AB+AC,(B+C)A=BA+CA$

For the identity matrix EEE EA=AE=AEA=AE=AEA=AE=A

The role of the identity matrix EEE in matrix multiplication is similar to that of the number 1

The power of the matrix

• ​
  $A^1=A$
• ​
  $A^2=A^1A^1$
• .
• .
• .
• ​
  $A^{k+1}=A^kA^1$

other

• ​
  $A^kA^l=A^{k+l}$
• 
  $(A^k)^l=A^{kl}$
• 
  $(A+B)^2=(A+B)(A+B)=AA+AB+BA+BB=A^2+AB+BA+B^2$
• ​
  $(A-B)(A+B)=AA+AB-BA-BB=A^2+AB-BA-B^2$

2.2.4 Transpose of a matrix

define

Transposed matrix

The transpose of rows and columns of a matrix yields a new matrix

M * n matrix A

Transpose the columns and columns, resulting in the transpose of A, called ATA^TAT

AAA is an M × N matrix and ATA^TAT is an N × M matrix

For example,

Such as:

then

Operation rule

The transpose satisfies

• ​
  $(A^T)^T=A$
• ​
  $(A+B)^T=A^T+B^T$
• ​
  $(\lambda A)^T=\lambda A^T$
• ​
  $(AB)^T=B^TA^T$

Proof: (AB) T = BTAT (AB) ^ ^ T = B TA ^ T T = BTAT (AB)

First of all set A = (aij) m * n, B = (bij) s * nA = (a_ {ij}) _ {m * n}, B = (b_ {ij}) _ {s * n} A = (aij) m * n, B = (bij) s * n, Remember AB = C = (cij) m * n, BTAT = D = (dij) n x mAB = C = (c_ {ij}) _ {m * n}, B ^ TA ^ T = D = (d_ {ij}) _ {n * m} AB = C = (cij) m * n, BTAT = D = (dij) n * m

That’s the matrix multiplication formula

$c_{ji}=\sum_{k=1}^s a_{jk}b_{ki}$

And the ith behavior of BTB^TBT (b1i… ,bsi)(b_{1i},… ,b_{si})(b1i,… ,bsi)

ATA^TAT column J (AJ1,.. ,ajs)(a_{j1},.. ,a_{js})(aj1,.. ,ajs)

so

$d_{ij}=\sum_{k=1}^s b_{ki}a_{jk}=\sum_{k=1}^s a_{jk}b_{ki}$

launch

$d_{ij}=c_{ji}$

namely

$D=C^T$

so

$B^TA^T=(AB)^T$

supplement

Symmetric matrices

If A is A square matrix of order n, and AT=AA^T=AAT=A, then A is A symmetric matrix, abbreviated as A symmetric matrix: its elements are equal with the main diagonal as the axis of symmetry.

2.2.5 Determinant of square matrix

content

The determinant of the elements of square matrix A of order n, where the positions of the elements remain the same, is called | A | or detA

Pay attention to

• The square matrix of order n is a table of n^2n ^2n2 numbers arranged in a certain way
• The determinant of order n is a number of these numbers determined by a certain algorithm

Operation rule

Is the operation law of

• ​$\begin{vmatrix}

A^T \end{vmatrix}=\begin{vmatrix} A \end{vmatrix}$

• ​$\begin{vmatrix}

\lambda A \end{vmatrix}=\lambda ^n\begin{vmatrix} A \end{vmatrix}$

• ​$\begin{vmatrix}

AB \end{vmatrix}=\begin{vmatrix} A \end{vmatrix}\begin{vmatrix} B \end{vmatrix}$

Adjoint matrix

Determinants ∣A∣ begin{vmatrix} A \end{vmatrix}∣∣ A∣∣ the algebraic cofactors of each element AijA_{ij}Aij constitute the following matrix

Called adjoint matrix of matrix A, or adjoint matrix (A is square matrix of order N)

Test certificate: AA ∗ = A ∗ A = ∣ A ∣ EAA A = = A ^ ^ * * \ begin A \ {vmatrix} end EAA ∗ = {vmatrix} A ∗ A = ∣ ∣ ∣ A ∣ ∣ ∣ E

That set A = (aij), AA ∗ = (bij) A = (a_ {ij}), AA ^ * = (b_ {ij}) A = (aij), AA ∗ = (bij)

According to the matrix multiplication formula:

$c_{ij} = a_{i1}b_{1j}+a_{i2}b_{2j} + … + a_{is}b_{sj}$

available

Bij = ai1Aj1 + ai2Aj2 +. + ainAjnb_ {ij} = a_ a_ {i1} {j1} + a_ a_ {i2} {j2} + \ cdots + a_ {in} {Jacqueline Nottingham} A bij = ai1Aj1 + ai2Aj2 +. + ainAjn again by the determinant theorem 3 D=ai1Ai1+ai2Ai2+… + ainAin (I = 1, 2,… , n) (where D is the value of the determinant is here on behalf of | A | “) D = a_ a_ {i1} {i1} + a_ a_ {i2} {i2} +… A_ + a_ {in} {in} (I = 1, 2,… D=ai1Ai1+ai2Ai2+… D=ai1Ai1+ai2Ai2+… + ainAin (I = 1, 2,… N)(where D is the value of the determinant, Here is actually represent | A | “) concluded that bij = ai1Aj1 + ai2Aj2 +. + ainAjn = ∣ A ∣ delta ijb_ {ij} = a_ a_ {i1} {j1} + a_ a_ {i2} {j2} + \ cdots + a_ {in} A {Jacqueline Nottingham} = | A | \ delta_ {ij} Bij = ai1Aj1 + ai2Aj2 +. + ainAjn = ∣ A ∣ delta ij

The delta ij = {1 I = = j0i! = j (I, j = 1, 2, 3… n)\delta_{ij} =\begin{cases} 1 & i==j\\ 0 & i! = {cases} j \ end (I, j = 1, 2, 3… N) delta ij = {10 I = = ji! = j (I, j = 1, 2, 3… n)

instructions

• When I = = j, bij = ∣ A ∣ b_ {ij} = | A | bij = ∣ ∣ A
• i! + = j, bij = 0 b_ {ij} = 0 bij = 0

In the case of the matrix, it’s just that the positive diagonal elements are equal to A, and the other elements are equal to 0

so

$AA^*=(b_{ij})=(|A|\delta_{ij})=|A|(\delta_{ij})=|A|E$

Note:

The absolute value of A is A constant and E is the identity matrix, the absolute value of A E turns out to be A matrix, where everything on the diagonal is the absolute value of A

In the same way A ∗ A = ∣ ∣ EA ^ A * A = | A | EA ∗ = ∣ A ∣ E A

Proof done!

2.2.6 Conjugate matrix

When A=(aij)A=(a_{ij})A=(aij) is A complex matrix,

4, ‾ij\overline a_{ij}aij represents the complex conjugate of AIja_ {ij}aij

remember

$\overline A=(\overline a_{ij})$

A‾\overline AA is called the conjugate matrix of AAA

You’re essentially replacing every member of A with its complex conjugate.

That the conjugate matrix satisfies

• 
  $\overline {A+B}=\overline A + \overline B$
• ​
  $\overline {\lambda A}= \overline \lambda \overline A$
• ​
  $\overline {AB} = \overline A \overline B$

conclusion

Description:

• Refer to textbook “linear algebra” fifth edition tongji University mathematics department
• With the book concept explanation combined with some of their own understanding and thinking

The essay is just a study note, recording a process from 0 to 1

Hope to help you, if there is a mistake welcome small partners to correct ~

I am haihong ଘ(੭, ᵕ)੭

If you think it’s ok, please give it a thumbs up

Thanks for your support ❤️