Machine learning, need certain mathematical foundation, need to master the basic knowledge of mathematics is particularly much, if you begin to learn from beginning to end, estimates that most people too late, I suggest to learn the basic knowledge of mathematics, knowledge can be divided into higher mathematics, linear algebra, probability and mathematical statistics, I finish the relevant mathematical foundation information:
Source file download: \
Github.com/fengdu78/Da…
Content abstract
Stanford University CS229 Fundamentals of Mathematics \
This is the basic material of CS 229 machine learning course of Stanford University, which is the mathematics foundation of each major artificial intelligence course of Stanford.
We translated the original tutorial into an online reading version.
(Click to see: 1. Linear Algebra, 2. Probability Theory)
Second, the essence of mathematics basic teaching materials in domestic universities
This is the material of the Chinese textbook that I sorted out when I took the postgraduate entrance examination and the doctoral entrance examination, which is divided into three parts: advanced mathematics, linear algebra, probability theory and mathematical statistics. I sorted out and published the mathematical knowledge related to machine learning.
This article is part of the linear algebra, recommended collection slowly read.
The determinant
1. Theorem of row (column) expansion of determinant
(1) Suppose, then:
Or among them:
(2) is set as square matrix, then, but may not be true.
(3) is square matrix of order.
(4) Set as square matrix of order, (if reversible),
(5), is square matrix, but.
(6) Van der Monde determinant
Suppose is the square matrix, is an eigenvalue of, then
matrix
Matrix: A table of numbers arranged in rows and columns is called a matrix, or. If, it is called order matrix or order square matrix.
Linear operations on matrices
1. Matrix addition
Let be two matrices, then the matrix is called the sum of the matrix and, denoted by.
2. Matrix multiplication
Let is a matrix, and is a constant, then the matrix is called the number times the matrix, and is denoted by.
3. Matrix multiplication
Let’s say lambda is the matrix, lambda is the matrix, so the matrix, where we call the product of lambda, lambda is lambda.
4. The relationship between,, and
(1)
(2)
But not necessarily.
(3),
But not necessarily.
(4)
5. Relevant conclusions
(1)
(2)
(3) If reversible, then
(4) If is square matrix of order, then:
6. Relevant conclusions
reversible
Can be expressed as a product of elementary matrices; .
7. Conclusions about matrix rank
(1) Rank = row rank = column rank;
(2)
(3);
(4)
(5) Elementary transformation does not change the rank of the matrix
(6) In particular:
(7) If exist if exist
If the if.
(8) there are only zero solutions
8. Block the inverse formula
; ;
;
In this case, we have invertible matrices.
vector
1. Linear representation of vector groups
(1) Linear correlation At least one vector can be linearly represented by the rest of the vectors.
(2) Linear independence, and linear correlation can be represented by unique linearity.
(3) can be expressed linearly.
2. Linear correlation of related vector groups
(1) Partial correlation, whole correlation; The whole is irrelevant, the parts are irrelevant.
(2) ① Dimension vectors are linearly independent and dimension vectors are linearly correlated.
② The vectors of dimensions are linearly correlated.
(3) If it is linearly independent, it is still linearly independent after adding components; Or a set of vectors that are linearly dependent when you remove some of the components.
3. Linear representation of vector groups
(1) Linear correlation At least one vector can be linearly represented by the rest of the vectors.
(2) Linear independence, and linear correlation can be represented by unique linearity.
(3) can be expressed linearly
4. The relationship between the rank of a vector group and the rank of a matrix
Suppose, the linear correlation between the rank of and the column and column vector group of is:
(1) If, the row vector set of is linearly independent.
(2) If, the row vector group of is linearly correlated.
(3) If, the column vector set of is linearly independent.
(4) If, the column vector group of is linearly correlated.
5. Basis transformation formula and transition matrix of dimensional vector space
If and are two sets of basis of vector space, the basis transformation formula is:
Where is the invertible matrix, called the transition matrix from basis to basis.
6. Coordinate transformation formula
If the coordinates of the vectors on the basis and the basis are,
Is, then the vector coordinate transformation formula is or, where the transition matrix is from the basis to the basis.
7. Inner product of vectors
8. Schmidt orthogonalization
If linearly independent, it can be constructed to make the pairwise orthogonal, and is only a linear combination of, and then it is normalized orthogonal vector group. Among them,,,
.
9. Orthogonal basis and normal orthogonal basis
If the vectors in a basis of a vector space are pairwise orthogonal, they are called orthogonal basis; If every vector in an orthogonal basis is a unit vector, it is called a canonical orthogonal basis.
System of linear equations
1. Clem’s rule
A linear system of equations, if the determinant of the coefficients is, the system has a unique solution, where is the determinant of the constant sequence of the right end of the system by substituting the first column element in.
2. Order matrix invertible only has zero solution. There’s always a unique solution, and in general, there’s only a zero solution.
3. The necessary and sufficient conditions for the existence of solutions for nonsingular linear equations, and the properties and structures of solutions for linear equations
(1) is set as the matrix, if, then there must be for, thus there is a solution.
(2) The solution set to, is still the solution of when; But when, is the solution of. Specially for solution; For the solution.
(3) No solution of inhomogeneous linear equations cannot be linearly represented by column vectors.
4. Basic solutions and general solutions of singular linear equations, solution space, general solutions of non-singular linear equations
(1) The homogeneous equations always have solutions (there must be zero solutions). When there is a non-zero solution, since any linear combination of solution vectors is still the solution vector of the homogeneous system, all of the solution vectors form a vector space, which is called the solution space of the system. The dimension of the solution space is, and a group of basis of the solution space is called the basic solution system of the homogeneous system.
(2) is the basic solution system of, namely:
- Is the solution;
- Linearly independent;
- Any solution of phi can be represented by linearity. Is the general solution of phi, where is an arbitrary constant.
Eigenvalues and eigenvectors of the matrix
1. Concepts and properties of eigenvalues and eigenvectors of matrices
(1) Suppose an eigenvalue of is, then one eigenvalue is respectively and the corresponding eigenvector is the same (exception).
(2) If is the eigenvalue of, then, thus there is no eigenvalue.
(3) The eigenvalues of is, and the corresponding eigenvectors are,
If:
Is:.
2. Concept and properties of similarity transformation and similarity matrix
(1) if, then
- To set up
3. The necessary and sufficient conditions for the similarity diagonalization of matrices
(1) Set as a square matrix, diagonalization can be performed for each multiple root eigenvalue, where
(2) Suppose diagonalizable, then by, thus
(3) Important conclusions
- If, then.
- If,, where is the polynomial about the square matrix.
- If is diagonalizable matrix, then the number of non-zero eigenvalues (repeated root calculation) = rank ()
4. Eigenvalues, eigenvectors and similar diagonal matrices of real symmetric matrices
(1) Similarity matrix: set as two square matrices. If there is an invertible matrix so that it is true, it is called matrix and similarity, denoted as.
(2) Properties of similarity matrix: If:
- (if, both reversible)
- (is a positive integer)
- And thus have the same eigenvalues
- So both reversible and irreversible
- Rank, not necessarily similar
quadratic
1. Quadratic homogeneous function of a variable
, which is called meta-quadratic type, abbreviated quadratic type. This quadratic form can be rewritten as a matrix vector if given. Which is called quadratic matrix, because, so the quadratic matrix is symmetric matrix, and the quadratic type corresponds to the symmetric matrix one by one, and the rank of the matrix is called the rank of quadratic type.
2. Inertia theorem, standard form and normal form of quadratic form
(1) Inertia theorem
For any quadratic form, no matter what contractual transformation is chosen to make it into the standard form containing only square terms, the positive and negative inertia exponents are independent of the chosen transformation, which is called the inertia theorem.
(2) standard
The quadratic form is transformed into a contract
Is called the canonical form of. In the general number field, the standard form of the quadratic form is not unique and depends on the contract transformation being performed, but the number of square terms whose coefficients are not zero is uniquely determined.
(3) the specification form
Any real quadratic form can be transformed into normal form through contract transformation, where the rank of is is positive inertia index, is negative inertia index, and the normal form is unique.
3. Using orthogonal transformation and coordination method to transform the quadratic form into standard form, and positive qualitative analysis of the quadratic form and its matrix
Set positive definite positive definite; Reversible; And,
Positive definite positive definite, but not necessarily positive definite
Positive definite
Are all greater than zero
Is greater than zero
Is the positive inertia exponent of
There is an invertible matrix
There is an orthogonal matrix so that
Where positive definite positive definite; Reversible; , and.
This article was first published on the public account “Machine Learning Beginners”
