Fundamentals of Machine Learning mathematics: Learn linear Algebra, don’t go astray! Recommend a correct learning route

The preface

I feel very sorry for the readers of machine Learning in Depth: Advanced Mathematics/Calculus and Python Implementation in Machine Learning. I am not satisfied with my writing.

As mentioned in the previous article, this kind of extremely basic knowledge is the most difficult to introduce. I am also thinking about how to change the way and explain the problem clearly. But the headlines don’t support mathematical formulas, and space is limited. Therefore, IN this article, I would like to introduce my own learning process and recommend some good textbooks and videos to you. In this way, we can take fewer detours and learn knowledge more comprehensively.

Also advice: bite off more than you can chew, and be more refined than you can chew. I am confident that linear algebra can definitely be solved after reading this book and video I recommend.

Linear algebra in machine learning

Linear algebra is an indispensable part of the field of machine learning. From the symbols describing algorithm operations to the realization of algorithms in code, linear algebra belongs to the scope of study. Linear algebra is used almost everywhere in machine learning.

  • Vector and its various operations, including addition, subtraction, number multiplication, transpose, inner product
  • Norm of vectors and matrices, L1 norm and L2 norm
  • The matrix and its various operations, including addition, subtraction, multiplication, and number multiplication
  • Definition and properties of inverse matrix
  • Definition and calculation of determinant
  • Definition of quadratic form
  • Positive characterization of matrices
  • Eigenvalues and eigenvectors of matrices
  • Singular value decomposition of matrices
  • Numerical methods for the solution of linear equations, especially the conjugate gradient method

The teaching material recommended

Learning linear algebra, you must not take out the textbook of Tongji University edition, this book is very bad, it is only suitable for examination review.

I recommend Linear Algebra Review and Reference

Andrew Ng-Mechine Learning is a recitation course on linear algebra for CS229. For those who are not good at English, there is no need to worry.

    1. Basic concepts and symbols
  • 1.1 Basic Symbols
  • Matrix multiplication
  • 2.1 Vector-vector multiplication
  • 2.2 Matrix-vector multiplication
  • 2.3 Matrix-matrix multiplication
  • 3 Operations and properties
  • 3.1 Identity matrix and diagonal matrix
  • 3.2 transpose
  • 3.3 Symmetric Matrix
  • 3.4 Trace of matrix
  • 3.5 norm
  • 3.6 Linear correlation and rank
  • 3.7 Inverse of the square matrix
  • 3.8 the orthogonal array
  • 3.9 Range and null space of the matrix
  • 3.10 the determinant
  • 3.11 Quadratic and semi-positive definite matrices
  • 3.12 Eigenvalues and eigenvectors
  • 3.13 Eigenvalues and eigenvectors of symmetric matrices
  • Matrix calculus
  • 4.1 the gradient
  • 4.2 Hessian matrix
  • 4.3 Gradient and Hessier matrix of quadratic and linear functions
  • 4.4 Least square method
  • 4.5 Gradient of the determinant
  • 4.6 Eigenvalue optimization

For electronic version please send me a private message: Linear Algebra WX: HTSA360

Video is recommended

Some students prefer to watch video, here I blow a course produced by the famous 3BLUE1Brown: “The Nature of Linear Algebra”, 3BLUE1Brown course, beautiful animation, vivid explanation, very suitable to help establish mathematical image thinking, worth watching repeatedly.

I’m sure you’ll be like, “What the hell was that linear algebra I studied?”

For example, the concept of the cross product cannot be applied more broadly in machine learning. 3B1B interprets it as: project W onto a line that v is on, and multiply the length of the projection of W onto V by the length of V, which is the value of its dot product

If you want to watch it, go to site B and search: AV6731067