download:Spark2.x+ collaborative filtering algorithm to develop enterprise-level personalized recommendation system

This is a comprehensive course of both algorithm principle and complete recommendation system architecture. Solve the problem of “lack of process from actual scene to model application” in ordinary algorithm class, so that people not only understand the principle of algorithm, but also know how to implement the implementation. The course combines the algorithm principle, uses Spark 2.x and mainstream technology stack, collects user behavior through Flume multi-level high availability log, uses HBase feature vector storage, uses algorithm principle, combines Spark and Storm to carry out offline and real-time recommendation, and realizes personalized recommendation system under the scene of book e-shopping mall.

1 to 2 years experience in big data, big data development engineers interested in recommendation algorithms Basic Spark (Spark Core, Scala knowledge) Basic Python (Numpy) Experience in Hbase, Storm, Hadoop, Flume, Kafka, and Hive have high number basis and probability statistics basis 1. Prepare the environment according to the official document Settings, my environment: Windows 10 64-bit

Node.js(64-bit, >= 8.12.0, < 9.0.0) 3.Python(version 2.7 only) is the native module device compiler that VSCode relies on after the above environment device ends

Option 1: Use the Windows Build Tools NPM module

Start Powershell as administrator and install Windows Build Tools NPM module (documentation).

npm install –global windows-build-tools –vs2015

Note: — DEBUG This flag is useful if you encounter any problems during installation.

Note: If you encounter an error you cannot find the build tool for V141 (Platform Toolset =’ V141 ‘). “You probably have the Visual Studio version installed. Either uninstall this version, or make sure to install VC ++ 2015.3 V14.00 (V140) Desktop Appliance Toolset (see below)

Option 2: Use Visual Studio 2017 (I have never tried this before, but I have Visual Studio 2019 native)

Install Visual Studio 2017 Community Edition
  1. Select C ++ to stop desktop development
  2. Select the desktop VC ++ 2015.3 V14.00 (V140) toolset on the right hand side
  3. Note: If you encounter an error with the build tool V141 (platform toolset = “V141”) that cannot be found “. Make sure you install the desktop VC ++ 2015.3 V14.00 (V140) toolset, the previous steps above is the official tutorial, next up my operation
  4. NPM install -g yarn
  5. Change source so far is not finished, must be hanged, because the script inside the official source, the above only do domestic local acceleration. After hanging, even if the environment is ready 2. Download compile operation source code open the website

When downloading, you must download the ZIP package, because I can not run directly clone, if you think you can try clone, here I will assume that the downloaded zip package downloaded after decompression, use powershell administrator form into the decompression of vscode main directory, run the command

yarn

There will be a long wait, about half an hour, is almost enough, after playing the command line will stop to allow the operation.

At this point, just run the victory to complete the powershell, listener

yarn watch

Run vscode for Windows

.\scripts\code.bat

Run vscode for the web

yarn watch web