Writing is not easy, without the permission of the author is prohibited in any form of reprint! <br/ thumb up > <br/ thumb up > <br/ thumb up > <br/ thumb up >

Continue to share technical blog posts, pay attention to WeChat public number 👉 front-end Leon

Bytedance recruitment is in progress, the recruitment code is 4FCV6BV, the front-end team of the game department can chat directly

The original link

Introduction to the

Load balancing means that the load (work task) is balanced according to a certain algorithm and distributed to multiple operation units for operation and execution, the common ones are Web server, enterprise core application server and other main task server, so as to cooperate with each other to complete the work task. Based on the original network structure, load balancing provides a transparent and effective method to expand the bandwidth of servers and network equipment, strengthen the network data processing capacity, increase throughput, improve the availability and flexibility of the network, and at the same time withstand a greater level of concurrency.

In simple terms, a large number of concurrent request processing is forwarded to multiple back-end node processing, reducing the work response time.

Avoid resource waste
Avoid service unavailability

A, classification,

Four-layer (transport layer)

Four-layer is the transport layer in the seven-layer OSI model, including TCP and UDP protocols. These two protocols include source IP and target IP, as well as source port number and target port number. Four-layer load balancer forwards the traffic to the application server by modifying the address information of the packet (IP + Port) after receiving the client request.

Seven layers (application layer)

Agent load balancing

The seven-layer is the application layer in the OSI seven-layer model. There are many protocols in the application layer, among which HTTP/HTTPS is the most commonly used. Seven layers of load balancing can load these protocols. There is a lot of interest in these application-layer protocols. For example, the load balancing of the same Web server can be determined according to the URL, Cookie, browser category, language and request type of the seven layers in addition to the load balancing based on IP + Port.

The essence of the fourth layer load balancing is forwarding, and the essence of the seventh layer load balancing is content exchange and proxy.

	Four-layer load balancing	Seven layers of load balancing
Based on the	IP + PORT	URL or host IP
similar	The router	Proxy server
The complexity of the	low	high
performance	High, no need to parse the content	In, an algorithm is needed to identify URL headers, cookies, etc
security	Low, unable to recognize a DDoS attack	High, can defend against SYN Flood attack
Extend the functionality	There is no	Content cache, image anti hotlinking, etc

Second, common algorithms

Pre-data structure

Interface urlObj{url:string, weight:number} urlDesc: urlObj[] interface urlcollectObj {count: collectObj URLCollect: URLCollectObj [] URLCollect: URLCollectObj []

Random

random

const Random = (urlDesc) => { let urlCollect = []; // Collect URLDESC.foreach ((val) => {UrlCollect. Push (val. Url); }); Const pos = ParseInt (Math.random() * UrlCollect. Length); return urlCollect[pos]; }; }; module.exports = Random;

Weighted Round Robin

Weight polling algorithm

const WeiRoundRobin = (urlDesc) => { let pos = 0, urlCollect = [], copyUrlDesc = JSON.parse(JSON.stringify(urlDesc)); While (copyUrlDesc.length > 0) {for (let I = 0; i < copyUrlDesc.length; i++) { urlCollect.push(copyUrlDesc[i].url); copyUrlDesc[i].weight--; if (copyUrlDesc[i].weight === 0) { copyUrlDesc.splice(i, 1); i--; }}} return () => {const res = URLCollect [Pos ++]; if (pos === urlCollect.length) { pos = 0; } return res; }; }; module.exports = WeiRoundRobin;

IP Hash & URL Hash

Source IP/URL Hash

const { Hash } = require(".. /util"); const IpHash = (urlDesc) => { let urlCollect = []; For (const key in urlDesc) {// UrlCollect. PUSH (urlDesc[key]. Url); } return (sourceInfo) = bb0 {// generate Hash const hashInfo = Hash(sourceInfo); // take remainder as const urlPos = Math.abs(hashInfo) % urlCollect. Length; // return urlCollect[urlPos]; }; }; module.exports = IpHash;

Consistent Hash

Consistency of the Hash

const { Hash } = require(".. /util"); const ConsistentHash = (urlDesc) => { let urlHashMap = {}, hashCollect = []; For (const key in urlDesc) {// Collect urlHash into array and generate hashMap const {url} = urlDesc[key]; const hash = Hash(url); urlHashMap[hash] = url; hashCollect.push(hash); } hashCollect = hashCollect. SORT ((a, b) => a-b); Return (sourceInfo) = bb0 {// generate Hash const hashInfo = Hash(sourceInfo); // iterate through the hash array to find the first hash that is greater than the hash value of the source information, HashCollect. Foreach ((val) => {if (val >= hashInfo) {return UrlHashMap [val]; }}); Return UrlHashMap [HashCollect [HashCollect. Length - 1]; }; }; module.exports = ConsistentHash;

Least Connections

Minimum number of connections

const leastConnections = () => { return (urlCollect) => { let min = Number.POSITIVE_INFINITY, url = ""; // For (const val = UrlCollect [Key]. Connection; if (val < min) { min = val; url = key; }} // return the return URL; }; }; module.exports = leastConnections;

Note: URLCollect is a data statistics object with the following attributes

Connection number of real-time connections
Count the number of requests processed
Costtime response time.

FAIR

Minimum response time

const Fair = () => { return (urlCollect) => { let min = Number.POSITIVE_INFINITY, url = ""; // For (const Key in UrlCollect) {const Urlobj = UrlCollect [Key]; if (urlObj.costTime < min) { min = urlObj.costTime; url = key; }} // return the return URL; }; }; module.exports = Fair;

See here is not the feeling algorithm is quite simple 🥱

Look forward to the implementation of module five 😏

Health monitoring

Health monitoring is health monitoring of the application server, and to prevent requests from being forwarded to an abnormal application server, health monitoring policies should be used. The strategy and frequency can be adjusted to meet different business sensitivities.

HTTP/HTTPS health monitoring steps (seven layers)

The load balancer sends a HEAD request to the application server.
The application server receives the HEAD request and returns the status code as appropriate.
If the return status code is not received within the timeout time, the timeout is judged and the health check fails.
If the returned status code is received within the timeout time, the load balancing node will compare it and judge whether the health check is successful.

TCP health check steps (four layers)

The load balancer sends TCP SYN request packets to the Intranet application server IP + Port.
After receiving the request, the Intranet application server will return the SYN + ACK packet if it is listening normally.
If no packets are received within the timeout, the service is determined to be unresponsive, the health check has failed, and the RST packet is sent to the Intranet application server to interrupt the TCP connection.
If a packet is received within the timeout period, the service is considered healthy and the RST packet is initiated to interrupt the TCP connection.

UDP health check steps (four tiers)

The load balancer sends UDP messages to the Intranet application server IP + Port.
Returns if the Intranet application server is not listening properlyPORT XX unreachableICMP error message, otherwise is normal.
If an error message is received within the timeout period, the service is determined to be abnormal and the health check failed.
If no error message is received within the timeout period, the service is judged to be healthy.

IV. VIP technology

Vrtual IP

Virtual IP

Under TCP/IP, all computers that want to connect to the Internet, in whatever form, do not need to have a unique IP address. In fact, an IP address is an abstraction of the physical address of the host hardware.
Basically, there are two kinds of addresses
- MAC physical address
- IP Logical Address
A virtual IP is an IP that is not assigned to a real host. That is, the host of the server that is being provided has a virtual IP in addition to a real IP, and either IP can be connected to the host.
- Through the virtual IP corresponding to the real host MAC address
Virtual IP is generally used to achieve the purpose of high availability. For example, the database link configuration in all projects is this virtual IP, and when the primary server fails to provide external services, the virtual IP is dynamically switched to the standby server.

Principle of Virtual IP

ARP is an address resolution protocol that converts an IP address into a MAC address.
Each host has an ARP cache, which stores the mapping relationship between IP address and MAC address in the same network. When the host sends data, it will first check the MAC address corresponding to 3 target IP from this cache and send data to this MAC address. The operating system maintains this cache automatically.
The ARP cache can be operated by the ARP command under Linux

For example, host A (192.168.1.6) and host B (192.168.1.8) exist. A serves as the primary server for the external service and B serves as the backup machine, and the two servers communicate via HeartBeat.
If the backup server does not hear it HeartBeat within the specified period of time, it is considered down.
At this point the backup server is upgraded to the primary server.
- Server B sends out its ARP cache and tells the router to modify the routing table and that the virtual IP address should point to 192.168.1.8.
- When the external accesses the virtual IP again, machine B becomes the primary server and machine A is downgraded to A backup server.
- In this way, the switch between the master and the machine is completed, which is non-perceptive and transparent to the outside.

Fifth, implement a simple load balancing based on NodeJS

Want to manually implement a load balancer/look at the source of the students can look at 👉
Code warehouse

The desired effect

After editing the config. Js
npm run startThe equalizer and the back-end service node can be started

UrlDesc: Backend service node configuration object, weight only plays a role in the WeightRoundRobin algorithm
Port: The equalizer listens on the port
Algorithm: name of the algorithm (all the algorithms in module 2 have been implemented)
Workernum: Number of processes open on the back-end service port, providing concurrency.
Balancernum: The equalizer port opens the number of processes, providing concurrency.
WorkerFilePath: Backend service node execution file, absolute path is recommended.

const {ALGORITHM, BASE_URL} = require("./constant"); module.exports = { urlDesc: [ { url: `${BASE_URL}:${16666}`, weight: 6, }, { url: `${BASE_URL}:${16667}`, weight: 1, }, { url: `${BASE_URL}:${16668}`, weight: 1, }, { url: `${BASE_URL}:${16669}`, weight: 1, }, { url: `${BASE_URL}:${16670}`, weight: 2, }, { url: `${BASE_URL}:${16671}`, weight: 1, }, { url: '${BASE_URL}:${16672}', weight: 4,},], port: 8080, algorithm: algorithm. Random, workerNum: 5, balancerNum: 5, workerFilePath: path. Resolve (__dirname, ". / worker. Js ")}

Architectural blueprint

Let’s take a look at the main process main.js

Initialize the load balancing statistics object BalanceDatabase
- BalanceDatabase is an instance of the Database class used to count load balancing data (described later).
Run the equalizer
- Multi-process model, providing concurrency capabilities.
Run the back-end service node
- Multi-threading + multi-process model, running multiple service nodes and providing concurrency capability.

const {urlDesc, balancerNum} = require("./config") const cluster = require("cluster"); const path = require("path"); const cpusLen = require("os").cpus().length; const {DataBase} = require("./util"); const {Worker} = require('worker_threads'); Const RunWorker = () => {const RunWorker = () => {const RunWorker = urlDesc.slice(0, cpusLen); // create child thread for (let I = 0; i < urlObjArr.length; i++) { createWorkerThread(urlObjArr[i].url); } const runBalancer = () => {cluster.setupMaster({exec: path.resolve(__dirname, "./balancer.js")}); Let Max if (Balancernum) {Max = Balancernum > cpusLen? cpusLen : balancerNum } else { max = 1 } for (let i = 0; i < max; i++) { createBalancer(); }} const BalanceDatabase = new Database (URLDESC); // Run the equalizer runBalancer(); // Run the back-end service node runWorker();

Create an equalizer (createBalancer function)

Create a process
Listen for process communication messages
- Listens for update response time events and executes the update function
  - Used for FAIR algorithm (minimum response time).
- Listener gets statistics object events and returns them
Listen for an exception exit and recreate the process daemon.

Const CreateBalancer = () => {// Create process const worker = cluster.fork(); worker.on("message", (MSG) = > {/ / listen to update events if response time (MSG) type = = = "updateCostTime") {balanceDataBase. UpdateCostTime (MSG) URL, MSG. CostTime)} if (MSG. Type === "getUrlCollect") {worker. Send ({type: "getUrlCollect"); balanceDataBase.urlCollect}) } }); On ("exit", () => {createBalancer(); }); }

Create a back-end service node (createWorkerThread function)

Create a thread
Resolve which ports you want to listen on
Communicate to a child thread, sending a port to listen on
Listening for subthread events through thread communication
- Listens for connection events and fires handlers.
- Listens for disconnection events and fires handlers.
- Used to count load balancing distribution and real-time connections.
Listen for an exception exit and recreate the thread daemon.

Const CreateWorkerThread = (ListenUrl) => {// Create thread const Worker = new Worker(path.resolve(__dirname,)); "./workerThread.js")); // get listenPort const listenPort = listenUrl.split(":")[2]; // Sends the port number Worker. postMessage({type: "port", port: listenPort}) to the child thread; Worker. on("message"), worker.on("message"), worker.on("message"), (MSG) => {if (MSG. Type === "connect") {balanceDatabase.add (msg.port); } // Listen for disconnection events and trigger count events else if (msg.type === "disconnect") {balanceDatabase.sub (msg.port); }}); On ("exit", () => {createWorkerThread(listenUrl); }); }

Take a look at the equalizer workflow balancer.js

Gets the getURL utility function
Listens for requests and proxies
- Gets the parameters that need to be passed to the getURL utility function.
- Get the destination URL of the balancing agent through the getURL utility function
- Record the start time of the request
- To deal with cross domain
- Returns a response
- A response time update event is triggered by process communication.

Note 1: LoadBalance function returns different getURL tool functions by algorithm name. See module 2: common algorithms for the implementation of each algorithm

Note 2: The getSource function processes the parameter and returns it. The getURL function is the tool function for getting the URL described above.

const cpusLen = require("os").cpus().length; const LoadBalance = require("./algorithm"); const express = require("express"); const axios = require("axios"); const app = express(); const {urlFormat, ipFormat} = require("./util"); const {ALGORITHM, BASE_URL} = require("./constant"); const {urlDesc, algorithm, port} = require("./config"); Const run = () => {const getUrl = loadBalance (UrlDesc. slice(0, cpusLen), algorithm); Get ("/", async (req, res) => {const source = await getSource(req); // getURL const URL = getUrl (source); // Res.redirect (302, URL) const start = date.now (); // Res.redirect (302, URL) Get (URL). Then (async (response) => {const UrlCollect = await getUrlCollect(); // handle cross-domain res.setHeader("Access-Control-Allow-Origin", "*"); response.data.urlCollect = urlCollect; Res.send (response.data); // update const costTime = date.now () -start; process.send({type: "updateCostTime", costTime, URL}) }); }); App. listen(port, () => {console.log(' Load Balance Server Running at ${BASE_URL}:${port} '); }); }; run(); const getSource = async (req) => { switch (algorithm) { case ALGORITHM.IP_HASH: return ipFormat(req); case ALGORITHM.URL_HASH: return urlFormat(req); case ALGORITHM.CONSISTENT_HASH: return urlFormat(req); case ALGORITHM.LEAST_CONNECTIONS: return await getUrlCollect(); case ALGORITHM.FAIR: return await getUrlCollect(); default: return null; }};

How to get the load balancing statistic object getUrlCollect in the equalizer

Through process communication, a fetch message is sent to the parent process.
At the same time, start listening for parent communication messages, receive them and return them using Promise Resovle.

Const getUrlCollect = () => {return new Promise((reject) => {try {process.send({type: const getUrlCollect = ()); "getUrlCollect"}) process.on("message", msg => { if (msg.type === "getUrlCollect") { resolve(msg.urlCollect) } }) } catch (e) { reject(e) } }) }

How to implement concurrent WorkerThread.js for service nodes

The multi-threading + multi-process model is used to provide concurrency capability for each service node.

Master process flow

Create the appropriate number of service nodes according to the configuration file.
- Create a process
- Listen for messages from the parent thread (the service node listens on the port) and forward them to the child process.
- Listens for child messages and forwards them to the parent thread (connection establishment, connection disconnection events).
- Listen for an exception to exit and rebuild.

const cluster = require("cluster"); const cpusLen = require("os").cpus().length; const {parentPort} = require('worker_threads'); const {workerNum, If (cluster.isMaster) {// Create a worker function const CreateWorker = () => {// Create a process const worker = cluster.fork(); // Listen for messages from the parent thread and forward them to the child process. parentPort.on("message", msg => { if (msg.type === "port") { worker.send({type: "port", port: Port})}) // Listen to the child and forward the message to the parent worker. On ("message", MSG => {parentPort.postMessage(MSG); }) // Listen for the process to exit and recreate Worker. on("exit", () => {createWorker(); })} let Max if (workerNum) {Max = workerNum > cpusLen? cpusLen : workerNum } else { max = 1 } for (let i = 0; i < max; i++) { createWorker(); }} else {// Backend server executor require(workerFilePath)}

Child process process Worker.js (config.workerFilePath)

Through interprocess communication, a message is sent to the parent process, triggering a connection establishment event.
Return the corresponding.
Through interprocess communication, a message is sent to the parent process to trigger a disconnect event.

var express = require("express"); var app = express(); let port = null; App. Get ("/", (the req, res) = > {/ / trigger connection process. The send ({type: "connect", the port}); // Prints the message console.log("HTTP Version: "+ req.httpVersion); console.log("Connection PORT Is " + port); const msg = "Hello My PORT is " + port; // return the response res.send({MSG}); Process.send ({type: "disconnect", port}); }); Process. on("message", (MSG) => {if (msg.type === "port") {port = msg.port; app.listen(port, () => { console.log("Worker Listening " + port); }); }});

Finally, look at the Database class

Members:

Status: Task queue status
URLCollect: data statistics object (provided for each algorithm to use/display data)
- Count: Number of requests processed
- Costtime: Response time
- Connection: Number of real-time connections
The add method
- Increases the number of connections and live connections
Sub way
- Reduce the number of live connections
UpdateCostTime method
- Update response time

class DataBase { urlCollect = {}; // constructor (constructor () {this.urlCollect[val] = {count: 0, costTime: 0, connection: 0); 0}; }); } // add (port) {const url = '${BASE_URL}:${port}'; this.urlCollect[url].count++; this.urlCollect[url].connection++; } sub (port) {const url = '${BASE_URL}:${port}'; this.urlCollect[url].connection--; } // UpdateCostTime (url, time) {this.urlCollect[url].costtime = time; }}

The final result

Made a visual chart to see the equilibrium effect (Random)✔️

Looks like the balance effect is good 🧐

Little homework