1. Node module mechanism

1.1 What are the modules in Node

In Node, each file module is an object, defined as follows:

function Module(id, parent) {
  this.id = id;
  this.exports = {};
  this.parent = parent;
  this.filename = null;
  this.loaded = false;
  this.children = [];
}

module.exports = Module;

var module = new Module(filename, parent);
Copy the code

All modules are instances of Module. As you can see, the current module (module.js) is also an instance of Module.

1.2 Please introduce the module loading mechanism of require

This question basically tells you how well the candidate understands the mechanism of Node module

  • 1. Calculate the module path first
  • 2, if the module is in the cache, take out the cache
  • 3. Load modules
  • 4. Export attribute of module
// require internally calls the module. _load method module. _load =function(request, parent, isMain) {// Calculate the absolute path var filename = module._resolveFilename (request, parent); Var cachedModule = module._cache [filename]; var cachedModule = module._cache [filename];if (cachedModule) {
    returncachedModule.exports; // Step 2: Whether it is a built-in moduleif (NativeModule.exists(filename)) {
    returnNativeModule.require(filename); } / * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * note here * * * * * * * * * * * * * * * * * * * * * * * * * * / / / the third step: Var Module = new Module(filename, parent); var Module = new Module(filename, parent); Module._cache[filename] = module; / * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * note here * * * * * * * * * * * * * * * * * * * * * * * * * * / / / the fourth step: There is a method called module.prototype.load try {module.load(filename); hadException =false;
  } finally {
    if(hadException) { delete Module._cache[filename]; } // Step 5: export the module exports propertyreturn module.exports;
};
Copy the code

We’ll pick up where we left off

When loading a Module, why does each Module have __dirname and __filename attributes? When loading a Module, why does each Module have __dirname and __filename attributes

// In this step, the module is wrapped in the following form. // In this step, the module is wrapped in the following form. // In this step, the module is wrapped in the following form.functionVar math = require(exports, require, module, __filename, __dirname) {'math');
  exports.area = function(radius){
      return Math.PI * radius * radius
  }
});

Copy the code

In other words, each module is passed __filename and __dirname, which are not in the module itself, but are passed from the outside

Exports = XXX and module. exports={}

  • Exports are module.exports
  • In fact, the code of 1.3 problems has already explained the problem, and then I quote liao Xuefeng’s explanation, hoping to make it more clear
Module. exports vs exports many times, you will see that there are two ways to export variables in a module in the Node environment: Method 1: assign a value to module.exports: // hello.jsfunction hello() {
    console.log('Hello, world! ');
}

function greet(name) {
    console.log('Hello, ' + name + '! '); } module.exports = { hello: hello, greet: greet }; Exports: // hello.jsfunction hello() {
    console.log('Hello, world! ');
}

function greet(name) {
    console.log('Hello, ' + name + '! ');
}

function hello() {
    console.log('Hello, world! '); } exports.hello = hello; exports.greet = greet; Exports = {hello: hello, greet: greet}; exports = {hello: hello, greet: greet}; First, Node executes the entire hello. Js file in a wrapper function called Load. Before executing the load() function, Node prepares the module variable: var module = {id:'hello', exports: {} }; The load() function finally returns module.exports: var load =function(exports, module) {// hello. Js file content... // The load function returns:returnmodule.exports; }; var exportes = load(module.exports, module); Exports by default, Node prepares the exports variable and module.exports variable as an empty {} object, so we can write: exports. Foo =function () { return 'foo'; };
exports.bar = function () { return 'bar'; }; You can also write: module.exports.foo =function () { return 'foo'; };
module.exports.bar = function () { return 'bar'; }; In other words, Node gives you an empty object {} by default, so you can add things directly to it. However, if we export a function or array, then we can only assign module.exports: module.exports =function () { return 'foo'; }; An assignment to exports is invalid because module.exports is still an empty {} object. If you want to export a key object {}, you can use exports to export an existing empty object {} and add a new key to it. If you want to export a function or array, you must directly assign a value to the module.exports object. Exports = {foo:function () { return 'foo'; }}; Or: module.exports =function () { return 'foo'; }; Ultimately, we strongly recommend using module.exports = XXX to export module variables so that you only need to remember one method.Copy the code

2. Asynchronous I/ OS of Node

Most of the answering ideas in this chapter are borrowed from “NodeJS in Simple Terms” by Daesung Park.

2.1 This section describes the process of Node event loop

  • When the process starts, Node creates a loop similar to while(true). Each time the body of the loop is executed, it becomes a Tick.

  • The process for each Tick is to see if there are events to be handled. If so, fetch the event and its associated callback function. It then enters the next loop and exits the process if there is no more event handling.

2.2 How do I determine whether an event needs to be processed during each TICK?

  • Each event loop has one or more observers, and the process of determining whether there is an event to process is to ask these observers if there is an event to process.

  • In Node, events mainly originate from network requests, file I/O, etc., and the corresponding observers of these events include file I/O observers and network I/O observers.

  • The event loop is a typical producer/consumer model. Asynchronous I/O, network request and other event producers continuously provide different types of events for Node. These events are passed to the corresponding observers, and the event loop takes the events from the observers and processes them.

  • Under Windows, this loop is created based on IOCP, and under *nix it is created based on multithreading

2.3 Describe the entire asynchronous I/O process

V8’s garbage collection mechanism

3.1 How Can I View the MEMORY Usage of V8

Using process.memoryUsage(), the following is returned

{
  rss: 4935680,
  heapTotal: 1826816,
  heapUsed: 650472,
  external: 49879
}
Copy the code

HeapTotal and heapUsed represent the memory usage of V8. External represents the memory usage of v8-managed C++ objects bound to Javascript. RSS, resident set size, is how much physical memory is allocated to the process (as part of the total allocated memory). This physical memory includes the heap, the stack, and the code snippet.

What is the memory limit of 3.2 V8, and why is V8 designed this way

This is 1.4GB for a 64-bit system and 0.7GB for a 32-bit system. Because of the 1.5GB of garbage collection heap memory, V8 takes more than 50 milliseconds to do a non-incremental garbage collection and more than a second. This is the event in garbage collection that causes Javascript threads to pause execution, and at this expense, the performance and impact of an application plummets.

Please briefly talk about the 3.3 V8 memory generation and reclamation algorithm

In V8, memory is divided into new generation and old generation. Young generation objects that live for a short time, old generation objects that live for a long time, or objects that live in memory.

3.3.1 Cenozoic

The Scavenge algorithm is the main method for garbage collection in the new generation. This is a garbage collection algorithm implemented by copying. It divides the heap memory into two parts, each part of space being called semispace. Of the two Semispace Spaces, only one is in use and the other is idle. A semispace space that is in use is called a From space, and a space that is idle is called a To space.

  • When garbage collection begins, the living objects in the From space are checked, and the living objects are copied To the To space, while the space occupied by non-living objects is freed. When replication is complete, a role swap takes place between the From space and the To space.

  • Since the life cycle of objects in the new generation is relatively short, it is more suitable for this algorithm.

  • When an object survives multiple copies, it is considered a long-lived object. Such long-lived objects from the Cenozoic will then be moved into the old generation.

3.3.2 rainfall distribution on 10-12 old generation

The old generation mainly adopts the garbage collection algorithm of mark removal. Unlike Scavenge, which replicates a living object, the mark clearing algorithm exploiture goes through all objects in the heap during the mark phase and tags the living objects, only clearing dead objects. Living objects make up only a small part of the new generation and dead objects make up only a small part of the old generation, which is why the marker clearing algorithm is used.

3.3.3 Mark the problem of the algorithm clearly

The main problem is that the memory space will appear discontinuous after each marked clear reclamation

  • This type of fragmentation can cause problems with subsequent memory allocation. It is possible that a large object will need to be allocated. In this case, all the debris will not be able to complete the allocation, triggering an early garbage collection, which is unnecessary.
  • To solve the fragmentation problem, tag collation is proposed. After the object is marked dead, in the process of sorting, the living object is moved to one end, and after the movement is complete, the memory outside the boundary is cleaned directly.

3.3.4 What Causes V8 To Fail to Reclaim Memory Immediately

Closures and global variables

3.3.5 Please talk about what memory leak is, the common causes of memory leak, and the troubleshooting methods

What is a memory leak

  • A Memory Leak is a situation in which, due to negligence or error, a program fails to free Memory that is no longer in use.
  • If the location of the leak is critical, it may hold more and more unused memory as the processing progresses, which can cause the server to respond more slowly.
  • Severe cases cause memory to reach a limit (possibly the upper limit of the process, such as v8’s upper limit; Or the maximum amount of memory the system can provide) can crash the application. Common causes of memory leaks There are several cases of memory leaks:

1. Global variables

a = 10; // No object is declared. global.b = 11; // The global variable reference is a simple reason. The global variable is directly attached to the root object and will not be cleared.Copy the code

Second, the closure

function out() {  
    const bigData = new Buffer(100);  
    inner = function() {}}Copy the code

Closures refer to variables in parent functions, which can cause a memory leak if closures are not released. In the example above, the inner is directly mounted on the root, so every time the out function is executed, the bigData will not be released, resulting in a memory leak.

Note that while this example is simply a reference hanging on a global object, the actual business situation may be caused by hanging on an object that can be traced from root.

Event listening

Node.js event listeners can also be exposed to memory leaks. For example, if you listen to the same event repeatedly and forget to remove it (removeListener), you will cause a memory leak. This can easily happen when events are added to reusable objects, so repeated event listeners may receive the following warning:

emitter.setMaxListeners() to increase limit 
Copy the code

For example, node.js may cause a memory leak when the keepAlive value of the Agent is true. When the keepAlive value of the Agent is true, the previously used socket is reused. If the event listener is added to the socket and you forget to clear the event listener, the reuse of the socket will result in repeated event listening and memory leakage.

The principle is the same as the previous event listener added when forgetting to clear. When using the Node.js HTTP module, it is not a problem not to reuse through keepAlive, which can cause memory leaks. Therefore, you need to be aware of the lifecycle of the objects that you are adding event listeners to, and be careful to remove them yourself.

Screening method

When you want to locate a memory leak, there are usually two situations:

  • For a memory leak that can be reproduced in normal use, this is a simple case that can be addressed in a test environment simulation.

  • For accidental memory leaks, they are usually related to a particular input. It is a time-consuming process to reproduce this input stably. If this particular input cannot be located through the code’s logging, it is recommended that the production environment print a memory snapshot.

  • Note that printing memory snapshots consumes a lot of CPU and may affect online services. The snapshot tool recommends heapdump to save memory snapshots and devtool to view memory snapshots.

  • When heapdump is used to save a snapshot of memory, only objects in the Node.js environment will be disturbed (if using the Nodes-inspector, the snapshot will be disturbed by variables in the front).

  • NPM install heapdump -target= node.js to install heapdump.

4. Buffer module

4.1 Will new Buffers Occupy the MEMORY allocated by V8

No, Buffer is an out-of-heap memory and is not allocated by V8.

4.2 Differences between buffer. alloc and buffer. allocUnsafe

The underlying memory of the Buffer instance created by allocUnsafe is uninitialized. The contents of the newly created Buffer are unknown and may contain sensitive data. Use buffer.alloc () to create a zero-initialized Buffer instance.

4.3 Buffer memory allocation Mechanism

In order to efficiently use the allocated memory, Node uses slab allocation mechanism. Slab is a dynamic memory management mechanism. Node uses the 8KB limit to distinguish between large and small buffers. If a Buffer is smaller than 8KB, it is small, and if it is larger than 8KB, it is large.

For example, the first allocation of a 1024 byte Buffer, buffer.alloc (1024), will use a slab for this allocation, and then if you continue to use buffer.alloc (1024), you will still use up the last slab, since it is 8KB in total. 1024+1024 = 2048 bytes, no 8KB, so continue using this slab to allocate space to the Buffer.

If the size is larger than 8kb, then use the C++ underlying SlowBuffer to provide space for the Buffer object.

4.4 Garbled Buffer Characters

For example, a file in test.md reads:

Before the bed bright moonlight, suspected to be frost on the ground, looked up at the moon, bowed his head to think of homeCopy the code

When we read it like this, we get garbled:

var rs = require('fs').createReadStream('test.md', {highWaterMark: 11}); // Before bed? Light, doubt??? Frost on the ground, look up?? The bright moon,????? First remembering homeCopy the code

In general, you only need to set rs.setencoding (‘utf8’) to solve the garble problem

5, webSocket

5.1 What are the advantages of webSocket over traditional HTTP

  • Only one TCP connection is required between the client and the server, using fewer connections than HTTP long polling
  • The webSocket server can push data to the client
  • Lighter protocol headers to reduce data transmission

5.2 What is webSocket protocol upgrade? Can you briefly describe it?

First, the WebSocket connection must be initiated by the browser because the request protocol is a standard HTTP request in the following format:

GET the ws: / / localhost: 3000 / ws/chat HTTP / 1.1 Host: localhost Upgrade: websocket Connection: Upgrade Origin: http://localhost:3000 Sec-WebSocket-Key: client-random-string Sec-WebSocket-Version: 13Copy the code

This request differs from a normal HTTP request in several ways:

  • Instead of an address like /path/, GET requests an address beginning with ws://.
  • The request headers Upgrade: WebSocket and Connection: Upgrade indicate that the Connection is to be converted to webSocket;
  • Sec-websocket-key is used to identify the connection, not to encrypt the data;
  • Sec-websocket-version Specifies the WebSocket protocol Version.

Then, if the server accepts the request, it returns the following response:

HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: server-random-string
Copy the code

The response code 101 indicates that the HTTP protocol is about to be changed. The changed protocol is Upgrade: The WebSocket protocol specified by WebSocket.

6, HTTPS

6.1 Which Ports are used for HTTPS communication and what are the uses of these ports

  • Port 443 is used to verify the identity of the server and client, for example, to verify the validity of certificates
  • Port 80 is used for data transfer (in case of valid authentication, for data transfer)

6.2 The authentication process involves the key, symmetric encryption, asymmetric encryption, the concept of summary, please explain

  • Key: A key is a parameter that is entered in the algorithm for converting plaintext to ciphertext or ciphertext to plaintext. Keys are divided into symmetric keys and asymmetric keys, which are used in symmetric encryption and asymmetric encryption respectively.

  • Symmetric encryption: Symmetric encryption, also known as private key encryption, means that the sender and receiver of a message use the same key to encrypt and decrypt data. Symmetric encryption is characterized by fast algorithm disclosure, encryption and decryption, and is suitable for encryption of large amounts of data. Common symmetric encryption algorithms include DES, 3DES, TDEA, Blowfish, RC5 and IDEA.

  • Asymmetric encryption: Asymmetric encryption is also called public key encryption. Asymmetric encryption is more secure than symmetric encryption. Both sides of a symmetrically encrypted communication use the same key. If one key is leaked, the entire communication will be cracked. Asymmetric encryption uses a pair of keys, public and private, that come in pairs. The private key is kept by itself and cannot be disclosed to others. A public key is a public key that can be obtained by anyone. One of the public or private keys is used for encryption and the other for decryption.

  • Abstract: The algorithm is also known as the hash/hash algorithm. It uses a function to convert data of any length into a string of fixed length (usually in hexadecimal format). The algorithm is not reversible.

6.3 Why do I Need the CA To Sign the Certificate

If you do not sign the certificate, there is a risk of manin – the – man attack. After you sign the certificate, you can ensure that the information in the certificate, such as the public key, server information, and enterprise information, is not tampered with and can verify the validity of the client and server.

6.4 HTTPS Authentication Is the process of AUTHENTICATION through TSL/SSL

A brief diagram is as follows

7. Process communication

7.1 Describe the multi-process architecture of node

Facing the situation that node’s single thread has insufficient use of multi-core CPU, Node provides child_process module to realize process replication. The multi-process architecture of Node is a master-slave mode, as shown below:

var fork = require('child_process').fork;
var cpus = require('os').cpus();
for(var i = 0; i < cpus.length; i++){
    fork('./worker.js');
}
Copy the code

In Linux, we use ps aux | grep worker. Js to check the process

7.2 What are the methods of creating a child process? Briefly describe the differences between them

The methods for creating a child process are as follows:

  • Spawn () : Starts a child process to execute the command
  • Exec (): Starts a child process to execute the command. Unlike spawn(), the interface has a callback function that knows the condition of the child process
  • ExecFlie (): Starts a child process to execute executable files
  • Fork (): similar to spawn(), except that it executes js files to create Node child processes
  • Spawn () differs from exec() and execFile() in that the timeout attribute can be specified to set the timeout period when creating the spawn() process, and the created process will be killed if the specified time period is exceeded
  • Exec () differs from execFile() in that exec() is suitable for executing existing commands and execFile() is suitable for executing files.

7.3 Do you know that there is an stdio option for the third argument to spawn when the child process is created? What does this option do? What is the default value?

  • The option is used to configure the pipes that are established between the parent and child processes.
  • By default, the stdin, stdout, and stderr streams of the ChildProcess are redirected to the corresponding subprocess. Stdin, subprocess.
  • This is equivalent to setting options.stdio to [‘pipe’, ‘pipe’, ‘pipe’].

7.4 How to implement the automatic restart code after a node child process is killed

  • When creating a child process, let the child process listen for the exit event. If it is killed, fork it again
var createWorker = function(){
    var worker = fork(__dirname + 'worker.js')
    worker.on('exit'.function(){
        console.log('Worker' + worker.pid + 'exited'); // Create a new worker createWorker()})}Copy the code

7.5 On the basis of 7.4, achieve limited restart. For example, I allow it to restart 5 times within 1 minute at most, and alarm will be sent to operation and maintenance if more than 5 times

  • The idea is to determine whether the created worker restarts more than 5 times within 1 minute when it is created
  • Therefore, every time a worker is created, the creation time of this worker should be recorded and put into an array queue. Every time a worker is created, the first five records in the queue should be fetched
  • If the time interval between these five records is less than one minute, it’s time to call the police

7.6 How Can I Share Status or Data between Processes

I have not used Kafka this kind of message queue tool, asked Java, can use similar tools to achieve interprocess communication, better methods welcome comments

8. Middleware

8.1 If you have used the two Node frameworks koA and Egg, please briefly describe the middleware principle, and it is better to express it in code

  • The middleware implementation is like an onion, with the earliest middleware used in the outermost layer. The processing sequence is from left to right. The left side receives a request and the right side outputs a response
  • Typically, middleware executes it twice, the first time before calling Next, which passes control downstream to the next middleware. When there is no more downstream middleware or the next function is not executed, the upstream middleware in turn will resume its behavior and let the upstream middleware execute the code after next
  • Take the following code for example
const Koa = require('koa') const app = new Koa() app.use((ctx, Next) => {console.log(1) next() console.log(3)}) app.use((CTX) => {console.log(2)}) app.listen(3001) If the result is 1=>2=>3Copy the code

Koa middleware implementation source code roughly thought is as follows:

/ / note which compose function, the function is the key to the realization of middleware model onion asynchronous promise / / / / the scene simulation const delay = async () = > {return new Promise((resolve, reject) => {
    setTimeout(() => { resolve(); }, 2000); }); } const fn1 = async (CTX, next) => {console.log(1); await next(); console.log(2); } const fn2 = async (ctx, next) => { console.log(3); await delay(); await next(); console.log(4); } const fn3 = async (ctx, next) => { console.log(5); } const middlewares = [fn1, fn2, fn3]; Const compose = (middlewares, CTX) => {let fn = middlewares[i];
    if(! fn){return Promise.resolve() }
    return Promise.resolve(fn(ctx, () => {
      return dispatch(i+1);
    }));
  }
  return dispatch(0);
}

compose(middlewares, 1);
Copy the code

9, other

We are now going over the main API for Node 12 and there are many new discoveries, such as

  • In the fs.watch module, the event callback function takes the name of the event that was triggered as an argument, but no matter what I add or delete, the event is triggered by rename (update, delete, rename). Later, I found a Node-watch module on the Internet. This module has corresponding events for addition, deletion and modification, and also supports recursive watch files efficiently.
  • The util module has a Promisify method that makes a function that follows an exception-first callback style, i.e. (err, value) =>… The callback function takes the last argument and returns a promise version of the function that returns a value.
const util = require('util');
const fs = require('fs');

const stat = util.promisify(fs.stat);
stat('. ').then((stats) => {// process 'stats'. }).catch((error) => {// Handle error. });Copy the code

9.1 noise to

  • Crypto module, you can investigate the basic knowledge of encryption, such as what are the abstract algorithm (MD5, SHA1, SHA256, salt MD5, SHA256, etc.), then you can ask how to use MD5 to simulate a salt MD5 algorithm, Then you can ask the difference between AES and EDS algorithms in crypto. CreateCiphe, what are the block encryption modes (such as ECB,CBC, why ECB does not recommend it), what is the block encryption mode in Node (CMM), what are the fillers and vectors in these encryption algorithms? You can then ask about the process of digital signature and HTTPS (why CA is needed, why symmetric encryption is used to encrypt public keys, etc.)
  • TCP/IP, you can ask a lot of basic questions, such as what protocol does the link layer use to get a physical address from an IP address (ARP), what is the gateway, what is ICMP in IP, what is TCP’s three-way handshake, what is the process of four-way breakups, how does TCP control retransmission, what does TCP do when the network is blocked, and so on. What is the difference between UDP and TCP? What is broadcast and multicast in UDP? What modules are multicast implemented in node?
  • OS, the operating system related basis, what is the process of IO (from the inside of the hard disk read data into the memory of the kernel, and kernel memory data into the IO to call the process of the application of memory), von neumann system, what are the difference between processes and threads, etc. (I’ve been to see Marco on Linux tutorial, because he was not trained, I have heard a lot of basic computer knowledge and benefited a lot. I suggest you go to Bilibili to see it.
  • Linux related operation knowledge (Node involves background, although it is to do the middle platform, does not involve the database, but the basic Linux operation or will be able to)
  • Node performance monitoring (learning this myself)
  • The tests, because they use the Egg framework, are well documented for learning unit tests, so skip this part
  • What is the default transaction level? What is the default transaction level? What is the problem? Mysql orM tool for Node, mysql orM tool for Node, mysql orM tool for Node, mysql orM tool for Node, mysql orM tool for Node, mysql orM tool for Node… (For example, I watch silicon Valley mysql beginner + advanced video, the book is to watch mysql must know must be, I myself out of hobby learning… No actual combat)