The birth of the Node
NodeJS was created in 2009 by Ryan Dahi. It is neither a language nor a framework, it is based on Google V8 ENGINE JS operating environment, so that JS has DOM operation (browser), I/O, file reading and writing, operation database (server) and other capabilities. NodeJS ‘package management tool, NPM, has become the largest ecosystem of open source package management in the world
In general, NodeJS has the following characteristics:
The event-driven non-blocking IO model (asynchronous) is lightweight and efficient
The Node core API
Buffer
Before TypedArray was introduced, javascript did not have a mechanism for handling binary data streams from media files. Later, NodeJS introduced buffers, which interact with octet streams in TCP streams, file system operations, and other contexts.
The characteristics of the Buffer
Buffer
It’s very similar to an array, but it can store binary dataBuffer
Is determined at creation time and cannot be changedBuffer
Class in global scope, not requiredrequireBuffer
The data is stored in binary, but displaying data in binary is long, so the console printsBuffer
It will be in hexadecimal format.Buffer
Elements (bytes) in the range 00-FF (two-digit hexadecimal range), 0-255(decimal range), 00000000-11111111 (binary range)
The computer’s memory is divided into countless cells, and each cell stores data in the form of an electrical signal (0 or 1). This cell is called a bit,8 bits = 1 byte.Buffer
An element of a byte
Buffer
The length of represents the size of memory,BufferThe English in Chinese is one byte and the Chinese is three bytes
The use of the Buffer
// Create a buffer of 10 bytes. By default, each byte is filled with 0
const buf1 = Buffer.alloc(10);
console.log("buf1", buf1);
// Fill the 0 byte bit with the base 10 value 88
buf1[0] = 88
// Fill the 1 byte bit with the decimal value 255
buf1[1] = 255
// Fill byte bit 2 with the hexadecimal value 0xaa
buf1[2] = 0xaa
// The number 3 byte bit is filled with the base 10 value 257, which is converted to binary 1 0000 0001 exceeds the maximum 1111 1111 by more than 8 bits to the left
buf1[3] = 257
console.log("After modifying buF1 bits", buf1);
// Create a buffer of 10 bytes. Each byte is filled with 1
const buf2 = Buffer.alloc(10.1);
console.log("buf2", buf2);
// Create an uninitialized buffer of length 10.
// This is faster than calling buffer.alloc (),
// But the returned buffer instance may contain old data,
// Fill (), write(),
// or other function overrides to populate the contents of the buffer.
const buf3 = Buffer.allocUnsafe(10);
console.log("buf3", buf3);
// Create a buffer containing bytes [1, 2, 3].
const buf4 = Buffer.from([1.2.3]);
console.log("buf4", buf4);
// Create a buffer containing bytes [1, 1, 1, 1],
// All entries are truncated with '(value & 255)' to fit the range 0 -- 255.
const buf5 = Buffer.from([257.257.5, -255.'1']);
console.log("buf5", buf5);
// Create a buffer containing utF-8 encoded bytes of the string 'test' :
// [0x74, 0xC3, 0xa9, 0x73, 0x74]
// [116, 195, 169, 115, 116] (decimal)
const buf6 = Buffer.from('tést');
console.log("buf6", buf6);
// Create a buffer containing latin-1 bytes [0x74, 0xe9, 0x73, 0x74].
const buf7 = Buffer.from('tést'.'latin1');
console.log("buf7", buf7);
let str = "Hello binary"
const buf8 = Buffer.from(str);
console.log("buf8", buf8);
console.log("String length", str.length, "The length of the buffer", buf8.length);
Copy the code
A character encoding
People speak human language, computers speak computer language. To interact with the machine language of a computer, which processes data in binary form, a human language needs a dictionary (character set).
The character set assigns each character a unique number by which the corresponding character can be found. The process of converting characters into unique numbers based on the character set is called character encoding and vice versa.
ASCII code comparison table
ASCII comparison relation table for character A:
BIN | Symbol |
---|---|
0100, 0001, | A |
ASCII provides a specification for each character number conversion and memory storage. ASCII code has eight digits, each of which is a bit, and eight bits is a byte. All seven bits can have either a 0 or a 1 except the first one, so ASCII can represent a total of 2^7, or 128 characters.
For English, 128 characters includes letters, arrays, punctuation, etc. But for other languages, such as Chinese, it is far from enough
Unicode unified coding
If there is a standard scheme for displaying all characters in all languages of the world, Unicode is a good solution to this problem. It is a large set of characters in which all the computer-supported characters are contained. It can hold millions of characters. Each character has a different number (code point). For example, U+0041 stands for the English capital letter A, and U+4E25 stands for the Chinese character Yan. You can view the Unicode character mapping table or the Unicode Character mapping table
The problem of Unicode
Unicode is just a set of symbols, and it only specifies the binary of a symbol, not how that binary should be stored.
For example, the Unicode of the Chinese character strict is the hexadecimal number 4E25, which has a full 15 bits to convert to binary (100111000100101), meaning that at least two bytes are required to represent this one character strict. Representing other characters may require three or four, or even more.
There are two issues involved:
- The encoding is ambiguous and cannot distinguish Unicode from ASCII. It is impossible for a computer to determine whether three bytes represent one character or three characters.
- A waste of storage space. Unicode characters may require 3 or 4 bytes, whereas English characters require only one byte. If all characters are represented in 4 bytes, the other two or three bytes in English are filled with zeros. It’s a huge waste of storage.
UTF-8
Unicode only provides character sets and does not specify how these code points are encoded. Utf-8 has since become the most widely used implementation of Unicode on the Internet. Other implementations include UTF-16 (characters in two or four bytes) and UTF-32 (characters in four bytes), but are rarely used on the Internet. Again, the relationship here is that UTF-8 is one of the implementations of Unicode.
One of the biggest features of UTF-8 is that it is a variable length encoding method. It can use 1 to 4 bytes to represent a symbol, varying the length of the byte depending on the symbol.
1) For a single-byte symbol, the first byte is set to 0 and the next 7 bits are the Unicode code for the symbol. So utF-8 encoding is the same as ASCII for English letters.
2) For n-byte symbols (n > 1), the first n bits of the first byte are set to 1, the n + 1 bits are set to 0, and the first two bits of the following bytes are set to 10. The remaining bits, not mentioned, are all Unicode codes for this symbol.
Dgram
The Dgram module provides an implementation of UDP datagram sockets.
- newdgram/server.jsAnd perform
node server.js
Run the service/ / the server const dgram = require('dgram') const server = dgram.createSocket('udp4') // An error occurred server.on('error'.(err) = > { console.log('Server exception: \n${err.stack}`) server.close() }) // Receive the request server.on('message'.(msg, info) = > { // Obtain the client IP address and port number console.log('The server receives from${info.address}:${info.port} 的 ${msg}`) }) server.on('listening'.() = > { const address = server.address() console.log('Server listening${address.address}:${address.port}`)})// Run the service on port 3005 server.bind(41234) Copy the code
- newdgram/client.jsAnd perform
node server.js
Access the service/ / the client const dgram = require('dgram') const message = Buffer.from('Some strings') const client = dgram.createSocket('udp4') console.log('Client executed'); client.send(message, 41234.'localhost'.(err) = > { // After the link is successfully established, close it in time otherwise it will occupy resources forever client.close() }) Copy the code
Event
Most of Node’s core apis are implemented through an asynchronous event-driven architecture, in which certain types of objects (called “triggers”) fire named events that cause Function objects (” listeners “) to be called. For example: The Net.server object fires an event every time there is a connection; Fs. ReadStream triggers an event when a file is opened; The stream fires events whenever there is data available to read. All objects that emit events are instances of the EventEmitter class. These instances provide EventEmitter. On (event type, event callback) to listen to at some time, and EventEmitter
The sample
const EventEmitter = require('events')
class MyEmitter extends EventEmitter {}const myEmitter = new MyEmitter()
myEmitter.on('sayHello'.() = > {
console.log("Trigger sayHello event");
})
myEmitter.emit('sayHello')
Copy the code
The event callback this points to
const EventEmitter = require('events')
class MyEmitter extends EventEmitter {}const myEmitter = new MyEmitter()
// If the event callback is a normal function, the this of the event callback points to myEmitter
myEmitter.on('sayHello'.function (a, b) {
console.log(a, b, this.this === myEmitter);
})
// If the event callback is an arrow function, the this pointer to the event callback is lost
myEmitter.on('sayHello'.(a, b) = > {
console.log(a, b, this.this === myEmitter);
})
myEmitter.emit('sayHello'.Parameters' 1 '.Parameters' 2 ')
Copy the code
Only one event is emitted
const EventEmitter = require('events')
class MyEmitter extends EventEmitter {}const myEmitter = new MyEmitter()
myEmitter.once('sayHellow'.function () {
console.log("Trigger the sayHello event");
})
myEmitter.emit('sayHellow')
Once an event is triggered, it is automatically unbound
myEmitter.emit('sayHellow')
Copy the code
fs
All file modules in NodeJS are implemented by FS module, including file directory creation, deletion and query as well as file reading and writing. In FS module, the method can be divided into synchronous and asynchronous methods, and the one with sync suffix is synchronous method
Permission bit mode
Fs is used to manipulate files, and this involves file permissions
Permission to a | read | write | perform |
---|---|---|---|
Character representation | r | w | x |
The figures show that the | 4 | 2 | 1 |
As shown in the table above, the operation permissions of articles are divided into three types: read, write and operation. Numbers are represented as octal digits, with octal digits of 4, 2, and 1 having permissions, but no permissions of 0
Identify a flag
In NodeJS, the identifier bits represent how files can be manipulated, such as readable, writable, both readable and writable. Below are the identifiers and their corresponding meanings
symbol | meaning |
---|---|
r | Reads the file and throws an exception if the file does not exist |
r+ | Reads and writes to a file, throwing an exception if the file does not exist |
rs | Reading and writing files instructs the operating system to bypass the local file system cache |
w | Write file, the file does not exist will be created. If yes, delete and write |
wx | Write file, open it in exclusive mode |
w+ | The file is read and written to. If the file does not exist, the file is created. If the file exists, the file is written to |
wx+ | Just like w+, open in exclusive mode |
a | Append write, create file if file does not exist |
ax | Similar to A, open in exclusive mode |
a+ | Read and append to, create if not present |
ax+ | Similar to a+, open in exclusive mode |
A brief summary is:
- R: read
- W: write
- S: synchronous
- + : Adds the reverse operation
- X: Exclusive
The difference between r+ and W + is that r+ does not create a file when it does not exist. Throws an exception twice, but w+ creates the file; If r+ exists in a file, the file will not be automatically cleared. If w+ exists in a file, the file will be automatically cleared
Read the file
-
Synchronous read
The synchronous read method readcFileSync, which takes two arguments:
- The first parameter is the read file path or file descriptor
- The second parameter isConfiguring objects (Options)orString (encoding), the default value is
null
encoding
: Encoding type. Default valuenull
flag
: Indicates the identifier bit. The default value isr
const fs = require('fs') const path = require('path') // Read files synchronously const buffer = fs.readFileSync(path.resolve(__dirname, './1.text')) // Use utF-8 encoding to read files const text = fs.readFileSync(path.resolve(__dirname, './1.text'), 'utf-8') console.log('Read results', buffer, text); Copy the code
-
Asynchronous read
The asynchronous read method readcFile, which takes three arguments:
- The first two parameters are the same as readcFileSync
- The third argument is the callback function, which takes two arguments, err and data.
const fs = require('fs') const path = require('path') // Read files asynchronously fs.readFile(path.resolve(__dirname, './1.text'), 'utf-8'.(err, data) = > { console.log('Read results', err, data); }) Copy the code
Written to the file
-
Synchronous write
The synchronous write method writeFileSync takes three arguments:
- The first parameter is the read file path or file descriptor
- The second argument is the data to be written, of type String or Buffer
- The third parameter is zeroConfiguring objects (Options)orString (encoding), the default value is
null
encoding
: Encoding type. Default valueutf-8flag
: Indicates the identifier bit. The default value iswmode
: Permission bit. The default value is0o666
const fs = require('fs') const path = require('path') const buffer = Buffer.from('hello code') // Write to String synchronously fs.writeFileSync(path.resolve(__dirname, '2.text'), 'hello world') // Write to Buffer synchronously fs.writeFileSync(path.resolve(__dirname, '3.text'), buffer) Copy the code
-
Asynchronous writes
The asynchronous write method writeFile takes four arguments:
- The first three parameters and
writeFileSync
The same - The fourth argument is the callback function, which takes one argument, err.
const fs = require('fs') const path = require('path') // Write the string asynchronously fs.writeFile(path.resolve(__dirname, '4.text'), 'hello world'.(err) = > { // Write is complete if(! err) {// Read what was just written const content = fs.readFileSync(path.resolve(__dirname, '4.text'), 'utf-8') console.log('content', content); }})Copy the code
- The first three parameters and
-
Synchronous appending
Write in front of all is written in the form of cover content, synchronous write for additional appendFileSync, he like writeFileSync also has three parameters
const fs = require('fs') const path = require('path') // Append the content fs.appendFileSync(path.resolve(__dirname, '1.text'), 'Hello Code') // Read the file const content = fs.readFileSync(path.resolve(__dirname, './1.text'), 'utf-8') console.log('content', content); Copy the code
-
Asynchronous appending
The asynchronous write method appendFile takes four arguments, each of which is used the same as writeFile
const fs = require('fs') const path = require('path') // Write the string asynchronously fs.appendFile(path.resolve(__dirname, '1.text'), 'hello zhangsan'.(err) = > { // Write is complete if(! err) {// Read what was just written const content = fs.readFileSync(path.resolve(__dirname, '1.text'), 'utf-8') console.log('content', content); }})Copy the code
fs
Module advanced usage
- Open the file
Open the file before operating the file.open
The open file descriptor method takes four arguments:- Path: indicates the file path
- Flag: indicates the flag bit
- Mode: indicates the permission bit. Default: 0O666
- Callback: A callback function that takes err (error) and fd (file descriptor) and executes after opening the file
const fs = require('fs')
const path = require('path')
fs.open(path.resolve(__dirname, '4.text'), 'r'.(err, fd) = > {
console.log('data', err, fs)
})
Copy the code
- Close the file
close
The method takes two arguments:- The first argument is: close the file descriptor for the file
- The second argument is the callback function, which has an argument, err, to close the file
const fs = require('fs')
const path = require('path')
fs.open(path.resolve(__dirname, '4.text'), 'r'.(err, fd) = > {
// Close the file descriptor
fs.close(fd, err= > {
console.log('Closed successfully')})})Copy the code
- Read the file
The read method can be used to read a file. Unlike readFile, it is usually used when the file is too large to be read into the Buffer at once or when the size of the file is unknown. The read method takes six parameters:
- Fd: file descriptor, which needs to be used first
open
Open the - Buffer: To read the contents into
Buffer
中 - Offset: integer, direction
Buffer
The initial location of the write - Length: indicates the length of the file to be read
- Position: integer, the initial position of the file to be read
- Classback: a callback function that takes three arguments
err
(wrong),bytesRead
(actual number of bytes read),buffer
(cache object being written), read execution after completion.
const fs = require('fs')
const path = require('path')
// Create a cache that occupies 6 memory
const buf = Buffer.alloc(6)
fs.open(path.resolve(__dirname, '1.text'), 'r'.(err, fd) = > {
console.log('fd', err, fd);
// Read the file
fs.read(fd, buf, 0.3.0.(err, bytesRead, buffer) = > {
console.log(bytesRead, buffer, buffer.toString()); })})Copy the code
- Create a readable stream
Also throughcreateReadStreamTo create a readable stream, which takes two arguments:- The first parameter is the file path or file descriptor
- The second parameter isA configuration objectBy default,
null
- Flags: flag bit, default: ‘r’
- Encoding: Indicates the encoding type
- .
const fs = require('fs')
const path = require('path')
const reader = fs.createReadStream(path.resolve(__dirname, '1.text'), { encoding: 'utf-8' })
Error reading file
reader.on('error'.(err) = > {
console.log('Exception occurs', err);
})
// The file to read has been opened
reader.on('open'.(fd) = > {
console.log('File open', fd);
})
// The file is in place for reading events
reader.on('ready'.() = > {
console.log('The files are ready... ');
})
// The file is being read
reader.on('data'.(chunk) = > {
console.log('Read file data', chunk);
})
// File reading is complete
reader.on('end'.() = > {
console.log('Read completed... ');
})
// The file is closed
reader.on('close'.() = > {
console.log('File closed... ');
})
Copy the code
The HTTP module
Create a simple HTTP service
const http = require('http')
// Create a service
const server = http.createServer()
// Listen for requests
server.on('request'.function (request, response) {
// Configure the response header
response.writeHead(200, { 'Content-Type': 'text/html; charset=utf-8' })
// Respond to data with request.write
response.write('<h1>Hello World</h1>')
// End the response with response.end to prevent the client from waiting
response.end(' Response ends
')})// Bind the port number to start the service
server.listen(3002.function () {
console.log('Service started on port 3002');
})
Copy the code
Create an HTTP2 service
const http2 = require('http2')
const fs = require('fs')
// http2 uses HTTPS by default
const server = http2.createSecureServer({
// Configure the public and private keys. These two generally need to go to a third party certification authority certification
Openssl can be used to generate public and private keys
key: fs.readFileSync('localhost-privkey.pem'),
cert: fs.readFileSync('localhost-cert.pem')}); server.on('error'.(err) = > console.error(err));
server.on('stream'.(stream, headers) = > {
stream.respond({
'content-type': 'text/html; charset=utf-8'.':status': 200
});
stream.end('<h1>Hello World</h1>');
});
// Run the service on port 8443
server.listen(8443);
Copy the code
HTTP history
HTTP was created primarily to send hypertext Markup Language (HTML) from a Web server to a client browser. With the popularization and development of the Internet, the WEB environment is becoming more and more complex. We had to optimize HTPP.
Basic optimizations for HTTP
There are two main factors that affect HTTP network requests: bandwidth and latency
Bandwidth: now network infrastructure is done very well, bandwidth has been greatly improved. We no longer worry about bandwidth affecting Internet speed. Delay:
- Browser block: The browser blocks requests for several reasons. The browser can only have four connections to the same domain name at the same time (this may vary depending on the browser’s kernel). If the maximum number of browser connections is exceeded, subsequent requests will be blocked.
- DNS query: When the browser accesses a domain name, the DNS query is used to convert the domain name to an IP address. This process can be optimized with DNS caching
- Establishing a connection: HTTP is based on TCP. Each HTTP request requires three handshakes to establish a connection. But these connections cannot be reused, resulting in three handshakes and a slow start per request.
HTTP 1.0 vs. 1.1
-
In HTTP1.0, if-modifyIEd-since and Expires are mainly used as the criteria for cache judgment. HTTP1.1 this introduces more cache control policies such as Entity Tag, if-unmodified-since, if-match, if-none-match, and more options for cache header control cache policies
-
HTTP1.0, there is some waste of bandwidth, such as the client only needs a part of an object, and the server will transfer the whole object, and does not support resumable breakpoint function, HTTP1.1 is introduced in the request header range header field, which allows only a part of the resource request, The return code is 206, which improves development freedom while maximizing bandwidth and connectivity
-
For example, 409 indicates that the requested resource conflicts with the current state of the resource. 410 indicates that a resource on the server has been permanently deleted
-
The Host header processing in HTTP1.0 assumes that each server is bound to a unique IP address. Therefore, the URL in the request message does not pass the hostname. However, with the development of virtual machine technology, there can be multiple virtual hosts on the same physical server, and they share the same IP address. For example, there is a server with the IP address 61.135.169.125 on which the websites of Google, Baidu, and Taobao are deployed. HTTP1.1 request messages and response messages should support the Host header field, And no Host header field in the request message will report a 400 error
-
Long link HTTP1.1 supports long connection and request pipelining-multiple requests and responses can be sent over a single TCP connection, reducing the wait and delay for establishing and closing connections. HTTP1.1 Connection: keep-alive is enabled by default
New features of HTTP2.0
- The resolution of the new binary format http1.x is text-based. Text-based format parsing has a natural defect. The variety of text representations must be considered according to its application scenarios. Binary is different, and only 0 and 1 are convenient and robust.
- Multiplexing Multiplexing can be seen as an upgrade to HTTP1.1 long connections, where only one request can be sent per connection and the connection is closed at the end of the request. HTTP1.1 long connections can send several requests in a connection, but there is a thread blocking problem, which is several requests queued in a single thread. When a request times out, subsequent requests can only be blocked. Multiplexing allows multiple requests in a connection to be randomly mixed together without affecting each other. When a task takes too much time, other requests will not be affected.
- Header The header of the compression request has a lot of information and is sent repeatedly. HTTP2.0 uses encoder methods to reduce the size of the header fields in transmission. The communication parties have a cached header fields table to compare the data in transmission. This avoids duplicate header transmission and reduces the size of transmitted data
- Server push means that the server has the server push function
HTTPS module
The API of the HTTPS module of Node is similar to that of the HTTP2 module. You can refer to the usage documentation of the HTTPS module to focus on the development of HTTPS. HTTP requests are transmitted in plaintext and data may be intercepted. To solve this problem, Netscape developed the HTTPS protocol, which encrypts and transmits data, known as ciphertext, to ensure network communication is secure even if the data is intercepted.
Fundamentals of cryptography
Before we dive into the HTTPS protocol, we need to know a little bit about cryptography.
-
Plaintext: Plaintext refers to raw data that has not been encrypted
-
Ciphertext: The plaintext becomes ciphertext after being encrypted by an encryption algorithm. Ciphertext can also be decrypted to obtain the original plaintext
-
Key: A key is a parameter that is converted between plaintext and ciphertext by an algorithm. The key is used as the parameter of the algorithm
-
Symmetric encryption: Symmetric encryption is also called private key encryption, that is, the data receiver and the data sender use the same key to encrypt and decrypt data. Symmetric encryption is characterized by fast algorithm disclosure, encryption and decryption, and is suitable for encrypting large amounts of data. Common algorithms are DES, 3DES, TDEA, Blowfish, RCS, and IDEA
- Encryption process: Plaintext + encryption algorithm + key => ciphertext
- Decryption process: Ciphertext + decryption algorithm + key => plaintext
The key used in symmetric encryption is called a private key, and the same private key is used for encryption and decryption. Because the algorithm is public, the private key can be easily cracked once the ciphertext is disclosed, so the disadvantage of symmetric encryption is that the security management of the key is difficult.
-
Asymmetric encryption: Asymmetric encryption, also known as public key encryption, is more secure than symmetric encryption. Asymmetric encryption uses a set of keys, the public key and the private key, and the two come in pairs. The private key is kept by itself and cannot be disclosed to each other. A public key is a public key that can be obtained by anyone, encrypted with either the public key or the private key, and decrypted with the other
HTTPS Communication Process
In the process of HTTPS data transmission, SSL/TLS is used to encrypt and decrypt the data, and HTTPS is used to transmit the encrypted data. It can be seen that HTTPS is accomplished by the cooperation of HTTP and SSL/TLS. That is, HTTPS = HTTP + SSL/TLS
To ensure efficiency and security, HTTPS uses both symmetric encryption and asymmetric encryption. Data is transmitted using symmetric encryption. During symmetric encryption, a client key is required. Symmetric encryption is used to encrypt data. The keys used in symmetric encryption are transmitted through asymmetric encryption.
The public and private keys of the server are used for asymmetric encryption. The random key generated by the client is used for symmetric encryption
The digital certificate
HTTP does not verify the identity of the communication parties, so the identity can be forged, causing security problems. Digital certificates were invented to solve this problem. The usage process is as follows:
- The server first applies for an identity certificate from an authoritative third party organization
- The client must request the server to obtain the server certificate before establishing communication with the server
- When the server receives the request, it sends the digital certificate to the client
- After obtaining the server certificate, the client authenticates it with the trusted third-party certificate. After the authentication succeeds, the client can communicate with the trusted third-party certificate.
A digital signature
HTTP does not verify the integrity of the data, so that if the data is tampered with during the communication, neither party can know about it, hence the digital signature technology
Digital signatures have two main functions:
- Verify that the data is emitted by the target object
- Verify data integrity to verify whether the data has been tampered with.
- The process of summarizing the sent content is very similar to the hashmap generated by webpack each time. The communication parties agree that the sender will get the hash value of the data through the hash algorithm each time, so that the new hash value will be generated every time the data changes, and the sender will send the hash value as the summary of the data to the other party. After receiving the data and abstract, the other party hashes the data through the agreed hash algorithm, and then compares the hash value with the transmitted abstract to judge the integrity of the data.
- Abstract information is signed to confirm the identity of the data sender, the signature technology uses asymmetric encryption principle