Nodejs advanced: A summary of the common API usage of the core module Buffer

An overview of the module

Buffer is the core module of Node. Developers can use it to handle binary data, such as reading and writing file streams and processing network requests.

There are many Buffer apis, this article only selects the more common/easy to understand APIS to explain, including Buffer instance creation, comparison, connection, copy, search, traversal, type conversion, interception, encoding conversion, etc.

create

new Buffer(array)
Buffer.alloc(length)
Buffer.allocUnsafe(length)
Buffer.from(array)

Through the new Buffer (array)

// Creates a new Buffer containing the ASCII bytes of the string 'buffer'
const buf = new Buffer([0x62.0x75.0x66.0x66.0x65.0x72]);
Copy the code

Validation:

var array = 'buffer'.split(' ').map(function(v){
    return '0x' + v.charCodeAt(0).toString(16)});console.log( array.join() );
/ / output: 0 x62, 0 x75, 0 x66, 0 x66, 0 x65, 0 x72
Copy the code

Through the Buffer. Alloc (length)

var buf1 = Buffer.alloc(10);  // A buffer of length 10 with the initial value 0x0
var buf2 = Buffer.alloc(10.1);  // A buffer of length 10 with the initial value 0x1
Copy the code

var buf3 = Buffer.allocUnsafe(10);  // A buffer of length 10 with an indeterminate initial value
Copy the code

var buf4 = Buffer.from([1.2.3])  // A buffer of length 3 with initial values 0x01, 0x02, 0x03
Copy the code

Through the Buffer. The from ()

Buffer. From (array)

// [0x62, 0x75, 0x66, 0x66, 0x65, 0x72] is the string "buffer"
// 0x62 is a hexadecimal, which in decimal form is 98, representing the letter B
var buf = Buffer.from([0x62.0x75.0x66.0x66.0x65.0x72]);
console.log(buf.toString());
Copy the code

Buffer.from(string[, encoding])

When creating a buffer from a string, keep the same encoding as when converting a buffer to a string, otherwise garbled characters will appear, as shown below.

var buf = Buffer.from('this is a tést');  // Utf8 is used by default

// Output: this is a test
console.log(buf.toString());  // The default encoding is UTf8, so print normally

// Output: this is a tC
console.log(buf.toString('ascii'));  // When converted to a string, the encoding is not UTF8, so garbled
Copy the code

The analysis of garbled codes is as follows:

var letter = 'é';
var buff = Buffer.from(letter);  // The default encoding is UTf8, which takes two bytes 
      
var len = buff.length;  / / 2
var code = buff[0]; // The first byte is 0xc3, which is 195: out of the maximum ASCII support range
var binary = code.toString(2);  // Binary of 195:10101001
var finalBinary = binary.slice(1);  // Discard the high 1 to: 0101001
var finalCode = parseInt(finalBinary, 2);  // The decimal value of 0101001 is 67
var finalLetter = String.fromCharCode(finalCode);  // The corresponding character of 67 is C

// in the same way, 0xa9 converts to ASCII characters.
// This is a tC)st
Copy the code

Example 3: Buffer. From (Buffer)

Create a new Buffer instance and copy the data from the Buffer to the new real example.

var buff = Buffer.from('buffer');
var buff2 = Buffer.from(buff);

console.log(buff.toString());  // Output: buffer
console.log(buff2.toString());  // Output: buffer

buff2[0] = 0x61;

console.log(buff.toString());  // Output: buffer
console.log(buff2.toString());  // Output: auffer
Copy the code

Buffer is

buf.equals(otherBuffer)

Determines whether two buffer instances store the same data, returning true if so, false otherwise.

// Example 1: Same encoding, same content
var buf1 = Buffer.from('A');
var buf2 = Buffer.from('A');

console.log( buf1.equals(buf2) );  // true

// Example 2: Same coding, different content
var buf3 = Buffer.from('A');
var buf4 = Buffer.from('B');

console.log( buf3.equals(buf4) );  // false

// Example 3: Different encoding, same content
var buf5 = Buffer.from('ABC');  // <Buffer 41 42 43>
var buf6 = Buffer.from('414243'.'hex');

console.log(buf5.equals(buf6));    //true

// Buf.equals (otherBuffer) returns true as long as the two items are the same
Copy the code

buf.compare(target[, targetStart[, targetEnd[, sourceStart[, sourceEnd]]]])

The two buffer instances are also compared, but the difference is:

You can specify a range for a particular comparison (specified by start, end)
The return value is an integer that meets the size relationship between BUF and target

Suppose the return value is

0: BuF and target have the same size.
1: buF is greater than target, which means that BUF should be ranked after target.
- 1: BUF is smaller than target, which means that BUF should be ranked before target.

Look at the example, the official example is very good, directly paste:

const buf1 = Buffer.from('ABC');
const buf2 = Buffer.from('BCD');
const buf3 = Buffer.from('ABCD');

// Prints: 0
console.log(buf1.compare(buf1));

// Prints: -1
console.log(buf1.compare(buf2));

// Prints: -1
console.log(buf1.compare(buf3));

// Prints: 1
console.log(buf2.compare(buf1));

// Prints: 1
console.log(buf2.compare(buf3));

// Prints: [ <Buffer 41 42 43>, <Buffer 41 42 43 44>, <Buffer 42 43 44> ]
// (This result is equal to: [buf1, buf3, buf2])
console.log([buf1, buf2, buf3].sort(Buffer.compare));
Copy the code

Buffer.compare(buf1, buf2)

Similar to buf.compare(target), it is used for sorting. Post official examples directly:

const buf1 = Buffer.from('1234');
const buf2 = Buffer.from('0123');
const arr = [buf1, buf2];

// Prints: [ <Buffer 30 31 32 33>, <Buffer 31 32 33 34> ]
// (This result is equal to: [buf2, buf1])
console.log(arr.sort(Buffer.compare));
Copy the code

From the Buffer. The from ([62])

Look at buffer.from (array) a little bit here. The API is described in the official documentation below, that is, each array element corresponds to one byte (8 bits) ranging from 0 to 255.

Allocates a new Buffer using an array of octets.

The array elements are numbers

Let’s start with a scenario where the element passed in is a number. The following are base 10, base 8, and base 16, as expected.

var buff = Buffer.from([62])
// <Buffer 3e>
// buff[0] === parseInt('3e', 16) === 62
Copy the code

var buff = Buffer.from([062])
// <Buffer 32>
// buff[0] === parseInt(62, 8) === parseInt(32, 16) === 50
Copy the code

var buff = Buffer.from([0x62])
// <Buffer 62>
// buff[0] === parseInt(62, 16) === 98
Copy the code

Array elements are strings

Now look at the scenario where the element passed in is a string.

0The initial string, in parseInt(‘062’), can be interpreted as either 62 or 50 (octal), as seen here using the first interpretation.
If parseInt() has any relation to the string scenario, it is just a guess. TODO (Find time to study)

var buff = Buffer.from(['62'])
// <Buffer 3e>
// buff[0] === parseInt('3e', 16) === parseInt('62') === 62
Copy the code

var buff = Buffer.from(['062'])
// <Buffer 3e>
// buff[0] === parseInt('3e', 16) === parseInt('062') === 62
Copy the code

var buff = Buffer.from(['0x62'])
// <Buffer 62>
// buff[0] === parseInt('62', 16) === parseInt('0x62') === 98
Copy the code

The size of an array element exceeds 1 byte

If you are interested, explore.

var buff = Buffer.from([256])
// <Buffer 00>
Copy the code

Buffer.from(‘1’)

Buffer.from(‘1’)[0] was initially equated with “1” when the code for “1” was 49.

var buff = Buffer.from('1')  // <Buffer 31>
console.log(buff[0= = =1)  // false
Copy the code

1 is a control character that means Start of Heading.

console.log( String.fromCharCode(49))/ / '1'
console.log( String.fromCharCode(1))// '\u0001'
Copy the code

Buffer connection: buffer.concat (list[, totalLength])

Note: I personally feel that totalLength is a redundant parameter. According to the official documentation, it is for performance improvement. However, the internal implementation just iterates through the list and adds the length to totalLength. From this point of view, the performance optimization is almost negligible.

var buff1 = Buffer.alloc(10);
var buff2 = Buffer.alloc(20);

var totalLength = buff1.length + buff2.length;

console.log(totalLength);  / / 30

var buff3 = Buffer.concat([buff1, buff2], totalLength);

console.log(buff3.length);  / / 30
Copy the code

In addition to the performance tuning mentioned above, there are two other things to note about totalLength. Assume that the sum of all buffers in the list is length

TotalLength > length: Returns a Buffer instance of totalLength length, filled with 0 for the portion exceeding the length.
TotalLength < length: Returns a Buffer instance of totalLength length, the rest of which is discarded.

var buff4 = Buffer.from([1.2]);
var buff5 = Buffer.from([3.4]);

var buff6 = Buffer.concat([buff4, buff5], 5);

console.log(buff6.length);  // 
console.log(buff6);  // <Buffer 01 02 03 04 00>

var buff7 = Buffer.concat([buff4, buff5], 3);

console.log(buff7.length);  / / 3
console.log(buff7);  // <Buffer 01 02 03>
Copy the code

Copy (target[, targetStart[, sourceStart[, sourceEnd]])

If you omit the last three parameters, copy the buF data to target, as shown below:

var buff1 = Buffer.from([1.2]);
var buff2 = Buffer.alloc(2);

buff1.copy(buff2);

console.log(buff2);  // <Buffer 01 02>
Copy the code

The other three parameters are more intuitive, directly look at the official example

const buf1 = Buffer.allocUnsafe(26);
const buf2 = Buffer.allocUnsafe(26).fill('! ');

for (let i = 0 ; i < 26 ; i++) {
  // 97 is the decimal ASCII value for 'a'
  buf1[i] = i + 97;
}

buf1.copy(buf2, 8.16.20);

// Prints: !!!!!!!! qrst!!!!!!!!!!!!!
console.log(buf2.toString('ascii'.0.25));
Copy the code

Buf.indexof (value[, byteOffset][, encoding])

As with array lookups, it is important to note that values can be of any type from String, Buffer, or Integer.

String: If it is a String, encoding is the encoding, which is utF8 by default.
Buffer: In the case of a Buffer instance, the full data in value is compared to buF.
Integer: If it is a number, value is treated as an unsigned 8-bit Integer ranging from 0 to 255.

Alternatively, byteOffset can be used to specify the starting search location.

Directly on the code, the official example of no problem, the patience to see it basically understand almost.

const buf = Buffer.from('this is a buffer');

// Prints: 0
console.log(buf.indexOf('this'));

// Prints: 2
console.log(buf.indexOf('is'));

// Prints: 8
console.log(buf.indexOf(Buffer.from('a buffer')));

// Prints: 8
// (97 is the decimal ASCII value for 'a')
console.log(buf.indexOf(97));

// Prints: -1
console.log(buf.indexOf(Buffer.from('a buffer example')));

// Prints: 8
console.log(buf.indexOf(Buffer.from('a buffer example').slice(0.8)));


const utf16Buffer = Buffer.from('\u039a\u0391\u03a3\u03a3\u0395'.'ucs2');

// Prints: 4
console.log(utf16Buffer.indexOf('\u03a3'.0.'ucs2'));

// Prints: 6
console.log(utf16Buffer.indexOf('\u03a3', -4.'ucs2'));
Copy the code

Write: buf.write(string[, offset[, length]][, encoding])

Writes sring to the buf instance, returning the number of bytes written.

The parameters are as follows:

String: the written character string.
Offset: starts from the buF bit. The default value is 0.
Length: Indicates the number of bytes to be written. The default value is buf. length-offset.
Encoding: Encoding of the string. The default value is UTF8.

Let’s do a simple example

var buff = Buffer.alloc(4);
buff.write('a');  / / returns 1
console.log(buff);  // Print 
      

buff.write('ab');  / / return 2
console.log(buff);  // Print 
      
Copy the code

Fill: buf.fill(value[, offset[, end]][, encoding])

The BUF is populated with value, often used to initialize the BUF. The parameters are described as follows:

Value: The content to fill. It can be Buffer, String, or Integer.
Offset: starts from the number of digits. The default value is 0.
End: the position to stop filling. The default is buf.length.
Encoding: ifvalueIs String, then isvalueThe default is UTF8.

Example:

var buff = Buffer.alloc(20).fill('a');

console.log(buff.toString());  // aaaaaaaaaaaaaaaaaaaa
Copy the code

ToString ([encoding[, start[, end]])

Decode buF into string, use more intuitive, look at the example

var buff = Buffer.from('hello');

console.log( buff.toString() );  // hello

console.log( buff.toString('utf8'.0.2));// he
Copy the code

Convert toJSON string: buf.tojson ()

var buff = Buffer.from('hello');

console.log( buff.toJSON() );  // { type: 'Buffer', data: [ 104, 101, 108, 108, 111 ] }
Copy the code

Traversal: buf.values(), buf.keys(), buf.entries()

Buf for… Of traversal, let’s look at the example.

var buff = Buffer.from('abcde');

for(const key of buff.keys()){
    console.log('key is %d', key);
}
// key is 0
// key is 1
// key is 2
// key is 3
// key is 4

for(const value of buff.values()){
    console.log('value is %d', value);
}
// value is 97
// value is 98
// value is 99
// value is 100
// value is 101

for(const pair of buff.entries()){
    console.log('buff[%d] === %d', pair[0], pair[1]);
}
// buff[0] === 97
// buff[1] === 98
// buff[2] === 99
// buff[3] === 100
// buff[4] === 101
Copy the code

Buf. slice([start[, end]])

Intercepts the BUF and returns a new Buffer instance. Note that the Buffer instance returned here points to the memory address of the BUF, so changes to the new Buffer instance will also affect the BUF.

var buff1 = Buffer.from('abcde');
console.log(buff1);  // <Buffer 61 62 63 64 65>

var buff2 = buff1.slice();
console.log(buff2);  // <Buffer 61 62 63 64 65>

var buff3 = buff1.slice(1.3);
console.log(buff3);  // <Buffer 62 63>

buff3[0] = 97;  // parseInt(61, 16) ==> 97
console.log(buff1);  // <Buffer 62 63>
Copy the code

TODO

Create, copy, intercept, transform, find
Buffer, ArrayBuffer, DataView, TypeDarray
Buffer vs coding
Buffer.from(), buffer.alloc (), buffer.alocunsafe ()
Buffer vs TypedArray

Document the

Dynamic allocation of buffer memory space

Instances of the Buffer class are similar to arrays of integers but correspond to fixed-sized, raw memory allocations outside the V8 heap. The size of the Buffer is established when it is created and cannot be resized.

A link to the

Unicode table unicode-table.com/cn/#control…

A character encoding notes: ASCII, Unicode and utf-8 www.ruanyifeng.com/blog/2007/1…