?

Buffer has been mentioned in several recent articles. Although I have used several apis, I would like to summarize the system, mainly for API, and the specific use will be a little less

Remote Procedure Call

WebSocket protocol and WS source code analysis

Why Buffer?

Prior to the introduction of TypedArray, the JavaScript language had no mechanism for reading or manipulating binary data streams. The Buffer class was introduced as part of the Node.js API to interact with octet streams in TCP streams, file system operations, and other contexts.

The most common scenarios are:

  • TCP flow: currently encountered in the packet and unpacket phase, is covered in the three articles mentioned

    • WebSocketData frame
    • RPCCommunication protocol
    • Etc., etc.

  • File system Operations

reference

Network order? Local order? Stupid can’t tell…

1. Create Buffer

1.1 alloc family

Simply create a size ofsizeA new Buffer of bytes

  • Buffer.alloc(size[, fill[, encoding]])

    • The allocated buffer is initialized with buf.fill to ensure that it will never contain sensitive data

    • It’s much slower than allocUnsafe because of the initialization

  • Buffer.allocUnsafe(size)

    • The distribution ofbufferMemory is uninitialized and may contain sensitive data

  • Buffer.allocUnsafeSlow(size)

    • The allocated buffer memory is uninitialized and may contain sensitive data

    • Create a buffer that is not pooled to prevent the garbage collection mechanism from overusing it by creating too many separate buffers

Three choices:

// Need a new buffer buffer. Alloc // need a buffer, don't care if sensitive data, it will be overwritten anyway, AllocUnsafe // Requires a Buffer that needs to be stored in the memory poolCopy the code

From the quintuplets

  • Buffer.from(string[, encoding])

    • Create one from the stringbufferIs the most commonly used

  • Buffer.from(buffer)

    • frombufferTo create abuffer, often used for replication

  • Buffer.from(array)

    • A common one from numeric arraysbufferIf it is not a number, it is automatically filled with 0

  • Buffer. From (arrayBuffer[, byteOffset[, length]]): Because it has not been used, it is not worth remembering

  • Buffer.from(object[, offsetOrEncoding[, length]])

    • Create a buffer from the object that actually looks like buffer.from (string)

    • Essentially calling the Symbol. ToPrimitive or valueOf method of the object

1: chestnut Buffer. The from (object)

class A {
  [Symbol.toPrimitive]() {
    return 'A'
  }
}
Buffer.from(new A(), 'utf-8') // Print <Buffer 41>Copy the code

2: chestnut Buffer. The from (array)

Buffer.from(['a'.'1', 2, 'b'.'3'] // Print <Buffer 00 01 02 03>Copy the code

2. Iteration of Buffer

Buffer instances can use for.. The of syntax iterates, and related functions include:

  • buf.keys()

  • buf.values()

  • buf.entries()

But it’s very boring

const buf = Buffer.from([1, 2, 3]);

for(const b of buf) { console.log(b); } // Print: // 1 // 2 // 3Copy the code

Index operators are more interesting than this: the index operator [index] can be used to get or set bytes at a specified location in buF

const buf = Buffer.from('Calabash')

console.log(buf[0]) // 67
Copy the code

3. Collision between two buffers

  • buf.compare

    • It is used to sort Buffer arrays and is recommended to skip

  • buf.copy(target[, targetStart[, sourceStart[, sourceEnd]]])

    • copybufData to a region intargetA region in

  • buf.equals(otherBuffer)

    • Compare the twobufferWhether the bytes are exactly the same

  • Buffer.concat(list[, totalLength])

    • Merge a listBufferAn array of

4. BE and LE

4.1 the size of the end

Consider an integer with w bits (bit) represented by [Xw-1, XW-2,…,X1, X0], where XW-1 is the most significant bit and X0 is the least significant bit. Assuming w is a multiple of 8, these bits can be grouped into bytes, with the most significant byte containing bits [xW-1, XW-2,… xW-8] and the least significant byte containing bits [X7, X6,… xW-8], and the size end referring to the arrangement of the significant bytes

  • Little endian: the least significant byte is first

  • Big endian: The most significant byte is first

Big end vs small end comparison

// If the following two bytes are binary 1111 1111 0000 0001 // convert to hexadecimal 0xFF 0x01 // the big end outputs 65281 console.log(buffer. from([0xff, 0x01]).uint16be (0).toString(10))Copy the code

TCP/IP defines a unified Network byte order for any integer data item: big-endian byte order

A little extra

  • Network ordering is big-endian byte order

  • The local order may be big-endian or small-endian, depending on the machine type

4.2 Read/write throttling

There are a total of 22 apis for reading byte streams, including the following two apis that require only one byte to represent and do not involve the size side

  • Buf. readInt8: Reads a signed 8-bit integer, 1 byte

  • Buf. readUint8: Read an unsigned 8-bit integer, 1 byte

Other numeric types include:

  • Read BigInt, 8 bytes:

    • readBigInt64BE
    • readBigInt64LE
    • readBigUInt64BE
    • readBigUInt64LE

  • Read 64 bit double precision floating point number, 8 bytes:

    • readDoubleBE
    • readDoubleLE

  • Read a 32-bit floating-point number, 4 bytes:

    • readFloatBE
    • readFloatLE

  • Read 16-bit integers, 2 bytes

    • readInt16BE
    • readInt16LE
    • readUInt16BE
    • readUInt16LE

  • Read a 32-bit integer, 4 bytes

    • readInt32BE
    • readInt32LE
    • readUInt32BE
    • readUInt32LE

  • Read the specified number of bytes. The maximum accuracy is 48 bits, 0 to 6 bytes

    • readIntBE
    • readIntLE
    • readUIntBE
    • readUIntLE

The write stream API is the same as abovereadSwitch towrite

4.3 Exchanging large and Small Ends

There are three methods involved, all of which parse buF into an array of unsigned X-bit integers and swap them in place in byte order

  • Swap16 (): buf.length must be a multiple of 2

  • Swap32 (): buf.length must be a multiple of 4

  • Swap64 (): buf.length must be a multiple of 8

Such as:

// 65281
Buffer.from([0xff, 0x01]).readUInt16BE(0).toString(10)
// 511
Buffer.from([0xff, 0x01]).readUInt16LE(0).toString(10)
// 511
Buffer.from([0xff, 0x01]).swap16().readUInt16BE(0).toString(10)
Copy the code

5. Small dessert

  • buf.toJSONReturns thebufJSON format when stringingBufferWhen,JSON.stringify()This function is called
const buf = Buffer.from([0x1, 0x2, 0x3, 0x4, 0x5]); const json = JSON.stringify(buf); console.log(json); // Print: {"type":"Buffer"."data": [1, 2, 3, 4, 5]}Copy the code

  • Buf. slice(start, end): create a new Buffer that points to the same memory as the original Buffer, but with clipping. Note that the memory allocated by the two objects overlaps, and modifying one will affect the other

  • Base64 and String conversion

// Y2FsYWJhc2g=
const base64Str = Buffer.from('calabash').toString('base64')
// calabash
Buffer.from(base64Str, 'base64').toString()
Copy the code

6. END

So far, I have basically finished learning the Buffer API. However, at my current level, THERE is no scenario where I need to unpack the Buffer API myself. I can only watch others unpack TAT