5. buffer&fs&stream

coding

The concept of buffer

Buffer is used to describe memory. Memory is binary and Buffer is hexadecimal

• On the server side, we need something to identify memory, but it can’t be a string, because strings don’t identify images
• Nodes use buffers to identify in-memory data and convert the contents to hexadecimal for display (because hexadecimal is short).
• Base 10 255 => base 2 0b11111111 => hexadecimal 0xFF => Buffer Each byte ranges from 0 to 0xFF
• The JavaScript language itself only has string data types, not binary data types. But when dealing with things like TCP streams or file streams, you must use binary data. Therefore, in Node.js, a Buffer class is defined to create a Buffer dedicated to binary data.
• (UTF-8) 1 Chinese character 3 bytes
• A buffer is similar to an array in that its elements are hexadecimal two-digit numbers ranging from 0 to 255
• Buffer and string conversion in node (garbled)

coding

• Node encodings currently support only UTF8, which takes up different bytes. For UTF8, a Chinese character is three bytes
• A byte is the smallest observable unit
• Bit A byte consists of eight bits (binary)
• Convert any base to base 10: the sum of the current bit * the current bit ^(bit -1) =>parseInt("1010", 2);
• To convert an integer to other bases and reverse output => corresponding method:10.. toString(2)

Base 2 starts with 0B base 8 starts with 0O base 16 starts with 0x

• Decimals are also converted to base 2
  • 0.5 in base 10 is 0.1 in base 2 (because 10 => 0.5 20x, 2 => 0.1 20x)
  • Decimal Method of converting a decimal to binary: The round by two method converts a decimal to base 2

// 0.1 + 0.2 problem
// The calculation does not add directly, but converts both values to base 2

0.1 * 2 = 0.2 0
0.2 * 2 = 0.4 0
0.4 * 2 = 0.8 0
0.8 * 2 = 1.6 1
0.6 * 2 = 1.2 1
0.1 * 2 = 0.2 0
0.2 * 2 = 0.4 0
0.4 * 2 = 0.8 0
0.8 * 2 = 1.6 1
0.6 * 2 = 1.2 1

// 0.1 to binary is an infinite decimal
// 0.1 is slightly larger than 0.1 when converted to binary storage
// The same goes for 0.2
// So 0.1 + 0.2 will be greater than 0.3

console.log(0.1+0.2) // This is a base conversion problem

// js does not convert decimals to base 2
// why is 0.2+0.2 ok?

// How to solve the accuracy problem?
Copy the code

Coding standards

• ASCII
  • A byte can be up to 255, that is, can represent 255 numbers, case and some special symbols used up to 127, a byte to do all the encoding
• GB8030/GBK
  • 255 is not enough for Chinese
  • From 127 after the compilation of their own, excluding other countries
• unicode
  • Organize a code
• UTF8
  • Implement your own set of mechanisms using Unicode
  • If it’s a character, it still follows ASCII
  • Character set
• Node does not support GBK, only UTF8

The common encoding implementation is Base64

Because in some systems only ASCII characters can be used. Base64 is a method used to convert non-ASCII data into ASCII characters.

• Base64 encoding: Base64 is just an encoding rule without encryption

• Base64 strings can be placed in links in any path, reducing the number of requests to be sent, but the file size will increase, and the base64 converted file will be 1/3 larger than the previous file

• Base64 comes from converting every byte to a value less than 64

• Base64 principle

  • A Chinese character has 3 bytes and 24 bits
  • To convert 24 bits of a Chinese character into 4 bytes, each byte has 6 bits, but a byte should have 8 bits, so 0 is not enough.

// Base64 method
// Call toString to convert to the specified encoding
// The Chinese character "bead" is 3 bytes, which is 54+g is 4 bytes
const r = Buffer.from('pearl').toString('base64');
Copy the code

Let buffer = buffer. from(" bead "); /* Base64 is a base conversion. Buffer.tostring ("base64") -1. Buffer is a hexadecimal system. To convert hexadecimal to base 2 => toString() (0xe7).toString(2); (0x8f).toString(2); (0xa0).toString(2); // 11100111 10001111 10100000 => convert to 4 bytes, each byte has 6 bits, // 111001 111000 111110 100000 // 00111001 00111000 00111110 00100000 // 00111001 00111000 00111110 00100000 // 00111001 00111000 00111110 00100000 That is, the converted value should not be greater than 64) -2. Convert these values to base 10, ParseInt () parseInt("00111001", 2) or parseInt(0b00111001) //57 parseInt("00111000", 2) //56 parseInt("00111110", 2) //62 parseInt("00100000", 2) //32 visible encoding: Standard base64 only 64 characters (English case, number, +, /) let STR = "ABCDEFGHIJKLMNOPQRSTUVWXYZ"; str += "abcdefghijklmnopqrstuvwxyz"; str += "0123456789"; str += "+/"; Console. log(STR [57] + STR [56] + STR [62] + STR [32]) // The base64 encoding is mapped without encryption */
Copy the code

Buffer

• A buffer represents memory. Memory is a “fixed space” that is a fixed amount of memory that cannot be added to it
• If you want to change the size of the buffer, you can intercept the memory if you want to change the size of the buffer. If you want to change the size of the buffer, you need to create a large memory space and copy the data to it
• Alloc is used to declare a buffer or convert a string to a buffer

const buffer1 = Buffer.alloc(5);

// Like an array, but unlike arrays, arrays can be expanded. Buffers cannot be expanded and can be indexed
console.log(buffer[0]); // Prints the decimal number of the byte in the current buffer

// Use hexadecimal
const buffer2 = Buffer.from([0x24.0x25.0x26]);

const buffer3 = Buffer.from('ha ha'); // Corresponding to 6 bytes
Copy the code

• The data obtained in the background are all buffers, including file operations

There are three ways to define buffers

1. let buf1 = Buffer.alloc(6);
2. Let buf2 = Buffer. From (' haha ');
3. Let buf3 = Buffer. The from (,66,67 [65]);

• Define buffers by length
  

  let buffer = Buffer.alloc(6)

  Specifies the size of a buffer. The parameter is a number up to 3, where the number refers to bytes

  let buffer = Buffer.allocUnsafe(6)

• Define buffers by arrays
  

  Let buffer = buffer. from([1,2,3,4]) automatically converts the decimal number in the array to hexadecimal

  The argument is an array, which tells you what to store in a buffer. There are very few scenarios. Only numbers 0-255 can be stored in the array

• String creates buffer – most used
  

  Let buffer = buffer. from(‘ ha ha ‘) => string to buffer

  Buffer. length refers to the length of a hexadecimal buffer

  Buffer.tostring () => Convert buffer to a string buffer.tostring () does not pass arguments. The default is UTF8

Buffer method

• The buffer function is used to store binary data, and each bit in the buffer represents a byte. If displayed in binary, each byte is displayed as 8 bits. If displayed in hexadecimal, each byte is displayed as two bits. This is why each element of the buffer (each byte) is displayed in hexadecimal
• Whether it’s base 2 or base 16, they represent the same thing
• Common methods of buffer
  

  Buffer. fill fills the contents of the buffer

  Buffer.tostring Converts a buffer to a string

  Slice Intercepts the desired buffer

  Buffer. The buffer copy copies

  Buffer. concat Concatenation method for buffers

  Buffer. isBuffer Checks whether it is a buffer

var buffer = Buffer.from([1.2.3]);
var newBuffer = buffer.slice(0.1); // Copy the memory address space
newBuffer[0] = 100; // Modify newBuffer, and the buffer also changes

var buf1 = Buffer.from('xi xi');
var buf2 = Buffer.from('ha ha');
var buf = Buffer.allocUnsafe(12);
// Copy buffer
// targetBuffer targetBuffer targetStart start of target sourceStart start of source sourceEnd end of source this.length
// Copy buf1/buf2 to buf
buf1.copy(buf, 0);
buf2.copy(buf, 6);
console.log(buf, buf.toString());
// If the buffer is too long, you need to calculate the location of the copy, and you cannot change the size of the buffer.
// So concat is more convenient. It automatically creates the sum of the two buffers to be concatenated and then concatenates them.
/ / connection buffer
Buffer.concat([buf1, buf2]).toString();

Buffer.prototype.MyCopy = function (
  targetBuffer,
  targetStart,
  sourceStart = 0,
  sourceEnd = this.length
) {
  for (let i = sourceStart; i < sourceEnd; i++) {
    targetBuffer[targetStart++] = this[i]; }}; Buffer.MyConcat =function (list, totalLength) {
  // Determine whether the length is passed, if passed, use the total length
  if (typeof totalLength === 'undefined') {
    totalLength = list.reduce((prev, next) = > prev + next.length, 0);
  }
  // Create a buffer of this size by length, buffer.alloc (len)
  let buffer = Buffer.alloc(totalLength);
  // Recirculate the list to copy each item to the large buffer buf.copy
  let offset = 0;
  list.forEach((buff) = > {
    if(! Buffer.isBuffer(buff))return;
    buff.copy(buffer, offset);
    offset += buff.length;
  });
  // If the length is too long, fill or use slice to intercept the valid length
  // Return a new buffer
  return buffer.slice(0, offset);
};
Copy the code

fs

• I/O Input Output Read files => Write operations (memory as a reference)
• When reading, the default encoding is buffer. If the file does not exist, an error is reported
• ToString (‘utf8’); toString(‘utf8’);
• The concept of reading is to read the contents of a file into memory, while writing is to read the contents of memory and write into a file

File status determination — fs.stat

fs.stat('1.txt'.function (err, stats) {
  if (err)
    // File does not exist
    stats.isFile(); // Whether it is a file
  stats.isDirectory(); // Whether it is a folder
});
Copy the code

Create a folder — fs.mkdir

Create folder D only when a/ B/C exists
fs.mkdir('a/b/c/d'.function (err) {});
// Create folders recursively
function makep(url, cb) {
  let urlArr = url.split('/');
  let index = 0;
  function make(p) {
    if (urlArr.length < index) return; // Terminate the loop
    // See if it exists before creating it. If not, create it
    fs.stat(p, function (err) {
      if (err) {
        // File does not exist
        fs.mkdir(p, function (err) {
          index++;
          make(urlArr.slice(0, index + 1).join('/'));
        });
      } else {
        // If so, skip to the next creation
        make(urlArr.slice(0, ++index + 1).join('/')); }}); } make(urlArr[index]); } makep('a/b/c/d'.function (err) {
  console.log('Created successfully');
});
Copy the code

Read folder

fs.readdir('a'.function(err,dirs){}) // Only one layer can be read
Copy the code

Delete folders

fs.rmdir("a".function(err){}) // Delete the folder, do not delete the folder when there is content under it
Copy the code

Self-implementation Deletes folders

• Asynchronous deletion of multilayer directories
  • Serial delete
  • Concurrent delete delete subfolders delete and then delete yourself
  • Breadth traversal and then reverse delete

// Serial delete
function rmdir(dir, cb) {
    fs.stat(dir, function(err, stat) {
        if(stat.isDirectory()) {
            fs.readdir(dir, function(err, dirs) {
                dirs = dirs.map(item= > path.join(dir, item));
                let index = 0;
                function step()  {
                    if(index ===  dirs.length) return fs.rmdir(dir, cb);
                    rmdir(dirs[index++], step)
                }
                step()
            })
        }else{
            fs.unlink(dir, cb)
        }
    })
}
// Serial promise
const fs = require("fs").promises;
const path = require("path");
async function rmdir(dir) {
    const statObj = await fs.stat(dir);
    if(statObj.isDirectory()) {
        let dirs = await fs.readdir(dir);
        dirs = dirs.map(item= > path.join(dir, item));
        for(let i = 0; i<dirs.length; i++) {
            await rmdir(dirs[i]);
        }
        await fs.rmdir(dir);
    }else{
        return fs.unlink(dir);
    }
}
rmdir("aaa").then(() = > {
    console.log("success zhuzhu")})// Concurrent delete
function rmdir(dir, cb) {
    fs.stat(dir, function(err, statObj)  {
        if(stat.isDirectory()) {
            fs.readdir(dir, function(err, dirs) {
                dirs = dirs.map(item= > path.join(dir, item));
                if(! dirs.length) {return fs.rmdir(dir, cb);
                }
                let v = 0;
                function done() {
                    if(++v === dirs.length) {
                        return fs.rmdir(dir, cb)
                    }
                }
                for(let i  = 0; i < dirs.length; i++) { // Execute concurrently
                    rmdir(dirs[i], done)
                }
            })
        }else{
            fs.unlink(dir, cb)
        }
    })
}

// Make a promise

const fs = require("fs").promises;
const path = require("path");
async function rmdir(dir) {
   const statObj = await fs.stat(dir);
   if(statObj.isDerectory()) {
       let dirs = await fs.readdir(dir);
       await Promise.all(dirs.map(item= > rmdir(path.join(dir, item))));
       await fs.rmdir(dir);
   }else{
       returnfs.unlink(dir); }}// serial width traversal
const fs = require("fs").promises;
const path = require("path");
async function rmdir(dir) {
    let stack = [dir];
    let index = 0;
    let currentNode;
    while(currentNode = stack[index++]){ // width traversal, put all folders and files
        let statObj = await fs.stat(currentNode);
        if(statObj.isDirectory()) {
            let dirs = await fs.readdir(currentNode);
            dirs = dirs.map(item= >path.join(currentNode, item)); stack = [...stack, ...dirs]; }}// Delete in reverse order
    let len = stack.length;
    while(len--) {
        let stat = await fs.stat(stack[len]);
        if(stat.isDirectory()) {
            await fs.rmdir(stack[len])
        }else{
            await fs.unlink(stack[len])
        }
    }
}

rmdir("aaa").then(() = > {
    console.log("success zhuzhu")})Copy the code

Read the file

1. To read files files must have contents that cannot be read through /, which represents the root directory
2. The default type to read is buffer

let result = fs.readFileSync('log.js');
// There are two ways to convert the result format:
result.toString();
fs.readFileSync('log.js'.'utf8');
Copy the code

Written to the file

1. The read type is buffer and the write type is UTF8
2. The read file must exist. If the file does not exist, it will be automatically created and its contents will be overwritten
3. The toString method is called for the second argument (the contents of the file) by default.

fs.writeFile('1.txt', document content,(err) = > {});
fs.writeFileSync('1.txt', document content);Copy the code

Reading and writing files (copying)

// Implement synchronous/asynchronous file reading and writing functions
let fs = require("fs");

fucntion copySync(source, target) { // Synchronize readFileSync + writeFileSync
    let result = fs.readFileSync(source, "utf8");
    fs.writeFileSync(target, result);
}
function copy(source, target, callback){ // Asynchronous readFile + writeFile
    fs.readFile(source, function(err, data) {
        if(err) return callback(err)
        fs.writeFile(target, data, callback)
    })
}
copy("1.txt"."2.txt".function(res) {
    console.log(res);
})
Copy the code

• The above method can only be written after reading, which is suitable for small files. Large files using this method will flood the available memory
• Read and write

// This code block can be ignored

// Implement reading and writing three at a time: 123456789
const fs = require('fs');
const path = require('path');
let buf = Buffer.alloc(3);
fs.open(path.resolve(__dirname, 'a.txt'), 'r'.function (err, fd) {
  // fd:file descriptor is a number type, which should be closed when the file descriptor is used
  console.log(fd, 'fd');
  // read a.txt writes the read contents to position 0 of the buffer, three times, starting from position 6 of the file
  fs.read(fd, buf, 0.3.6.function (err, bytesRead) {
    // bytesRead is the actual number read
    console.log(bytesRead, 'bytesRead', buf);
    // Open and w will empty the file and always write the first 3 bytes
    fs.open(path.resolve(__dirname, 'b.txt'), 'w'.function (err, wfd) {
      // Read buffer data from 0 to the 0th position of the three written files
      fs.write(wfd, buf, 0.3.0.function (err, written) {
        console.log(written);
        // Internal recursion.....
        // Close the file
        fs.close(fd, () = > {});
        fs.close(wfd, () = > {});
      });
    });
  });
});
Copy the code

• W Write operation R read operation A append operation R + Write operation w+ Read operation
• Permissions: 3 groups (permissions of the current user Permissions of the user group Permissions of others) RWX combination (read, write, and execute) 421 => 7 => Highest permissions 777 (base 8)
• Fs.open (source,”r”,0o666, function(){}

Flow based on file system operations

• The following implementation reads and writes are coupled together
• Fs is a file stream, a self-implemented stream in file operations. File streams inherit from stream, and the underlying implementation uses fs.open

fs.read fs.write…

// Copy three bytes at a time
const fs = reuqire('fs');
function copy(source, target, cb) {
  const BUFFER_SIZE = 3;
  const buffer = Buffer.alloc(BUFFER_SIZE);
  const r_offset = 0;
  const w_offset = 0;
  // Write some data while read some data
  fs.open(source, 'r'.function (err, rfd) {
    fs.open(target, 'w'.function (err, wfd) {
      // The callback method is used recursively
      // The synchronization code can use a while loop
      function next() {
        fs.read(rfd,buffer,0,BUFFER_SIZE,r_offset,function (err, bytesRead) {
            if (err) return cb(err);
            if (bytesRead) {
              fs.write(wfd,buffer,0,bytesRead,w_offset,function (err, written){ r_offset += bytesRead; w_offset += written; next(); }); }else {
              fs.close(rfd, () = > {});
              fs.close(wfd, () = >{}); cb(); }}); } next(); }); }); } copy('./a.txt'.'./b.txt'.function (err) {});
Copy the code

To decouple read and write, adopt a publish-subscribe model and have readable/writable streams

A readable stream

• If a byte in buffer displays 30 => hexadecimal 30 to 10:48 => find the corresponding in THE ASCII code
• The concept of flow has nothing to do with FS
• Fs extends a file read/write method based on the stream module
• Fs.createreadstream () creates a readable stream object rs, which listens for events
• If you do not listen for the “data” event, it is in non-flowing mode, equivalent to buying a hose, but the water will not flow out
• When listening to the “data” event, from non-flowing mode => flowing mode, the data event callback function will be triggered repeatedly, and the highWaterMark value will be fetched each time
• The end event is raised when the file is finished reading
• So streams are event-based
• Readable stream has method: the data/end/error/resume/pause, as long as you have these methods is called a readable stream
• Open and close are methods unique to file streams

Use of readable streams

const fs = require('fs');
// Returns a readable stream object
const rs = fs.createReadStream('./a.txt', {
  // Create a readable stream without passing its own parameters
  flags: 'r'.// r reads for fs.open
  encoding: null.// The buffer is read by default
  autoClose: true.// Close the stream fs.close after reading
  emitClose: true.// Trigger a close event after reading
  start: 0.highWaterMark: 3.// The default value is 64 x 1024 bytes
});
// If you do not listen for the "data" event, it is in non-flowing mode, equivalent to buying a pipe, but the water does not flow out
// It will listen to the user, bind the data event will trigger the corresponding callback, constantly firing
// open/close is unique to the file stream
rs.on('open'.function (fd) {
  // Open file event
  console.log(fd);
});
rs.on('data'.function (chunk) { // If data is bound, data will be triggered continuouslyEvents pass internal data out// Read file event
  console.log(chunk);
  rs.pause(); // Pause does not continue to fire data events
});
rs.on('end'.function () {
  // File read completion event
  console.log('end');
});
rs.on('close'.function () {
  // Close the file event
  console.log('close');
});
rs.on('error'.function (err) {
  console.log(err, 'error');
});

setInterval(() = > {
  rs.resume(); // Restore the event that triggered data
}, 1000);
Copy the code

Implementation of readable streams

The idea here is worth learning

• The open method is asynchronous, and the implementation calls the read method to read the file when the open execution is complete

const fs = require('fs');
const EventEmitter = require('events');
class ReadStream extends EventEmitter {
  constructor(path, options = {}) {
    super(a);this.path = path;
    this.flags = options.flags || 'r';
    this.encoding = options.encoding || null;
    this.autoClose = options.autoClose || true;
    this.start = options.start || 0;
    this.end = options.end;
    this.highWaterMark = options.highWaterMark || 64 * 1024;

    this.open(); // File open operation Note that this method is asynchronous

    // Note that the user listens for the data event to read the file
    // this.read();

    // EventEmitter emits newListeners internally as long as the method is bound
    // This will allow you to monitor which events the user is bound to
    // newListener execution is triggered when the binding method name is not newListener
    this.on('newListener'.function (type) {
      console.log(type); // The newListener event is raised when an event is bound
      if (type === 'data') {
        this.read(); }}); }open() {
    fs.open(this.path, this.flags, (err, fd) = > {
      if (err) {
        return this.destroy(err);
      }
      this.fd = fd; // Used by the read method
      this.emit('open', fd); // Trigger the open event, when the file is opened, you can retrieve the fd
    });
  }
  read() {
    // The read method takes the fd
    // The fd is not available until the open callback is executed
    if (typeof this.fd ! = ='number') {
      // return this.once("open", (fd) => {
      // console.log(fd);
      // })
      return this.once('open'.() = > this.read());
    }
    console.log(this.fd); // You can get fd
  }
  destroy(err) {
    if (err) {
      this.emit('error', err); }}}module.exports = ReadStream;
Copy the code

// Complete implementation
const fs = require('fs');
const EventEmitter = require('events');
class ReadStream extends EventEmitter {
  constructor(path, options = {}) {
    super(a);this.path = path;
    this.flags = options.flags || 'r';
    this.encoding = options.encoding || null;
    this.autoClose = options.autoClose || true;
    this.start = options.start || 0;
    this.end = options.end;
    this.highWaterMark = options.highWaterMark || 64 * 1024;

    / / the offset
    this.offset = this.start;
    
    / / switch
    // false Non-flowing mode, the contents of the file have not been read. The subsequent pause and resume would update this flowing property
    this.flowing = false;

    this.open(); // File open operation Note that this method is asynchronous

    // Note that the user listens for the data event to read the file
    // this.read();

    // EventEmitter emits newListeners internally as long as the method is bound
    // This will allow you to monitor which events the user is bound to
    // newListener execution is triggered when the binding method name is not newListener
    this.on('newListener'.function (type) {
      console.log(type); // The newListener event is raised when an event is bound
      if (type === 'data') {
        this.flowing = true;
        this.read(); }}); }open() {
    fs.open(this.path, this.flags, (err, fd) = > {
      if (err) {
        return this.destroy(err);
      }
      this.fd = fd; // Used by the read method
      this.emit('open', fd); // Trigger the open event, when the file is opened, you can retrieve the fd
    });
  }
  read() {
    // The read method takes the fd
    Fd this.open is asynchronous
    if (typeof this.fd ! = ='number') {
      // return this.once("open", (fd) => {
      // console.log(fd);
      // })
      return this.once('open'.() = > this.read());
    }
    console.log(this.fd); // You can get fd
    const buffer = Buffer.alloc(this.highWaterMark);
    // Read the contents of the file, this.highwatermark one at a time
    let howMuchToRead = this.end
      ? Math.min(this.end - this.offset + 1.this.highWaterMark)
      : this.highWaterMark;
    fs.read(this.fd,buffer,0,howMuchToRead,this.offset,(err, bytesRead) = > {
        if (bytesRead) {
          this.offset += bytesRead;
          this.emit('data', buffer.slice(0, bytesRead));
          if (flowing) {
            this.read(); / / to read}}else {
          this.emit('end');
          this.destroy(); }}); }destroy(err) {
    if (err) {
      this.emit('error', err);
    }
    if (this.autoClose) {
      fs.close(this.fd, () = > this.emit('close')); }}pause() {
    this.flowing = false;
  }
  resume() {
    if (!this.flowing) {
      this.flowing = true;
      this.read(); }}}module.exports = ReadStream;
Copy the code

Summary of implementation process:

The constructor sets the default parameters, opens the file directly, and listens for the user’s events and starts reading if the user is bound to a data event

However, if the file is not open during the reading process, binding an open event and calling read itself after the file is opened is equivalent to delaying the read method

When the file is opened, open will be called, and read will be called, and fd will be ready to read the file

Each read file is put into buffer and thrown out

If it is in flow mode, the next read continues until the read is complete, triggering the end/close event

Streams can be written

• fs.createWriteStream(“./b.txt”)
• ws.on(‘open’)
• ws.write
• ws.on(‘drain’)
• ws.end

const fs = require('fs');
const ws = fs.createWriteStream('./b.txt', {
  flags: 'w'.encoding: 'utf8'.autoClose: true.start: 0.highWaterMark: 3.// The highWaterMark for writable streams is different from the highWaterMark for readable streams, indicating how much memory is expected to be used to write
});

ws.on('open'.(fd) = > {
    console.log('open', fd);
})


let flag = ws.write('1');
flag = ws.write('2');
flag = ws.write('345');
ws.end(); // Write +end
// Since the write method is asynchronous, if multiple write methods operate on the same file at the same time, there will be an error.
// Write the first time, queue the rest of the queue, put it on the stack, finish first, clear the cache,
// If the cache size is too large, memory will be wasted, so an expected value will be set to control, and the write method will not be called after the expected value is reached


let index = 0;
function write(){ // Will waste memory, only the first write is real, the subsequent are put into the cache
    while(index ! =5 ){
        ws.write(index++ + ' ')
    }
}
write();


let index = 0;
function write(){
    let flag = true;
    while(index ! =5 && flag){
        flag = ws.write(index++ + ' ')
    }
}
write();

ws.on('drain'.function(){ / / to dry up. This alarm is triggered when the expected data has been cleared and written to a file
    write();
    console.log('drain');
})

setTimeout(() = >{
    ws.end('Can write content'); Ws.write ()+fs.close
})
Copy the code

Implementation of writable streams

const fs = require('fs');
const EventEmitter = require('events');
class WriteStream extends EventEmitter {
  constructor(path, options = {}) {
    super(a);this.path = path;
    this.flags = options.flags || 'w';
    this.encoding = options.encoding || null;
    this.autoClose = options.autoClose || true;
    this.emitClose = options.emitClose || true;
    this.start = options.start || 0;
    this.highWaterMark = options.highWaterMark || 16 * 1024;

    this.open();

    // Customize the attributes written to the stream
    this.len = 0; // Record the total number of writes
    this.offset = this.start; // Write the offset
    this.cache=[]; // Cache write operations
    this.needDrain = false; // Whether to fire the drain event
    this.writing = false; // Indicates whether writing is in progress. If writing is in progress, put it in the cache
  }
  open() {
    fs.open(this.path, this.flags, (err, fd) = > {
      if (err) {
        return this.destroy(err);
      }
      this.fd = fd; // Used by the read method
      this.emit('open', fd); // Trigger the open event, when the file is opened, you can retrieve the fd
    });
  }
  write(chunk, encoding=null, cb=() => {}) {
    chunk = Buffer.isBuffer(chunk)? chunk : Buffer.from(chunk);
    this.len += chunk.length;
    this.needDrain = this.len >= this.highWaterMark;

    let oldCb = cb;
     After each successful write, the successful callback is called and another callback is fetched from the cache to continue writing
     // Until the cache is empty
    cb = () = > {
        oldCb();
        this.clearBuffer();
    }

    if(this.writing) {
        // Put it in the cache
        this.cache.push({chunk, encoding, cb});
    }else{
        // Write to the file
        this.writing = true;
        this._write(chunk, encoding, cb);
    }
    return !this.needDrain;
  }
  _write(chunk, encoding, cb) {
      if(typeof this.fd ! = ='number') {
          return this.once('open'.() = > this._write(chunk, encoding, cb));
      }
      fs.write(this.fd, chunk, 0, chunk.length, this.offset, (err,writen) = > {
          this.offset += writen;
          this.len -= writen; cb(); })}clearBuffer() {
      // Clear the cache after writing
      let obj = this.cache.shift();// The bottom layer uses linked lists to optimize queues
      if(obj) {
        this._write(obj.chunk, obj.encoding, obj.cb);
      }else{
          this.writing = false;
          if(this.needDrain) {
              this.needDrain = false;
              this.emit("drain"); }}}}module.exports = WriteStream;
Copy the code

pipe

• readSteam.pipe(writeStream);
• This method is asynchronous, reads and writes point by point, and supports large file operations

const fs = reuqire("fs");

const rs = fs.createReadStream("1.txt", {highWaterMark: 3});
const ws = fs.createWriteStream("2.txt", {highWaterMark: 1});

rs.on("data".function(chunk) {
    console.log(chunk);
    let flag = ws.write(chunk);
    if(! flag) { rs.pause(); } }) ws.on("drain".function() {
    console.log("drain");
    rs.resume();
})

// A simple way to implement the above function
rs.pipe(ws);
// So there is a method called pipe in ReadStream, based on the above code
Copy the code

The list

• Unidirectional list to achieve the increase, delete, change and check function
• Reverse a one-way linked list

class Node{
    constructor(element, next) {
        this.element = element;
        this.next = next; }}class LinkedList {
    constructor() {
        this.head =  null;
        this.size  =  0;  // The total length of the list
    }
    _getNode(index) {
        let head = this.head;
        for(let i = 0; i <= index; i++) {
            head = head.next;
        }
        return head;
    }
    / / to add
    add(index, element) { // Index can be passed without index
        if(arguments.length === 1) {
            element = index;
            index =  this.size;
        }
        if(index == 0) { // Modify the header of the list
            let oldHead = this.head;
            this.head = new Node(element, oldHead);
        }else {
            let preNode = this._getNode(index - 1); // Always find the previous node of the current index
            preNode.next = new Node(element, preNode.next);
        }
        this.size++;
    }
    / / delete
    remove(index) {
        let removeNode;
        if(index == 0) {
            removeNode = this.head;
            if(removeNode == null) return;
            this.head = this.head.next;
        }else{
            let preNode = this._getNode(index - 1);
            if(! preNode)return;
            removeNode = preNode.next;
            preNode.next = preNode.next.next;
        }
        this.size --;
        return removeNode;
    }
    / / change
    update(index, element) {
        let node = this._getNode(index);
        node.element = element;
    }
    / / check
    getNode(index) {
        return this._getNode(index);
    }
    // Reverse the one-way list recursion
    reverse() {
        function n(head) {
            if(head == null || head.next == null) return head;
            let newHead = n(head.next);  // The new head becomes the next
            head.next.next = head.next; // Let the next next point to the old head
            head.next = null; // The next pointer to the old header is null
            return newHead;
        }
        n(this.head);
    }
    // Reverse unidirectional lists to be non-recursive
    reverse(){
        let head = this.head;
        if(head == null || head.next == null) return head;
        let newHead = null;
        while(head) {
            let n = head.next;
            head.next = newHead;
            newHead = head;
            head = n;
        }
        returnnewHead; }}let ll = new LinkedList();
ll.add(0.1);
ll.add(0.2);
ll.add(0.3);
console.dir(ll, {depth:1000});
Copy the code

• Use linked lists to create queues

class Queue{
  constructor() {
    this.ll  = new LinkedList();
  }
  add(element) {
    this.ll.add(element)
  }
  peak() {
    return this.ll.remove(0); }}Copy the code

Binary search tree

• Create a binary search tree

class Node {
  constructor(element, parent) {
    this.element = element;
    this.parent = perent;
    this.left = null;
    this.right = null; }}class BST {
  constructor() {
    this.root  =  null;
  }
  add(element) {
    if(this.root == null) {
      this.root = new Node(element, null);
      return;
    }else{
      let current = this.root;
      let parent = null;
      let compare = true;
      while(current) {
        parent = current;
        compare = current.element - element > 0;
        if(compare) {
          current = current.left;
        }else{ current = current.right; }}let newNode = new Node(element, parent);
      if(compare) {
        parent.left = newNode;
      }else{ parent.right = newNode; }}}}Copy the code

• A more general way to compare
• Tree traversal
  • Sequential traversal – visits the root node first, then the left subtree, then the right subtree
    • Traversal a tree, encountered a node on the processing of a node can be used in advance traversal
  • Middle order traversal — left subtree root node right subtreeThe tree can be processed either in ascending or descending order in a middle-order traversal
    • In order
    • According to the binary search tree property: traversal results are in ascending order
  • Back-order traversal — left subtree right subtree root node
    • You deal with your son before you deal with yourself
  • Sequence traversal
  • Breadth-first traversal
  • Depth-first traversal
• Invert the binary tree
  • The four traversal implementations below can all implement inverted binary trees

class Node {
  constructor(element, parent) {
    this.element = element;
    this.parent = perent;
    this.left = null;
    this.right = null; }}class BST {
  constructor(compare) {
    this.root  =  null;
    let internalCompare = this.compare;
    this.compare = compare || internalCompare;
  }
  compare(current, element) {
    return current - element > 0;
  }
  add(element) {
    if(this.root == null) {
      this.root = new Node(element, null);
      return;
    }else{
      let current = this.root;
      let parent = null;
      let compare = true;
      while(current) {
        parent = current;
        compare = this.compare(current.element, element);
        if(compare) {
          current = current.left;
        }else{ current = current.right; }}let newNode = new Node(element, parent);
      if(compare) {
        parent.left = newNode;
      }else{ parent.right = newNode; }}}// order traversal first
  preorderTraversal(cb){
    const traversal = node= > {
      if(node == null) return;
      cb(node);
      traversal(node.left);
      traversal(node.right);
    }
    traversal(this.root);
  }
  // middle order traversal
  inorderTraversal(cb){
    const traversal = node= > {
      if(node == null) return;
      traversal(node.left);
      cb(node);
      traversal(node.right);
    }
    traversal(this.root);
  }
  // after the sequence traversal
  postorderTraversal(cb){
    const traversal = node= > {
      if(node == null) return;
      traversal(node.left);
      traversal(node.right);
      cb(node);
    }
    traversal(this.root);
  }
  // sequence traversal
  levelorderTraversal(cb) {
    let stack =  [this.root];
    let i  = 0;
    let currentNode;
    while(currentNode = stack[i++]) {
      cb(currentNode);
      if(currentNode.left) {
        stack.push(current.left);
      }
      if(currentNode.right) { stack.push(currentNode.right); }}}// Invert the binary tree
  invertTree(cb) {
    const traversal = (node) = > {
      if(node == null) return;

      let r =  node.right;
      node.right = nodee.left;
      nodee.left = r;

      traversal(node.left);
      traversal(node.right);
      cb(node);
    }
    traversal(this.root)
    return this.root; }}let bst = new BST((a,b) = > {
  return a.age - b.age > 0
});
let arr = [{age: 15}, {age: 18}, {age: 9}, {age: 12}, {age: 23}];
arr.forEach(item= >  {
  bst.add(item);
})

bst.preorderTraversal((node) = > {
  console.log(node.element)
})
Copy the code