Read more articles in the series at myMaking a blog, please visit the sample codeHere,.

The File System is faulty

We usually use the File System module to read the File, as follows:

fs.readFile('./test.txt', (error, buffer) => {
  if (error) {
    console.error(error)
  } else{// File read successfully res.write(buffer)}})Copy the code

This is simple and effective, but there are a few problems:

  1. Memory usage Fs reads a file, which reads all the contents of the file into memory at one time and sends it to the client at one time, so it consumes a lot of memory.
  2. Low resource usage During file reading from the disk, the disk is busy and the network is idle. After the disk is read, when the file is sent, the situation is reversed. The network is busy while the disk is idle.

The Stream flow

Compared with File System, a Stream Stream reads a File one time and sends a File another time. The writing operation of Stream Stream also has the same characteristics, so it can solve the two problems mentioned above in File System.

Next, implement a simple stream that writes the contents of the 1.txt file to 2.txt:

Example code: /lesson24/stream.js

const fs = require('fs') // Create a readable stream. constreadStream = fs.createReadStream('./1.txt') // Create a writable stream. const writeStream = fs.createWriteStream('./2.txt'// Write the contents of 1.txt to 2.txt by pushing the data read from the readable stream through the pipe.readStream.pipe(writeStream) // Raises an error event when a read error occurs.readStream.on('error', (error) => {console.error(error)}) // When the write is complete, the Finish event is emitted. writeStream.on('finish', () => {
  console.log('finish')})Copy the code

Use Zlib to compress the file

You can use the Zlib module, in conjunction with the Stream, to achieve file compression, as follows:

Example code: /lesson24/gzip.js

const fs = require('fs'Const zlib = require() const zlib = require() const zlib = require()'zlib') // Create a readable stream. constreadStream = fs.createReadStream('./google.jpg') // Create a writable stream. const writeStream = fs.createWriteStream('./google.jpg.gz'Const Gzip = zlib.createGzip() // Create a Gzip object that compresses the file to a.gz file const Gzip = zlib.createGzip() // Push the data that can be read from the stream to Gzip first through the pipe and then to the write stream. // Compress the data from the readable stream and push it to the writable stream.readStream.pipe(gzip).pipe(writeStream) // An error event is raised when a read error occurs.readStream.on('error', (error) => {console.error(error)}) // When the write is complete, the Finish event is emitted. writeStream.on('finish', () => {
  console.log('finish')})Copy the code

Use streaming to transfer files to the foreground

By learning to stream, we can transfer files to the foreground more efficiently:

Example code: /lesson24/server.js

const http = require('http')
const zlib = require('zlib')
const url = require('url')
const fs = require('fs')

const server = http.createServer((req, res) => {
  const {
    pathname
  } = url.parse(req.url, true) // Create a readable stream. constreadStream = fs.createReadStream(`./${pathname}Const Gzip = zlib.createGzip() // Pushes the read content through the pipe to res without compressionreadStream.pipe(res) // Handles readable Stream errors, preventing requests for files that do not existreadStream.on('error', (error) => {
    console.error(error);
    res.writeHead(404)
    res.write('Not Found')
    res.end()
  })
})

server.listen(8080)
Copy the code

But as you can see, in this example, although the implementation uses streaming the file, it does not use GZIP compression, and still consumes more network resources when transferring the file. Gzip compression can be introduced next.

The file is compressed by GZIP and transferred to the foreground

However, if you simply use readstream.pipe (gzip).pipe(res) to transfer the file, the browser will not be able to open the file directly, but will trigger the file download.

This is because the request header attribute Content-Encoding is not set, so the gzip-compressed file is not recognized by the browser. This requires modifying the request header Res.setheader (‘ Content-encoding ‘, ‘gzip’) to make it recognizable to the browser.

This allows the browser to open the file, but if the browser accesses a file that does Not exist, the browser will say “This site cannot be accessed” because the request header attribute content-Encoding has been set to Gzip, but the server sent the browser a Not Found string, which the browser cannot recognize.

In this case, you can use the fs.stat method to check whether the file exists. If the file does Not exist, the system returns Not Found. If the file does exist, the transmission continues.

Example code: /lesson24/server_gzip.js

const http = require('http')
const zlib = require('zlib')
const url = require('url')
const fs = require('fs')

const server = http.createServer((req, res) => {
  const {
    pathname
  } = url.parse(req.url, true// Relative path of the file const filepath = './${pathname}Fs. stat(filepath, (error,stat) = > {if (error) {
      console.error(error);
      res.setHeader('content-encoding'.'identity')
      res.writeHead(404)
      res.write('Not Found')
      res.end()
    } else{// Create a readable stream. constreadStream = fs.createreadStream (filepath) // Create a Gzip object to compress the file into a.gz file const Gzip = zlib.createGzip() // send the Gzip compressed file to the browser, Set the response header. Otherwise, the browser cannot identify the response header and automatically downloads it. res.setHeader('content-encoding'.'gzip') // The content to be read is compressed by gZIP and pushed to res through a pipeline. Res inherits from Stream and can receive pushes from pipes as well.readStream.pipe(gzip).pipe(res) // Handles readable Stream errors to prevent files from being deleted or errors resulting in errors.readStream.on('error', (error) => {
        console.error(error);
        res.setHeader('content-encoding'.'identity')
        res.writeHead(404)
        res.write('Not Found')
        res.end()
      })
    }
  })
})

server.listen(8080)
Copy the code