Static file server implementation

Nodejs can be used not only to write server-side interfaces, but also to replace Nginx with static file servers. Without further ado, first the code:

var server=http.createServer(function (req,res){
    fs.createReadStream(Path.resolve(__dirname,"."+req.url)).pipe(res);
})
Copy the code

Create a hello. HTML file in the project root directory to test the hello. HTML file:

<h1>hello,world</h1>
Copy the code

Node app. Js running, open a browser visit: http://localhost/hello.html

Fs.createreadstream reads the local file to create a readable Stream (an instance of the ReadStream class), and pipes it into the RES response Stream. Res is an instance of the http.ServerResponse class and is a writable Stream that inherits from the Stream class

The http.ServerResponse class inherits as follows:

Safety considerations

After the above code implements static file server, means the project all the files in the root directory (recursive) can be directly through the browser to access and download, it will bring some security problems, think about it, your server side code and configuration files can be directly downloaded from the browser, so you need to add some limitations in the code, For example, only files in a specific directory and files with a specific extension can be accessed. This is not enough. Refer to OWasp Top 10 Security Risks (Article 4 – Direct references to Insecure objects). /.. / directory backtrace method to access other directories, for access paths containing.. Filter out all of them.

Realize the mime type

Mime Type refers to the Content-Type field in the HTTP response header. It determines how the browser parses the file, whether it is displayed as plain (text/plain), rendered as HTML (text/ HTML), or downloaded as a binary file. If the correct mine type is not output, the picture file cannot be displayed, the font file is invalid, and the video file cannot be played. It is also very simple to implement, just need to make a mapping table, different file extensions, output the corresponding mine type in the content-Type field of the response header.

The complete code is as follows:

const http=require("http");
const Path=require("path");
const fs=require("fs");

var server=http.createServer(function (req,res){
    const fileName=Path.resolve(__dirname,"."+req.url);
    const extName=Path.extname(fileName).substr(1);

    if (fs.existsSync(fileName)) { // Check whether the local file exists
        var mineTypeMap={
            html:'text/html; charset=utf-8'.htm:'text/html; charset=utf-8'.xml:"text/xml; charset=utf-8".png:"image/png".jpg:"image/jpeg".jpeg:"image/jpeg".gif:"image/gif".css:"text/css; charset=utf-8".txt:"text/plain; charset=utf-8".mp3:"audio/mpeg".mp4:"video/mp4".ico:"image/x-icon".tif:"image/tiff".svg:"image/svg+xml".zip:"application/zip".ttf:"font/ttf".woff:"font/woff".woff2:"font/woff2",}if (mineTypeMap[extName]) {
            res.setHeader('Content-Type', mineTypeMap[extName]);
        }
        var stream=fs.createReadStream(fileName);
        stream.pipe(res);
    }

    
})
server.listen(80);
Copy the code

Realize the gzip

For text files, such as HTML, JS, and CSS, gzip compression can greatly reduce the amount of transmission and improve the transmission performance of the server. Of course, it will cost the CPU performance of the server. If the client browser supports Gzip compression, PIP is a new layer in the stream. The pilot stream is exported to the gzip stream, and then exported to the RES stream. Also add content-Encoding to gzip in the response header so that the browser correctly recognizes that the HTTP body is gZIP compressed and automatically decompresses it.

The code is as follows:

const zlib = require('zlib');

if (req.headers["accept-encoding"].indexOf("gzip") > =0 && (extName=="js" || extName=="css" || extName=="html"))) {
     res.setHeader('Content-Encoding'."gzip");
     const gzip = zlib.createGzip();
     stream.pipe(gzip).pipe(res);
 }
Copy the code

Client cache

The HTTP cache negotiation process takes a long time. In the end, the response header generates two parameters to control the cache expiration time: expire (absolute time) and cache-control (relative time). The next time the browser requests this file, the parameters are classified as follows:

  1. If the expiration date is not reached, the browser does not request that the file be read directly from the cache
  2. If the expiration date is reached, the last-Modified field in the request header carries the last modified date of the file, and if the comparison timestamp matches the server file, HTTP returns 304: Not Modified
  3. If f5 refresh is pressed, the expiration time of the cache is carried in the if-Modified-since field in the request header, and HTTP returns 304: Not Modified if the comparison timestamp is consistent with the server file
  4. CTRL + F5 refresh, the request header contains cache-control: no-cache, forcibly disabling cache. Redownload the file

There are more logical branches, but all are date comparison, it is easier to write out the process of cache negotiation, interested students can achieve their own

High performance static file server optimization

If you want to do a high performance static file server only realize gzip is not enough, and the cache consultation involves local file read frequently, high concurrent I/O bound to become a bottleneck, considering that the file on the server is rarely updated, you can use the file stream Buffer cache into memory, each request to find a match in the memory, If a hit is returned directly from memory, avoiding disk reading, gzip does not need to be compressed, and returns directly with the compressed file stream, which can improve performance exponentially. Of course, if there are too many files, the memory will soar, so we need to consider the elimination algorithm, cache only the files with high access times, and eliminate the files with low access.

Fs. watch is used to monitor the changes of directory files and delete the cache if the files are updated.

summary

The built-in pipe method of Node.js can easily output the local file of the server to the HTTP response flow. Gzip compression can also be achieved through PIPE, and the static server with the output of mine type can already meet the use of general business. If you want to achieve high performance static file server, also need to implement client side cache, server side cache functions (this article provides ideas, follow the blueprint is not difficult).

Finally, I recommend my personal open source project, Node.js Web development framework, which includes the static file server function of this paper. Webcontext: github.com/windyfancy/…