Application of HTTP request headers and response headers

This article is about the cute head of the browser… Boys and girls.

Chap1 found headers

When we open a random website (such as Baidu, which we often use to test the Network), open Network, we will see the following request header, response header:

Chap2 headers purposes

2.1 the content-type

Content-type Specifies the Content Type of the request header or response header. It can be used for body-parser when used as a request header. Sooo~ What is body-parser?

Body-parser is a common node middleware. It is used to:

Parse incoming request bodies in a middleware before your handlers, available under the req.body property.

That is, the body of the POST request is parsed by middleware before the data is processed. An example of body-parser is:

The following example shows how to add a Body Parser to a route. In general, this is the most recommended method of using Body-Parser in Express.

var express = require('express')
var bodyParser = require('body-parser')
var app = express()
// create application/json parser
var jsonParser = bodyParser.json()
// create application/x-www-form-urlencoded parser
var urlencodedParser = bodyParser.urlencoded({ extended: false })
// POST /login gets urlencoded bodies
app.post('/login', urlencodedParser, function (req, res) {
  if(! req.body)return res.sendStatus(400)
  res.send('welcome, ' + req.body.username)
})
// POST /api/users gets JSON bodies
app.post('/api/users', jsonParser, function (req, res) {
  if(! req.body)return res.sendStatus(400)
  // create user in req.body
})
Copy the code

The core source of body-parser is:

  // this uses a switch for static require analysis
  switch (parserName) {
    case 'json':
      parser = require('./lib/types/json')
      break
    case 'raw':
      parser = require('./lib/types/raw')
      break
    case 'text':
      parser = require('./lib/types/text')
      break
    case 'urlencoded':
      parser = require('./lib/types/urlencoded')
      break
  }
Copy the code

Take JSON as an example:

var contentType = require('content-type') / /... /** * Get the charset of a request. * * @param {object} req * @api private */function getCharset (req) {
  try {
    return (contentType.parse(req).parameters.charset || ' ').toLowerCase()
  } catch (e) {
    returnundefined } } //... / / assert charset per RFC 7159 SEC 8.1 var charset = getCharset (the req) | |'utf-8'
if(charset.substr(0, 4) ! = ='utf-') {
  debug('invalid charset')
  next(createError(415, 'unsupported charset "' + charset.toUpperCase() + '"', {
    charset: charset,
    type: 'charset.unsupported'
  }))
  return
}
Copy the code

By analyzing the Type of the content-Type in the request header, we can process the corresponding data according to different types.

Step1: create server.js:

req.on('end',function (params) { let r = Buffer.concat(arr).toString(); // body-parser parses requests, If (req.headers['content-type'] === www.js){let queryString = require(' queryString '); r = querystring.parse(r); // a=1&b=2 console.log(r,1); } else if (req.headers['content-type'] === 'application/json'){ console.log(JSON.parse(r),2); } else{ console.log(r,3); } res.end('end'); })Copy the code

Step2: Client simulated request:

let opts = {
  host:'localhost',
  port:3000,
  path:'/hello',
  headers:{
    'a': 1,'Content-Type':'application/json'."Content-Length":7 // Use a length when simulating, otherwise the client will assume that no data is passed}}let http = require('http');
let client = http.request(opts,function (res) {
  res.on('data'.function(data) { console.log(data.toString()); })}); client.end("{\"a\":1}"); // Send out the requestCopy the code

Step3: test. {a: 1} 2. The relationship between Content-Type and body-Parser is analyzed here. Next we look at the request header.

Range: 2.2 bytes

The request header uses Range:bytes to request a portion of a resource. Use this field to simulate a partial read. As follows:

http.createServer(function (req, res) { let range = req.headers['range']; })Copy the code

server:

let http = require('http');
let fs = require('fs');
let path = require('path');
// The current size of the file to download
let size = fs.statSync(path.join(__dirname, 'my.txt')).size;
let server = http.createServer(function (req, res) {
  let range = req.headers['range']; / / 0 to 3
  if (range) {
    Curl -v --header "Range:bytes=0-3" http://localhost:3000
    let [, start, end] = range.match(/(\d*)-(\d*)/);
    start = start ? Number(start) : 0;
    end = end ? Number(end) : size - 1; // 10 bytes size 10 (0-9)
    res.setHeader('Content-Range'.`bytes ${start}-${end}/${size - 1}`);
    fs.createReadStream(path.join(__dirname, 'my.txt'), { start, end }).pipe(res);
  } else {
    // Write the contents of the file to the client
    fs.createReadStream(path.join(__dirname, 'my.txt')).pipe(res);
    // A readable stream can be channeled to a writable stream through pipe}}); server.listen(3000);
Copy the code

client:

let opts = {
  host:'localhost'.port:3000.headers:{}
}
let http = require('http');
let start = 0;
let fs = require('fs');
function download() {
  opts.headers.Range = `bytes=${start}-${start+3}`;
  start+=4;
  console.log(`start is ${start}`)
  let client = http.request(opts,function (res) {
      let total = res.headers['content-range'].split('/') [1];
      // console.log(half)
      res.on('data'.function (data) {
        fs.appendFileSync('./download1.txt',data);
      });
      res.on('end'.function () {
        setTimeout((a)= > {
          if((! pause)&&(start < total)) download(); },1000); })}); client.end(); } download()Copy the code

Piecewise reading adds pause function to listen for user input

let pause = false;
process.stdin.on('data'.function (data) {
  if (data.toString().includes('p')){
    pause = true
  }else{
    pause = false;
    download()
  }
})
Copy the code

Test results:

Segmented reading has the following benefits:

Improve reading speed, multithreading parallel, block reading
Breakpoint continuingly

Simulation parallel download:

let halfFlag = 20
function download() {
  opts.headers.Range = `bytes=${start}-${start+3}`;
  start+=4;
  console.log(`start is ${start}`)
  let client = http.request(opts,function (res) {
      let total = res.headers['content-range'].split('/') [1];
	  let halfFlag = Math.floor(total/2)
      // console.log(half)
      res.on('data'.function (data) {
        fs.appendFileSync('./download1.txt',data);
      });
      res.on('end'.function () {
        setTimeout((a)= > {
          if((! pause)&&(start < halfFlag)) download(); },1000); })}); client.end(); }let half = halfFlag

function downloadTwo() {
	opts.headers.Range = `bytes=${half}-${half+3}`;
	half+=4;
	console.log(`half is ${half}`)
	let client = http.request(opts,function (res) {
		let total = res.headers['content-range'].split('/') [1];
		res.on('data'.function (data) {
			fs.appendFileSync('./download2.txt',data);
		});
		res.on('end'.function () {
			setTimeout((a)= > {
				if(! pause&&half < total) downloadTwo(); },1000); })}); client.end(); } download(); downloadTwo();Copy the code

As a result, the original file is divided into two parts and downloaded to download1.txt and download2.txt. Testing:

2.3 Cache-Control and Expires mandatory caching

Response Header Cache-Control: max-age=1233 in the Response Header can set a mandatory Cache for the current time, and the associated Expires can set an absolute point in time to limit the time the Cache can be read. Simulation implementation:

let url = require('url'); // http://username:password@hostname:port/pathname? querylet server = http.createServer(async function (req,res) {
	console.log(req.url)
  let { pathname,query} = url.parse(req.url,true); 
  // trueIs to convert a Query to an objectlet readPath = path.join(__dirname, 'public', pathname);
  try {
  let statObj = await stat(readPath); // The root client says to cache within 10 seconds res.setheader ('Cache-Control'.'max-age=10');
  res.setHeader('Expires',new Date(Date.now()+10*1000).toGMTString()); // All requests within 10s will return 200 from the cacheif (statObj.isDirectory()) {
      let p = path.join(readPath, 'index.html');
      await stat(p); Fs.createreadstream (p).pipe(res); fs.createreadstream (p).pipe(res); }else {
      fs.createReadStream(readPath).pipe(res);
    }
  }catch(e){
    res.statusCode = 404;
    res.end(`Not found`);
  }
}).listen(3000);
Copy the code

Testing:

2.4 Compare last-modified and if-modified-since caches

Comparing the response header last-modified and with the request header if-modified-since, the file modification time can be used to see If the file has been Modified, thus deciding whether to rerequest or remove the cache. The simulation is as follows: step1 do not set mandatory cache

 res.setHeader('Cache-Control'.'no-cache');
Copy the code

Step2 apply the file modification time to check whether the file is modified.

   res.setHeader('Last-Modified', statObj.ctime.toGMTString());
      if (req.headers['if-modified-since'] === statObj.ctime.toGMTString()) {
        res.statusCode = 304;
        res.end();
        return; / / cache
      }
      fs.createReadStream(readPath).pipe(res);
Copy the code

Testing:

2.5 Compare cache Etag and if-none-match

Compare response header :Etag with request header: if-none-match. If Etag and if-none-match are equal, 304 is returned. How do I add an eTag?

Based on the file contents, generate an MD5 digest and tag the entity.

This method is more performance intensive, but allows a more accurate comparison of whether a file has been modified. Relying on file modification times for comparison is not accurate enough. Sometimes a file is last-modified, but the contents of the file may not have changed at all. So this scenario is superior to 2.4.

Implementation method:

let rs = fs.createReadStream(p);
let md5 = crypto.createHash('md5'); // Can not write the response body before writing the header
let arr = [];
rs.on('data'.function (data) {
    md5.update(data);
    arr.push(data);
});
Copy the code

Set the Etag

rs.on('end'.function () {
let r = md5.digest('base64');
res.setHeader('Etag', r);
if (req.headers['if-none-match'] === r ){
    res.statusCode = 304;
    res.end();
    return;
}
res.end(Buffer.concat(arr));
})
Copy the code

Testing:

2.6 the Accept – Encoding

Depending on the request header: accept-encoding: gzip, deflate, BR tells the server what data format is acceptable. The server returns with the data format marked by the response format via Content-Encoding. When the client accepts the GZIP format, the back end can be passed through file compression processing for improved performance. The Node API provides the zlib module:

The Zlib module provides compression via Gzip and Deflate/Inflate

Let’s apply zlib and request header Accept-encoding to compress.

let zlib = require('zlib');
let fs = require('fs');
let path = require('path');
function gzip(filePath) {
  lettransform = zlib.createGzip(); Fs.createreadstream (filePath).pipe(transform).pipe(fs.createWritestream (filePath+)'.gz'));
}
 gzip('2.txt')
Copy the code

Extract:

function gunzip(filePath) {
  let transform = zlib.createGunzip();
  fs.createReadStream(filePath).pipe(transform).pipe(fs.createWriteStream(path.basename(filePath,'.gz')));
}
Copy the code

Path. basename(filePath,’.gz’) is used to remove the suffix. Gz from the filePath file name.

Depending on the type accepted by the request header:

if(req.url === '/download'){
    res.setHeader('Content-Disposition'.'attachment' )
    return fs.createReadStream(path.join(__dirname, '1.html')).pipe(res);
  }
Copy the code

let http = require('http');
let fs = require('fs');
let path = require('path');
let zlib = require('zlib');
http.createServer(function (req,res) {
  if(req.url === '/download'){
    res.setHeader('Content-Disposition'.'attachment' )
    return fs.createReadStream(path.join(__dirname, '1.html')).pipe(res);
  }
  let rule = req.headers['accept-encoding'];
  if(rule){
    if(rule.match(/\bgzip\b/)){
      res.setHeader('Content-Encoding'.'gzip');
      fs.createReadStream(path.join(__dirname, '1.html'))
      .pipe(zlib.createGzip())
      .pipe(res);
    } else if (rule.match(/\bdeflate\b/)){
      res.setHeader('Content-Encoding'.'deflate');
      fs.createReadStream(path.join(__dirname, '1.html'))
        .pipe(zlib.createDeflate())
        .pipe(res);
    }else{
      fs.createReadStream(path.join(__dirname, '1.html')).pipe(res); }}else{
    fs.createReadStream(path.join(__dirname, '1.html')).pipe(res);
  }
}).listen(3000);
Copy the code

test deflate:

curl -v --header "Accept-Encoding:deflate" http://localhost:3000
* Rebuilt URL to: http://localhost:3000/
*   Trying 127.0.0.1. * TCP_NODELAY set * Connected to localhost (127.0.0.1) port 3000 (# 0)
> GET / HTTP/1.1
> Host: localhost:3000
> User-Agent: curl/7.54.0
> Accept: */*
> Accept-Encoding:deflate
> 
< HTTP/1.1 200 OK
< Content-Encoding: deflate
< Date: Thu, 23 Aug 2018 03:01:13 GMT
< Connection: keep-alive
< Transfer-Encoding: chunked
Copy the code

test others:

curl -v --header "Accept-Encoding:nn" http://localhost:3000 * Rebuilt URL to: http://localhost:3000/ * Trying 127.0.0.1... * TCP_NODELAY set * Connected to localhost (127.0.0.1) port 3000 (#0) > GET/HTTP/1.1 > Host: Localhost :3000 > user-agent: curl/7.54.0 > Accept: */* > accept-encoding :nn > < HTTP/1.1 200 OK < Date: Thu, 23 Aug 2018 03:02:51 GMT < Connection: keep-alive < Transfer-Encoding: chunked < <! DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <meta name="viewport" content="width=device-width, <meta HTTP-equiv =" x-UA-compatible "content=" IE =edge"> <title>Document</title> </head> <body </body> * Connection #0 to host localhost left intact </html>%Copy the code

2.7 referer

let http =  require('http');
let fs = require('fs');
let url = require('url');
let path = require('path'); // This is baidu serverlet server = http.createServer(function (req,res) {
  let { pathname } = url.parse(req.url);
  let realPath = path.join(__dirname,pathname);
  fs.stat(realPath,function(err,statObj) {
    if(err){
      res.statusCode = 404;
      res.end();
    }else{
      let referer = req.headers['referer'] || req.headers['referred'];
      if(referer){
        let current = req.headers['host'] // Referer = url.parse(referer).hostif (current === referer){
          fs.createReadStream(realPath).pipe(res);
        }else{
          fs.createReadStream(path.join(__dirname,'images/2.jpg')).pipe(res); }}else{
        fs.createReadStream(realPath).pipe(res);
      }
    }
  })
}).listen(3000);
Copy the code

2.8 the Accept – Language

Accept-language: zh-cn,zh; Q =0.9 Multiple languages are separated by ‘,’, weight reuse ‘=’ for ‘, no default weight is 1

The backend accepts a lookup based on the language weight of the request, returns if it finds one, and uses the default language if it cannot find one

let langs = {
  en:  'hello world'.'zh-CN':Hello world.zh:'hello'.ja: 'Kosher view of the world'
}
let defualtLanguage = 'en'
// Multilingual server solution: do it (the browser will send a header)
// Implement multiple languages through URLS
let http = require('http');
http.createServer(function (req,res) {
    let lan = req.headers['accept-language'];
    / / [[useful, q = 0.9], [useful - CN]] = > [{name: 'useful - CN, q = 1}, {name:' useful ', q: 0.9}]
    if(lan){
      lan = lan.split(', ');
      lan = lan.map(l= >{
        let [name,q] = l.split('; ');
        q = q?Number(q.split('=') [1) :1 
        return {name,q}
      }).sort((a,b) = >b.q-a.q); // Outputs the weight array

      for(let i = 0; i <lan.length; i++){// Take out everyone's name
        let name= lan[i].name;
        if(langs[name]){ // Go to the language package to find it
          res.end(langs[name]);
          return;
        }
      }
      res.end(langs[defualtLanguage]); // Default language
    }else{
      res.end(langs[defualtLanguage]); // Default language
    }
}).listen(3000);
Copy the code

Testing:

conclusion

Request headers and response headers are often used in front – end interfacing. Understand their clever use before and after the end of the coordination will be more harmonious smooth ~

Author: Yanni Jia Nickname: Very rabbit Email: [email protected]

Application of HTTP request headers and response headers

Chap1 found headers

Chap2 headers purposes

2.1 the content-type

Range: 2.2 bytes

2.3 Cache-Control and Expires mandatory caching

2.4 Compare last-modified and if-modified-since caches

2.5 Compare cache Etag and if-none-match

2.6 the Accept – Encoding

2.7 referer

2.8 the Accept – Language

conclusion

Related Posts

Change the Node version and update YARN

Implementation principle of Redux Middleware

HTML display time, and refresh in real time (detailed code)