How does TCP implement HTTP

Goal:

1. Understand the CONTENT of HTTP
2. Understand the basic use of TCP in Node
3. Net to achieve an HTTP protocol

HTTP usually runs on the TCP/IP stack, so HTTP must be implemented based on TCP. How does it work? Let’s take a look.

Start a Node HTTP server.

HTTP/TCP implementation not

const http = require('http');
const fs = require('fs');
const path = require('path');
const server = http.createServer(function (req, res) {
  if (['/get.html'].includes(req.url)) {
    res.writeHead(200, {
      'Context-type': "text-html"
    });
    res.end(fs.readFileSync(path.join(__dirname, 'static', req.url.slice(1))));
  } else if (req.url === '/get') {
    res.writeHead(200, {
      'Context-type': "text-plain"
    });
    res.end('get'); }}); server.listen(8081);
Copy the code

TCP. Port == 8081; The following two graphs are obtained:

Diagram of request messages:

Graph of response message:

Browser diagram:

Conclusion:

Component of a request message

GET /get HTTP/1.1
Host: localhost:8081
Connection: keep-alive
sec-ch-ua: "Google Chrome"; v="89"."Chromium"; v="89"."; Not A Brand"; v="99"
age: 10
name: zhufeng
sec-ch-ua-mobile: ?0
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10 _15_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.04389.128. Safari/537.36
Accept: */* Sec-Fetch-Site: same-origin Sec-Fetch-Mode: cors Sec-Fetch-Dest: empty Referer: http://localhost:8081/get.html Accept-Encoding: gzip, deflate, br Accept-Language: zh-CN,zh; Q = 0.9, en. Q = 0.8Copy the code

Components of a request message:

1. The request line consists of three parts:
   1. Request method: is a verb, such as GET/POST, indicating an operation on a resource;
   2. Request target: Usually a URI that marks the resource on which the request method will operate;
   3. Version: Indicates the HTTP version used by the packet.

These three sections are usually separated by a space and terminated with a CRLF newline.

Method SP URL SP Version CRLF

2. Header field set: describes the packet in more detail in key-value format.

Context-type: text-plain

Date: Tue, 20 Apr 2021 12:20:53 GMT

Field Name : Field Value CRLF

3. Response body

To draw a picture:

With an overview of the protocols in HTTP, let’s take a look at TCP in Node.

Net module of Node

Net module in Node is implemented based on TCP, so logically speaking, we can implement our HTTP server based on NET module.

Running the TCP module

const net = require('net');
const fs = require('fs')
const server = net.createServer((socket) = > {
  socket.on('data'.(data) = > {
    socket.write(134)
    socket.end();
  });
})
server.on('error'.(err) = > {
  console.error(err);
});

server.listen(5555.() = > {
  console.log('Server started', server.address());
});
Copy the code

Browser to access http://localhost:5555/get will find no content page. This is because although we started the TCP server, we did not parse and send according to the HTTP protocol, so the browser did not display properly.

Parsing the HTTP Protocol

HTTP/TCP implementation demo2

const net = require('net');
const fs = require('fs')
const server = net.createServer((socket) = > {
  socket.on('data'.(data) = > {
    let request = data.toString();
    // Parse the request line
    let [requestLine, ...headerRows] = request.split('\r\n');
    // Get the request method, request path
    let [method, path] = requestLine.split(' ');
    // Parse the header field set, get the header;
    let headers = headerRows.slice(0, -2).reduce((memo, row) = > {
      let [key, value] = row.split(':');
      memo[key] = value;
      return memo;
    }, {});
    console.log('method', method);
    console.log('path', path);
    console.log('headers', headers);
    var obj = {
      method,
      path,
      headers
    }
    let rows = [] 
    if(path==='/get') {// Return as a response message;
         rows.push(OK ` ` HTTP / 1.1 200);
         rows.push(`Context-type: text-plain`);
         rows.push(`Date: The ${new Date().toGMTString()}`);
         rows.push(`Connection: keep-alive`);
         rows.push(`Transfer-Encoding: chunked`);
          let responseBody = `{"name":"124"}`;
          rows.push(`\r\n${Buffer.byteLength(responseBody).toString(16)}\r\n${responseBody}\r\n0\r\n\r\n`);
    }
 
    let response = rows.join('\r\n');
    socket.write(Buffer.from(response))
    socket.end();
  });
})
server.on('error'.(err) = > {
  console.error(err);
});

server.listen(5555.() = > {
  console.log('Server started', server.address());
});
Copy the code

Through the browser to http://localhost:5555/get; The browser returns {“name”:”124″}, which means we parsed the message successfully.

Parsing the POST Protocol

Since the parameters passed in the POST request are placed in the request body, parsing HTTP is a lot more difficult, so we use a “state machine” approach to parsing strings.

HTTP/TCP implementation demo3State machine logic analysis:By analyzing characters one by one, enter different analysis according to the state of characters, and parse out the information you want;

Parser. Js state machine code:

let LF = 10.// Line feed
  CR = 13.// carriage return
  SPACE = 32./ / space
  COLON = 58; / / the colon
let PARSER_UNINITIALIZED = 0./ / not parsing
  START = 1.// Start parsing
  REQUEST_LINE = 2.// Parse the request row
  HEADER_FIELD_START = 3.// Parse the request header
  HEADER_FIELD = 4,
  HEADER_VALUE_START = 5,
  HEADER_VALUE = 6,
  READING_BODY = 7;
class Parser {
  constructor() {
    this.state = PARSER_UNINITIALIZED;
  }
  parse(buffer) {
    let self = this,
      requestLine = "",
      headers = {},
      body = "",
      i = 0,
      char,
      state = START, // Start parsing
      headerField = "",
      headerValue = "";
    // console.log(buffer.toString());
    for (i = 0; i < buffer.length; i++) {
      char = buffer[i];
      switch (state) {
        case START:
          state = REQUEST_LINE; // Parse the request row
          self["requestLineMark"] = i; // Request line opening letter mark
        case REQUEST_LINE:
          if (char == CR) {
            / / a newline
            //POST /POST HTTP/1.1 0 to 19 Obtain the request line
            requestLine = buffer.toString("utf8", self["requestLineMark"], i);
            break;
          } else if (char == LF) {
            / / return
            state = HEADER_FIELD_START; // Parse the request header
          }
          break;
        case HEADER_FIELD_START:
          if (char === CR) {
            state = READING_BODY; // All that remains is the body data
            self["bodyMark"] = i + 2;
            break;
          } else {
            state = HEADER_FIELD;
            self["headerFieldMark"] = i;
          }
        case HEADER_FIELD:
          if (char == COLON) { // Check the colon to see if the key has been obtained
            headerField = buffer.toString("utf8", self["headerFieldMark"], i);
            state = HEADER_VALUE_START;
          }
          break;
        case HEADER_VALUE_START:
          if (char == SPACE) {
            break;
          }
          self["headerValueMark"] = i;
          state = HEADER_VALUE;
        case HEADER_VALUE:
          if (char === CR) { // If it is a newline character, value has been obtained
            headerValue = buffer.toString("utf8", self["headerValueMark"], i);
            headers[headerField] = headerValue;
            headerField = "";
            headerValue = "";
          } else if (char === LF) {
            state = HEADER_FIELD_START;
          }
          break;
        default:
          break; }}let [method, url] = requestLine.split("");
    // console.log(self["bodyMark"], i, JSON.stringify(buffer.toString("utf8", self["bodyMark"] - 2, i)))
    body = buffer.toString("utf8", self["bodyMark"], i);
    return{ method, url, headers, body, }; }}module.exports = Parser;

Copy the code

Use:

const net = require('net');
const Parer = require('./Parser');
const fs = require('fs')
const server = net.createServer((socket) = > {
  socket.on('data'.(data) = > {
    let parser = new Parer(); 
    let { method, url,headers,body } = parser.parse(data); 
    if (url==='/post') {let rows = [];
         rows.push(OK ` ` HTTP / 1.1 200);
         rows.push(`Context-type: text-plain`);
         rows.push(`Date: The ${new Date().toGMTString()}`);
         rows.push(`Connection: keep-alive`);
         rows.push(`Transfer-Encoding: chunked`);
        // console.log(body)
         rows.push(`\r\n${Buffer.byteLength(body).toString(16)}\r\n${body}\r\n0\r\n\r\n`);
         let response = rows.join('\r\n');
        // console.log(response)socket.end(response); }}); }) server.on('error'.(err) = > {
  console.error(err);
});

server.listen(8082.() = > {
  console.log('Server started', server.address());
});
Copy the code

The appeal code gets the browser’s data and returns it to the browser;

The above code is our own HTTP server based on TCP implementation;

Consider: How to implement upload:

If we upload information, how do we use TCP to parse it in the background?

References and codes: Codes