Introduction to the HTTP Method

To implement an HTTP server (or client) via Netty, you first need to understand the HTTP protocol.

HTTP is used as a request-response protocol in the client-server computing model.

For example, a Web browser can be a client, and an application running on a computer hosting a Web site can be a server. The client submits an HTTP request message to the server.

The server provides resources such as HTML files and other content, or performs other functions on behalf of the client, returning a response message to the client. The response contains completion status information about the request and can also include the requested content in its message body.

What are HTTP methods?

Those of you who have written web forms are familiar with GET and POST, but do you know what GET and POST are? ? Today’s web design tools are so advanced that they don’t even need to touch THE HTML syntax to complete a large website, but more and more people have forgotten the underlying implementation principles of HTTP, resulting in the failure to properly debug in the case of errors.

In the early days of HTML form syntax, the following was written, but most software engineers use POST for form transfer.

<form action="" method="POST/GET">
</form>
Copy the code

However, to obtain Form variables in our web application, we only need to call methods already wrapped in the system, such as PHP using $_REQUEST, JAVA using getParameter(), ASP using request.form () and so on. From the above approach, it seems that using POST or GET is not important. Many Web engineers remember the use of the method form as “POST sends more data “, “POST is used when forms send files “, “POST is safer than GET “, and other strange concepts.

In HTTP 1.1, there are eight methods defined, as follows:

  • OPTIONS

  • GET

  • HEAD

  • POST

  • PUT

  • DELETE

  • TRACE

  • CONNECT

Day ah! These methods seem strange. The form we used used only two of these methods (GET/POST). The other methods are rarely used, but in a RESTful design architecture more methods are used to simplify the design.

GET and POST methods

For example, if HTTP stands for how we send letters in real life today.

Speaker: So the envelope format is HTTP. Let’s call the outside of the envelope http-header and the inside of the envelope message-body, so HTTP Method is the rule you tell the mailman to send a letter.

If GET means no letter can be sent inside the envelope, like a postcard, you can write the information you want to send on the envelope (HTTP-header) until it is full, which is cheaper. However, POST is a way to send a letter inside an envelope (the envelope has content). Not only can the envelope write something, but also the message-body can put information or files you want to send, which is relatively expensive.

With GET, the data we are sending is appended to the URL we are sending as a Query String (a Key/Vaule encoding) and sent to the postman for delivery.

In the case of POST, the mailing address (URL) is written on an envelope, and the information to be sent is written on another piece of paper, which is then placed inside the envelope and handed to the postman for transmission.

The GET method

Then LET me tell you how it actually works:

Let’s take a look at how GET transmits data. When we send a GET form, here’s an example:

<form method="get" action="">
<input type="text" name="id" />
<input type="submit" />
</form>
Copy the code

When the form is submitted and the browser URL becomes “xxx.toright.com/?id=010101”, the browser automatically converts the form content into a Query String and adds it to the URL.

Take a look at the contents of the HTTP Request packet:

GET /? HTTP/1.1 Host: XXX.toright.com User-agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-TW; Rv :1.9.2.13) Gecko/20101203 Firefox/3.6.13 GTB7.1 (.net CLR 3.5.30729) Accept: text/html,application/xhtml+xml,application/xml; Q = 0.9 * / *; Q = 0.8 Accept - Language: useful - tw, en - us; Q = 0.7, en. Q =0.3 Accept-encoding: Gzip,deflate Accept-Charset: UTF-8,* keep-alive: 115 Connection: keep-aliveCopy the code

In HTTP GET Method, message-body is not allowed.

From the browser’s url column, you can see the information our form is sending. If you want to send a password, wouldn’t it be “in full view “……. That’s why people talk about security.

POST method

Let’s look at POST data

<form method="post" action="">
<input type="text" name="id" />
<input type="submit" />
</form>
Copy the code

The url column has not changed, so let’s look at the contents of the HTTP Request package:

User Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-TW; Rv :1.9.2.13) Gecko/20101203 Firefox/3.6.13 GTB7.1 (.net CLR 3.5.30729) Accept: text/html,application/xhtml+xml,application/xml; Q = 0.9 * / *; Q = 0.8 Accept - Language: useful - tw, en - us; Q = 0.7, en. Q =0.3 Accept-encoding: gzip,deflate Accept-Charset: UTF-8,* keep-alive: 115 Connection: keep-alive Content-Type: application/x-www-form-urlencoded </code><code>Content-Length: 9 id=020202Copy the code

See why? The original POST is to send the form data to message-body, which seems safer without peeking at the packet……. : yellow_heart:. In addition, multi-part coding is used when sending files, which are sent in message-body along with other form fields. That’s the difference between a GET and a POST form.

Netty HTTP codec

To process HTTP requests using Netty, codecs are required first.

NettyHTTP codec

public class HttpHelloWorldServerInitializer extends ChannelInitializer<SocketChannel> { @Override public void initChannel(SocketChannel ch) { ChannelPipeline p = ch.pipeline(); /** * or use HttpRequestDecoder & HttpResponseEncoder */ p.ddlast (new HttpServerCodec()); */ p.addlast (new HttpObjectAggregator(1024*1024)); p.addLast(new HttpServerExpectContinueHandler()); p.addLast(new HttpHelloWorldServerHandler()); }}Copy the code
  • Line 8: Call the **#new HttpServerCodec()** method. The codec supports partial RESOLUTION of HTTP requests, such as HTTP GET requests that pass parameters contained in urIs, so the request parameters can be resolved via HttpRequest.
    • HttpRequestDecoder decodes ByteBuf to HttpRequest and HttpContent.
    • HttpResponseEncoder is an HttpResponse or HttpContent encoded into ByteBuf.
    • HttpServerCodec is a combination of HttpRequestDecoder and HttpResponseEncoder.

However, for HTTP POST requests, the parameter information is placed in the Message Body (HttpMessage for Netty), so the above codec does not fully parse HTTP POST requests.

What to do about this situation? Don’t panic, Netty provides a handler to handle this.

  • Line 12: Call **#new HttpObjectAggregator(1024*1024)** to aggregate HttpMessage and HttpContent into FullHttpRequest or FullHttpResponse (depending on whether you’re processing a request or a response), and it also helps you ignore whether it’s a “block” transfer when decoding.

    Therefore, be sure to include HttpObjectAggregator in the ChannelPipeline when parsing HTTP POST requests. (Please refer to the code for details)

  • Line 13: The HTTP 100-continue method is used by the client to check whether the server is processing the POST data before sending it to the server. If the server is not processing the POST data, the client will not upload the POST data. If the server is processing the POST data, the client will upload the POST data. In real-world applications, the 100-continue protocol is used only when Posting big data

Implementation of HTTP response messages

The process of encapsulating Java objects into binary packets based on HTTP is called encoding, and the process of parsing Java objects out of binary packets is called decoding. Before learning how to use Netty for HTTP codec, Let’s start by defining the Java objects that the client communicates with the server.

Java object

We define Java objects during communication as follows

@Data
public class User {
    private String userName;

    private String method;

    private Date date;
}
Copy the code
  1. The above is an abstract class of Java objects during communication. As you can see, we define a user name (the default is Sanshengshui) along with an HTTP request method and the current date and time.
  2. The @data annotation is provided by Lombok and automatically generates getter/setter methods, reducing the amount of repetitive code. It is recommended

With Java objects defined, we then need to define a rule for converting a Java object into binary data. This rule is called Serialization of Java objects.

serialization

We define the serialization interface as follows

/** * serialize */ public Interface Serializer {/** * serialize(Object Object); <T> T deserialize(Class<T> clazz, byte[] bytes); /** * deserialize(Class<T> clazz, byte[] bytes); }Copy the code

The serialization interface has two methods, serialize() converts a Java object into a byte array, and deserialize() converts a byte array into a Java object of some type. In the project, we use the simplest json serialization method, Alibaba fastjson is used as serialization framework.

public class JSONSerializer implements Serializer {
    @Override
    public byte[] serialize(Object object) {
        return JSON.toJSONBytes(object);
    }

    @Override
    public <T> T deserialize(Class<T> clazz, byte[] bytes) {
        returnJSON.parseObject(bytes,clazz); }}Copy the code

coding

		User user = new User();
        user.setUserName("sanshengshui");
        user.setDate(new Date());
        user.setMethod("get"); JSONSerializer jsonSerializer = new JSONSerializer(); Byte [] content = jsonSerializer.serialize(user); byte[] content = jsonSerializer.serialize(user); FullHttpResponse response = new DefaultFullHttpResponse(HTTP_1_1, OK, Unpooled.wrappedBuffer(content)); response.headers().set(CONTENT_TYPE,"text/plain");
        response.headers().setInt(CONTENT_LENGTH, response.content().readableBytes());

        boolean keepAlive = HttpUtil.isKeepAlive(request);
        if(! keepAlive) { ctx.write(response).addListener(ChannelFutureListener.CLOSE); }else {
            response.headers().set(CONNECTION, KEEP_ALIVE);
            ctx.write(response);
           }
Copy the code

HTTP GET parsing practices

As mentioned above, the parameters of the HTTP GET request are contained in the URI, which can be resolved as follows:

HttpRequest request = (HttpRequest) msg;
String uri = request.uri();
Copy the code

In particular, HTTP requests from browsers are often interfered with by uri = “/favicon.ico”, so it is best to treat this special:

if(uri.equals(FAVICON_ICO)){
    return;
}
Copy the code

The next step is to parse the URI. Here we need QueryStringDecoder:

Splits an HTTP query string into a path string and key-value parameter pairs.
This decoder is for one time use only.  Create a new instance for each URI:
 
QueryStringDecoder decoder = new QueryStringDecoder("/hello? recipient=world&x=1; y=2");
assert decoder.getPath().equals("/hello");
assert decoder.getParameters().get("recipient").get(0).equals("world");
assert decoder.getParameters().get("x").get(0).equals("1");
assert decoder.getParameters().get("y").get(0).equals("2");
 
This decoder can also decode the content of an HTTP POST request whose
content type is application/x-www-form-urlencoded:
 
QueryStringDecoder decoder = new QueryStringDecoder("recipient=world&x=1; y=2".false);
Copy the code

As you can see from the above description, QueryStringDecoder splits HTTP URIs into path and key-value pairs, Can also be used to decode HTTP posts with Content-Type = “application/ X-www-form-urlencoded “. It is important to note that the decoder can only be used once.

The parse code is as follows:

String uri = request.uri();
HttpMethod method = request.method();
if(method.equals(HttpMethod.GET)){ &emsp; &emsp; QueryStringDecoder queryDecoder = new QueryStringDecoder(uri, Charsets.toCharset(CharEncoding.UTF_8)); &emsp; &emsp; Map<String, List<String>> uriAttributes = queryDecoder.parameters(); &emsp; &emsp; // Only request parameters are printed here (you can customize according to your business needs) &emsp; &emsp;for(Map.Entry<String, List<String>> attr : uriAttributes.entrySet()) { &emsp; &emsp; &emsp; &emsp;for(String attrVal : attr.getValue()) { &emsp; &emsp; &emsp; &emsp; &emsp; &emsp; System.out.println(attr.getKey() +"="+ attrVal); &emsp; &emsp; &emsp; &emsp; } &emsp; &emsp; }}Copy the code

HTTP POST parsing practices

As mentioned earlier, always use HttpObjectAggregator to parse the Message body of an HTTP POST request. But is it necessary to convert MSG to FullHttpRequest? The answer is no, and read on.

Let’s first explain what FullHttpRequest is:

Combinate the HttpRequest and FullHttpMessage, so the request is a complete HTTP request.
Copy the code

FullHttpRequest contains both HttpRequest and FullHttpMessage, and is the completion of an HTTP request.

The method of converting MSG to FullHttpRequest is simple:

FullHttpRequest fullRequest = (FullHttpRequest) msg;
Copy the code

The next step is to parse the content-Types.

private void dealWithContentType() throws Exception{ String contentType = getContentType(); // HttpJsonDecoder can be usedif(contentType.equals("application/json")){
            String jsonStr = fullRequest.content().toString(Charsets.toCharset(CharEncoding.UTF_8));
            JSONObject obj = JSON.parseObject(jsonStr);
            for(Map.Entry<String, Object> item : obj.entrySet()){
                logger.info(item.getKey()+"="+item.getValue().toString()); }}else if(contentType.equals("application/x-www-form-urlencoded"){// QueryStringDecoder String jsonStr = fullRequest.content().toString(Charsets.tocharset (charencoding.utf_8)); QueryStringDecoder queryDecoder = new QueryStringDecoder(jsonStr,false);
			Map<String, List<String>> uriAttributes = queryDecoder.parameters();
            for (Map.Entry<String, List<String>> attr : uriAttributes.entrySet()) {
                for (String attrVal : attr.getValue()) {
                    logger.info(attr.getKey() + "="+ attrVal); }}}else if(contentType.equals("multipart/form-data"){//TODO for file uploads}else{
            //do nothing...
        }
    }
    private String getContentType(){
        String typeStr = headers.get("Content-Type").toString();
        String[] list = typeStr.split(";");
        return list[0];
    }
Copy the code

A functional test

I am using Postman to netty implementation of the HTTP server request, if you can feel, you can download.

A Get request

Postman:

Server:

] [nioEventLoopGroup 16:58:59. 130-3-1 the DEBUG io.net ty. Util. Recycler - Dio.net. Ty Recycler. MaxSharedCapacityFactor: 2 16:58:59. [130] nioEventLoopGroup - 3-1 the DEBUG io.net ty. Util. Recycler - Dio.net. Ty Recycler. LinkCapacity: 16 16:58:59. [130] nioEventLoopGroup - 3-1 the DEBUG io.net ty. Util. Recycler - Dio.net. Ty Recycler. Thewire: 8 / / print request url 16:58:59. [159] nioEventLoopGroup - 3-1 the INFO com.sanshengshui.net ty. The uri HttpHelloWorldServerHandler - HTTP: /Copy the code

A Post request

Postman:

Server:

] [nioEventLoopGroup 16:58:59. 130-3-1 the DEBUG io.net ty. Util. Recycler - Dio.net. Ty Recycler. MaxSharedCapacityFactor: 2 16:58:59. [130] nioEventLoopGroup - 3-1 the DEBUG io.net ty. Util. Recycler - Dio.net. Ty Recycler. LinkCapacity: 16 16:58:59. [130] nioEventLoopGroup - 3-1 the DEBUG io.net ty. Util. Recycler - Dio.net. Ty Recycler. Thewire: 8 16:58:59. 159] [nioEventLoopGroup - 3-1 the INFO com.sanshengshui.net. Ty HttpHelloWorldServerHandler - HTTP uri: [nioEventLoopGroup / 17:03:59. 813-2-1] INFO io.net. Ty handler. Logging. LoggingHandler - [id: 0x0f3f5fdd, L:/0:0:0:0:0:0:0:0:8888] READ: [id: 0xfd00cb1b, L: / 0:0:0:0:0:0:0:1:8 888 - R: / 0:0:0:0:0:0:0:1:45 768] 17:03:59. 813 INFO [nioEventLoopGroup - 2-1] io.netty.handler.logging.LoggingHandler - [id: 0x0f3f5fdd, L:/ 0:0:0:0:0:0:0:0:0:8888] READ COMPLETE // Print the URL of the POST request 17:03:59.817 [nioeventloopgroup-3-2] INFO com.sanshengshui.netty.HttpHelloWorldServerHandler - http uri: /tttCopy the code

Gatling performance, load test

If you’re not familiar with the Gatling test tool, take a look at my previous post:

  1. Load, performance test tool -Gatling

  2. Gatling simply tests the SpringBoot project

The performance test report is as follows:

================================================================================ ---- Global Information -------------------------------------------------------- > request count 1178179 (OK=1178179 KO=0 ) > min response time 0 (OK=0 KO=- ) > max response time 12547 (OK=12547 KO=- ) > mean response time 1 (OK=1 KO=- ) > std deviation 32 (OK=32 KO=- ) > response time 50th percentile 0 (OK=0 KO=- ) > response time 75th percentile 1 (OK=1 KO=- ) > response time 95th Percentile 2 (OK=2 KO=-) > Response time 99th percentile 5 (OK=5 KO=-) > Mean requests/ SEC 10808.982 (OK = 10808.982 KO = -), the Response Time Distribution -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- > t < 800 ms 1178139 (100%) > 800 ms < t < 1200 ms 0 ( 0%) > t > 1200 ms 40 ( 0%) > failed 0 ( 0%) = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =Copy the code

other

That concludes the discussion of Netty’s implementation of a high-performance HTTP server.

Netty implements high-performance HTTP server project Address: github.com/sanshengshu…

Original is not easy, if you feel good, I hope to give a recommendation! Your support is the biggest motivation for my writing!

Copyright Notice:

Author: Mu Shuwei

Blog garden source: www.cnblogs.com/sanshengshu…

github.com/sanshengshu…

Sanshengshui.github. IO /