Small knowledge, big challenge! This article is participating in the creation activity of “Essential Tips for Programmers”.

原 文 : Fetching in Deno

I also live whole:

I just finished reading asynchronous Rust and wanted to write some simple projects about asynchronous Rust from scratch, preferably related to the front end. However, I didn’t have any suitable projects in mind, nor did I find any on Build Your Own X, so I decided to take a look at Deno. It is best to water a contribute 😉

Bird’s eye view of Deno

Deno is divided into three parts:

The first part is the CLI, which includes all the apis used by users like FMT, REPL, RUN, compile, doc, and so on. Some of the functionality is implemented by other projects that are called. Cli is integrated into denO. For example, Dprint is used in FMT and typescript uses the official fock to implement transpile with type checking. SWC was used for transpile only. The CLI of the JS execution environment is a portal for initializing modules and runtime. Most of the unit tests, integration tests, and benchmark tests are also located here.

The second part is core, which relies heavily on rusty_V8. Rusty_v8 is a layer of binding for the V8 engine using Rust. It basically calls the V8 API directly, without too much encapsulation. If you are familiar with Rust Async, you should know that the asynchronous API implementation in Rust has two parts: One part of the poll method implements the Future trait, and the other part implements executor. In core, only the poll method is implemented. Executor can use Tokio, Smol, and other libraries. But everywhere else you call core, you use Tokio as an executor; Finally, core implements the communication method between Rust side and JS side: Ops, where Rust registers the OPS to be used by JS, JS can make synchronous call through opSync or asynchronous call through opAsync, and the transmitted data can be serialized and deserialized through Serde_V8, which can be understood as a kind of RPC.

The third part is runtime, which relies on core to implement the JS execution environment of Deno and all ops and JS packages related to operating systems, such as OS, FS, HTTP, process, etc. Other OPS, such as Fetch, Timers, NET, webGPU, etc. are separately placed in Ext. Rust code is registered in CLI and invoked through JS.

fetch

A lot of the apis in Deno are implemented according to the Web standards, so some of your code can run on both Deno and the browser. Of course, this is just a beautiful vision. Web standards are not that easy to implement, and Github has dedicated Web Platform Tests. Deno also passed only part of the test, with Rust’s WPT, a browser engine written by Servo, only a third of the TODO.

Also, some DETAILS of THE API implemented by Deno are not up to standard, such as the implementation of HTTP header value in FETCH, which is also the source of my PR.

Let’s take a look at the issue. In a simple way, when Rust parses header values, some values will cause panic. This panic will cause the JsRuntime of the entire Deno to hang. Imagine your server is running fine, then some fetch request is sent, and your server hangs…

Take a look at what the FETCH specification says about valid header values:

A value is a byte sequence that matches the following conditions:

  • Has no leading or trailing HTTP tab or space bytes.
  • Contains no 0x00 (NUL) or HTTP newline bytes.

Your header value has no space between 0x20 (SP) and 0x7E (~) and does not contain 0x00, 0x0A or 0x0D. The append header specification normalizes the header value, that is

const headers = new Headers()
headers.append("val1"." hah")
headers.append("val2"."hah ")
fetch("xxx", {
  headers: { val: " hha "}})Copy the code

If you normalize a header value with a space before or after it, it will still give you a valid header value. If you normalize a header value with a space before or after it, you can normalize it to a valid header value.

hyperium/http

Hyperium/HTTP is the HTTP library that ops implemented by Rust at the lower level of FETCH in Deno relies on. Header values are parsed by methods in this library. The JS side will detect header values according to the FETCH specification, and any header values that do not meet the specification will throw error to prevent requests from being sent to the Rust side.

Panic is caused by unwrap after the header value is parsed by this library. It can be expected that the JS side will not pass illegal header value, so the unwrap should be safe.

But a look at the Hyperium/HTTP documentation reveals:

If the argument contains invalid header value bytes, an error is returned. Only byte values between 32 and 255 (inclusive) are permitted, excluding byte 127 (DEL).

This means that charcodes between 32 and 255 are valid except for 127, which results in some characters conforming to the FETCH specification but not conforming to hyperium/ HTTP header value parsing requirements. Unwrap causes panic.

I also found related issues in this library, which was also used by Servo and this library has the same problem, haha. Although Deno and Servo are corrupted by this library, there is nothing wrong with the implementation of this library, because its resolution rules are implemented according to HTTP RFC7230, while fetch’s header value is defined according to HTTP RFC2616. And RFC2616 has been scrapped by RFC7230, so it is due to specification differences caused by the pit…

I also put a question on Zhihu, want to know the deeper reason (the reason for the difference between the two RFC, why the FETCH specification is not updated), interested leaders can answer it ~

Finally, I repaired the issue and submitted the first PR I sent to Deno. The content of PR is very simple, that is to remove unwrap and return this Err, so that Err returned from OPS will also be serialized to JS side, and JS side will throw error. This allows the user to catch the error without failing to handle panic and causing the entire Runtime to hang. After two more tests are added, the WPT test results are merged successfully.

Afterword.

Although I haven’t written a single line of Rust asynchronous code in the past few days, I have learned a lot about Deno’s excellent workflow and emphasis on testing. I have also learned a lot from watching masters chat in Discord. I have also learned a lot about v8 Snapshot (Deno Compile). How to implement a JS runtime environment and even glimpse the future of the front-end toolchain. I learned a lot of these things when I was exposed to Rust. I like this quote from Doodlewind:

With a little C/C++ and modern JavaScript, you can take the traditional Web stack out of the browser

Similarly, as long as a small amount of low-level knowledge with the front-end technology stack, you can make yourself out of the browser, front-end engineers are also engineers, not only limited to the front-end, front-end is the starting point, but not the end.

❤️ Thank you all

  1. If you think this article is good, please give it a thumbs-up. Your thumbs-up is the motivation for my writing.
  2. Keep tabs on the public account “Ahab’s Front-end Essays”.