Why esbuild?

I don’t know if you’ve had this problem,

<— Last few GCs —>

[59757:0x103000000] 32063 ms: Mark-sweep 1393.5 (1477.7) -> 1393.5 (1477.7) MB, 109.0/0.0 ms Allocation Failure GC in old space requested

<— JS stacktrace —>

==== JS stack trace =========================================

Security context: 0x24d9482a5ec1

.

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed – JavaScript heap out of memory

1: node: : Abort () [/ Users/XXX/NVM/versions/node/v10.13.1 / bin/node]

2:…

We all know that Node is based on the V8 engine, and there are no limitations on basic memory usage in common back-end development languages. ** You can only use a fraction of the memory in Node when using JavaScript (about 1.4 GB on 64-bit systems, 0.7 GB on 32-bit systems) ** So regardless of how much memory your computer actually has, the amount of memory that Node uses when running JavaScript packages and compiles, It doesn’t change because the actual memory size of your system changes

Or you’re stuck at 92%,

Low Webpack █ █ █ █ █ █ █ █ █ █ █ █ █ █ █ █ █ █ █ █ █ █ █ █ █ the chunk asset optimization TerserPlugin (92%)

As the product gets bigger, it takes longer and longer to compile and CI, and one-third or more of that time is spent doing compression. OOM problems also usually come from compression, how to solve the problem of slow compression and memory, has been very troublesome

Which brings us to today’s hero, Esbuild

The main goal of the EsBuild project is to usher in a new era of build tool performance and create a modern packer that is easy to use.

Compare with other tools

Configure the difficulty

  • Vite: The simplest, almost zero configuration. Vite has many templates that you can use directly, but even if you don’t use a template, you only need to configure a React hot update plugin. However, if the framework you’re using doesn’t have plug-ins, or if you don’t want to use the default configuration, then you’ll need to read the documentation carefully, and you’ll need to study the documentation for Vite and Rollup and even esBuild to do what you want.

  • Esbuild: It’s easy. Esbuild doesn’t have many configuration items; you just need to add a servedir to use its development server.

  • Webpack: It’s complicated. Here I use webpack5, with minimal configuration and code splitting optimizations, but still a lot of third-party packages to install. The rest of the configuration needs to be configured step by step according to the documentation, which is not particularly friendly for beginners.

Startup speed (use the same code and entry)

  • Esbuild: fastest.

  • Vite: Nearly 10 times slower than Esbuild, but not as strong in actual perception, still very fast.

  • Webpack: More than 10 times slower than Vite, actually aware and waiting.

Hot update and **react-router** routing configuration

  • Vite: Supports hot update and react-router after configuring official plug-ins

  • Webpack: Hot update needs to be enabled in devServer and react-router also needs to be configured.

  • Esbuild: The development server does not support hot update and react-router configuration

Code separation and packaging speed

  • Esbuild: Fastest, but worst code separation. As you can see, the largest JS file is 4.8MB, which cannot be deployed in a production environment. If you need to optimize, there is no relevant configuration in the configuration project, you need to modify your code.

  • Webpack: Average speed, good separation of code.

The disadvantage of esbuild && esbuild – loader

Esbuild isn’t perfect either (if it is, why hasn’t it been widely used?).

  1. To ensure efficient compilation of esbuilds, esBuilds do not provide AST capabilities. So some babel-plugins that process code through the AST don’t have a good way to transition into esbuild. So, if your project uses babel-plugin-import, or some custom babel-plugin. At present, there is no good migration scheme.
  2. At present, the built-in Loader of Esbuild can process a large number of format files, such as text, URL, wASM, etc. But it cannot process style files. If you want to use Esbuild to process style files, it cannot.
  3. The esbuild-loader is loaded asynchronously, but because of the Onion model, all script files are not concurrent at the beginning, but concurrent according to the beginning of the dependency tree
  4. Both Vite and Webpack support*.config.tsFormat the configuration file whereas esbuild has no correspondingcliStart the tool, so it needs to be usednode *.jsCommand to start, so ts is not suitable.
  5. Esbuild: Esbuild is really fast. But other than fast experience is not good, this is an immature tool, expect its function to be perfect.

When to use esbuild

It is possible to use esbuild in a production environment. Build tools like Snowpack, Vite and others already use EsBuild as a code handling tool (stability enough). If you must use it, check to see if it meets the following criteria

  1. Not using some custom babel-plugin (e.g., babel-plugin-import)
  2. No need to be compatible with older browsers (esbuild can only convert code to ES6)

Don’t be afraid to use esbuild-Loader to improve your projects

Why so fast?

Esbuild is known for its speed, taking only 2-3 percent of webPack’s time.

There is a picture and there is a truth

(Image from Github)

Several reasons:

  • It is written in the Go language, which compiles to native code

(Webpack uses Javascript; Esbuild uses Go.)

So why do different languages have different speeds?

This comes from the design of Go and Javascript

  • Go is designed for parallelism, whereas Javascript is single-threaded

  • Go shares memory between threads, whereas Javascript must serialize data between threads

  • Go compiles directly to machine code, does not rely on other libraries, and is necessarily faster than JIT.

  • Heavy use of parallel operations.

Esbuild’s algorithm has been carefully designed to be roughly divided into three stages:

Parse => link => code generation. Parsing and code generation are parallelized

Parsing and code generation are the bulk of the work, and can be completely parallel (linking is inherently serial tasks in most cases).

Because all threads share memory, it is easy to share work when bundling imports into different entry points of the same JavaScript library.

  • Customization (code is written by yourself, no other dependencies are used)

In tools like Webpack and Rollup, we have to use a lot of additional third-party plug-ins to address various engineering needs, such as:

  • Use Babel for ES version translation

  • Use ESLint for code checking

  • TSC is used to achieve TS code translation and code inspection

  • Use CSS preprocessing tools such as LESS, stylus, and SASS

Esbuild gets a head start, complete selection! Completely rewrite all the tools needed for the entire compilation process! This means that it needs to rewrite the loading, parsing, linking, and code generation logic of js, TS, JSX, JSON, and other resource files.

Performance customizes the stages of compilation for the highest priority, such as:

  • Rewrite the TS translation tool, completely abandon TS type checking, only code conversion

  • Most packaging tools disassemble lexical analysis, syntax analysis, symbol declaration and other steps into multiple processing units with high cohesion and low coupling. Each module has clear responsibilities, high readability and maintainability. Esbuild adheres to the principle of performance first and uses a counterintuitive design pattern to blend multiple processing algorithms together to reduce the performance loss caused by data flow during compilation

  • Consistent data structures, and efficient caching strategies derived from them

  • Efficient use of memory

Memory access speed can seriously affect performance if you are processing large amounts of data.

The fewer times the data is traversed (and the fewer different representations needed to transform the data into data), the faster the compiler will be.

When using babel-Loader to process JavaScript code in Webpack, multiple data conversions may be required:

  • Webpack reads the source code as a string

  • Babel parses the source code and converts it into AST form

  • Babel transforms the source AST into a lower version AST

  • Babel will generate low version AST as low version source code, string form

  • Webpack parses the lower-version source code

  • Webpack packages multiple modules into the final product

The source code needs to go through string => AST => AST => String => AST => String, repeatedly jumping between the string and AST.

Esbuild overwrites most translation tools to share similar AST structures at multiple compilation stages, minimizing string-to-AST conversions and improving memory efficiency.

Another benefit of Go is that it can store content tightly in memory, which allows it to use less memory and hold more content in the CPU cache.

All object field types and fields are tightly wrapped together, such as several Boolean flags that take up only one byte each.

Go also has value semantics that allow you to embed one object directly into another, so it is’ free ‘and does not require additional allocation.

use

Based on using

First taste — Hello World

Install esbuild first

NPM I ESbuild-g (Global)

Take the React project as an example

Install the react/react-dom dependencies.

We then execute esbuild to build

Command line build

esbuild src\app.jsx --bundle --outfile=out.js

We execute the packaged file

Of course, we can configure package.json to simplify our commands

We just need to execute NPM run build

API

Before we can use it, we need to understand a few apis, the most important of which are Transform and Build

transform

This API will transform ts, JSX and other files into JS files. The Transform API calls operate on a single string without accessing the file system. Great for use in environments without file systems or as part of another tool chain.

Supported files are:

export type Loader = 'js' | 'jsx' | 'ts' | 'tsx' | 'css' | 'json' | 'text' | 'base64' | 'file' | 'dataurl' | 'binary' | 'default';
Copy the code

They are used together with Loaders. Some Loaders are described later

Example:

Output:

build

Build API calls operate on one or more files in the file system. This allows files to refer to each other and be compiled together.

We can also package through JS code

Entry. Js as follows:

require("esbuild")
  .build({
    entryPoints: ["src/app.jsx"],
    bundle: true,
    outfile: "build/out.js",
  })
  .catch(() => process.exit(1));
Copy the code

Then we execute Node Entry.js and see the package

build

Bundling for the browser

Esbuild is packaged for the browser by default, so it can be packaged for the browser without any additional configuration. Of course, we can package it with other configuration items. If we want to minification, we can configure — Minify, If you want to use the sourcemap feature, you can configure –sourcemap. Perhaps you need to specify a specific browser to be packaged for use, we can configure –target

Demonstrate a Minify

esbuild app.jsx --bundle --minify

Demonstrate the Sourcemap functionality

Of course, in addition to configuration on the command line, we can also package in package.json or configuration JS code

JS code

require('esbuild').buildSync({
  entryPoints: ['app.jsx'],
  bundle: true,
  minify: true,
  sourcemap: true,
  target: ['chrome58', 'firefox57', 'safari11', 'edge16'],
  outfile: 'out.js',
})
Copy the code
Bundling for the node

Bundling automatically converts TS types, converting ES modules to CommonJS modules and converting higher versions of ES code to lower versions of ES code.

We just need to add –platform=node on the command line

Esbuild app.js --bundle --platform=node --target=node10.4

This is where you can specify the node version

Of course, you can also configure packaging through JS code

Require ('esbuild').buildsync ({entryPoints: ['app.js'], bundle: true, platform: 'node', target: ['node10.4'], outfile: 'out.js', })Copy the code

loader

There are a lot of loaders built into esBuild, but there are a few that aren’t, which I’ll cover later

  1. Js-loader

This loader defaults to.js,.cjs, and.mjs files. The.cjs extension is used by Node for CommonJS modules and the.mjs extension is used by Node for ECMAScript modules, although esBuild does not distinguish between the two.

Esbuild supports all modern JavaScript syntax. Note that by default, the output of esbuild will take advantage of all modern JS features. However, newer syntax may not be supported by older browsers, so you may want to configure the target option to tell ESBuild to convert newer syntax to the appropriate older syntax.

For example, the following situations will occur:

a ! == void 0 && a ! == null ? A: B is converted to A? B) it may lead to some bugs.

Note, however, that ES5 is not very well supported and there is currently no support for converting ES6+ syntax to ES5.

  1. Ts or TSX-loader (handles TS code or TSX-loader)

This loader is enabled by default for.ts and.tsx files, which means esBuild has built-in support for parsing TypeScript syntax and discarding type comments. However, esbuild does not do any type checking, so you still need to run tSC-noemit in parallel in ESbuild to check the type. Note that esBuild does not do type checking at compile time, which should be done using the IDE before compiling.

  1. Jsx-loader

This loader is enabled by default for.jsx or.tsx files

For example, JSX code becomes js code below

import Button from './button'
let button = <Button>Click me</Button>
render(button)


================================================================
import Button from "./button";
let button = React.createElement(Button, null, "Click me");
render(button);
Copy the code

Jsx/Jsx/Jsx/Jsx/Jsx/Jsx/Jsx/Jsx/Jsx/Jsx/Jsx/Jsx/Jsx/Jsx/Jsx/Jsx/Jsx/Jsx/Jsx/Jsx

import * as React from 'react'
import Button from './button'
let button = <Button>Click me</Button>
render(button)
Copy the code

The react. createElement is called during the conversion, otherwise an error will be reported

  1. Css-loader

Css-loader interpret @import and URL (), import/require() and resolve (). This loader is enabled by default for.css files. It loads files as CSS syntax. CSS is a Category 1 content type in ESBuild, which means esBuild can compile CSS files directly without importing your CSS from JavaScript code.

esbuild --bundle app.css --outfile=out.css

  1. File-loader

The loader copies the file to the output directory and inserts the file name as a string into the compilation. This string is exported using the default export method.

Used in complex scenarios

More configuration items

  1. Serve

It is inconvenient to manually rerun Esbuild before reloading the code in the browser.

Esbuild serve uses a Web server that is rebuilt on every request to serve your code

The advantage of this approach over other approaches is that the Web server can delay browser requests until the build is complete. Reloading your code before the latest build is complete will never run the code generated by the last build. These files serve in memory and are not written to the file system to ensure that stale files are not listened on.

Provides services for all content built by esBuild

Provide a directory for esbuild called servedir, which provides additional content in addition to the files generated by esBuild. This is useful for simple cases where you create some static HTML pages and want to use ESbuild to package JavaScript and/or CSS.

In this way, we can reference the packaged file through an external file, and each HTTP request will cause ESBuild to rebuild your code.

Example:

Index. Js as follows:

const a = 4;
console.log(a);
Copy the code

Entry.js is as follows :(used to execute the wrapper)

require("esbuild")
  .serve(
    {
      servedir: "www",
    },
    {
      entryPoints: ["src/index.js"],
      outdir: "www/js",
      bundle: true,
    }
  )
  .then((server) => {
    console.log(server);
    // Call "stop" on the web server to stop serving
    //server.stop();
  });
Copy the code

We then reference the external resources in index.html

<! DOCTYPE html> <html lang="en"> <head> <title>Document</title> </head> <body> <script src="http://localhost:8000/js/index.js"></script> </body> </html>Copy the code

When we make the changes to the source program and save it, we don’t need to re-execute entry.js. We just need to refresh the referenced file to see the repackaged file. The packaged code is not actually stored in the local file system, but in memory. (Default port is 8000 or larger)

The advantage is that you can use exactly the same HTML page in both development and production. During development, you can run esbuild with –servedir=, and esbuild will supply the generated output file directly. For production, you can omit this flag, and esBuild writes the generated files to the file system. In both cases, you should use exactly the same code in development and production to get exactly the same results in the browser. And the URL structure of the Web server generated by ESbuild is exactly the same as the URL structure of the output directory.

  1. Watch

Enable listening mode in the Build API and tell ESBuild to listen for changes to the file system and then rebuild

require('esbuild').build({
  entryPoints: ['src/index.tsx'],
  outdir: 'build',
  bundle: true,
  watch: {
    onRebuild(error, result) {
      if (error) console.error('watch build failed:', error)
      else console.log('watch build succeeded:', result)
    },
  },
}).then(result => {
  // Call "stop" on the result when you're done
  result.stop()
})
Copy the code
  1. Bundle

Packaging a file means inlining any imported dependencies into the file. The process is recursive because the dependencies of the dependencies (and so on) will also be inlined. Note that packaging is different from file wiring. Passing multiple input files to esBuild when packaging is enabled creates two separate bundles rather than connecting the input files together. To package a series of files together using esbuild, introduce all the files in a single entry point file, and then package them as if they were a single file.

require('esbuild').buildSync({
  entryPoints: ['in.js'],
  bundle: true,
  outfile: 'out.js',
}){ errors: [], warnings: [] }
Copy the code
  1. Entry points

The configuration package entry file, which is an array, is packaged into a separate script

require('esbuild').buildSync({
  entryPoints: ['home.ts', 'settings.ts'],
  bundle: true,
  write: true,
  outdir: 'out',
})
Copy the code

The package produces two files, out/home.js and out/settings.js

  1. Format

Sets the generated JS output format, with three optional values: IIFE, CJS, and ESM.

The IIFE format stands for “Call the function expression now

require("esbuild").buildSync({
  entryPoints: ["src/home.ts", "src/index.ts"],
  bundle: true,
  format: "iife",
  outdir: "bundle",
});
Copy the code

Then the output is as follows:

(() => { // src/home.ts var a = 1; console.log(a); }) ();Copy the code

The CJS format package represents “CommonJS” and runs in the Node environment. Entry points with exports in ECMAScript module syntax are converted to a module, with a getter on the “exports” of each export name. When you set platform to Node, CJS is the default format.

require("esbuild").buildSync({
  entryPoints: ["src/home.ts", "src/index.ts"],
  bundle: true,
  format: "cjs",
  outdir: "bundle",
});
Copy the code

After the package:

var __defProp = Object.defineProperty;
var __reflectGet = Reflect.get;
var __reflectSet = Reflect.set;
var __markAsModule = (target) => __defProp(target, "__esModule", { value: true });
var __export = (target, all) => {
  __markAsModule(target);
  for (var name in all)
    __defProp(target, name, { get: all[name], enumerable: true });
};

// src/home.ts
__export(exports, {
  a: () => a
});
var a = 1;
console.log(a);
Copy the code

Esm format stands for “ECMAScript Module “. Entry points with exports in CommonJS module syntax will be converted to a single default export with module.exports values.

After the package:

// src/home.ts
var a = 1;
console.log(a);
export {
  a
};
Copy the code
  1. Loader

This configuration item changes the way input files are parsed. [js] (https://esbuild.docschina.org/content-types/#javascript), for example, the loader will file parsing of JavaScript, (CSS) (https://esbuild.docschina.org/content-types/#css), the loader will file parsing for CSS.

  1. Tree shaking

Tree Shaking in ESBuild is always enabled during binding and cannot be turned off because removing unused code makes the resulting file smaller without changing observable behavior.

Tree Shaking is the easiest example to explain. Consider the following files. There is a used function and an unused function:

// input.js
function one() {
  console.log('one')
}function two() {
  console.log('two')
}
one()
Copy the code

If you package the file with esbuild –bundle input.js –outfile=output.js, unused functions will be automatically destroyed, producing the following output for you:

// input.js
function one() {
  console.log("one");
}
one();
Copy the code

This works even if we split the functions into separate library files and import them using import statements:

// lib.js export function one() {console.log('one')}export function two() {console.log('two') import * as lib from './lib.js' lib.one()Copy the code

How can I interwork with other tools

Use esbuild directly

Sometimes we don’t want to use scaffolding to build a project and want to configure it ourselves. Here is a simple example:

The new file

Index.html refers to the two packaged files

The index. The TSX is as follows:

import React from 'react'; import ReactDOM from 'react-dom'; import './style.css'; const root = document.createElement('div'); root.className = 'root'; document.body.appendChild(root); const App = () => { return ( <div> <h1 className='esbuild'>Hello, Esbuild! </h1> <h1 className='react'>Hello, React! </h1> </div> ); }; ReactDOM.render( <App />, root, );Copy the code

Index. CSS is as follows:

.esbuild {
    color: rgb(247, 209, 71);
}

.react {
    color: rgb(97, 218, 251);
}
Copy the code

And then do the packaging

Add “build”: “esbuild SRC /index.jsx –outfile=build/index.js –bundle” to the script tag of your package.json file. Then enter the NPM run build command in the terminal

Then reference our packaged file in index.html to see what the page looks like

However, it is obvious that every time we change the referenced file we need to manually execute a build, which is very inconvenient. Can we do hot update?

We can either add a –watch command to the build command in package.json or add a watch command “watch”: “Esbuild SRC /index.tsx –outfile=build/index.js –bundle –watch” with vs Code’s Live Server plugin.

There is also an API mentioned above, Server, which we can use to listen for file changes

Let’s write a JS file that wraps the entry, entry.js

require("esbuild")
  .serve(
    {
      port: 8022,
    },
    {
      entryPoints: ["src/index.tsx"],
      bundle: true,
      outdir: "www",
    }
  )
  .then((server) => {});
Copy the code

We then re-reference it in index.html, but note:!!! In this case, change the path of the reference to

Localhost :8022/ Plus the packed file name you want to reference (if not specified, the packed file name is the same as the packed file name)

<! DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8" /> <link rel="stylesheet" href="http://localhost:8022/index.css" /> <title>Title</title> </head> <body> <script src="http://localhost:8022/index.js"></script> </body> </html>Copy the code

We open the page and see the same effect as above.

Of course, this is only a simple example, for more complex projects, or already defined framework of the project, we can not all refactor, we can use esbuild-Loader to complete some optimizations

The application of Esbuild – loader

Esbuild-loader, developed by Hiroki Osame, is a WebPack loader built on esBuild. It allows users to swap ts-loaders or Babel-loaders through it, which greatly improves build speed.

So how do you migrate existing projects to ESBuild?

Using either babel-loader or TS-Loader allows you to migrate a project to esbuild-Loader very directly. First, install dependencies:

npm i -D esbuild-loader
Copy the code

If you are using babel-loader, modify your webpack.config.js as follows:

module.exports = {
    module: {
    rules: [
-       {
-         test: /\.js$/,
-         use: 'babel-loader',
-       },
+       {
+         test: /\.js$/,
+         loader: 'esbuild-loader',
+         options: {
+           loader: 'jsx',  // Remove this if you're not using JSX
+           target: 'es2015'  // Syntax to compile to (see options below for possible values)
+         }
+       },
Copy the code

To demonstrate the use of esbuild-Loader in the React project, I created a new application

npx create-react-app my-app --template typescript

This will use TS to build a React project in the my-app directory, and we’ll use the original template application to perform the build

time npm run build

It takes about 22.08 seconds

Since the Create React App uses babel-Loader behind the scenes, we need to customize the Create React App. One way to do this is with the Craco tool.

Add the dependency first

npm install @craco/craco esbuild-loader --save-dev

Then replace package.json to use Craco

"start": "craco start",

"build": "craco build",

"test": "craco test",

Then configure a craco.config.js file in the project root directory. Babel-loader (esbuild-loader, esbuild-loader, esbuild-loader

Then run time NPM run build

Our full build, TypeScript type checking, translation, minification, and so on all took 13.85 seconds. By migrating to esbuild-Loader, we reduced our overall compile time by about a third.