background

Recently, some students have joked that the packaging of the existing development environment is too slow. The words are as follows

  • Student A: The XXX project started cold. The timing was about 3 minutes and 45 seconds. Let’s see if there is room for optimization.

The video is the XXX project F-XXX-6196 branch (yarn Dev enter to start timing)

  • Student B: The local operation is too slow
  • S: Our project started slowly. What should we do
  • Student D: The compilation time of XXX project is too long, so it is easy for me to change something. I can’t rule out my computer problems, and MY experience is very poor

The screenshot shows the gitLab information of the project

And indeed, 3 minutes and 45 seconds 🐢🐢🐢 wait time who can stand 😣! So what’s wrong with the scaffolding we have now?

Existing scaffolding is packaged based on Webpack, which actually does cache optimizations like Babel, TS, and even persistent caching like Hard-source. However, due to the rapid iteration of requirements, all branches resulted in a large number of node_modules dependency changes and needed to regenerate a new cache, which made each packaging cache optimization and other invalid, at this time I also planted the seeds of hatred nopack (not packaging ❌📦)!

So when the existing optimization means are hit, the time can barely be reduced to about 40s 🐢.

Webpack5 also tries to solve this problem by adding a heavyweight persistent caching feature, but according to the official data from esbuild, webpack5 is the slowest! 😢

This is probably the worst webpack5 hack ever… Since part of the first package is generated in the cache, why not use the time after the second cache takes effect to compare? In this case, upgrading webpack5 may not solve the pain point, branch back and forth back to pre-liberation!

When the project is expanding, it seems to have reached the bottleneck of Webpack. The recent popularity of Vite and Snowpack may be the real solution?

Vite, snowpack

Finally, I came up with the idea of using Vite or Snowpack to solve existing problems. There are many good articles about how these tools work, so I won’t go into too much detail here.

Soon after the release of Snowpack, I read his code with great curiosity and found that there were a lot of bad things in it. For example, define was implemented by replacing strings, which would obviously hurt innocent people. CSS import was also not handled, and the author recently said that he would leave it to the community to maintain. I’m out of my depth.

Real-world testing is super important. I’m sure that sounds cliche, but its true. We had a few starter projects that we could test Snowpack against, but they were all small and simple. This created a huge experience gap between our internal projects and our actual users.

6 More Things I Learned Building Snowpack to 20,000 Stars (Part 2) while we are closer to the business, with nearly 100 large projects, including all kinds of systems and H5, we can almost step on every hole.

To be honest, I’m not sure where Snowpack goes from here. I burnt out on it at the end of last year, and haven’t found the energy to return. Usage and downloads began to trend down and the community has gotten quieter.

At the same time, Vite (that Snowpack alternative that now powers SvelteKit) is taking off. To their credit, they do a lot of things really well. The good news is that two tools are very similar and easy to switch out. Even Astro is experimenting with moving from Snowpack to Vite in a future release.

Still, Snowpack is a respectable product 🔥 ~

Therefore, after comprehensive experience, Vite is still reliable. If it is a new project at this time, THERE is no doubt that I recommend everyone to use Vite, but this is an old project…

What is the old project? I upgraded a big version of TS-Loader before, and two project test environments went blank. I encountered problems such as cssModules parsing error and JS SCSS import parsing path failure when TRYING to access Vite.

So far I think vite is not any problem, some strange, a hundred flowers blossom problem code you have to change, of course, the actual continue to struggle may find more strange problems, the cost of communication and time to solve the cost is impossible to estimate.

nopack

Eventually I decided to develop it myself, and in the interim I wanted to be able to switch between the new ESM development mode and the existing WebPack mode.

Nopack will exist in the form of a global installation, that is, zero intrusion into existing projects, zero risk to the online environment, and zero modification of the project source code that can be accessed.

Q: How to change project 0 access?

  • The processing of most parts of Vite jsResolver, Sass and so on is biased to rollup system. In order to seamlessly accommodate the _moduleAliases configured in the current project and various imports in SASS, I used parts of Webpack for this part, mainly the processing logic of enhanced- Resolve and Sas-Loader.
  • Less important is the fact that everyone likes to communicate with Websocket, vite, Snowpack, webpack-dev-Server, while Nopack uses SSE.

Q: What does the name nopack mean?

  • The original intention is just to do a translation service, try not to package ❌📦. The original intention was to do only one translation and return of TS and JSX file requests, that is, only one AST operation in total. The idea is naive, since CJS to EJS is a non-valid thing, require is runtime, import is static, and perhaps you have thought that the import() function is also runtime? But import() returns a Promise!

  • Facts prove that the initial thought of just doing translation is not tenable 👇

Q: How to eliminate CJS syntax completely?

  • You’d just have to prebuild package it, synthesize a file, and require’s code would just be shoved in the right place, so you wouldn’t have anything like require

Q: Isn’t Nopack determined not to pack?

  • Nopack will try to determine if the NPM package is an ESM package. For example, package.json has the module, browser fields. Or import es, esM directory files such as “packageA/es/a.js” may also be esM does not pre-build. As a result, the project XXX 🔍 scanned 336 packages, and only 199 packages were pre-built at 📦.

Q: So it’s faster than vite?

  • Fast is fast, but there is a lot of deceptive behavior of the package, such as a package also declared module field, but I am actually CJS code. What’s more, most of the files in a package are ESM files, and 1 ~ 2 files are CJS 🤯! No way, you are not nopack, these packages I bear you, I hardcode into an array to blacklist you. So a fully pre-built Vite as a general-purpose tool might be a no-go.

Q: What is the effect of the final access?

  • The project that is claimed to be 3 minutes and 45 seconds takes about 8s to pre-build through esBuild, and then it takes about 4s to translate ts and JSX under SRC through esbuID when the page is refreshed, so it takes only 12s to finally start pre-build to fully display the page. Esbuild is much faster than I thought ⚡️⚡️ port.

Q: Disadvantages of ESM development?

  1. The wait time to refresh the page is slower
    • Esm mode emphasizes translation. When the browser code runs from the entry file to the bottom, the import syntax will continuously initiate new file requests. For JSX and TS files, the request will also need to be translated through esbuild. Dart-sass translation is required for SCSS requests until all code has been run.
    • Webpack mode emphasizes packaging. In the packaging stage, the entry file will rely on analysis, and finally package a main.js for the browser to run, so the browser will run much faster.

So you can do a strong cache optimization for node_modules files and hot update via react-refresh to avoid page refresh

  1. There are certain performance requirements
    • When one of the projects was connected, the students using Windows development said that it was fast, but there was a dot block 🤔 when the page was refreshed. During the investigation, we found that more than 3000 JS requests were sent in the project network, and chrome CPU was full. So nopack had to go back to 😢 and had to fully pre-build node_modules packages to merge more files, bringing the number of requests down to 700, and Chrome was down for a while even when the CPU was full. So in the end NoPack compromised again and went full pre-build. It turns out that file requests reduce the amount of white screen time during page refreshes, and prebuilds only add about 2s to the development experience.

If the full pre-built node_modules didn’t solve the problem, I also developed a version based on the packaged esbuild ⚡️ development environment, like WebPack, The SRC /index.js entry was packaged as a main.js file, and the esbuild ⚡️ package for this 3:45s project only took about 15s!

  1. Esbuild does not check for ts types
    • See fork-ts-checker-webpack-plugin to implement a plugin that works with the ESbuild check

4. Inconsistency between development environment and production environment * ESM page display speed is slow due to excessive JS file requests. The production environment still has to follow the existing packaging logic (file merge, ugly, etc.), and the inconsistency between the development environment and the production environment may result in no local problems and no test production problems.

Q: Is the production environment ready for use?

  • Nopack also supports production builds, but core old projects don’t use ⚠️⚠️⚠️ (they won’t lose their jobs for three minutes). However, new internal projects will use NoPack by default (you can use Vite here, just use NoPack to avoid duplicate tools). In addition, esbuild ⚡️ is used for noPack and Rollup 📦 is used for Vite.

Q: Is it time for ESM development?

  • I think the wait-and-see students can also join. By 2021-10-29, 9 projects have been connected to NOpack, and basically all are connected and run with 0 changes. Although a variety of problems were exposed during the period, pioneer Vite basically had a variety of solutions to refer to.

Q: Vite, Snowpack, unbundled which do you recommend?

  • My personal recommendation is v choose V.

SWC, esbuild

When choosing a compiler, I researched SWC and ESBuild

swc

Nextjs has experimented with SWC.

// https://github.com/vercel/next.js/blob/canary/packages/next/build/webpack-config.ts#L442 const getBabelOrSwcLoader = (isMiddleware: boolean) => { return useSWCLoader ? { loader: 'next-swc-loader', options: { isServer: isMiddleware || isServer, pagesDir, hasReactRefresh: ! isMiddleware && hasReactRefresh, }, } : { // ... }Copy the code

esbuild

Vite and Snowpack are both esbuilds to use.

Currently, we only find problems with assigning generic functions to variables for compilation.

const Demo = <T>(num): T => {
  return num
}
Copy the code

The test data

Below is the test data related to the ESbuild and SWC in Webpack mode

  • Project: XXX

  • A69d4e version: 91

  • Compile speed: build time in the first no cache state

    • babel
      • Yarn start 0.23s user 0.06s System 0% CPU 1:05.51 total
      • Yarn start 0.23s user 0.09s System 0% CPU 1:05.47 total
    • swc
      • Yarn Start 39.25s user 5.70s System 123% CPU 36.407 Total
      • Yarn Start 40.17s user 5.97s system 134% CPU 34.420 Total
    • esbuild
      • Yarn Start 32.52s user 5.03s System 111% CPU 33.757 total
      • Yarn Start 30.70s user 4.90s System 120% CPU 29.616 Total
    • Esbuild (TS mode compiling TS and JS simultaneously)
      • Yarn Start 47.81s user 6.96s System 120% CPU 45.556 Total
      • Yarn Start 47.90s user 6.62s system 126% CPU 43.037 total

  • Compile compatible:

    • SWC compile componets/ModalForFirstTime js file exists problems, such as other routing page appear normal.
    • Esbuild js compiler decorator is not supported, so switch to TS mode to compile JS files. Other routing pages behave normally.

@evanw decorator spec unfortunately still seems like it’s a long way off, both based on low activity and based on how many things it looks like they still have to figure out.

  • Summary:
    • Esbuild is almost as fast as SWC, but esBuild is more compatible with older projects than SWC.

eggs

October 27 saw Bytes bundled with esbuild-based tools unbundled, when I thought if I had to learn English well, not To use Chinglish, but also referred to not always called unbundled 😄

Read more: github.com/xiaoxiaojx/… Welcome to 🌟 Star at 🌟