background

Developing and maintaining large front-end projects requires some code refactoring from time to time. Take a simple 🌰 :

  • There is an NPM packagean-npm-package-containing-constants, used to maintain the string constant used when the project buried point, the main contents of the code are as follows:
export const ConstantsForTrack = {
  Track1: 'track_in_scene_1'.Track2: 'track_in_scene_2'. };Copy the code
  • The caller uses this package as follows:
import { ConstantsForTrack } from 'an-npm-package-containing-constants'; track(ConstantsForTrack.Track1, ... otherParams);Copy the code
  • As the business iterated, this package grew in size and began to impact the overall performance of the project. Buried identity strings for all businesses are concentrated in this package, so we change how the code is exported:
export const Track1 = 'track_in_scene_1';
export const Track2 = 'track_in_scene_2'; .Copy the code
  • Each business controls the code volume by introducing it on demand:
import { Track1 } from 'an-npm-package-containing-constants/es/constants'; track(Track1, ... otherParams);Copy the code

Refactoring the code inside the package is not particularly complicated, but it can be a bit of a headache to change the way the package is introduced and called at buried points throughout the project. The IDE’s global replacement is obviously not up to the job; Manual replacements violate the programmer’s DRY principle and are tedious and error-prone; It’s not impossible to write a rege-based replacement script, but such scripts tend to be less maintainable and reusable, and may have bad cases (such as matches in comments or string literals, and needs to consider different code styles).

I was reminded of a concept I’d heard about earlier called Codemod, which was supposed to be a way to batch modify code; React, Vue, and Ant Design all provide their own official codemod to help users migrate to newer versions without having to deal with interface changes manually. It’s a good idea to try something new this time, just to get a foot in the door for this kind of refactoring in the future.

Introduction to related Concepts

Codemod

Codemod is a tool/library to assist you with large-scale codebase refactors that can be partially automated but still require human oversight and occasional intervention.

Codemod is a concept that was created inside Facebook and stands for code modification. As stated in the official introduction, the scenario codemod targets is refactoring in a large code base. Codemod provides a fast, reliable, semi-automatic tool for refactoring all relevant code in the code base to help developers iterate quickly when a significant change to an interface that is frequently used in code is not forward compatible.

jscodeshift

Jscodeshift is a JavaScript/TypeScript refactoring tool based on the codemod concept that parses JS/TS code into an Abstract Syntax Tree (AST). It also provides a series of apis for accessing and modifying the AST for automated code refactoring. Jscodeshift combines Babel Parser, AST-types (for quickly creating new AST nodes) and RecAST (for maintaining code style information of generated code) to provide a quick and easy operation interface. At the same time, it also provides the function of multi-task parallel execution, so that the reconstruction operation of massive code files can be run in parallel, which makes full use of multi-core CPU computing power and shortens the execution time of reconstruction task.

Abstract syntax tree

You may have seen the concept of abstract syntax trees in the principles of compilation course. Here is a wikipedia description:

In computer science, an Abstract Syntax Tree (AST), or Syntax Tree for short, is an Abstract representation of the syntactic structure of source code. It represents the syntactic structure of a programming language as a tree, with each node in the tree representing a structure in the source code. The syntax is “abstract” because it does not represent every detail that occurs in real grammar.

Let’s take another simple code example:

if (1 + 1= =3) {
  alert('time to wake up! ');
}
Copy the code

To see more concretely what an abstract syntax tree looks like :(provided by nhiro.org/learn_langu… Generated)

In JavaScript engineering, it’s not just JavaScript engines that parse code that are involved in AST, in code translation (Babel), static analysis (ESLint), packaged builds (Webpack, rollup…). , parses the code into an AST for further operations.

Go ahead and write a codemod

Jscodeshift is good and powerful, but the official documentation provides relatively limited information to help you write a codemod quickly. So the best way to get your hands dirty is to write your own Codemod by referencing some existing Codemod scripts and online articles. Taking the problem mentioned in the background section as an example, we will write a Codemod to complete the introduction and modification of an-Npm-package-containing -constants.

Before getting started, let’s introduce a powerful AST visualization tool, the AST Explorer. As shown in the figure below, we paste the code we want to modify on the left and immediately see the syntax tree from parsing the code on the right and view the properties of each node in it. With this tool, we can get a more intuitive view of the AST; While writing Codemod, you can also use this tool to quickly locate nodes that need to be modified. With the “Transform” switch on, you can also write codemod scripts directly in your browser and see the effect of codemod transformations in real time.

With an overview of the code and its AST, we can break down the replacement effort into the following steps:

  1. Through the file, the code file “AN-Npm-package-containing – Constants” is screened out;
  2. Find all pairs in the filtered fileConstantsForTrackObject member and willConstantsForTrackMember access expressions are replaced directly with expressions with constant names;
  3. Collect the constant names used in the code, generate a new import statement and replace the old one.

Now we can start writing Codemod! According to the official documentation, the Codemod code exports a function, and jscodeshift will inject apis and related information through arguments when running this function. From this we can write a preliminary framework for Codemod:

module.exports = function (
  fileInfo, // Information about the file being processed, including the file path and content
  api, // jscodeshift provides an interface
  options // A parameter passed through jscodeshift CLI
) {
  const { source, path } = fileInfo;
  const { jscodeshift: j } = api;
  const root = j(source);  // Parse the code to get the AST

  // Write the code to operate on the AST here

  return root.toSource({ quote: 'single' }); // Returns the AST converted to a code string
}
Copy the code

Note that the toSource() function at the end of the code can pass in some code style-specific configuration. When recast parses code, it maintains the code style information in the syntax tree and restores the code to its original form during toSource(). The nodes inserted after the code is converted to the syntax tree do not have these specific code style information. See the interface definitions in this file for details on how to style this code.

Step 1: Filter the files that need to be modified

Since we only need to focus on the code file with an-NpM-package-containing constants, Import {ConstantsForTrack} from ‘AN-Npm-package-containing -constants’; The root of these subtrees is of type ImportDeclaration, The value field of the source property in the node is the node characteristic “AN-Npm-package-containing -constants” that we are looking for.

The next step is to write code to filter out such nodes. Because of the data structure nature of the AST, both the parser and recAST apis perform traversal of the AST based on the visitor pattern. Jscodeshift further encapsulates the API based on the concept of Collection, making it easier for users to filter and modify AST nodes. Some common Collection apis will be used in the following practice. For a complete list of apis, please refer to the following source code:

Set basic operations: github.com/facebook/js… Access and Modify AST nodes: github.com/facebook/js…

Here we use find() to get the set of target nodes from the syntax tree, and skip the file if the set is empty. The code is as follows:

const trackConstantsImportDeclarations = root.find(j.ImportDeclaration, {
  source: { value: 'an-npm-package-containing-constants'}});if(! trackConstantsImportDeclarations.length) {// Return undefined to indicate that this file does not need to be modified
  return;
}
Copy the code

Step 2: Collect access to all relevant constants in the code and replace them

Constantsfortrack. Track2:

You can see that this expression is parsed to a node of type MemberExpression, whose object property is an Identifier node whose name is “ConstantsForTrack”, Property is an Identifier node with name “Track2”.

We can replace all of our constant fortrack.[constant name] with [constant name] simply by using the replaceWith() interface provided by jscodeshift, Replace the corresponding MemberExpression node with the Identifier node in its property property. In addition, we need to collect the name attribute of the Identifier in order to introduce these constants in the next step. The code is as follows:

let usedKeys = [];
const trackConstantsMemberExpressions = root.find(j.MemberExpression, {
  object: { name: 'ConstantsForTrack'}}); trackConstantsMemberExpressions.replaceWith((nodePath) = > {
  // replaceWith The type of argument passed in the callback function for traversing the collection is NodePath
  // NodePath contains the context information of the node in addition to its own information, so it needs to be extracted from it first
  const { node } = nodePath;
  const keyId = node.property;
  if (keyId.name) {
    usedKeys.push(keyId.name);
    returnkeyId; }});if(! usedKeys.length) {return;
}
Copy the code

Step 3: Replace the import statement

The goal of this step is to import {ConstantsForTrack} from ‘AN-Npm-package-containing -constants’; Replace with import {Track2} from ‘AN-npm-package-containing -constants/es/constants’; . Let’s examine the syntax tree for the latter:

As you can see, the ImportDeclaration SpeciFIERS attribute records what the import statement introduces as an ImportSpecifier array. We can now reconstruct a new ImportDeclaration node as follows:

usedKeys = [...new Set(usedKeys)];

const keyIds = usedKeys.map((key) = > j.importSpecifier(j.identifier(key)));
const trackConstantsEsImportDeclaration = j.importDeclaration(
  keyIds,
  j.literal("an-npm-package-containing-constants/es/constants"));// Replace the original import statement
trackConstantsImportDeclarations.at(0).replaceWith(
  () = > trackConstantsEsImportDeclaration
);
Copy the code

Jscodeshift provides a build interface for various nodes based on ast-types, in the form shown in the code above, in the form of a hump nomenclature (beginning with a lowercase letter), distinguished from the node types used for filtering in the first step (PASCAL nomenclature, beginning with a capital letter). Specific parameters for the different node-building methods can be found in the source code, and code hints are provided in the AST Explorer.

Before running, you need to make sure you have jscodeshift installed globally via NPM install -g jscodeshift.

Automatic refactoring completed! Here are some common CLI parameters. For more information, see here:

-c, --cpus=N Enables at most N child processes to run codemod in parallel (Max (total number of CPU cores -1, 1) by default) -d, --(no-)dry, Not to actually modify files (off by default) - extensions = EXT needs to handle the file extensions (with a ", "multiple, and the default of js) - parser = Babel | Babylon | flow | | ts TSX parse the file using the parser, Defaults to Babel - t - transform = FILE codemod script path or URL, defaults to ". / transform. Js "- v, -- verbose = 0 | 1 | 2 show codemod relevant information in the process of executionCopy the code

conclusion

Jscodeshift: jscodeshift: jscodeshift: jscodeshift:

  • Search and filter of AST:find(),filter()
  • The Collection access:get(),at()(The difference is that the former returnsNodePath, which returnsCollection)
  • Node insertion and modification:replaceWith(),insertBefore(),insertAfter()

Collection API: github.com/facebook/js… https://github.com/facebook/jscodeshift/blob/master/src/collections/Node.js

AST node construction parameters: github.com/benjamn/ast…

Jscodeshift CLI parameter: github.com/facebook/js…

I hope this article provides some inspiration for your code refactoring and helps you free yourself from repetitive code modifications.

reference

facebook/jscodeshift: A JavaScript codemod toolkit.

Effective JavaScript Codemods

Christoph Nakazawa: Evolving Complex Systems Incrementally | JSConf EU 2015

Fun with Codemod & AST

Write Code to Rewrite Your Code: jscodeshift

Jscodeshift

Getting started with jscodeshift

Join us

Feishu – Bytedance’s enterprise collaboration platform is a one-stop enterprise communication and collaboration platform integrating video conferencing, online documents, mobile office and collaboration software. At present, feishu business is developing rapidly, and we have R&D centers in Beijing, Shenzhen and other cities. There are enough HC in front-end, mobile terminal, Rust, server, testing and product positions. We are looking forward to your joining us and doing challenging things together (please link: future.feishu.cn/recruit).

We also welcome the students of flying book to communicate with each other on technical issues. If you are interested, please click on flying book technology exchange group to communicate with each other