Refactoring Code background

Some of our business components need to support multiple languages, these separate the components of the translation document usually contracting out the maintenance in some project folder, and each need to be translated text need to hand in your code to use the intl. Get (key) package to achieve runtime interpreter, developers also need according to the document one by one go to Google for a variety of language translation, Then write it into the project. This virtually increases the cost of development for component developers, and if the translation is wrong, the PM and operations students cannot go to the code repository to fix the translation. So business components need a more standard, lightweight multilingual solution to replace these manual tasks. After some research, we decided to adopt the company’s Translate – CLI + international translation platform + external open source React-Intl-Universal to complete the multi-language component.

Now that the decision has been made to adopt the new multilingual solution, the existing components that currently maintain the translation copy locally need to be adapted to incorporate the new multilingual solution. However, there are more than 30 existing components. If the developers of these components are required to modify each component file by file, it will bring great cost. Since the changes to all components are the same, it is possible to develop a CLI command to enable developers to execute the multi-language solution migration of components in their own component root directory. This can greatly reduce the developer’s transformation cost.

Some of the industry’s React project multilingual practices explored

Some of GitHub’s more mature multilingual libraries are as follows:

react-i18next

react-intl

react-intl-universal

These libraries are by far the most widely used in the industry for multilingual React projects. I will not introduce the usage of the above libraries, but their API design and usage are very similar. Basically, the project needs to maintain the copywriting of each language in different files, similar to the figure below, one file is maintained in English and Japanese respectively, and each additional language needs to maintain an additional JSON

The content of each file is as follows: similar to the form of key: value. Key is the code we define to identify a certain clause, and value is the translation copy of the copy in Chinese, English and Japanese.

These libraries then provide functions such as Init, translate, etc

Initialization (init)

The init function is called at project initialization, passing in the current language and its corresponding text JSON. This is equivalent to registering the translation copy inside these tools

import intl from 'react-intl-universal';
const locales = {
    "en": require('./locales/en.json'),
    "zh": require('./locales/zh.json')}; intl.init({currentLocale: 'en',
     locales
})
Copy the code

Call (translate)

Then, on the side of the code that requires multiple languages, use the translate function provided by these tools, intl.get below. The translate function takes the key from the parameter and fetches the corresponding translation copy from the JSON registered with the tool at the previous step.

import intl from 'react-intl-universal';

class Test extends Component {
    render() {
    return (
        <div>{intl.get('INPUT_MOBILE')}</div>)}}Copy the code

This makes it easy to internationalize a project.

disadvantages

There are some drawbacks to the industry’s current solutions

  1. If a project requires a multi-language copy, the developer will need to manually replace it with the Translate function.
  2. Developers also need to manually maintain the translation copy of en.json,ja.json and other languages.
  3. The developer needs to make sure that the key in lang.json and the key in intl.get(key) correspond correctly. If there are hundreds or thousands of documents that require multiple languages, it is easy to make translation errors in intl.get(key). This also creates an implicit mental burden for developers.
  4. Because most of our projects have products and operations, general translation is also checked and maintained by products and operations. Json is maintained within the project. If the product needs to modify the translation, it needs to contact the developer to modify it. This also increases the maintenance cost of translation.

Noun explanation

Because the article uses some of the company’s not open source technology, so give these technologies temporarily some pseudonyms, if the following technologies open source, you are welcome to use. The following is mainly to explain some nouns for the convenience of readers

International Translation Platform

The intelligent international translation platform developed by bytedance international team provides efficient and professional one-stop multi-language solutions for business. The platform is mainly responsible for maintaining the translation documents of various languages used in the company’s overseas business system. It is still in the internal use stage and is expected to open to the public this year

Translate – CLI (not open source yet)

A command-line tool developed by bytedance’s international team, the main functions are one-click copy scanning of code, machine translation, copy uploading to the international translation platform, copy replacement, etc., to solve the problem of manual maintenance of copy translation by developers

Document scanning

Translate – CLI is used to scan the project code and scan out the Chinese in the code. And the browser will show you how much Chinese is not wrapped in translate in the code

After scanning, a multi-language key will be generated according to the Chinese document, and the document will be translated in multiple languages (Google Translate can be), and finally the following JSON will be generated

// en.json
{
    "component.template.multilingual_test": "Multilingual test"
}
Copy the code

Copy to upload

Translate – THE CLI uploads the key generated from the copy scan, Chinese copy, and JSON(en,zh, JA, etc.) to an international translation platform. The platform is shown in the following figure and maintains the key and translation corresponding to the copy in each language.

Copy to replace

In the code automatically translate – cli ‘multilingual’ replacement for intl. Get (‘ component. The template. Multilingual_test ‘) or other tools provide translate function, So that the code can display the corresponding translation copy at run time

Document download

Translate – CLI is used to download the copy under the corresponding project of the international translation platform to the local, which solves the problem that the developer manually maintains the translation JSON, and also solves the problem that the product students cannot modify the translation copy maintained in the code

Use the CLI tool to reconstruct component code

CLI Tool Selection

CLI tool You can use commander to achieve the effect of the CLI tool.

As for implementing the CLI command to modify project files, after some research, facebook’s open source Jscodeshift was selected to implement the modification of component code.

Advantages of jscodeshift include:

  1. Also supports JavaScript or TypeScript parsing
  2. The API is simple, and code changes can be made through only a few apis
  3. Provides visualization of code->AST on the website: astexplorer.net/. Simply paste in our JS or TS code to convert it into an AST syntax tree. Using this syntax tree and the API, it is easy to find the code snippet specified by the file through the JS code to modify.

What the CLI helps component developers do

Now that the tools to modify the component code have been identified, the next step is to determine how to modify the component code to enable the migration of multilingual solutions

The following lists the changes to the component

1. Modify component package.json

Since the multilingual component uses translate- CLI, you need to provide translate-cli related commands in package.json for the developer to execute.

The solution

To modify package.json, it is relatively simple, just need to complete the script modification of package.json through the following code.

const data = require(`${process.cwd()}/package.json`)
data.scripts.i18n =
"translate scan && translate upload && translate replace --force && translate scan --fallback && translate clean"
Copy the code

2. Install the translate cli on the component and add the translate. Config. js configuration file

The solution

To install dependencies on components, use the exec function of node child_process to execute NPM or YARN commands

import { exec } from 'child_process'
exec('yarn add dependenceName', {}, fallback)
Copy the code

The CLI tool also provides a generic translate. Config. js file that is written to the component root directory through the Node FS module when executing CLI commands.

3. Modify the component SRC code

Point to specific to transform: maintains a tool within a component before class intl, contain the init and the get method, used for dictionary initialized and translation. It is now replaced with react-Inl-universal, in line with the team tools specification. That code uses the relative path import local class statement: import intl from ‘.. Import intl from ‘react-intl-universal’

The solution

This can be done using the jscodeshift tool we mentioned above. Import Intl from ‘.. /i18n’ import Intl from ‘react-intl-universal

You can see the ‘.. /i18n’ corresponds to a node whose type is ‘StringLiteral’ in the right AST syntax tree. And the value of node is ‘.. /i18n’

To add ‘.. ‘to the import declaration in component code /i18n’ to ‘react-intl-universal’

The specific code is as follows:

const jscodeshift = require('jscodeshift')
const j = jscodeshift.withParser('tsx')

const modifyCommonFile = (source) = > {
  // source is the source code, obtained from node fs.readfile string
  const ast = j(source)
  const imports = ast.find(j.StringLiteral)
  imports.filter(item= > item.value.value.includes('/i18n')).replaceWith("'react-intl-universal'")
  return ast.toSource()
}
// Both writeFile and readFile are functions wrapped around fs.writeFile and fs.readFile
const originData = readFile('filePath')
const newData = modifyCommonFile(originData)
writeFile('filePath', newData)
Copy the code

This completes a file modification. If you recursively traverse the component SRC directory, get the path of all SRC files, and then execute the above code, you have completed the modification of the entire component SRC file.

Finally, we put the implementation codes of the above three transformation schemes into our encapsulated CLI tools, and then execute cli commands in the root directory of the component to complete the one-click migration of multi-language tools to translate- CLI and react-Intl-universal.

Multilingual practice

Best multilingual practice? !

Translate – Use the CLI tool

Now that the migration of the component’s multilingual scheme has been completed through the CLI command above, add the script of “i18n” to package.json. Then we can use the i18n command in package.json to translate to cli for multilingual component work

"script": {
   "i18n": "translate scan && translate upload && translate replace --force && translate scan --fallback && translate clean"
}
Copy the code

The i18n command performs copy scan, copy upload, copy replacement, copy download sequentially.

This saves the component developer from having to manually wrap the translated text with intl.get(key) translate function, and from having to manually maintain json and keys for translation in multiple languages. Developers just need to write Chinese in the code and then execute NPM run i18n when the development is complete. Translate – CLI will help developers solve the problem. If PM or operation thinks some machine translation is inaccurate and wants to modify it, it only needs to modify the translation copy on the international translation platform, and then the developer can run NPM run i18n again to pull the latest copy from the international translation platform to the local. This is convenient for both developers and products.

conclusion

To sum up, we don’t need to worry about multilingual support any more. These tools will complete the multilingual process from compilation + runtime. The PM only needs to check the results of machine translation on the international translation platform. Wouldn’t that kill two birds with one stone?

Looking forward to

Based on the current situation and problems of multi-language projects, this paper lists some multi-language practices in the industry, and summarizes a set of solutions to multi-language projects at the lowest cost based on some internal practices of the company. It also describes how to use Commander +jscodeshift to develop a CLI to help a component to complete the code transformation, so as to realize the component multi-language solution migration

I hope that after reading this article, it will be helpful to readers in using CLI to batch transform project code or project multi-language.

Recruitment hard wide

Founded in 2014, bytedance’s commercial realization department is responsible for the realization of the entire product matrix of TikTok, Toutiao, Watermelon, TikTok and other bytes, as well as the realization of revenue at home and abroad, which has contributed to the company’s extremely rapid revenue growth for several years.

If you want to join us, please send your resume to [email protected] and I will push it for you as soon as possible.