TL; DR

The SDK project repository corresponding to this article is universal-SDK-by-TSdx.

If you need to create an SDK quickly, you can copy the project and build your own SDK, and come back to this article if you run into problems.

If you’d rather read the code to understand the implementation, then cloning the project and reading the source code is for you. Executing git Log in this project will show detailed practical steps that are very helpful in understanding the iterative process of the SDK.

background

The project needs to implement an SDK, which needs to meet the following conditions:

  1. Use TS language with support for Lint and Test.
  2. Supports node.js running and the lowest version is V10.
  3. Browsers are supported and can be configured using Browserslist.

Choose TSDX

The reason for choosing TSDX is that it can quickly set up TS configurations, including:

  • Lint and test
  • Prettier formatting
  • Vscode can friendly prompt lint errors
  • Can directly build CJS/ESM/UMD format output file

Create the project through TSDX

npx tsdx create universal-sdk-by-tsdx
#You can just choose BASIC

cd universal-sdk-by-tsdx
Copy the code

Refer to Commit for the created project.

After creating the project, we need to verify the functionality we need.

Verify eslint 😑

When vscode changes the ‘boop’ string in SRC /index.ts to double quotes, no error is reported.

You need to run yarn lint –write-file, which will generate the.eslintrc.js file in the project directory. Restart VScode and you will see the error message prompted by ESLint.

Verify that VScode is formatted ✅

Format vsCode as the prettier plug-in for the formatting to take effect. Because the package.json#prettier field defines the prettier configuration for the project, vscode does not use the default prettier configuration for the project.

The build artifact supports both the browser and Node.js

The SDK supports both the browser and node.js environment. There are several problems.

  1. Different apis are exposed. Generally, browser environments support fewer apis than node.js environments. For example, OSS has some limitations on the browser side.
  2. The browser side needs to build the UMD format so that the user can directly<script />Label.
  3. Different code logic is executed depending on the runtime environment. For example, you want to print in the console when the SDK is used in the browsing environmentHello World, but not in node.js environment.
  4. Native has different capabilities. Assume that the SDK needs to rely on Crypto for encryption and decryption. In the browser environment, because the browser does not provide crypto ability, so need to rely on the three-party package implementation, such as: browser-crypto. However, in the node.js environment, you can’t use the three-party library, because node.js crypto is implemented in C++, which is much higher performance.

Next, we will discuss each of these problems and give solutions.

Problem 1: Different apis are exposed

This can be done by setting up different entry files for the browser environment.

  1. createsrc/browser-index.ts, whose export andsrc/index.tsDifferent, referencecode.
  2. inpackage.jsonaddbuild:node 和 build:browserScript, referencecode.
  3. createtsdx.config.jsIf the build target is a browser, modify the build entry fileconfig.inputReference,code.

After modification, run yarn build:browser and yarn build:node to generate different products.

For details about how to modify this problem, see Commit.

Since a package can only have one type definition in the TypeScript language, Therefore, you cannot specify a different type definition for the browser environment and the Node.js environment (see issue-typings for the main and browser property in package.json).

Issue 2: The browser environment needs to be built in UMD format

The format parameter of TSDX can be used to configure the format of the build product. Modify the build:browser script.

For details about how to modify this problem, see Commit.

Problem 3: Different code is executed depending on the runtime environment

Usually we judge process.env.node_env implementations to execute different code in the development and production environments. For example, a code that prints error messages only in the development environment would look like this:

if ("development" === process.env.NODE_ENV) {
  console.error(err)
}
Copy the code

We can also use this approach to distinguish the environment in which the build artifacts run. Based on the target environment at build time, add a variable named TARGET_ENVIRONMENT to process.env and replace it with a string constant via @rollup/plugin-replace. The environment can then be distinguished by process.env.target_environment.

This approach has the very powerful advantage of removing dead code, meaning that code from the Node environment does not appear in the browser environment artifacts.

For details about how to modify this problem, see Commit.

Problem 4: Native ability is different

This is an interesting question. Let me list the solutions that come to mind and then compare them.

Solution 1: rollup-plugin-node-builtins

This solution uses the rollup-plugin-nod-builtins plug-in to package the modules built into Node.js into the UMD product. In this scenario, the node.js built-in modules need to be packaged by the user if they are using CJS or ESM artifacts. Usually Webpack handles these modules well.

For details about how to modify this problem, see Commit.

In this revision, SRC /browser-index.ts, SRC /index.ts, SRC /use-crypto.ts and index. HTML were added for testing purposes. You can test whether the UMD artifact is working properly by accessing the index.html file in a browser.

The modification points in the tsdx.config.js file include:

  1. Modify theconfig.external. Since all dependencies need to be packaged into umD artifacts, this isexternalAlways need to befalse.
  2. Add globals and Builtins plug-ins so that crypto features can be used in a browser environment.
  3. Replace the safer-buffer with a local file for this issue.
  4. Convert the local safer buffer using CommonJS. The TSDX commonJS plug-in configuration only works for modules in node_modules when we use the local onesafer-bufferAfter this, the module will not be converted by the CommonJS plug-in, resulting in an error (require is undefined).
  5. Use the Browser field in package.json to locate the module, otherwise it does not meetResolution rules for browser-related dependencies. The code is referenced hereTSDX source, increasing thebrowser: trueAnd modified mainFields. This is a bug in TSDX.

Scenario 2: Require dynamically according to the TARGET_ENVIRONMENT environment variable

For details about the modified code, see Commit.

First look at the SRC /use-crypto.js file. In this file, dynamically load the specified module according to process.env.target_environment.

Then use the commonjs plugin to process use-crypto. Js in the tsdx.config.js file.

Depending on the commonJS plug-in’s processing flow, if dynamic require depends on EITHER A or B, both require statements will be converted to ES6 import statements first. Both dependencies A and B are treated as dependencies, packaged into the final product. This increases the product construction time and volume.

But the problem can be solved gracefully. Simply put the Replace plug-in in front of the CommonJS plug-in and the dead code is removed before the code is processed by CommonJS. There is no impact on build time.

See [replace plugin’s readme.md]{github.com/rollup/plug… Typically, @rollup/plugin-replace should be placed in plugins before other plugins so that they may apply optimizations, such as dead code removal.

In fact, after the Commit, if you run yarn build:browser and open the index. HTML file in the browser, the following error is found.

This means that solution 2 needs to handle all of node.js’s built-in modules just like solution 1, so solution 2 is dependent on Solution 1. ** However, option 2 is useful in certain scenarios where you need to use the functionality involved in the browser environment in the Node.js environment. ** For example, if you want the Node.js environment to use the DOM API, you can use the environment to determine whether to introduce libraries (such as jsDOM) that simulate DOM operations in the Node.js environment.

For the code that will eventually run in the browser, see Commit.

Option 3: Customize the Polyfill file

Similar to Ali OSS. Provide a polyfill file for Crypto in your project, and then use the polyfill when packaging in your browser environment.

The crypto polyfill used by OSS is crypto-browserify. However, it only selectively includes the methods required by OSS (e.g., SHA1 and MD5) and does not include other algorithms (e.g., SHA256). The benefits of using this approach are that you can keep the amount of code in a Polyfill to a minimum, reduce the package size, and increase build speed. ** But even with this approach, we still need to deal with node.js’ built-in modules (e.g., Buffer), because crypto-browserify still uses buffer.

Once you have customized the polyfill file, there are two ways to determine whether to import the polyfill based on the current environment.

  1. throughTARGET_ENVIRONMENTEnvironment variable dynamic require.
  2. By adding a plug-in to tsdx.config.js@rollup/plugin-aliasAnd map the specified package to a Polyfill file in the browser environment.

I won’t go into the first way, but the second way. In fact, I prefer the first approach because it is more intuitive at the code level, whereas the second approach is configured through a rollup plug-in and requires a higher cost of understanding.

Refer to Commit for the implementation code.

Compare the above schemes

If you use node.js modules on the browser side, you need to package the underlying built-in modules, so option one is mandatory. It’s just that complex built-in modules like Crypto can be implemented with a custom polyfill. By comparing scheme 1 and Scheme 3, scheme 1 builds the built-in crypto into the UMD product, which takes 20 seconds, and the SIZE of the UMD product is 1MB, while scheme 3 builds the custom polyfill into the UMD product, which takes only 10 seconds, and the size of the UMD product is only 70KB.

Solution 2, when combined with the Replace plug-in, imports only the dependencies specified by the target environment and does not increase build time or product size. In scenarios other than simple Node.js built-in modules, Solution 2 is a great tool, such as combining solution 2 with Solution 3.

Although it is theoretically possible to “import the browser version of the Node.js built-in module into the browser environment” through Solution 2, there is a downside to this: if a third party relies on using the node.js built-in module (e.g. Crypto-browserify uses buffers internally) and it is not possible to modify the code that these three parties rely on to import the browser version of Buffer. In fact, it is not possible to import the browser version of the node.js built-in module into the browser environment with solution 2 alone. But it can be done through the Alias plug-in, which is essentially the same as using the Builtin plug-in.

yarn buildBuild both the browser and node.js environment artifacts

Previously we added build:node and build:browser script directives to package.json. Ideally, the two build statements should be executed in parallel. However, because TSDX has the following two limitations, we can only use custom scripts to get around.

  1. Build artifacts can only be stored in the dist directory (although rollupconfig.output can be modified, the internal logic of TSDX is coupled to the default artifact path).
  2. TSDX deletes the last build artifact each time it performs a build (–noClean does not take effect at build).

Refer to Commit for the implementation code.

The end product is in browser/ and dist/. Why not put the browser environment product in dist/browser? The reasons are as follows:

  1. src/browserThe TS file in the directory will be placed indist/browserDirectory, so there is a possibility of file conflicts.
  2. Allow the user to passuniversal-sdk-by-tsdx/browserImport the product of the browser environment, and the TS type file is also used at this timebrowser/index.d.ts.
  3. The source location after sourcemap is guaranteed to remain the same. If I put the product indist/browserThen the sourcemap mapped source path will bedist/src/...Rather thansrc/....

The SDK supports the lowest version of the environment

The SDK must indicate the operating environment that it supports. It is better to configure the operating environment to facilitate subsequent upgrades.

Node.js supports the lowest version v10

By looking at the source code of TSDX, it is found that the hard code of node.js is the lowest version v10.

To make it easier for us to configure the supported version of Node.js ourselves, we override it in the tsdx.config.js file.

Refer to Commit for the implementation code

The nullish-coalescing-operator syntax and exponentiation-operator syntax exist in the SRC/test-nod-version. ts file. Nullish -coalescing-operator is supported at node@v14, while exponentiation-operator is supported at node@v7. Therefore, after the yarn build:node command is executed, the nullish-coalescing-operator syntax is translated, but the exponentiation-operator syntax is not translated.

Exponentiation-operator will also be translated if the supported node.js version is changed to ‘6’.

Babel-preset -env configures the environment version required by the ES feature in the babel-compat-data/data/plugins.json file. On the Node. green website, it is also clear how well ES features are supported in the Node.js environment.

Configure supported browsers

For the modification, see Commit.

You can configure the sdk-supported browsers by configuring the package.json#browserslist field value. The current value is “Chrome >= 70”. After the yarn build:browser command is executed, the Nullish -coalescing-operator syntax is translated, but the exponentiation-operator syntax is not translated.

If package.json#browserslist is changed to “Chrome >= 40”, both nullish-coalescing-operator and exponentiation-operator are translated.

Will the Node.js environment and the browserslist configuration of the browser environment affect each other?

Because in the node.js environment we set the targets parameter for babel-preset-env, the package.json#browserslist configuration is not used in the node.js environment. In the browser environment we do not set targets parameter, so we will only use package.json#browserslist configuration.

Refer to the original website. By default @babel/preset-env will use browserslist config sources unless either the targets or ignoreBrowserslistConfig options are set.

test

Unit testing

The importance of unit testing goes without saying, an SDK without single testing will fail, and I won’t use it.

For the modification, see Commit

Because we need to do unit tests of node.js environment and browser environment, we created two script instructions test:browser and test: Node in package.json. Modify the test script to run a single test of the two environments, and at the end merge the test coverage with scripts/ mapcoverage.js.

Can test code refer directly to the product of a build?

If you reference the code in SRC/in the test file, you can use the code directly after the build. It has the following advantages:

  1. Direct references to the code in the artifact allow you to identify problems early in the build process. After all, it’s a test, so you should try to simulate a scenario where the user is using the SDK.
  2. Because we use environment variables in our codeTARGET_ENVIRONMENTTo judge, so if referencedsrc/, we need to set up before all test casesTARGET_ENVIRONMENTEnvironment variables, there is some coupling. But referencedist/The code in the

But after referencing the artifact code, is it impossible to locate the source code in SRC/during code execution or debugging? No, the Sourcemap file is automatically used by the Yarn test even if it refers to the product packaged in dist/. When console.log() executes or runs incorrectly, it points to the real SRC/file. So this is not a weakness of reference artifact code, but it has another, more serious weakness: poor test coverage. So if the SDK requires a test coverage report, it cannot reference the built product code for testing.

Demo of the browser environment

As mentioned earlier, if the user references the CJS or ESM artifacts of the browser environment, then the node.js built-in module used by the SDK needs to be packaged by the user. However, this scenario is not covered by unit tests (since the single test runs in node.js), so Demo needs to be set up to verify the availability of the SDK in this scenario.

For the modification, see Commit. For the Demo creation process, see webpack-getting Started.

In example/webpack. Config. Js, due to the inability of webpack @ 5 default package Node. Js polyfill built-in modules, so you need to use the Node – polyfill webpack — the plugin. In order to be able to view the SOURCE code of the SDK in the browser’s DevTools, source-map-Loader is used.

In package.json, the exposed module fields are grouped together and the “browser” field is added. If the “browser” field is not added, the SDK used by Example will be a different version of Node.js than expected.

conclusion

This paper records the process of building a general SDK, summarizes the problems encountered in the implementation of a general SDK, and gives solutions.

If you have similar needs, just copy the project and start writing the SDK.

Okay, I’m going to write the SDK

Original is not easy, don’t forget to encourage oh ❤️