Problem is introduced into
Start with a sharp question: how do MPA multi-page applications or micro-front-end architectures handle the common parts of a page?
The reason why it is sharp, because not only our company, including Tencent, many domestic and foreign first-line technical teams have encountered.
Like this blog post: Tencent documentation dilemma
Let’s take MPA applications for example, such as the menu section, which would normally pass when rendering traditional back-end templates
// .NET
@Html.Partial("Header")
Copy the code
or
// Java
<%@ include file="header.jsp"% >Copy the code
Introduce public templates so that public parts are rendered directly when a page is visited.
However, if it is a modern project (such as React), and the MPA project is not separated from the front and back ends (the page is still rendered by the back end, and Dom rendering is taken over by React), we will copy the constructed resource files to the back end project and introduce script and style into the page for rendering.
This is where the problem comes into play. The common part of the React rendering is the header.
The general practice is to build the common parts directly into each page-level project as components, and well, Tencent documentation does the same.
Doing so brings the following disadvantages:
- Build redundancy, which is packaged into each page-level project when it is built, wastes load bandwidth gratuitously.
For example, if the Header section is individually built to 400KB, then each page-level build will be 400KB larger than the existing size (ignoring the common library dependencies, assuming the DllReferencePlugin is used uniformly). There’s no exaggeration here, we have a lot of functionality in the Header, and with chunks it’s really close to 500 kilobytes.
- If the public part is modified, all projects that reference it must be rebuilt.
Especially for a common part like the Header, which is used on every page, all page-level applications must be rebuilt with minimal modification.
For example, the notification center of Tencent document is shown below:
Before webpack5, this seemed like an impossible thing to do!
Tencent document front-end people have also studied this problem, but from the research process described in the paper, mainly for the packaged __webpack_require__ method buried tick, and has not come up with an effective solution.
To be honest, the webpack underlayer was quite complex, unfamiliar and uncertain, so we were slow to actually do it.
— From The Dilemma of Tencent Documents
However, after a lot of exploration, we solved this problem perfectly in July 2019 with webpack4’s existing features! Coincidentally, the Wepback team also added the module-federation Feature for this scenario in the latest V5 release.
Let’s start the dry goods!
Webpack4 solution
The reason Tencent docs doesn’t want to do anything with __webpack_require__ is because it’s too complicated to change and cause other problems.
In fact, at the beginning of their direction is wrong, is the so-called snake hit seven inches, if not hit seven inches will cause a series of problems, or dare not play.
So let’s take a look at the “seven-inch” externals attribute (which you already know what it does).
Because it is the bridge between the inside of Webpack (NPM + build) and the outside references, I think this is where the knife is most appropriate!
reviewexternals
与 umd
Recall that we used externals to configure the CDN third-party library, such as React, as follows:
externals: {
'react-dom': 'ReactDOM'.'react': 'React'
}
Copy the code
The React variable will bind to the window variable in the browser environment. The umD build is compatible with commonJS, CommonJs2, AMD, Windows, etc.
Externals is used to: When WebPack builds, / / Import ReactDOM from ‘React’/import ReactDOM from ‘React’ / / import ReactDOM from ‘React’ / / import ReactDOM from ‘React’ / / import ReactDOM from ‘React’ / / import ReactDOM from ‘React’ / / import ReactDOM from ‘React’ / / import ReactDOM from ‘React’ This mapping value (ReactDOM and React) is found in the window variable.
Here are two charts to prove it:
Why did I spend so much time foreshadowing externals? Because this is the bridge, the bridge to the external module!
Let’s make a bold assumption: ideally, my public part would have a Header component! Suppose you build it as a standalone UMD package, configured as externals, with import Header from ‘Header ‘; How about importing it and using it as a component?
I have done the experiment and there is no problem!!
But the best-case scenario doesn’t exist, and the odds are as low as winning the welfare lottery.
Most of our situations look like this:
import { PredefinedRole, PredefinedClient } from '@core/common/public/enums/base';
import { isReadOnlyUser } from '@core/common/public/moon/role';
import { setWebICON } from '@core/common/public/moon/base';
import ErrorBoundary from '@core/common/public/wrapper/errorBoundary';
import OutClick from '@core/common/public/utils/outClick';
import { combine } from '@core/common/entry/fetoolkit';
import { getExtremePoint } from '@core/common/public/utils/map';
import { cookie } from '@core/common/public/utils/storage';
import Header from '@common/containers/header/header';
import { ICommonStoreContainer } from '@common/interface/store';
import { cutTextForSelect } from '@common/public/moon/format';
import { withAuthority } from '@common/hoc'; .Copy the code
References like this can be found in dozens of projects, and the use of aliases, in particular, can lead to dozens of references!
PS: We are monorepo architecture, @core/ Common is a common dependency project, where utility methods, enums, AXIos instances, public components, menus, etc are maintained, so we managed to build this project independently.
Externals is configured to convert only the following case, which matches the module name exactly:
import React from 'react'; = >'react': 'React' => e.exports = React;
import ReactDom from 'react-dom'; = >'react-dom': 'ReactDOM' => e.exports = ReactDOM;
Copy the code
Third party library names cannot be followed by/paths! For example, the following is not supported:
import xxx from 'react/xxx';
Copy the code
One good thing came out of
I thought it was unlikely that WebPack developers would be so rigid about the API that there must be hidden portals. Sure enough! A closer look at the official documentation gave me a hint: it also supports functions!
The import () function allows you to control any import statement!
We can try printing the value of request in this function. The result is as follows:
All import references are printed out! So, we can manipulate @common’s references to @core/common at will! Such as:
function(context, request, callback) {
if (/^@common\/? . * $/.test(request) && ! isDevelopment) {return callback(
null,
request.replace(/@common/.'$common').replace(/\//g.'. ')); }if (/^@moon$/.test(request) && ! isDevelopment) {return callback(null.'$common.Moon');
}
if (/^@http$/.test(request) && ! isDevelopment) {return callback(null.'$common.utils.http');
}
callback();
}
Copy the code
As an explanation, callback is a callback function (which also means it supports asynchronous judgment), and its first argument has an unspecified purpose. The second argument is a string that will execute the expression on the window, such as $common.moon, which will look for window.$common.moon.
So the purpose of this code is clear: replace @common with $common, and replace/in the reference path with. Look it up on Windows instead.
Variable names are not allowed to start with an @ sign, so I changed the library value to $common
So, now that you’re building a page-level project, you can strip the public part of it out of the window, which doesn’t have the $common object yet!
Build public projects independently
First, at the end of the last section, our requirement was clear: we need to build a $common object on the window, which we can build using the UMD, Window, or Global form. However, $common should have a set of subattributes that can be hierarchical according to the import path, such as:
import $http, { Api } from '@http';
import Header from '@common/containers/header/header';
import { CommonStore } from '@common/store';
import { timeout } from '@packages/@core/common/public/moon/base';
import * as Enums2 from '@common/public/enums/enum';
import { Localstorage } from '@common/utils/storage';
Copy the code
We need $common to have this structure:
So how do you build this hierarchical $common object? The answer is very simple, export a corresponding structure for the compiler entry object!
Post the code directly:
// webpack.config.js
output: {
filename: "public.js".chunkFilename: 'app/public/chunks/[name].[chunkhash:8].js'.libraryTarget: 'window'.library: '$common'.libraryExport: "default",},entry: ".. /packages/@core/common/entry/index.tsx".Copy the code
// @core/common/entry/index.tsx
import * as baseEnum from '.. /public/enums/base';
import * as Enum from '.. /public/enums/enum';
import * as exportExcel from '.. /public/enums/exportExcel';
import * as message from '.. /public/enums/message';
import commonStore from '.. /store';
import * as client from '.. /public/moon/client';
import * as moonBase from '.. /public/moon/base';
import AuthorityWrapper from '.. /public/wrapper/authority';
import ErrorBoundary from '.. /public/wrapper/errorBoundary';
import * as map from '.. /containers/map';
import pubsub from '.. /public/utils/pubsub';
import * as format from '.. /public/moon/format';
import termCheck from '.. /containers/termCheck/termCheck';
import filterManage from '.. /containers/filterManage/filterManage';
import * as post from '.. /public/utils/post';
import * as role from '.. /public/moon/role';
import resourceCode from '.. /public/moon/resourceCode';
import outClick from '.. /public/utils/outClick';
import newFeature from '.. /containers/newFeature';
import * as exportExcelBusiness from '.. /business/exportExcel';
import * as storage from '.. /public/utils/storage';
import * as _export from '.. /public/utils/export';
import * as _map from '.. /public/utils/map';
import * as date from '.. /public/moon/date';
import * as abFeature from '.. /public/moon/abFeature';
import * as behavior from '.. /public/moon/behavior';
import * as _message from '.. /public/moon/message';
import * as http from '.. /public/utils/http';
import Moon from '.. /public/moon';
import initFeToolkit from '.. /initFeToolkit';
import '.. /containers/header/style.less';
import withMonthPicker from '.. /public/hoc/searchBar/withMonthPicker';
import withDateRangePickerWeek from '.. /public/hoc/searchBar/withDateRangePickerWeek';
import withDateRangePickerClear from '.. /public/hoc/searchBar/withDateRangePickerClear';
import MessageCenterPush from '.. /public/moon/messageCenter/messageCenterPush';
import { AuthorityBusiness, ExportExcelBusiness, FeedbackBusinessBusiness,
FilterManageBusiness, HeaderBusiness, IAuthorityBusinessProps,
IExportExcelBusiness, IFeedbackBusiness, IFilterManageBusinessProps,
IHeaderBusinessProps, IMustDoBusinessProps, INewFeatureBusinessProps,
MustDoBusiness, NewFeatureBusiness } from '.. /business';
import {
Header, FeedBack, MustDoV1, MustDoV2, Weather,
withSearchBarCol, withAuthority,
withIconFilter, withExportToEmail, withSelectExport, withPageTable, withVisualEventLog
} from '.. /async';
const enums = {
base: baseEnum,
enum: Enum,
exportExcel,
message
};
const business = {
exportExcel: exportExcelBusiness,
feedback: FeedbackBusinessBusiness,
filterManage: { FilterManageBusiness },
header: { HeaderBusiness },
mustDo: { MustDoBusiness },
newFeature: { NewFeatureBusiness },
authority: { AuthorityBusiness },
};
const containers = {
map,
feedback: FeedBack,
newFeature,
weather: Weather,
header: { header: Header },
filterManage: { filterManage },
termCheck: { termCheck },
mustdo: {
mustdoV1: { mustDo: MustDoV1 },
mustdoV2: { mustDo: MustDoV2 },
}
};
const utils = {
pubsub,
post,
outClick,
storage,
http,
export: _export,
map: _map
};
const hoc = {
exportExcel: {
withExportToEmail: withExportToEmail,
withSelectExport: withSelectExport
},
searchBar: {
withDateRangePickerClear: withDateRangePickerClear,
withDateRangePickerWeek: withDateRangePickerWeek,
withMonthPicker: withMonthPicker,
withSearchBarCol: withSearchBarCol,
},
wo: {
withVisualEventLog: withVisualEventLog
},
withAuthority: withAuthority,
withIconFilter: withIconFilter,
withPageTable: withPageTable,
withVisualEventLog,
withSearchBarCol,
withMonthPicker,
withDateRangePickerWeek,
withDateRangePickerClear,
withSelectExport,
withExportToEmail,
};
export default {
enums,
utils,
business,
containers,
hoc,
initFeToolkit,
store: commonStore,
Moon: Moon,
wrapper: {
authority: AuthorityWrapper,
errorBoundary: ErrorBoundary,
},
public: {
enums,
hoc,
moon: {
date,
client,
role,
MessageCenterPush,
resourceCode,
format,
abFeature,
behavior,
message: _message,
base: moonBase,
}
}
};
Copy the code
The code is long, but not difficult to read. Our goal is to build an export object whose hierarchy exhausts all import path possibilities!
And once we add a public file for other projects to use, we have to maintain this file, because it is the real entrance!
This file is so long, partly because there are so many common features, but also because we use WebPack alias, which makes it a bit more likely to have a lot of different references (withSearchBarCol, for example, has two import methods, So it happens twice in the structure). Therefore, if you want to use this program, it is better to recommend a standard control.
Use a combination of
The common parts are built independently, and the page application pulls them out, so how do they work together?
Directly in order to quote it!
How to debug
A careful child might ask, such a page application refers to the packaged public. Js, the actual development of the development environment debugging?
When the page application is built or run, I add the isDevelopment variable to control it and only pull it out when building the production environment. Otherwise, call callback() directly and return as is, without doing anything.
In this way, when writing code in the development environment, the actual reference is still the local project under node_modules.
For the local project dependencies of the Monorepo architecture, Lerna establishes soft connections.
In fact, with webpack4 existing features to do this degree, or is not easy, after all, people at home and abroad the first line of technical teams have been a headache for several years!
Next, let’s take a look at webpack5, a solution that will blow their minds!
Webpack5 solution
Module Federation
Webpackage 5 brings us a built-in plugin: the ModuleFederationPlugin
The authors define it as follows:
Module federation allows a dynamically load code from another Application — in the process sharing dependencies, If an application Consuming a Federated module does not have a dependency needed by the federated code — Webpack will download the missing dependency from that federated build origin.
Module Federation enables JavaScript applications to load code dynamically from another JavaScript application — while sharing dependencies. If the Federated Module consumed by an application does not have the dependencies required in federated Code, Webpack will download the missing dependencies from the Federated build source.
terms
Several terms
-
Module Federation: Same idea as Apollo GraphQL Federation — but applies to JavaScript modules running in a browser or node.js.
-
Host: the Webpack that is first initialized during page loading (when the onLoad event is fired);
-
Remote: another Webpack build partially consumed by “host”;
-
Bidirectional hosts: When a bundle or Webpack build runs as a host or remote, it either consumes or is consumed by other applications — both at runtime.
-
Orchestration Layer: This is a specially designed Webpack Runtime and entry Point, but it is not a normal application entry point and is only a few KB.
Configure the parsing
Here’s a list of how to use it, and we’ll dig into the details later:
// app1 webpack.config.js
const ModuleFederationPlugin = require("webpack/lib/container/ModuleFederationPlugin"); . plugins: [new ModuleFederationPlugin({
name: "app1".library: { type: "var".name: "app1" },
remotes: {
app2: "app2"
},
shared: ["react"."react-dom"]]}),// app1 App.tsx
import * as React from "react";
import Button from 'app2/Button';
const RemoteButton = React.lazy((a)= > import("app2/Button"));
const RemoteTable = React.lazy((a)= > import("app2/Table"));
const App = (a)= > (
<div>
<h1>Typescript</h1>
<h2>App 1</h2>
<Button />
<React.Suspense fallback="Loading Button">
<RemoteButton />
<RemoteTable />
</React.Suspense>
</div>
);
export default App;
// app2 webpack.config.js
...
plugins: [
new ModuleFederationPlugin({
name: "app2",
library: { type: "var", name: "app2" },
filename: "remoteEntry.js",
exposes: {
Button: "./src/Button",
Table: "./src/Table"
},
shared: ["react", "react-dom"]
})
]
Copy the code
This demonstrates how to use the Button and Table components shared by App2 in App1.
Explain the meanings of these configuration items:
-
ModuleFederationPlugin from webpack/lib/container/ModuleFederationPlugin, is a plugin.
-
The ModuleFederationPlugin needs to be initialized for both host and remote.
-
Any module can act as host or remote or both.
-
Name specifies the name to be used as the orchestration layer filename of the current project if the filename attribute is not configured
-
Filename Specifies the name of the file for the orchestration layer. If this parameter is not specified, the name attribute value is used.
-
Library mandatory, defines the layout layer module structure and variable names, similar to output’s libraryTarget function, but only for the layout layer.
-
An exposes item (shared module mandatory), key-value pair, import Button from ‘app2/Button’; The value value is the actual path in the app2 project.
-
Import Button from ‘app2/Button’; App2: library -> name = app2: library -> name = app2
-
Shared Shared module used to share third-party libraries. For example, app1 loads first and shares a component in App2 that relies on React. When loading the component in App2, it will check the shared component in App1 for react dependencies. If there are react dependencies, it will use them first. It will not load its own fallback.
Finally introduced in index.html in app1
<script src="http://app2/remoteEntry.js"></script>
Copy the code
Can.
With these configurations, app1 is free to use app2/Button and app2/Table.
Build file profiling
So how does the ModuleFederationPlugin implement this dark magic?
The answer lies in the following built code:
__webpack_require__.e = (chunkId) = > {
return Promise.all(Object.keys(__webpack_require__.f).reduce((promises, key) = > {
__webpack_require__.f[key](chunkId, promises);
returnpromises; } [])); }; __webpack_require__.e(/* import() */ "src_bootstrap_tsx").then(__webpack_require__.bind(__webpack_require__, 601));
Copy the code
Src_bootstrap_tsx bootstrap_tsx bootstrap_tsx bootstrap_tsx bootstrap_tsx bootstrap_tsx
Object.keys(__webpack_require__.f).reduce((promises, key) = > {
__webpack_require__.f[key](chunkId, promises);
return promises;
}, [])
Copy the code
This is going through all the methods on the f object.
Overridables remotes j:
/ * * * * * * / /* webpack/runtime/overridables */
/ * * * * * * / ((a)= > {
/ * * * * * * / __webpack_require__.O = {};
/ * * * * * * / var chunkMapping = {
/ * * * * * * / "src_bootstrap_tsx": [
/ * * * * * * / 471./ * * * * * * / 14
/ * * * * * * / ]
/ * * * * * * / };
/ * * * * * * / var idToNameMapping = {
/ * * * * * * / "14": "react"./ * * * * * * / "471": "react-dom"
/ * * * * * * / };
/ * * * * * * / var fallbackMapping = {
/ * * * * * * / 471: (a)= > {
/ * * * * * * / return __webpack_require__.e("vendors-node_modules_react-dom_index_js").then((a)= > () => __webpack_require__(316))
/ * * * * * * / },
/ * * * * * * / 14: (a)= > {
/ * * * * * * / return __webpack_require__.e("node_modules_react_index_js").then((a)= > () => __webpack_require__(784))
/ * * * * * * / }
/ * * * * * * / };
/ * * * * * * / __webpack_require__.f.overridables = (chunkId, promises) = > {
/ * * * * * * / if(__webpack_require__.o(chunkMapping, chunkId)) {
/ * * * * * * / chunkMapping[chunkId].forEach((id) = > {
/ * * * * * * / if(__webpack_modules__[id]) return;
/ * * * * * * / promises.push(Promise.resolve((__webpack_require__.O[idToNameMapping[id]] || fallbackMapping[id])(a)).then((factory) = > {
/ * * * * * * / __webpack_modules__[id] = (module) = > {
/ * * * * * * / module.exports = factory();
/ * * * * * * / }
/ * * * * * * / }))
/ * * * * * * / });
/ * * * * * * / }
/ * * * * * * / }
/ * * * * * * /}) ();/ * * * * * * / /* webpack/runtime/remotes loading */
/ * * * * * * / ((a)= > {
/ * * * * * * / var chunkMapping = {
/ * * * * * * / "src_bootstrap_tsx": [
/ * * * * * * / 341./ * * * * * * / 980
/ * * * * * * / ]
/ * * * * * * / };
/ * * * * * * / var idToExternalAndNameMapping = {
/ * * * * * * / "341": [
/ * * * * * * / 731./ * * * * * * / "Button"
/ * * * * * * /]./ * * * * * * / "980": [
/ * * * * * * / 731./ * * * * * * / "Table"
/ * * * * * * / ]
/ * * * * * * / };
/ * * * * * * / __webpack_require__.f.remotes = (chunkId, promises) = > {
/ * * * * * * / if(__webpack_require__.o(chunkMapping, chunkId)) {
/ * * * * * * / chunkMapping[chunkId].forEach((id) = > {
/ * * * * * * / if(__webpack_modules__[id]) return;
/ * * * * * * / var data = idToExternalAndNameMapping[id];
/ * * * * * * / promises.push(Promise.resolve(__webpack_require__(data[0]).get(data[1])).then((factory) = > {
/ * * * * * * / __webpack_modules__[id] = (module) = > {
/ * * * * * * / module.exports = factory();
/ * * * * * * / }
/ * * * * * * / }))
/ * * * * * * / });
/ * * * * * * / }
/ * * * * * * / }
/ * * * * * * /}) ();/ * * * * * * / /* webpack/runtime/jsonp chunk loading */
__webpack_require__.f.j = (chunkId, promises) = >{.../ * * * * * * /}) ();Copy the code
The last f.j method, which I won’t mention in detail, is jsonp loading, which dates back to wepback4.
We mainly focus on f. rotes and F. Overridables, two new webpackage 5 methods. Zack Jackson chose this place to go under the knife. Unlike External, which is a build-to-outside connection, this is a post-build connection.
As we’ll see in a moment, the way you actually interact with the outside world is exactly the same way I discussed it in webpack4 in the last video, which is by using global variables to get through references.
Let’s start with what reduce does in the previous code: It basically iterates through these three methods to see if a dependency exists!
overridables
Shared public third-party dependencies, react, and react-dom are parsed here. During the construction of APP1, the two files will be independently constructed. The reception module in App2 will preferentially find the shared dependency under APP1 when loading. If there is one, it will be used directly; if not, its own will be used.
remotes
Remotes dependence, configuration of the remotes will be key to generate in idToExternalAndNameMapping variables, and then the key point is:
Post two pictures and let’s analyze them one by one:
__webpack_require__. E will search overridables remotes j one by one. When it finds remotes, it will enter the remotes method as shown in the figure above.
ChunkId variable value is src_bootstrap_tsx at this time, so, the first layer will traverse the 341 and 980, and then through the two values, find idToExternalAndNameMapping, 341 = [731, “Button”] and 980 = [731, “Table”].
__webpack_require__ highlighted in the figure of this line of code (data [0]). The get (data [1]) purpose is to take 731 this module, and calls it the get method, parameters for the Button | Table, to get the Button or the Table component.
So the question is, what module is this 731? Why does it have a get method on it?
Moving on to the second image above, I highlighted the 731 module, which internally references the 907 module and overrides the React React-DOM module. Points to 14 and 471 (which happen to come from the idToNameMapping mapping defined in the Overridables method).
The 907 module refers to the global variable APP2!
Why does the app2 variable have a get method? We didn’t define this method when we built App2. Let’s take a look at the results of the app2 build:
Click on remoteentry.js and you’ll find out:
The ModuleFederationPlugin defines two methods on the choreographer layer: Get and Override:
Get is used to find its own moduleMap map (from an object configuration), and it is this global variable app2 + whose GET method connects two unrelated modules!
Override is used to find shared third-party dependencies, which is also extremely subtle. Why? In the code posted above, we look at the orchestration layer of App1 and find the __webpack_require__.O object, defined when the Overridables method is run, with an initial value of {}, But it is empty when __webpack_require__.f.overridables is officially executed. This allows App1 to execute directly using fallbackMapping (i.e., local third party dependencies).
However, in module 731 mentioned above, the override method provided by App2 was used to copy references in App1 of React and React-DOM into app2. We moved to the choreography layer of App2 (all the codes of the choreography layer are consistent). Overridables in app2 uses the react and react-dom dependencies in __webpack_require__.
As you can see, the Override method in App2 overwrites the third-party dependencies in app1 passed in by the external call to the __webpack_require__.O variable!
This is why the authors emphasize that there is almost no dependency on redundancy:
There is little to no dependency duplication. Through the shared option — remotes will depend on host dependencies, if the host does not have a dependency, the remote will download its own. No code duplication, but built-in redundancy.
As of 2010/05/13, it was found that the plug-in code of Webpack5 was still merging new submissions to the Master branch. After a brief look at the last two submissions, I found that there are still a lot of changes in packaging (configuration items remain unchanged for the time being), so the code of packaging results above is just for reference, and the principle can be roughly understood. The code of building results I tested today has been different.
conclusion
ModuleFederationPlugin brings us unlimited imagination. There are many application scenarios, such as dependency sharing of micro-applications on micro-front-end, module sharing, etc.
Two drawbacks I can think of:
-
One is the need for an additional exposes configuration for the modules to be exposed (not suitable for our own scene in the previous section of this article, as the entry export structure is too complex), and the need to notify all modules in use of the correct configuration.
-
After configuring the NPM link or LERNA local dependencies, you also need to configure the webpack alias and tsconfig paths for Remotes.
However, combining the two solutions of wepback4 and webpack5 and using them in a scenario is nearly perfect!
Oh, I forgot to mention that with these two schemes, the compilation performance is greatly improved because the common parts are skipped and no compilation is required; And for separate shared files can also do cache, load performance is also improved. Data will not be posted, their application scenarios are different, clear in mind.
The resources
Webpack 5 Module Federation: A game-changer in JavaScript architecture