“This is the first day of my participation in the Gwen Challenge in November. Check out the details: The last Gwen Challenge in 2021”
Theia startup process
Before we look at the number of Theia plugin child processes, we need to delve into the startup process of Theia. In Build Your own IDE, a development project based on Theia, we see that package.json actually executes the Theia start command when NPM run dev.
"scripts": {
"prepare": "yarn run clean && yarn build && yarn run download:plugins"."clean": "theia clean"."build": "theia build --mode development"."start": "theia start --plugins=local-dir:plugins"."download:plugins": "theia download:plugins"
},
Copy the codeThis command is actually registered in global Node.js by Theia by building a command based on the Node.js CLI. Its main source in theia is the **@theia/ CLI ** package
@theia/cli
In package.json of @theia/cli, register theia’s global commands by specifying the bin field:
// dev-packages\cli\package.json
"bin": {
"theia": "./bin/theia"
},
Copy the code./bin/theia:
#! /usr/bin/env node
require('.. /lib/theia')
Copy the codeThe entry file is in lib/theia, the source code is in SRC /theia.ts, and there is a closure function inside that executes immediately
const projectPath = process.cwd();
const appTarget: ApplicationProps.Target = yargs.argv['app-target'];
const manager = new ApplicationPackageManager({ projectPath, appTarget });
const target = manager.pck.target;
function commandArgs(arg: string) :string[] {
const restIndex = process.argv.indexOf(arg);
returnrestIndex ! = = -1 ? process.argv.slice(restIndex + 1) : [];
}
yargs
.command({
command: 'start'.describe: 'start the ' + manager.pck.target + ' backend'.handler: async() = > {try {
manager.start(commandArgs('start'));
} catch (err) {
console.error(err);
process.exit(1); }}})/ / to omit...
// see https://github.com/yargs/yargs/issues/287#issuecomment-314463783
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const commands = (yargs as any).getCommandInstance().getCommands();
const argv = yargs.demandCommand(1).argv;
const command = argv._[0];
if(! command || commands.indexOf(command) === -1) {
console.log('non-existing or no command specified');
yargs.showHelp();
process.exit(1);
} else {
yargs.help(false);
}
Copy the codeInternally you can see the command line arguments processed through yargs and given to the Start method to ApplicationManager.
// dev-packages\application-manager\src\application-package-manager.ts
start(args: string[] = []): cp.ChildProcess {
if (this.pck.isElectron()) {
return this.startElectron(args);
}
return this.startBrowser(args);
}
startElectron(args: string[]): cp.ChildProcess {
let appPath = this.pck.projectPath;
if (!this.pck.pck.main) {
appPath = this.pck.frontend('electron-main.js');
}
const { mainArgs, options } = this.adjustArgs([appPath, ...args]);
const electronCli = require.resolve('electron/cli.js', { paths: [this.pck.projectPath] });
return this.__process.fork(electronCli, mainArgs, options);
}
startBrowser(args: string[]): cp.ChildProcess {
const { mainArgs, options } = this.adjustArgs(args); options.detached = process.platform ! = ='win32';
return this.__process.fork(this.pck.backend('main.js'), mainArgs, options);
}
Copy the codeThe start method determines whether the current startup environment is browser or electron base based on the parameter type. And load the corresponding entry file in the project. Note at the end of the code that startBrowser and startElectron both call this.__process.fork, which actually generates a new process by calling node.js’s cp function. Using startElectron as an example, the new process will specify the main process execution file (electron/cli.js) to startElectron and load the entry file in the front main process src-gen/fronted.
src-gen/fronted
// examples\electron\src-gen\frontend\electron-main.js
async function start() {
const application = container.get(ElectronMainApplication);
await application.start(config);
}
module.exports = Promise.resolve()
.then(function () { return Promise.resolve(require('@theia/api-samples/lib/electron-main/update/sample-updater-main-module')).then(load) })
.then(start).catch(reason= > {
console.error('Failed to start the electron application.');
if (reason) {
console.error(reason); }});Copy the codeThis file calls the start method to execute the ElectronMainApplication’s start function.
async start(config: FrontendApplicationConfig): Promise<void> {
this._config = config;
this.hookApplicationEvents();
const port = await this.startBackend();
this._backendPort.resolve(port);
await app.whenReady();
await this.attachElectronSecurityToken(port);
await this.startContributions();
await this.launch({
secondInstance: false.argv: this.processArgv.getProcessArgvWithoutBin(process.argv),
cwd: process.cwd()
});
}
protected async startBackend(): Promise<number> {
const noBackendFork = process.argv.indexOf('--no-cluster')! = = -1;
process.env.THEIA_APP_PROJECT_PATH = this.globals.THEIA_APP_PROJECT_PATH;
process.env.THEIA_ELECTRON_VERSION = process.versions.electron;
if (noBackendFork) {
process.env[ElectronSecurityToken] = JSON.stringify(this.electronSecurityToken);
const address: AddressInfo = await require(this.globals.THEIA_BACKEND_MAIN_PATH);
return address.port;
} else {
const backendProcess = fork(
this.globals.THEIA_BACKEND_MAIN_PATH,
this.processArgv.getProcessArgvWithoutBin(),
await this.getForkOptions(),
);
return new Promise((resolve, reject) = > {
// The backend server main file is also supposed to send the resolved http(s) server port via IPC.
backendProcess.on('message'.(address: AddressInfo) = > {
resolve(address.port);
});
backendProcess.on('error'.error= > {
reject(error);
});
app.on('quit'.() = > {
// Only issue a kill signal if the backend process is running.
// eslint-disable-next-line no-null/no-null
if (backendProcess.exitCode === null && backendProcess.signalCode === null) {
try {
// If we forked the process for the clusters, we need to manually terminate it.
// See: https://github.com/eclipse-theia/theia/issues/835
process.kill(backendProcess.pid);
} catch (error) {
// See https://man7.org/linux/man-pages/man2/kill.2.html#ERRORS
if (error.code === 'ESRCH') {
return;
}
throwerror; }}}); }); }}Copy the codeThis function creates the main process and renderer instance of Electron and also starts the Backend service for Theia. At this time, we learn that when the target is electron, in addition to the main process and rendering process of electron, the main process forks a new process to load main.js from src-gen/ Backend and start the Backend service. According to the previous article, loading of plug-ins is usually done in Backend. In this section, we will take a look at how the loading function of plug-ins is scheduled in Backend.
src-gen/backend
The full path of main.js here is src-gen/backend/main.js, which is generated by backend-generator.ts in the @theia/application-manager module. This is the entry file for Backend startup. During initialization, src-gen/ Backend /server.js is scheduled to bind some dependency injection modules. At the same time, the start method of BackendApplication is called to start the back-end application completely.
function load(raw) {
return Promise.resolve(raw.default).then(
module= > container.load(module)); }function start(port, host, argv = process.argv) {
if(! container.isBound(BackendApplicationServer)) { container.bind(BackendApplicationServer).toConstantValue({configure: defaultServeStatic });
}
return container.get(CliManager).initializeCli(argv).then(() = > {
return container.get(BackendApplication).start(port, host);
});
}
module.exports = (port, host, argv) = > Promise.resolve()
.then(function () { return Promise.resolve(require('@theia/core/lib/electron-node/keyboard/electron-backend-keyboard-module')).then(load) })
.then(function () { return Promise.resolve(require('@theia/core/lib/electron-node/token/electron-token-backend-module')).then(load) })
.then(function () { return Promise.resolve(require('@theia/core/lib/electron-node/hosting/electron-backend-hosting-module')).then(load) })
.then(function () { return Promise.resolve(require('@theia/plugin-ext/lib/plugin-ext-backend-electron-module')).then(load) })
/ /... omit
)
.then(() = > start(port, host, argv)).catch(error= > {
console.error('Failed to start the backend application:');
console.error(error);
process.exitCode = 1;
throw error;
});
Copy the codeThat’s how theIA started. Finally, the flow chart is as follows:
The process of establishing communication based on webSocket:
As shown above, FrontendApplication and BackendApplication exist in the startup process of THEIA. How do the two communicate and access? How is the process of communication established? In analyzing server.js, we found that when loading various dependent modules, Then (function () {return Promise. Resolve (the require (‘ @ sensics/plugin – ext/lib/plugin – ext – backend – electron – module ‘)). Then (load)}), This module is mainly used to load and bind modules of back-end extensions:
// packages\plugin-ext\src\plugin-ext-backend-module.ts
export default new ContainerModule(bind= > {
bindMainBackend(bind);
bindHostedBackend(bind);
});
Copy the codeBack-end plug-in modules are classified into two types: primary plug-in module and third-party plug-in module, which correspond to bindMainBackend and bindHostedBackend respectively. Extension modules of third-party plug-ins are bound in bindHostedBackend
// packages\plugin-ext\src\hosted\node\plugin-ext-hosted-backend-module.ts
const commonHostedConnectionModule = ConnectionContainerModule.create(({ bind, bindBackendService }) = >{... BindBackendService <HostedPluginServer, HostedPluginClient>(server, client) = > {
server.setClient(client);
client.onDidCloseConnection(() = > server.dispose());
returnserver; }); . Omit});export function bindCommonHostedBackend(bind: interfaces.Bind) :void {... Omit the bind (ConnectionContainerModule). ToConstantValue (commonHostedConnectionModule); . Omit}export function bindHostedBackend(bind: interfaces.Bind) :void {... Omit bindCommonHostedBackend (bind); . Omit}Copy the codeWhen bindHostedBackend binds third-party plug-in extension modules, the inversify Container API binds and instantiates the key logic of loading third-party modules on the backend. In commonHostedConnectionModule method, binding a back-end service bindBackendService, this raises the question: why do you want to bind a back-end service? The reason is that Theia is based on RPC communication for both front-end and back-end communication. This idea follows VS Code’s back and forth communication mechanism. In VS Code, both front and back end communication is based on RPC and follows a standard specification: JSON-RPC specification. Json-rpc is a stateless and lightweight remote procedure call (RPC) protocol. This specification mainly defines some data structures and related processing rules. It allows you to run in the same process based on socket, HTTP, and many other different messaging environments. It uses JSON (RFC 4627) as the data format. It was launched for one purpose only: simple communication! In fact, many vendors are following the JSON-RPC specification for service design of front-end and back-end interface invocation. For example, Language Server Protocol Node (LSP) provided by VS Code is widely used to define the front-end and back-end communication of a plug-in. The back and forth RPC communication in Theia is based on the VS Code JSON-RPC websocket plugin: VS Code-WS-JSONRPC. The front and back end RPC communication is connected based on Websocket, Theia will create a Websocket channel in each module that needs the front and back end communication to ensure two-way communication. The commonHostedConnectionModule code is specified for the backend inside a service request to associate a front-end, request service service – the url is: hostedServicePath, understandable into Java servlet in a path. Each time a client sends a request to this path, it intercepts and forwards the request. It also binds a backend service instance, HostedPluginServer, and binds a client according to vs code-WS-JsonRPC’s proxy service. So where is this client declared and bound? Let’s go back to the binding code for the front-end plug-in scheduling module:
// packages\plugin-ext\src\main\browser\plugin-ext-frontend-module.ts
bind(HostedPluginServer).toDynamicValue(ctx= > {
const connection = ctx.container.get(WebSocketConnectionProvider);
const hostedWatcher = ctx.container.get(HostedPluginWatcher);
return connection.createProxy<HostedPluginServer>(hostedServicePath, hostedWatcher.getHostedPluginClient());
}).inSingletonScope();
Copy the codeIn the process of front-end module loading, will create a WebSocketConnectionProvider, namely through vscode here – ws – jsonrpc to create a websocket path. It creates a proxy that specifies the current client, Client. In this case, HostedPluginServer is bound to a HostedPluginServer instance through bind(HostedPluginServer). Once the front-end function calls the HostedPluginServer instance for RPC invocation of the backend method, this proxy will be triggered to schedule the relevant module functions of the backend. I’ll write a follow-up article to explain the details. In Theia, the service terminal plug-in process is started in the HostedPluginServer object, so we start calling the service from the client side and create a child process to the server side to track the process.
Invoke the plug-in child process creation process
When Theia’s front-end service started creating editor views/controls, it followed VS Code loading: plug-ins were loaded sequentially based on front-end plug-in extension points. As mentioned at the end of the first article, THE extension of VS Code’s front-end plug-in is required to declare that the configuration for each module loaded by the editor is defined by Contributes Points. Each defined editor control itself needs to inherit FrontendApplicationContribution interface, its internal defines the controls in the process of loading the lifecycle callback, need to implement in the development of specific control.
// packages\core\src\browser\frontend-application.ts
export interface FrontendApplicationContribution {
/** * Called on application startup before configure is Called. * The Initialize lifecycle method is triggered before the configure() method is Called after the application is started. * /initialize? () :void;
/** * Called before commands, key bindings and menus are initialized. * Should return a promise if it runs asynchronously. * Is called before command panel initialization, before hotkey binding, and before menu initialization, and returns a Promise */ if asynchronousconfigure? (app: FrontendApplication): MaybePromise<void>;
/** * Called when the application is started. The application shell is not attached yet when this method runs. * Should Return a promise if it runs asynchronously. * Will be called when the application starts. This method is executed before the shell form of the entire application has been instantiated. If the method is asynchronous, return a Promise */onStart? (app: FrontendApplication): MaybePromise<void>;
/** * Called on `beforeunload` event, right before the window closes. * Return `true` in order to prevent exit. * Note: No async code allowed, this function has to run on one tick. * This function is called when the beforeUnload event is triggered before the window closes. * /onWillStop? (app: FrontendApplication):boolean | void;
/** * Called when an application is stopped or unloaded. * * Note that this is implemented using `window.beforeunload` Which doesn't allow any asynchronous code anymore. * I.e. this is the last tick. * called when the app is stopped or timed */onStop? (app: FrontendApplication):void;
/** * Called after the application shell has been attached in case there is no previous workbench layout state. * Should Return a promise if it runs asynchronously. * Called after the shell form of the entire application is instantiated. * /initializeLayout? (app: FrontendApplication): MaybePromise<void>;
/** * An event is emitted when a layout is initialized, but before the shell is attached. */onDidInitializeLayout? (app: FrontendApplication): MaybePromise<void>;
}
Copy the codeIn the process of front-end plug-in instantiation, by implementing FrontendApplicationContribution interface, onstart method, used to load the front control.
// packages\plugin-ext\src\hosted\browser\hosted-plugin.ts
onStart(container: interfaces.Container): void {
this.container = container;
this.load();
this.watcher.onDidDeploy(() = > this.load());
this.server.onDidOpenConnection(() = > this.load());
}
Copy the codeThe front-end control is loaded first by synchronizing a list of loaded plug-in ids.
// packages\plugin-ext\src\hosted\browser\hosted-plugin.ts
protected async doLoad(): Promise<void> {
const toDisconnect = new DisposableCollection(Disposable.create(() = > { /* mark as connected */ }));
toDisconnect.push(Disposable.create(() = > this.preserveWebviews()));
this.server.onDidCloseConnection(() = > toDisconnect.dispose());
// process empty plugins as well in order to properly remove stale plugin widgets
await this.syncPlugins();
}
protected async syncPlugins(): Promise<void> {
let initialized = 0;
const syncPluginsMeasurement = this.createMeasurement('syncPlugins');
const toUnload = new Set(this.contributions.keys());
try {
const pluginIds: string[] = [];
const deployedPluginIds = await this.server.getDeployedPluginIds(); }}Copy the codeSyncPlugins in this. Server. GetDeployedPluginIds (); Method, as described earlier, connects to the backend HostedPluginServer instance through a HostedPluginSupport proxy object declared by the front-end service. That’s the Server object in your code. The server proxy object getDeployedPluginIds() method is then called and the scheduling process initiates a Websocket request to access the server proxy object instance getDeployedPluginIds() method based on the VScode-JSON-RPC plug-in. Let’s look at an implementation of this method on the plug-in server side:
// packages\plugin-ext\src\hosted\node\plugin-service.ts
async getDeployedPluginIds(): Promise<string[] > {const backendMetadata = await this.deployerHandler.getDeployedBackendPluginIds();
if (backendMetadata.length > 0) {
this.hostedPlugin.runPluginServer(); }}Copy the codeThis method creates a plug-in service on the server side: runPluginServer()
// packages\plugin-ext\src\hosted\node\hosted-plugin-process.ts
public runPluginServer(): void {
if (this.childProcess) {
this.terminatePluginServer();
}
this.terminatingPluginServer = false;
this.childProcess = this.fork({
serverName: 'hosted-plugin'.logger: this.logger,
args: []});this.childProcess.on('message'.message= > {
if (this.client) {
this.client.postMessage(PLUGIN_HOST_BACKEND, message); }}); }Copy the codeIn runPluginServer, a separate Node.js process is created using the this.fork method:
private fork(options: IPCConnectionOptions): cp.ChildProcess {
const childProcess = cp.fork(this.configuration.path, options.args, forkOptions);
return childProcess;
}
Copy the codeTo start the front-end plug-in child process. After that, let’s run the process through the interruption point and observe the number of processes:
// dev-packages\electron\electron-cli.js
require('electron/cli.js');
Copy the codeAs you can see from the above code, electron-cli.js enables the startup of the entire application by requiring cli.js. Cli.js will first create a browser_init child process to call electron. Exe (in blue), At the same time, a development example project “– app-proje-path =D:\\git\\theia-1.16.0\\theia-1.16.0/examples/electron” was specified in a source code repository. It is the Theia development project mentioned at the beginning of this article. The entry file specified in package.json is:
//examples\electron\package.json
{
"main": "src-gen/frontend/electron-main.js",}Copy the codeThe Electron Frontend command starts to create. After that, the front-end and back-end modules will be loaded step by step according to the logic analyzed above, and the plug-in sub-process plugin-host.js will be created. From the above, we can clearly draw a conclusion that Theia starts a host process through backend-service to host two sets of plug-in systems respectively for both front-end and back-end plug-ins. This paper mainly analyzes how to create a whole set of plug-in sub-processes from the whole theIA startup process and the front and back end service communication mode, and creates several. Although looked from the conclusion, sensics only launched a child process used to host all the front-end and back-end plugin, but from the point of view of construction of our own product demand, the train of thought, we can also divergent development, according to the needs of the business scenario, the plug-in classified by type, is not limited to only create the plug-in the number of child processes.
reference
- Theia technology reveals JSON-RPC communication
- Communication via JSON-RPC
- JSON xml-rpc Specification 2.0
- microsoft/vscode-languageserver-node